Support for polymorphic types is unclear in the spec

While trying to work through apigee-127/swagger-tools#241 it became clear that the spec is not clear on how polymorphic types are intended to work. @webron will add more details.

In 2.0 we changed how inheritance is declared, but it seems that it is not well defined.
Take the following example from the spec:
definitions:
  Pet:
    discriminator: petType
    properties:
      name:
        type: string
      petType:
        type: string
    required:
    - name
    - petType
Cat:
  description: A representation of a cat
  allOf:
  - $ref: '#/definitions/Pet'
  - properties:
      huntingSkill:
        type: string
        description: The measured skill for hunting
        default: lazy
        enum:
        - clueless
        - lazy
        - adventurous
        - aggressive
    required:
    - huntingSkill
Dog:
  description: A representation of a dog
  allOf:
  - $ref: '#/definitions/Pet'
  - properties:
      packSize:
        type: integer
        format: int32
        description: the size of the pack the dog is from
        default: 0
        minimum: 0
    required:
    - packSize

The Pet defines a discriminator field, petType. The discriminator field is meant to help some strongly typed languages identify how to map the input to the right class.
The problem is that the `discriminator

In 2.0 we changed how inheritance is declared, but it seems that it is not well defined.
Take the following example from the spec:
{
  "definitions": {
    "Pet": {
      "discriminator": "petType",
      "properties": {
        "name": {
          "type": "string"
        },
        "petType": {
          "type": "string"
        }
      },
      "required": [
        "name",
        "petType"
      ]
    }
  },
  "Cat": {
    "description": "A representation of a cat",
    "allOf": [
      {
        "$ref": "#/definitions/Pet"
      },
      {
        "properties": {
          "huntingSkill": {
            "type": "string",
            "description": "The measured skill for hunting",
            "default": "lazy",
            "enum": [
              "clueless",
              "lazy",
              "adventurous",
              "aggressive"
            ]
          }
        },
        "required": [
          "huntingSkill"
        ]
      }
    ]
  },
  "Dog": {
    "description": "A representation of a dog",
    "allOf": [
      {
        "$ref": "#/definitions/Pet"
      },
      {
        "properties": {
          "packSize": {
            "type": "integer",
            "format": "int32",
            "description": "the size of the pack the dog is from",
            "default": 0,
            "minimum": 0
          }
        },
        "required": [
          "packSize"
        ]
      }
    ]
  }
}

The Pet defines a discriminator field, petType. The discriminator field is meant to help some strongly typed languages identify how to map the input to the right class.
The problem is that for the discriminator field to work, we need to know what value is assigned to that field for each instance of its inheritor. This part is currently not well defined in the spec or the samples.
I can think of four alternatives to solve it:

The value of the discriminator can be implicit by the model definition name under definitions. There are two problems here - this will not work with inline-defined models (though may not be relevant), and may not work with some remotely referenced models as well. This also forces the model name to be the type which is not always desirable.
The value of the discriminator needs to be explicitly overridden in the inheritor's definition. This specific option is ugly, verbose and error-prone, but allows the flexibility. Again, if not declared, the fallback could still be option 1 (as in, the model name as defined). (example below)
Like the first option, use the title property of the schema. The problem is that the usage of title in general is not well defined in the spec and this may end up being abuse of actual intent.
The last option is to add a new field, like we added the discriminator, such as class (or whatever), and its value would determine the value, in case the user doesn't want to assume the model name used under definitions. While this is a more elegant solution, it is technically a breaking change in the spec (even though it only adds, and doesn't mutate/remove from it). (example below)

Example for option 2:
Dog:
  description: A representation of a dog
  allOf:
  - $ref: '#/definitions/Pet'
  - properties:
      petType:
        type: string
        enum:
          - dog
      packSize:
        type: integer
        format: int32
        description: the size of the pack the dog is from
        default: 0
        minimum: 0
    required:
    - packSize

Example for option 4:
Dog:
  description: A representation of a dog
  class: dog
  allOf:
  - $ref: '#/definitions/Pet'
  - properties:
      packSize:
        type: integer
        format: int32
        description: the size of the pack the dog is from
        default: 0
        minimum: 0
    required:
    - packSize

Other considerations to take into account:

Options 2 and 4 can be used to allow for composition and inheritance of the same base model if we say that the class assignment has to be explicit (that is the model name will never be used by default). That way, if a model definitions allOf's the same base class but doesn't declare the discriminator value, it will not be considered a child of it and will simply be a composition of both models' properties.
We need to make sure the logic we choose applies for deep inheritance as well.
While not directly related to this issue, there have been requests of being able to mention that the base model is 'abstract' and cannot be used directly as input even if referenced. In the example above, that would mean that if Pet is referenced as a body parameter, only Cat and Dog would be applicable. If we'll end up going with option 4, we may want to add an additional property declaring that.

I want to caution against doing abstract classes. That takes a very language specific view of the world. Not all languages have classes or inheritance. Most of them have polymorphism but classes and especially hierarchies with abstractness etc are only available in a small subset of languages. Java does happen to be one of them.
I think inheritance and the structuring thereof is well outside the realm of describing remote API's.
Even the discriminator field is questionable to have in the spec, this seems to be an application specific something and not really the API.

One issue with the examples above is that it doesn't allow multiple values to map onto the same subclass, which could lead to unnecessary duplication within the swagger file. A potential 5th option would be to define the mapping on the parent class, so something like this:
  "definitions": {
    "Pet": {
      "discriminator": "petType",
      "discriminatorMap": {
        "cat": {"$ref": "#/definitions/Cat"},
        "dog": {"$ref": "#/definitions/Dog"}
      }
      "properties": {
        "name": {
          "type": "string"
        },
        "petType": {
          "type": "string"
        }
      },
      "required": ["name","petType"]
    }
}

This also has the advantage of being able to construct the entire inheritance tree from the same place where you see the discriminator defined, rather than having to search all defined schemas to see which of them inherit from the parent class.
Also, w/r/t option 4, I find class to be a fairly unintuitive name for that value. It seems like it could be something more straightforward like discriminatorValue

There is a 6th option (completely compatible with the existing JSON schema): using enum constraints for matching. That would allow to separate the model name from the discriminator property value. In the example below, the petType is in lowercase while the model name is capitalized.
    "Cat": {
          ...
          properties: {
            "petType": {
              "type": "string",
              "enum": [
                 "cat"
              ]
            },
          ...
        },

Full document:
{
  "definitions": {
    "Pet": {
      "discriminator": "petType",
      "properties": {
        "name": {
          "type": "string"
        },
        "petType": {
          "type": "string"
        }
      },
      "required": [
        "name",
        "petType"
      ]
    },
    "Cat": {
      "description": "A representation of a cat",
      "allOf": [
        {
          "$ref": "#/definitions/Pet"
        },
        {
          "properties": {
            "petType": {
              "type": "string",
              "enum": [
                 "cat"
              ]
            },
            "huntingSkill": {
              "type": "string",
              "description": "The measured skill for hunting",
              "default": "lazy",
              "enum": [
                "clueless",
                "lazy",
                "adventurous",
                "aggressive"
              ]
            }
          },
          "required": [
            "huntingSkill"
          ]
        }
      ]
    },
    "Dog": {
      "description": "A representation of a dog",
      "allOf": [
        {
          "$ref": "#/definitions/Pet"
        },
        {
          "properties": {
            "petType": {
              "type": "string",
              "enum": [
                 "dog"
              ]
            },
            "packSize": {
              "type": "integer",
              "format": "int32",
              "description": "the size of the pack the dog is from",
              "default": 0,
              "minimum": 0
            }
          },
          "required": [
            "packSize"
          ]
        }
      ]
    }
  }
}

@dolmen - that's option 2...

From the perspective of a polymorphic API designer (not a tool builder), I would prefer options 1 & 4. That is, I would prefer that the discriminator is implicitly considered to be the model definition name as defined under definitions (perhaps lowercase'd), unless an explicit class (or discriminatorValue or whatever) is defined.
That said, I'm not sure I understand @toomuchpete's critique. @toomuchpete: can you give an example where multiple values would map onto the same subclass, requiring unnecessary duplication?
@webron: how do you see proceeding with this issue? I would love to define my polymorphic API with Swagger!

We may implement support for option 4 in a non-breaking way by using a vendor extension x-class, however I can't give more details as to if/when it would happen.

It seems to me that option 4 is an extension of option 1. If option 4 is the decided goal, and option 1 is non-breaking, could we declare option 1 to be the currently supported option (that doesn't support inline-defined models and remotely referenced models), then add support for additional use-cases through the addition of x-class later?
Serious apologies if this is a noob comment and I am misunderstanding the effects of the change.

In a way, I believe we have to assume option 1 anyways for existing implementations.

With respect to APIs not yet created but that are dedicated to using Swagger, this discussion mostly boils  down to the user-friendliness of Swagger itself. The APIs can be crafted around whatever Swagger's limitations are.
However, if the intent is that Swagger be used to describe a large set of existing APIs, care needs to be taken not to make an API impossible to define in Swagger.
Option 1 above does this. Consider a simple case where two different types of objects in the same API are polymorphic and their discriminator values overlap. In a small API this might be unlikely, but in larger APIs, fields like "type" and "status" can often have very repetitive values even across different objects. Option 1 requires the overlapping child definitions be located at the same location in the Swagger. It's not particularly suitable for many existing, real-world APIs.
To address @dhoelle's question about Option 2: I didn't fully flesh out this thought in my comment above. It definitely solves the problem of mapping multiple values to a single definition. My concern with option 2 is that it raises all of the issues that JSON Schema's "oneOf" with the added bonus of needing to access every definition in the Swagger document to determine the result and it leaves undefined the result of multiple definitions that match a particular discriminator value. This is already a bit of an issue with using "allOf" for inheritance, since there's no way know which of the schemas in that array is the real "parent".
If our goal is to address the lack of clarity in the spec without making any breaking changes, then I suppose some combination of options 1 and 2 are the ones that we'd have to choose. For that, I like option 2 without the option 1 fallback. IMO, if the value of the discriminator isn't found in a subclass's definition, the parent class definition should be used by itself. That reduces the possibility of weird and potentially dangerous bugs with data values being able to link into other places within the document.
Both option 1 and option 2 do have serious deficiencies, though, and I'd love to see a v2.1 even if all that does is address the polymorphism complexity. If not, I hope this topic will be revisited in whatever the next version of Swagger is.
An explicit mapping would make the author's intent clear, it would make implementation of parsers and other systems far less complex, and it allow the system to remain flexible. There are definitely other reasonable approaches, though.
For now, I'll probably need to use some custom extensions for an active project. Once I have a swagger doc I can share publicly I'll post a link here.

Hi,
@wing328 pointed me to this issue. I haven't read through all the comments yet and not sure I understand all the edge cases yet either.
I just wanted to point you towards some work I did on implementing polymorphism in the PHP client: https://github.com/swagger-api/swagger-codegen/compare/master...arnested:php-polymorphism-client (please note that it requires a snapshot release of swagger-parser because of https://github.com/swagger-api/swagger-parser/pull/128).
As I wrote I haven't read/understood every aspect of this yet, but for my (possibly limited) use case the above fork/branch solves our problem of having an API endpoint return a specific class (with a discriminator property defined) and all subclass inheriting from it (through allOf).
I'll try to look closer in to this issue to see if I have something valuable to add. I might be limited in time by a work deadline.

FYI. Swagger parser 1.0.12 has been released several hours ago: https://github.com/swagger-api/swagger-parser/releases

Thank you for the hint, @wing328. I've updated my fork/branch.

Tackling PR: #741

From what I understand, inheritance relation is defined by allOf: the fact that Cat has a allOf field with a ref to Pet is interpreted as "Cat inherits from Pet" and this is the one and only way to represent inheritance in the spec. Then, in case of polymorphism, discriminator points to the right schema.
I might be missing something, but what about a case where the child not only adds but also overrides properties? I don't have a pet-store-simple example. While overriding methods is frequent in OOP, overriding attributes is less common in my limited experience but legitimate AFAIK.
In this case, Child still inherits from Parent but the spec can't read
  "definitions": {
    "Child": {
      "type": "object",
      "allOf": [
        {
          "$ref": "#/definitions/Parent"
        }
        [...]

because some attributes from Parent are redefined (different type, validators (range,...), properties (readOnly),...) so Child fields can't validate against Parent schema.
In this case, I don't know how to represent inheritance.
The easy solution that comes to mind is to use a x-inherits or x-childOf attribute to define inheritance when allOf can't.
Is the use case I'm presenting illegitimate? Is there another way to deal with it already?
FWIW, related post on SO: http://stackoverflow.com/a/42474248/4653485.
(With a inherits/childOf attribute introduced, I'd be tempted to go even further and rely solely upon it to define inheritance, in which case allOf would just be a tool to avoid redundancy, with no semantic meaning. After all, I can also imagine a class having all the attributes from another one without any inheritance link. Currently, I can't use allOf there to cut redundancy because there is no actual parent/child relationship between the two classes. Obviously, this would be a totally breaking change, probably not justified by the limited potential benefit.)

@lafrech allOf in JSON schema means that the object to be validated must be valid for all of the listed schemas. So you can't have an object of a sub type which is not also a valid object of the parent type. If you have properties with same name but different type in both, there will be no objects matching both schemas, so it is basically an empty type. If you have validations, they simply are merged (i.e. all of them must apply).

Thanks @ePaul for your comment.
I totally understand that. It means that

either allOf can't represent all polymorphism cases by itself and there should be another way (for instance an explicit link to parent like inherits or childOf,

either overriding an element when inheriting, rather than just adding elements, is a wrong use of polymorphism. Is it?

So far as follows from here https://github.com/OAI/OpenAPI-Specification/blob/master/versions/2.0.md#composition-and-inheritance-polymorphism only a half of "variant 2" has been included into spec, the one which demands coincidence between discriminator value and the definitions field name. Which requires discriminator value to be unique among all the definitions and is highly impractical as @toomuchpete said. Also it doesn't allow you to derive schema hierarchy from the swagger definition, unless you make an assumption that if schema has a single reference in allOf section, then this schema is sub-type of one that is referenced which is a weak assumption because allOf was primarily designed for composition not specifying hierarchies.
Would it be legitimate for me to take the approach with specifying discriminator values via enums instead?
And what about allOf and field overriding? Is it allowed? From the first glance looks like it isn't.

+1 for using enum to specify the child's discriminator value. As for using the enum in the parent class... well
If we include this information in the parent class, then developing a child class requires modifying the parent class. However, if the information is only in the child class, then you don't have control over which classes are acceptable -- simply adding a new definition can effectively change the API of multiple paths.
I'm starting to think the root of this problem is that we can't specify parameters as anyOf. If we could, then there would be no need for discriminator for many applications. You could just declare the "child classes" with allOf, and then list the ones you want to accept in the various paths of your API.
Let's say Cat, Dog and HoneyBee are Pets, but because of regulations due to the decline in bee populations, we have to provide separate endpoints for HoneyBees now (let's say we want our gateway to load balance differently for them). If we use anyOf in parameter definitions, then we only have to remove the HoneyBee entry from our existing paths. If we use something akin to discriminator instead, then we have to make it so HoneyBee is not a Pet anymore, which doesn't make much sense.
On top of the anyOf solution we could add support for discriminator for reifying that type information must be passed by the clients in requests. All we would need is a string in the "parent" class specifying what's the name of the discriminator parameter, and then in the "child classes" a definition of this parameter is required, and it has to have an enum entry with at least one value. If discriminator definitions are not compatible, or if two classes define the same enum value for their discriminator, then it's an error in the API.
You 'd need this discriminator if you want to generate code in a language without much support for polymorphism or union types, but for those languages that don't need it, you don't necessarily need the discriminator. This would help a lot with documenting existing APIs.

Please take a look at https://github.com/OAI/OpenAPI-Specification/pull/1093

I'm looking for docs tooling that displays polymorphism either as defined in version 2 spec or version 3 spec.
In my research the only docs tooling that handles and displays polymorphism is Swagger UI using the v3 spec.
The other docs tools that I've come across only support version 2 of the spec but I can't seem to find any that handle polymorphism as defined in the v2 spec.
Any help is appreciated!

In addition to swagger-ui, Rebilly ReDoc supports discriminator based polymorphism, and recent alphas support OpenAPI 3.0.x

Related to this ambiguity, the spec is not very clear about how to represent lists of polymorphic types. For example, suppose the "Pet Store" API can return a list of pets:
store:
  properties:
    pets:
      description: The collection of pets in this store
      type: array
      items:
        - $ref: '#/components/schemas/Pet'

When the API designer writes this spec, they could have two separate intents:

Return a non-polymorphic list of pets, e.g. [{name, petType}, {name, petType}...]
Return a polymorphic list of pets, where each item in the array is a "subclass" of pet, i.e. list of dogs, cats.
Possibly a list that includes a mix of 1 and 2.

For #2, using "oneOf" is one way to solve the problem, but it gets unwieldy when there are lots of "subclasses". Also, "oneOf" makes it challenging because the list of $ref in oneOf may be a subset of all subclasses, and it doesn't seem possible to specify any "subclass of".
store:
  properties:
    pets:
      description: The collection of pets in this store
      type: array
      items:
        oneOf:
          - $ref: '#/components/schemas/Dog'
          - $ref: '#/components/schemas/Cat'

Can I use a defined enum property as discriminator?
"PetType": {         "type": "string",         "enum": [           "CAT",           "DOG"         ]  }, "Dog": {       "description": "A representation of a dog",       "allOf": [         {           "$ref": "#/definitions/Pet"         },         {           "properties": {             "petType": {                    "$ref": "#/definitions/PetType"}           }         }       ]   }

This was fixed by a PR in 2017, so I'm closing it.  To answer the three subsequent questions:

our only additional docs are on learn.openapis.org (to which you can contribute; we don't keep track of other documentation/advice sites
as the point of discriminator is to tell tools how to instantiate the correct subclass, that is what it does regardless of whether it is in a collection or not, for each instance to which that schema object applies
yes, you can definitely use type: string, enum: [...] for your discriminating property


[$500] mWeb- Chat - Android chrome crashes on opening password protected PDF

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Open the app
Open any report with password protected PDF sent to it
Open the password protected PDF
Observe that app crashes

Expected Result:
App should not crash on opening password protected PDF
Actual Result:
App crashes on opening password protected PDF on android chrome
Workaround:
Unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[x] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[ ] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number: 1.3.74-0
Reproducible in staging?: Y
Reproducible in production?: Y
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation

Expensify/Expensify Issue URL:
Issue reported by: @dhanashree-sawant
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1695401444195489
View all open jobs on GitHub
Upwork Automation - Do Not Edit

Upwork Job URL: https://www.upwork.com/jobs/~016f798c5b0834a975
Upwork Job ID:<PHONE_NUMBER>647206912
Last Price Increase: 2023-10-02
Automatic offers:

Ollyws | Reviewer | 27099989
ZhenjaHorbach | Contributor | 27099993
dhanashree-sawant | Reporter | 27099994

Hmm, works for me on my phone running Chrome 117

Melvin was acting up, offers have been created now!

Issue not reproducible during KI retests. (First week)

I'm OOO Oct 16-23. Adding leave buddy
Status: waiting to issue payment https://github.com/Expensify/App/issues/28179#issuecomment-1765323839

@Ollyws review + bonus = $750 - please accept the offer
@ZhenjaHorbach fix + bonus = $750 - paid ✔
@dhanashree-sawant report = $50 - paid ✔

Haha oh no, let me go find the one you've accepted

Paid ✔


dark mode

Hi, if it is possible to allow users to select dark mode?

thanks for the suggestion , will plan
On Tue, Nov 14, 2023 at 11:01 AM 神谷実 @.***> wrote:

Hi, if it is possible to allow users to select dark mode?
—
Reply to this email directly, view it on GitHub
https://github.com/midhunz/gmail-notifier/issues/11, or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAHPLT7LWKMCUBLLUMNGD4DYEMJODAVCNFSM6AAAAAA7KJZ2ISVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE4TEMJRGM3TGMI
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

--
Thanks & Regards
Midhun P


Do we want wider country infoboxes?

Description
Since we modified the "Report an error" button on the country pages, we have an empty column to the right of dark blue infobox.
Do we want to expand that dark blue to get full witdh?

Or turn the external links (Wikipedia/website) into buttons and add some spacing between each line in the box for readability?

I like that. And perhaps pull in one or two more facts about the country that might be interesting in the Govdirectory context? I am thinking perhaps executive body (P208) and legislative body (P194).

Or something like / in combination #187

Or turn the external links (Wikipedia/website) into buttons and add some spacing between each line in the box for readability?

And is it an exclusive feature for the country page or is it intended to be implemented site-wide? Because the organization page (if all boxes are checked) already has enough buttons.

And is it an exclusive feature for the country page or is it intended to be implemented site-wide? Because the organization page (if all boxes are checked) already has enough buttons.

Oh, that is actually a good argument for not turning those into buttons. We don't want to overflow the organization pages and I wouldn't want content to be in different places on org and country pages.

Or we can do the same as the 2 small buttons at the top; With the Wikipedia logo, but I'm afraid it won't be clear enough...

I think we should move the website (for both countries and orgs) to a right-aligned button to give the link more visibility. The downside might be that we would be moving away from the idea of the buttons being "actions".


Xml server status

Adding xml server status to the xml api section of the documentation.
Please let me know if i have done anything incorrect.

Hey man,
Thank you very much for the PR. As this is your first PR would you like to add yourself to the CONTRIBUTORS file?

Thanks mate, I just added myself to the CONTRIBUTORS file.
Everything else look good with the PR? I rarely use GitHub so sometimes forget what i am doing.

Yup! Thank you very much. :)


Implement conversion from a Kubernetes store snapshot to a graph

Implement a function/stuct converting a Kubernetes store snapshot into its graph representation. The graph should use Kubernetes objects as its vertices and the edges between them should define dependencies (Ingress depending on a Service, Plugin depending on an Ingress/Service/Consumer, etc.).
The graph will be used to determine what objects need to removed/replaced during the fallback configuration build process - traversing the graph should allow identifying an invalid object's dependants and removing/replacing them along with it.
Blocked by https://github.com/Kong/kubernetes-ingress-controller/issues/5928.
### Tasks
- [ ] Implement ResolveDependency for Kong CRDs

I've implemented dependency resolving for Gateway API and Ingress types - Kong CRDs are left to be done: https://github.com/Kong/kubernetes-ingress-controller/blob/main/internal/dataplane/fallback/graph_dependencies_kong.go


feat: Add a service layer to get AppsmithType from client-type (#16508)

This PR takes care of the following things

It creates a service layer to get the target AppsmithType from the client-side data type and the evaluated value
This service layer is currently not consumed by any plugins
A full JUnit test suit for MySQL specific data types

Description
Fixes #16508
Type of change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

JUnit

Checklist:

[x] My code follows the style guidelines of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes

/ok-to-test sha=708c987

/ok-to-test sha=2d97675

/ok-to-test sha=140dcda

/ok-to-test sha=7b1d726

/ok-to-test sha=6a350a7

/ok-to-test sha=607f214

/ok-to-test sha=58d6f3c

/ok-to-test sha=4fe79c8

/ok-to-test sha=cadc4c4


Add dockerTools test: on top of pulled image

Motivation for this change
This is to ensure a pulled image can be used as base image. This is not trivial since our current docker tooling relies on the legacy Docker image format.
cc @Profpatsch
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option build-use-sandbox in nix.conf on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Fits CONTRIBUTING.md.

@GrahamcOfBorg test docker-tools

Failure on x86_64-linux (full log)
Partial log (click to expand)
error: while evaluating anonymous function at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/nixos/release.nix:19:72, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/lib/attrsets.nix:282:43:
while evaluating ‘hydraJob’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/lib/customisation.nix:162:14, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/nixos/release.nix:19:80:
while evaluating the attribute ‘drvPath’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/lib/customisation.nix:179:13:
while evaluating the attribute ‘drvPath’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/lib/customisation.nix:146:13:
while evaluating the attribute ‘buildCommand’ of the derivation ‘vm-test-run-docker-tools’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘buildCommand’ of the derivation ‘nixos-test-driver-docker-tools’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘buildCommand’ of the derivation ‘nixos-vm’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating the attribute ‘text’ of the derivation ‘run-nixos-vm’ at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating anonymous function at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/pkgs/build-support/closure-info.nix:9:1, called from /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/nixos/modules/virtualisation/qemu-vm.nix:105:13:
assertion failed at /var/lib/gc-of-borg/.nix-test-rs/repo/38dca4e3aa6bca43ea96d2fcc04e8229/builder/ogden/pkgs/build-support/closure-info.nix:11:1

Success on aarch64-linux (full log)
Partial log (click to expand)

The test fails because ofborg is using nix 1.11.


Support for daemon, enarx-wasmldr

This is a big merge from enarx/enarx.  Requires docs.

@MikeCamel This code doesn't belong here. It belongs in enarx-keepmgr.

@MikeCamel This code doesn't belong here. It belongs in enarx-keepmgr.

Which bits of this code?  enarx-keepmgr holds code for the central keep manager process, which manages and coordinates Keep loader instanes.  Anything which is per instance lives here: that's how I segmented the code (as we agreed, I believe).  If we want to have a different split, we need to discuss how that works.
I can see moving all of the daemon pieces out into enarx-keepmgr makes sense if the idea is for keepldr to hold only implementations of Keeps.  Is that what you had in mind?

@MikeCamel This code doesn't belong here. It belongs in enarx-keepmgr.

Which bits of this code? enarx-keepmgr holds code for the central keep manager process, which manages and coordinates Keep loader instanes. Anything which is per instance lives here: that's how I segmented the code (as we agreed, I believe). If we want to have a different split, we need to discuss how that works.

We agreed to a different split. enarx-keepmgr holds all the system-wide state. enarx-wasmldr holds all the per-instance state. The enarx-keepldr has a very minimal job: start and run a static pie binary in a keep. Nothing else.

I can see moving all of the daemon pieces out into enarx-keepmgr makes sense if the idea is for keepldr to hold only implementations of Keeps. Is that what you had in mind?

Yes. It is supposed to be bare bones. enarx-keepmgr and enarx-wasmldr need to be integrated to each other. But enarx-keepldr should be a (mostly) invisible middle man. This is why I described enarx-keepmgr as being able to work by spawning /usr/bin/enarx-wasmldr directly OR via enarx-keepldr exec /usr/bin/enarx-wasmldr. The result should be (mostly) the same.

@MikeCamel This code doesn't belong here. It belongs in enarx-keepmgr.
Which bits of this code? enarx-keepmgr holds code for the central keep manager process, which manages and coordinates Keep loader instances. Anything which is per instance lives here: that's how I segmented the code (as we agreed, I believe). If we want to have a different split, we need to discuss how that works.

We agreed to a different split. enarx-keepmgr holds all the system-wide state. enarx-wasmldr holds all the per-instance state. The enarx-keepldr has a very minimal job: start and run a static pie binary in a keep. Nothing else.

NOTE - please read the below, but don't get irate until you've read the "Important bit" (which will hopefully mean you don't need to get irate).  I kept what I've written in between because I think it raises some useful points in the meantime.
I think we may have agreed to different things.  :-)  Part of the reason for the design as I've done it to this point is to allow state to be held by keeploader so that it can be pre-warmed at an early stage, and manage comms in a (guaranteed) multi-threaded manner with the keepmanager or other possible entities.  This allows us the broadest set of options for different attestation, billing and other protocols in the future, and why there's a state machine in here.  It's also why I wasn't planning on losing the originating process (though I think I'm probably OK with this once that wasmtime piece is loaded and running).
This, btw, is why I wanted to go through the protocol design and work out what we needed, rather than designing on the fly: doing this allows a data model for exactly what data is owned by which components, and when.  It seems that we've come up with conflicting ideas of what sits where, and how best to manage future options, and we need to come to an agreement or we'll go through this again: there are multiple moving parts which need coordination, not stand-alone components, and building distributed systems needs more design work.  In particular, I still think we need to write out exactly what the early SEV protocol version is, as it's an important corner case.

I can see moving all of the daemon pieces out into enarx-keepmgr makes sense if the idea is for keepldr to hold only implementations of Keeps. Is that what you had in mind?

Yes. It is supposed to be bare bones. enarx-keepmgr and enarx-wasmldr need to be integrated to each other. But enarx-keepldr should be a (mostly) invisible middle man. This is why I described enarx-keepmgr as being able to work by spawning /usr/bin/enarx-wasmldr directly OR via enarx-keepldr exec /usr/bin/enarx-wasmldr. The result should be (mostly) the same.

Design principle: don't replicate functionality.  The keeploader (or associated daemon - see below) should be the thing that knows how to create and pre-provision Keeps (including "nil Keeps" or "runtimes which can take workloads").  The keepmgr interacts with the client to gather enough information to launch a Keep, which job it hands off to the keeploader.  The client then talks to the Keep.
Design principle: maintain state close to where it's needed, and allow it to be owned by the component which needs it and can change it most directly.  Keep-specific state should be managed individually, not centrally (this also allows for recovery in the case of a failed keepmanager.)
I'm not saying that we necessarily need the daemon model, but we need to be sure we don't need it before we design it out, and I'm not sure that the (possibly) single-threaded nature of some Keeps means that the wasmldr is where this sits.
Important bit - on re-reading what I've written and what you've written, I'm unclear how far apart we actually are.  I think my main concerns are around a) multi-threading in wasmldr & b) possible protocol futures.  It may be that all you're advocating is moving the daemon part out of enarx-keepldr into enarx-keepmgr, in which case that's pretty easy.  If so, then please acknowledge!

NOTE - please read the below, but don't get irate until you've read the "Important bit" (which will hopefully mean you don't need to get irate). I kept what I've written in between because I think it raises some useful points in the meantime.

Nobody's irate. :)

Important bit - on re-reading what I've written and what you've written, I'm unclear how far apart we actually are. I think my main concerns are around a) multi-threading in wasmldr & b) possible protocol futures. It may be that all you're advocating is moving the daemon part out of enarx-keepldr into enarx-keepmgr, in which case that's pretty easy. If so, then please acknowledge!

I don't think we're far apart.
The design of keepldr is to be an adapter. It makes keeps look like normal Linux processes. It has to have a minimal amount of specialized knowledge in order to handle the pre-attestation workflow. But that should be it. Other than that, its job is to make things look normal and then get out of the way.
We agree that state should be held close. This is why per-instance state is in wasmldr (the closest it can be!) and per-host state is in keepmgr.
I'm not worried about the lack of threading in wasmldr. In fact, I think avoiding threading there altogether is desirable. Using tokio (as we're doing) helps us here significantly. With tokio we can handle multiple clients at the same time without threads, which is really all we need.


Optimize constants

Motivation:
Optimize constants

Pull Request Test Coverage Report for Build 551

8 of 15 (53.33%)  changed or added relevant lines in 3 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.03%) to 75.815%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

sofa-tracer-plugins/sofa-tracer-datasource-plugin/src/main/java/com/alipay/sofa/tracer/plugins/datasource/tracer/StatementTracerInterceptor.java
1
2
50.0%

sofa-tracer-plugins/sofa-tracer-datasource-plugin/src/main/java/com/alipay/sofa/tracer/plugins/datasource/tracer/DataSourceClientStatJsonReporter.java
2
8
25.0%

Totals

Change from base Build 550:
0.03%

Covered Lines:
605

Relevant Lines:
798

💛  - Coveralls


Please use a standard open source license

There's no way most of us in big companies are going to be able to use something with a nonstandard license. You say this is open source. If that's true, can you switch to something that's widely understood like Apache 2.0 or BSD or etc.? There's a great list of licenses to choose from at https://opensource.org/licenses/
When projects are put up on GitHub with a bespoke license, a lot of people will interpret it as meaning that the authors don't really have any experience with open source, and that's probably not the signal you want to send for this interesting project.

Thanks, @Ambrosiussen, that's really helpful, and I'm happy to see this change so quickly. (To be honest, I didn't expect it!)
Just so you know where I'm coming from on this, organizations such as mine have already spent a lot of time analyzing most of the standard licenses such as BSD, MIT, Apache 2.0, and essentially we're "pre-cleared" to use software with those licenses without needing to check back with the legal department. In contrast, we would need to ask our legal affairs to sign on off on our using software with any "new" licenses we haven't encountered before, and that can sometimes be a months-long round trip through the legal departments of ourselves and also our parent company. So anyway, I really appreciate the adjustment that makes this interesting software substantially more practical for us to try.


ConvertTo-PnPClientSidePage does not name sections in migrated page

Thank you for reporting an issue or suggesting an enhancement. We appreciate your feedback - to help the team to understand your needs, please complete the below template to ensure we have the necessary details to assist you. Thanks!
Category
[ X] Bug
[ ] Enhancement
Problem Area
[ ] Page Transformation: Error during the setup/use of the Page Transformation UI solution (did you check our troubleshooting guide?)
[ X] Page Transformation: Error during the use of page transformation from PnP PowerShell
[ ] Page Transformation: Error during the use of page transformation from .Net
[ ] Page Transformation: Page is not looking correct after transformation
[ ] Modernization Scanner: something went wrong...
Expected or Desired Behavior
Each Section in the Migrated Page should be named (eg Column1 or Section1)
Observed Behaviour
The Sections in the migrated page are not named.  so when click on "+"  you get "add a new webpart in" rather than "add new webpart in column 1".  So you do not know where you are adding the webpart.
Steps to Reproduce
Take webpart page and run ConvertTo-PnPClientSidePage against it then  edit page and then click on the "+" sign.
Thanks for your contribution! Sharing is caring.

Hey @NPrice99 ,
Sections in modern pages don't have names and columns are always "column one", "column two", ...when you click on the plus in the desired column you can add the web part or text in that column.
Can you clarify what actually goes wrong compared to editing a completely manually created page?

Hi Bert
The Sections in the migrated page are not named. so when click on "+" you get "add a new webpart in" rather than "add new webpart in column 1" when a page is created manually.. So you do not know where you are adding the webpart.
Regards
Nigel

Hi Both, this this might be a SPO bug rather transform, just converted pages in my dev tenant and see the full message.

I also see the full message...and these names are not even persisted in the page...seems indeed to be a temp SharePoint glitch.

@NPrice99 : will be closing this issue as it's not related to page transformation


Number of expected partitions mismatch

Scenario:

Kafka has a certain topic with 3 partitions.
I'm booting a service that produces to this partitions and the configured partition count is 100 (of course with autoAddPartitions = true).
The following exception is raised:

java.lang.IllegalStateException: The number of expected partitions was: 100, but 3 have been found instead at org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner$2.doWithRetry(KafkaTopicProvisioner.java:260) ~[spring-cloud-stream-binder-kafka-core-1.2.1.RELEASE.jar!/:1.2.1.RELEASE] at org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner$2.doWithRetry(KafkaTopicProvisioner.java:246) ~[spring-cloud-stream-binder-kafka-core-1.2.1.RELEASE.jar!/:1.2.1.RELEASE] at org.springframework.retry.support.RetryTemplate.doExecute(RetryTemplate.java:286) ~[spring-retry-1.2.0.RELEASE.jar!/:na] at org.springframework.retry.support.RetryTemplate.execute(RetryTemplate.java:163) ~[spring-retry-1.2.0.RELEASE.jar!/:na] at org.springframework.cloud.stream.binder.kafka.provisioning.KafkaTopicProvisioner.getPartitionsForTopic(KafkaTopicProvisioner.java:246) ~[spring-cloud-stream-binder-kafka-core-1.2.1.RELEASE.jar!/:1.2.1.RELEASE] at org.springframework.cloud.stream.binder.kafka.KafkaMessageChannelBinder.createProducerMessageHandler(KafkaMessageChannelBinder.java:149) [spring-cloud-stream-binder-kafka-1.2.1.RELEASE.jar!/:1.2.1.RELEASE] at org.springframework.cloud.stream.binder.kafka.KafkaMessageChannelBinder.createProducerMessageHandler(KafkaMessageChannelBinder.java:88) [spring-cloud-stream-binder-kafka-1.2.1.RELEASE.jar!/:1.2.1.RELEASE] at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindProducer(AbstractMessageChannelBinder.java:112) [spring-cloud-stream-1.2.2.RELEASE.jar!/:1.2.2.RELEASE] at org.springframework.cloud.stream.binder.AbstractMessageChannelBinder.doBindProducer(AbstractMessageChannelBinder.java:57) [spring-cloud-stream-1.2.2.RELEASE.jar!/:1.2.2.RELEASE] at org.springframework.cloud.stream.binder.AbstractBinder.bindProducer(AbstractBinder.java:152) [spring-cloud-stream-1.2.2.RELEASE.jar!/:1.2.2.RELEASE] at org.springframework.cloud.stream.binding.BindingService.bindProducer(BindingService.java:124) [spring-cloud-stream-1.2.2.RELEASE.jar!/:1.2.2.RELEASE] at org.springframework.cloud.stream.binding.BindableProxyFactory.bindOutputs(BindableProxyFactory.java:238) [spring-cloud-stream-1.2.2.RELEASE.jar!/:1.2.2.RELEASE] at org.springframework.cloud.stream.binding.OutputBindingLifecycle.start(OutputBindingLifecycle.java:57) [spring-cloud-stream-1.2.2.RELEASE.jar!/:1.2.2.RELEASE] at org.springframework.context.support.DefaultLifecycleProcessor.doStart(DefaultLifecycleProcessor.java:175) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE] at org.springframework.context.support.DefaultLifecycleProcessor.access$200(DefaultLifecycleProcessor.java:50) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE] at org.springframework.context.support.DefaultLifecycleProcessor$LifecycleGroup.start(DefaultLifecycleProcessor.java:348) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE] at org.springframework.context.support.DefaultLifecycleProcessor.startBeans(DefaultLifecycleProcessor.java:151) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE] at org.springframework.context.support.DefaultLifecycleProcessor.onRefresh(DefaultLifecycleProcessor.java:114) [spring-context-4.3.7.RELEASE.jar!/:4.3.7.RELEASE]

Even though the service failed to start,  when I executed ./kafka-topics.sh --describe --zookeeper zookeeper:2181 --topic my-topic, the number of partitions that returned is 100.

Some notes:

When I was trying the same scenario but with 6 partitions and not 100 this hasn't happened and the service had started successfully.
After waiting for about 10 minutes in debugging the fact that the code that generated the exception is wrapped inside a RetryTemplate this exception wasn't raised again (I saw that the instance of the partitonByTopic field that is inside the Cluster object has changed and included the new partition infos).

@YossiShasha The fact that you saw this exception thrown when the number of partitions were 100 and didn't see this when it was 6 could be an indication that there might be some timing (asynchronous) issues hiding there. The call to increase partitions happens before you come to this getPartitionsForTopic check (where you got the above exception) and since the partition count is larger, it still hasn't finished adding all the partitions. Looks like a bug. We will do further investigation.

OK, that's exactly what I suspected.

@YossiShasha What version of Kafka are you running? I am unable to reproduce the scenario that you described using the latest spring-cloud-stream and binder snapshots and Kafka version <IP_ADDRESS>. Any chance you can test on the latest and confirm if you still see this same behavior?

@sobychacko Sure, but I will only be able to try it on Sunday.

Good news :)
Although I wasn't able to try to reproduce the error with the latest versions due dependencies problems,
I was able to run it on spring-cloud-stream Ditmars which uses spring-cloud-stream-binder-kafka 1.3.0.REALEASE (previously I used spring-cloud-stream Chelsea SR2 and  spring-cloud-stream-binder-kafka 1.2.1.REALEASE). With the new versions the error wasn't reproduced and I also think that I know the reason:
The method createProducerMessageHandler in KafkaMessageChannelBinder in spring-cloud-stream-binder-kafka 1.3.0.REALEASE has changed and now in the end of each invocation of the call() method you close the producer and destroy the producerFB instances. By doing this in the next time that the method createProducer is called and a new Producer instance is created. In one of the subsequent invocations the Metadata inside the Producer contains the updated Cluster info.

Thank you @YossiShasha for the confirmation. Glad it worked for you with the latest release. I am closing this issue based on your findings and analysis.

Hi. I am getting similar problem.
I am using embedded kafka to test my kafka consumer. I have used KafkaMessageListenerContainer. The tests passed in my local however it fails in the CI pipeline with the following error
java.lang.IllegalStateException: Expected 1 but got 0 partitions.
I have referred: https://blog.mimacom.com/testing-apache-kafka-with-spring-boot-junit5/ for the code


Feature/synapse io website

Creating the synapse io website
Description
Creating the synapse io website available at https://americanexpress.io/synapse/
This site contains our mission statement as well as the team, and description of what is synapse and how to onboard.
Changes in this pull request
the images
the stylesheets
the fonts
the javascript and parallax logic

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.ffgutt seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Q: Leader election rebalancing?

Hi I have one question to ask. Yesterday I had few under-replicated partitions. I restarted broker that couldn't replicate and problem was fixed. Later I realized I have not even distribution of leader for partitions across cluster. It's like 130,180,10 leader partition per broker or so.
Here is screenshot of what happened before under-replication has occurred.
SCREENSHOT
Also how rebalancig works it this case. With previous setup, producers got list of brokers eg., kafka-0,kafka-1,kafka-2 and all 3 brokers had similar usage. Now when connecting producers and consumers to headless-service of kafka, one of the brokers use significantly more page cache than other two, like twice more.
ad 2. Out of my OCD just to be sure is it ok to use headless-service as connection point for producer and consumer right?

After some investigation I found out that it was cause by huge peak in one of the topics (see image below) and cruise control suddenly started doing his job. Yet I ened up with anothe 10 underreplicated partitions after another peak in topic.
graphs
Another question is whether it will come to normal after a while or I need to restart that broker that does not replicate again.

Out of my OCD just to be sure is it ok to use headless-service as connection point for producer and consumer right?
Yes

I'm not entirely sure about the other questions though, I might leave that for someone else to weigh in on.

CC also stuck during this after I restart thath broker.
10126300 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - Starting 218 inter-broker partition movements. 10138023 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 20/218 (9.17%) inter-broker partition movements completed. 5/2230 (0.22%) MB have been moved. 10149015 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 40/218 (18.35%) inter-broker partition movements completed. 5/2230 (0.22%) MB have been moved. 10159634 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 60/218 (27.52%) inter-broker partition movements completed. 5/2230 (0.22%) MB have been moved. 10170108 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 80/218 (36.70%) inter-broker partition movements completed. 5/2230 (0.22%) MB have been moved. 10180261 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 90/218 (41.28%) inter-broker partition movements completed. 23/2230 (1.03%) MB have been moved. 10190403 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 101/218 (46.33%) inter-broker partition movements completed. 24/2230 (1.08%) MB have been moved. 10200614 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 111/218 (50.92%) inter-broker partition movements completed. 36/2230 (1.61%) MB have been moved. 10210714 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 121/218 (55.50%) inter-broker partition movements completed. 41/2230 (1.84%) MB have been moved. 10220902 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 131/218 (60.09%) inter-broker partition movements completed. 44/2230 (1.97%) MB have been moved. 10231126 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 141/218 (64.68%) inter-broker partition movements completed. 61/2230 (2.74%) MB have been moved. 10241220 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 149/218 (68.35%) inter-broker partition movements completed. 61/2230 (2.74%) MB have been moved. 10271325 [posalExecutor-0] INFO  ruisecontrol.executor.Executor  - 150/218 (68.81%) inter-broker partition movements completed. 1085/2230 (48.65%) MB have been moved. 10272580 [omalyDetector-2] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10274240 [omalyDetector-2] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10282584 [omalyDetector-1] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10292579 [omalyDetector-2] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10302579 [omalyDetector-3] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10312579 [omalyDetector-4] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10322583 [omalyDetector-1] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10332580 [omalyDetector-3] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10342580 [omalyDetector-2] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10344798 [lingScheduler-0] INFO  trol.monitor.task.SamplingTask  - Skip sampling because the load monitor is in PAUSED state due to Paused-By-Cruise-Control-Before-Starting-Execution (Date: 2019-12-26_23:37:12 UTC).. 10352582 [omalyDetector-4] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10362586 [omalyDetector-1] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10372580 [omalyDetector-3] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10382583 [omalyDetector-2] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10392580 [omalyDetector-1] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10402579 [omalyDetector-4] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state. 10412579 [omalyDetector-3] INFO  .detector.AnomalyDetectorUtils  - Skipping anomaly detection because the executor is in INTER_BROKER_REPLICA_MOVEMENT_TASK_IN_PROGRESS state.


Feat: refine the logic of security type AUTO

Dear V2Fly Maintainers,
This patch will change the logic of cipher selection for security type "auto", which based on codes migrated from go standard library crypto/tls.
In the previous, this took mistakes on devices which does not support AES-GCM hardware acceleration but have installed a 64-bit system, such as J1900 mini-PC with x86_64 system and Raspberry with aarch64 kernel. It influenced me and my friends so we used to specify chacha20-poly1305 manually. And this might solve the problem.
I will keep following on this, thank you!
Best regards,
Hellojack
Refer:
https://cs.opensource.google/go/go/+/master:src/crypto/tls/cipher_suites.go;drc=9e6cd3985dbcdcfe0ed2075be6dbe8c5d6de59cb;l=364

Please run go generate format.go

Please run go generate format.go

Nothing changed with uncountable number of exit status 1🤔

Please run go generate format.go

Nothing changed with uncountable number of exit status 1🤔

Hmm, try to use go imports to format the code. You can also format by yourself.


HIVE-25046: Log CBO plans right after major transformations

What changes were proposed in this pull request?
Places DEBUG logging information after every major transformation step of the CBO planner.
Why are the changes needed?
More DEBUG info and overcomes the following shortcomings:

If there is a failure (exception) in some transformation step
intermediate plans are lost, thus losing track of which plan led to the
problem.
Intermediate logs are useful for identifying plan problems while
working on a patch; unfortunately the logs are explicitly disabled in
test mode which means that in order to appear the respective code needs
to change every time we need to see those logs.
Logging at the end necessitates keeping additional local variables
which makes the code slightly harder to read.

Does this PR introduce any user-facing change?
DEBUG logs are slightly different.
How was this patch tested?
No need for tests.

Tests are green can we get this in @jcamachor ?


Lazily evaluated env template variables in path entries

Setup:

HOME=/Users/john
A global config.tomlfile with:

a _.path  template entry with value "{{env.GEM_HOME}}/bin"
an env entry: GEM_HOME = "{{env.HOME}}/.gem"

A .mise.toml file in some directory under the home directory with:

an env entry: GEM_HOME = "{{env.HOME}}/.mise-gemsets/foo"

Today, env entries in path templates are expanded using the environment variables defined in the same mise config file, either global or local.
The idea behind a lazily expanded reference in path entries is to only expand the path env reference after both the global and local mise configs have been read, and use whatever value the env variable has at that point.
In the above example, this would mean that in the specific directory a path entry valued /Users/john/.mise-gemsets/foo/bin would be added, whereas elsewhere the entry /Users/john/.gem/bin would be added instead.
Today this doesn't work because the path entry has to be set in the global config where it is always added with the global value, which is not what we want here.
If there's another way to achieve this with existing mise facilities I'd love to learn about it. Thanks!

Here's a demo of my current branch, which allows lazy path entries to be specified using { value = "lazy_path_entry" } in addition to the existing "normal_path_entry". The effect is that the same positional PATH entry is either controlled by the global mise config, or by a local mise config in some directory, as can be seen below:
~/.config/mise/config.toml:
[env]
GEM_HOME = "{{env.HOME}}/.gem"
_.path = [
  { value = "{{env.GEM_HOME}}/bin" },
]

~/github/josb/.mise.toml:
[env]
GEM_HOME = "{{env.HOME}}/.mise-gemsets/foo"

~% mise env
export GEM_HOME=/Users/josb/.gem
export PATH='/Users/josb/.gem/bin:/opt/homebrew/opt/ruby/bin:...
~% cd ~/github/josb
~/github/josb% mise env
export GEM_HOME=/Users/josb/.mise-gemsets/foo
export PATH='/Users/josb/.mise-gemsets/foo/bin:/opt/homebrew/opt/ruby/bin:...
~/github/josb%

Normal _.path entries also still work, of course.
Feedback welcome.

there's another ticket (didn't have time to search for it) where we've talked about being able to configure some env vars to be parsed after tools are loaded that I think would be a prerequisite for doing this

Do you mean https://github.com/jdx/mise/issues/1982? That seems to not concern env var evaluation in env.path entries?

Oh sorry, I thought you wanted it to run after the tools are loaded, this makes sense

I seem to not be able to view your branch, not sure if that's because I'm on my phone

Hm. I'll open a PR in your repo later today so you take a look.

oh it works on my computer, go ahead and submit a PR—first glance it looks solid 👍

@jdx FYI https://github.com/jdx/mise/pull/2310

Thank you, @jdx! Much appreciated. 🙏


Cannot find 'aurelia-validation' module

Hi,
I'm submitting a bug report
I guess it's a bug, if not please prove me wrong, would be more than happy ;) I cut the issue template a bit because my rig specification is irrelevant.

Library Version:
0.13.1
Language:
TypeScript 2.0+

Current behavior:
Can't import 'aurelia-validation' module, example:
import { ValidationRules } from 'aurelia-validation';
The reason is that there's no single ambient module declaration in any of *.d.ts files in a distribution package, like before the plugin 'merge' there was such definition in aurelia-validationjs, take a look at https://github.com/aurelia/validatejs/blob/master/lib/validatejs.d.ts. So if you follow aurelia validation documentation then you just can't import module. To make it work I have to include sources directly, unless I'm missing something.
I don't like also the way that *.d.ts files are inside the distribution package, like aurelia used to have in some early versions. I guess it would be better to keep only transpiled code in a distribution package and *.d.ts should be manage by typings manager (outside of distribution package).
Expected/desired behavior:
There should be an ambient module declaration for 'aurelia-validation' plugin.

Sounds like you have a problem with your setup.

What does your tsconfig look like- especially the module resolution setting?
How are you installing your dependencies? npm? jspm?
What editor are your using?
Are you using typings, tsd or the new @types?

Hi Jeremy, thanks for reply.

Basically, it's aurelia skeleton so module resolution is 'node'
According to http://aurelia.io/hub.html#/doc/article/aurelia/validation/latest/validation-basics I'm installing aurelia-validation with jspm
vscode
typings


Error on command

Everytime someone does a command, I get an error from eZProtector which has been flooding my console for a few months now.
Here is the error: http://pastebin.com/XcafZQxq
Tried with regular spigot and paperspigot, both versions 1.8.8. I've had this issue for a few months.

Sorry for the inconvenience! This error has been fixed. Please update.


[Feature]: support for Python 3.9

What would you like the software to do?
I would like nPrintML to work with as little set-up as possible.
As operating systems increasingly ship with default versions of Python greater than 3.8, more users will be inconvenienced by nPrintML's lack of support for these. Moreover, it is ideal for nPrintML to support whichever is Python's most recent stable version, for the convenience of users who do install Python themselves.
Existing workarounds
I can install a version of Python that nPrintML supports. (In theory anyway.)
Implementation notes
This work appears to depend on #66. AutoGluon v0.1.0 does not support Python v3.9+. (Moreover, only "pre-release" versions of AutoGluon suggest support for Python v3.9+. As such, this issue might currently be blocked.)

The now-latest version of autogluon – 0.6.0 – supports Python versions 3.7-3.9.


Survey: Best Practices for Updating requirements.in

Hello fellow pip-tools users,
I'm new to pip-tools and would appreciate if you could share how you update your requirements.in file. I'm currently updating it manually for a small project but am entertaining the idea of adopting a, hopefully, simple process to automatically or semi-automatically update requirements.in -- an automation I foresee will possibly be quite convenient for using pip-tools in large projects.

How are you updating requirements.in in your projects?

Is there a command, or sequence of commands to list packages that have yet to be captured in requirements.in so that the user may consider their inclusion in requirements.in?

Speaking of automatically listing python packages in an environment, I understand that there is the vital task of "ignoring" packages which are purely dependencies while only surfacing "top-level" packages. E.g., conda-minify analyzes the dependency graph of a given python environment and extracts only "top-level" packages.
Discussions here could possibly be fed into documentation? Noticed #1794 calling for more detailed documentation 😄

Appreciate sharing how you use pip-tools @ThiefMaster. After reading your comment, things appear to make more sense. Prior to this, the unhelpful mental model I held as someone who was new to pip-tools was relying on conda (or another environment manager) mainly while only coming to pip-tools when confronted with a need to distribute the python environment (say via requirements.in, requirements.txt). I think the right mental model to is to use pip-tools primarily by editing requirements.in and using pip-tool commands to install/remove packages (basically synchronize) the environment.

I have a similar question regarding best practices.
Under the readme
The pip-compile command lets you compile a requirements.txt file from your dependencies, specified in either pyproject.toml, setup.cfg, setup.py, or requirements.in.
So which is the preferred way for django project, django plugin, regular python library?

pyproject.toml
setup.cfg
setup.py
requirements.in

Thank you

Hi, I was also wondering about this but haven't come to a conclusion yet.
So for my projects, I like to install them in develop mode for developing (duh!) from setup.py or pyproject.toml more recently. I like to have my project code available as an absolute import.
So for adapting pip-tools my impression is that I would now need three files:

requirements.in: To define all dependencies without version
requirements.txt: To define all dependencies with locked version
pyproject.toml: To include the dependencies from requirements.txt

Does that make sense?
I never worked with a larger project where you should freeze requirements like that so it seems a bit weird to spread everything over 3 files.

@aranvir
You probably wouldn't want to include the contents of requirements.txt in pyproject.toml, which should be as liberal as possible with its version requirements.
You also don't necessarily need requirements.in, as you can compile pyproject.toml to requirements.txt.
You may find it useful to use more files, for example a local-requirements.txt like:
-e .[dev]
-c requirements.txt


fix secretsmanager PutSecretValue on empty secret

Hello! It seems a case has been missed with #6720, when a secret would be created empty then updated with PutSecretValue. This would raise a KeyError:
  File "/opt/code/localstack/.venv/lib/python3.10/site-packages/moto/secretsmanager/models.py", line 131, in reset_default_version
    self.versions[previous_current_version_id]["version_stages"] = ["AWSPREVIOUS"]  # type: ignore
KeyError: None

I don't know secretsmanager too well, but maybe when creating an empty secret, it does not create a version?
So I've added a check to see if there was a previous version before trying to override it as AWSPREVIOUS.
Added a test case checking this specific case.
This issue was reported in the LocalStack Slack with cdktf, I was able to reproduce against LocalStack with a stripped down Terraform file:
main.tf
resource "aws_secretsmanager_secret" "example" {
  name = "example"
}

resource "aws_secretsmanager_secret_version" "example-version" {
  secret_id     = aws_secretsmanager_secret.example.id
  secret_string = "example-string-to-protect"
}

Hey @bentsku! Someone beat you to it - this has just been fixed in #6771 :slightly_smiling_face:

Ah oops, didn't check the open PR before! Thanks a lot for the fix!


No module named AudioConverted

hello, I like your package, but there is a problem after installing it according to the instructions listed in README file: No module named AudioConverted.
After checking your setup.py file, I found that there missing a key in setup dict called py_modules, which specifies the name of the module. I then moved the converter.py file out of AudioConvert folder and specified py_modules=['converter'], and install it in my local virtual env, everything is ok.
here is code snippet of setup.py :
setup(name='AudioConverter',
      setup_requires=['setuptools_scm'],
      description='Audio Converter CLI',
      long_description=long_description,
      long_description_content_type='text/markdown',
      author='lamdav',
<EMAIL_ADDRESS>      license='MIT',
      python_requires='>=3.5',
      py_modules=['converter'],
      entry_points={
        'console_scripts': [
            'audioconvert=converter:cli'
        ]
      },

This is just a simple fix, you may consider doing some fix related to the module problem in following releases.

Thanks for creating an issue.
I updated the setup.py file with the missing packages key and a call to find_packages(). I had also added the missing __init__.py for my python package 😅 . This should be fixed in version 0.0.5. Please reinstall the latest version and see if this fixes the issue for you.
The new version also contains a few internal clean ups but it shouldn't have impacted core functionalities. Feel free to reopen this issue if the problem persist.


Feature/wiki custom html generation

Add feature for dynamic generation of wiki-style HTML pages with hyperlink tracking on custom HTTP server
Use
python3 wikigengpt.py --user_concept "Fictional Concept"
Example

Good job! Thank you for contributing! :)


if pyppeteer supported

Well the code is in Node, so it uses the puppeteer framework. The use of the Python version of it is therefore not needed ?!


Improve preferredVideoCodecs

Currently a full chain comparison is done, but in video codecs, you may want H.265/VP9 with no profile preference, with this change it is possible to do this.
Examples:
'hvc1' instead of 'hvc1.1.2.L123.80' and 'hvc1.1.2.L153.80'
'vp09' instead of 'vp<IP_ADDRESS>.<IP_ADDRESS>.00'
Issue: #2179

Note: this is only implemented in video codecs because that is where it makes sense, in audio codecs this would not make sense in many cases.

Note: this is only implemented in video codecs because that is where it makes sense, in audio codecs this would not make sense in many cases.

Why not?  There is also mp4a.40.2, which  would be easier to write as mp4a.

Note: this is only implemented in video codecs because that is where it makes sense, in audio codecs this would not make sense in many cases.

Why not? There is also mp4a.40.2, which would be easier to write as mp4a.

@TheModMaker Do you want the change in this PR or create a new one?


Performance improvement for exports.getType()

Hy there,
While investigating performance issues in my application I found a place in knockout mapping plugin which could be improved slightly from both performance and memory allocation perspectives.
It is exports.getType(). In fact it is quick enough but since this function called on the bottom of the mapping recursion call stack, the number of invocations can reach tens of thousands which makes a difference. My suggestion is to replace existing code:
exports.getType = function(x) {
	if ((x) && (typeof (x) === "object")) {
		if (x.constructor == (new Date).constructor) return "date";
		if (Object.prototype.toString.call(x) === "[object Array]") return "array";
	}
	return typeof x;
}

with something like:
exports.getType = function(x) {
	if (x && typeof (x) === "object") {
		if (x.constructor == Array) return "array";
		if (x.constructor == Date) return "date";
	}
	return typeof (x);
}

I didn't spot any difference in returning values in both functions (tried IE6-9, FF 18, Chrome 24, Opera12.01, Safari 5.1.7).
Besides simplified code it also perform less object allocations and accordingly causes less garbage collections and thus makes better whole page performance (especially significant in single page apps).
I have created basic performance test for that: http://jsperf.com/gettype-test. It emulates almost the same number of invocations for 2 to and from mappings of my view model. Under that stress test the difference is 10-35% in different browsers. Perhaps, the suggested code could be improved further though.
Cheers,
Tavriets

It'd be great if someone could revert this change. I've just had to back out of using knockout.mapping in a project at the last minute because foreach wasn't working in IE. (I'm using exactly the combination that illinar describes: knockoutjs + SignalR + Internet Explorer.) It took me a while to track this down!


choppy microphone input

I am using Windows 10 and trying to use Skype.  It is very strange, I get great webcam performance with USB redirection and can stream 1080p videos from youtube and play pandora.  For the life of me though I cannot get the microphone to smoothly connect.  Every time I try to record audio I get only bits and pieces of the recording.  When I try to use skype its all choppy.
This is my build configuration:
[geiseri@luna ~]$ xfreerdp /buildconfig
This is FreeRDP version 2.0.0-dev (git 7d6f9581c)
Build configuration: BUILD_TESTING=OFF BUILTIN_CHANNELS=ON HAVE_AIO_H=1 HAVE_EVENTFD_H=1 HAVE_EXECINFO_H=1 HAVE_FCNTL_H=1 HAVE_INTTYPES_H=1 HAVE_JOURNALD_H=TRUE HAVE_MATH_C99_LONG_DOUBLE=1 HAVE_POLL_H=1 HAVE_PTHREAD_MUTEX_TIMEDLOCK=ON HAVE_PTHREAD_MUTEX_TIMEDLOCK_LIB=1 HAVE_PTHREAD_MUTEX_TIMEDLOCK_SYMBOL= HAVE_SYSLOG_H=1 HAVE_SYS_FILIO_H= HAVE_SYS_MODEM_H= HAVE_SYS_SELECT_H=1 HAVE_SYS_SOCKIO_H= HAVE_SYS_STRTIO_H= HAVE_TIMERFD_H=1 HAVE_TM_GMTOFF=1 HAVE_UNISTD_H=1 HAVE_XI_TOUCH_CLASS=1 WITH_ALSA=ON WITH_CCACHE=ON WITH_CHANNELS=ON WITH_CLIENT=ON WITH_CLIENT_AVAILABLE=1 WITH_CLIENT_CHANNELS=ON WITH_CLIENT_CHANNELS_AVAILABLE=1 WITH_CLIENT_COMMON=ON WITH_CLIENT_INTERFACE=OFF WITH_CUPS=ON WITH_DEBUG_ALL=ON WITH_DEBUG_CAPABILITIES=OFF WITH_DEBUG_CERTIFICATE=OFF WITH_DEBUG_CHANNELS=OFF WITH_DEBUG_CLIPRDR=OFF WITH_DEBUG_DVC=OFF WITH_DEBUG_KBD=OFF WITH_DEBUG_LICENSE=OFF WITH_DEBUG_MUTEX=OFF WITH_DEBUG_NEGO=OFF WITH_DEBUG_NLA=OFF WITH_DEBUG_NTLM=OFF WITH_DEBUG_RAIL=OFF WITH_DEBUG_RDP=OFF WITH_DEBUG_RDPEI=OFF WITH_DEBUG_REDIR=OFF WITH_DEBUG_RFX=OFF WITH_DEBUG_RINGBUFFER=OFF WITH_DEBUG_SCARD=OFF WITH_DEBUG_SND=OFF WITH_DEBUG_SVC=OFF WITH_DEBUG_SYMBOLS=OFF WITH_DEBUG_THREADS=OFF WITH_DEBUG_TIMEZONE=OFF WITH_DEBUG_TRANSPORT=OFF WITH_DEBUG_TSG=OFF WITH_DEBUG_TSMF=OFF WITH_DEBUG_WND=OFF WITH_DEBUG_X11=OFF WITH_DEBUG_X11_CLIPRDR=OFF WITH_DEBUG_X11_LOCAL_MOVESIZE=OFF WITH_DEBUG_XV=OFF WITH_DIRECTFB=OFF WITH_EVENTFD_READ_WRITE=1 WITH_FFMPEG=ON WITH_GPROF=OFF WITH_GSM=ON WITH_GSTREAMER_0_10=OFF WITH_GSTREAMER_1_0=ON WITH_IPP=OFF WITH_JPEG=ON WITH_KRB5=OFF WITH_LIBRARY_VERSIONING=ON WITH_LIBSYSTEMD=ON WITH_MACAUDIO=OFF WITH_MACAUDIO=OFF WITH_MACAUDIO_AVAILABLE=0 WITH_MANPAGES=ON WITH_MBEDTLS=OFF WITH_OPENH264=OFF WITH_OPENSLES=OFF WITH_OPENSSL=ON WITH_OSS=ON WITH_PCSC=ON WITH_PROFILER=OFF WITH_PULSE=ON WITH_SAMPLE=OFF WITH_SANITIZE_ADDRESS=OFF WITH_SANITIZE_ADDRESS_AVAILABLE=1 WITH_SANITIZE_LEAK=OFF WITH_SANITIZE_LEAK_AVAILABLE=1 WITH_SERVER=OFF WITH_SERVER_INTERFACE=ON WITH_SMARTCARD_INSPECT=OFF WITH_SSE2=ON WITH_THIRD_PARTY=OFF WITH_VALGRIND_MEMCHECK=OFF WITH_VALGRIND_MEMCHECK_AVAILABLE=1 WITH_WAYLAND=ON WITH_X11=ON WITH_X264=ON WITH_XCURSOR=ON WITH_XEXT=ON WITH_XFIXES=ON WITH_XI=ON WITH_XINERAMA=ON WITH_XKBFILE=ON WITH_XRANDR=ON WITH_XRENDER=ON WITH_XSHM=ON WITH_XV=ON WITH_ZLIB=ON
Build type:          RELWITHDEBINFO
CFLAGS:               -fPIC -Wall -Wno-unused-result -Wno-unused-but-set-variable -Wno-deprecated-declarations -fvisibility=hidden -Wimplicit-function-declaration -Wredundant-decls -g -msse2
Compiler:            GNU, 6.3.1
Target architecture: x64

and here are my connection arguments:
xfreerdp /v:<IP_ADDRESS> /u:usbtest /d:development /size:90% /usb:id,dev:04f2:b451 /sound:sys:pulse /multimedia:sys:pulse /echo /network:lan /gfx:AVC444 /disp +clipboard /bpp:32 /nsc /microphone:sys:pulse

I am also using Arch linux, but I have duplicated this on Ubuntu as well as other PCs.  I am also using pulse audio, but even when I try to use alsa directly I get the same choppy input.

does it work with /microphone:sys:alsa ?

I tried it with /microphone:sys:alsa but Skype and Google Hangouts are still choppy and scrambled.

I just noticed that on my system if I set PULSE_LATENCY_MSEC to a value of 60 then I get something that I can at least make out sounds on the other end.  Is it possible that there is some sort of issue with how FreeRDP reads from pulse?

@geiseri Ok, will investigate. Sadly my headset just broke, will take a day or two ;)

As an update with alsa, I did notice on my configuration alsa is routed via pulse so it may have not been a good test.  I am going to try on a system with pulse disabled completely and use alsa.  This is not ideal for my final configuration since I rely on pulse for my desktop, but it might narrow down the root cause.

@geiseri Ok, tried it locally. Working without glitches, but I suspect there is a bandwidth issue with your setup. USB redirection is quite demanding, so does the sound redirection also have these issues when not using USB redirection?

@geiseri Oh, did try pulse and alsa just for reference.

What OS are you on?

@geiseri Using debian jessie with backports. CPU/RAM is quite negligible for this (except when trying to run 4k monitors). Audio is Audio device: Intel Corporation 82801JI (ICH10 Family) HD Audio Controller
Remote end is a windows 10 machine connected with 1Gbit lan.

@akallabeth Did you make any changes to your pulse/daemon.conf and pulse/default.pa?  Also what desktop environment are you running if any?

@akallabeth on the remote host, is that a physical machine, or a virtual machine?

@geiseri

Running gnome or cinnamon (tested both).
The remote end is a physical windows 10 machine.
Default configuration of debian for pulse

I have an interesting experiments.  The first thing I did was set load-module module-udev-detect tsched=0 in my /etc/pulse/default.pa.  This seemed to radically help sound input quality.
Then I tried setting the format to different types.  One commandline is:
xfreerdp /v:10.0.5.xx /u:xxx /p:xxx  /size:90% /microphone:sys:pulse,format:17 /sound:sys:pulse,format:17 /multimedia:sys:pulse /echo /network:lan /gfx:rfx /disp +clipboard /bpp:32 +compression +gfx-thin-client +async-input -aero -themes -wallpaper /gdi:sw /usb:id,dev:04f2:b451
the other is:
xfreerdp /v:10.0.5.xx /u:xxx /p:xxx  /size:90% /microphone:sys:pulse /sound:sys:pulse /multimedia:sys:pulse /echo /network:lan /gfx:rfx /disp +clipboard /bpp:32 +compression +gfx-thin-client +async-input -aero -themes -wallpaper /gdi:sw /usb:id,dev:04f2:b451
I have attached the resulting files from Voice Recorder.
automatic format selection recording.zip
This one is smoother, but has random drops.
0x11 format selection recording.zip
This one lacks the random drops, but has audio artifacts.

@geiseri Ok, I think I got it. Currently microphone is only redirecting PCM, ADPCM and A/ULAW compressed format. In https://www.ietf.org/rfc/rfc2361.txt possible formats are listed, but only those above (high bandwidth/high quality and low bandwidth/low quality) are supported. Something covering the middle ground is currently not implemented, so it is quite prone glitches.

If I use A/ULAW (format:6/format:7 I think) I do not get any audio input or output.  If I set it to format:1 is this the same as not setting format at all?  Also I noticed in the auidin_pulse.c:252 there is a switch on 8 or 16bits.  How is that propagated?  Or do you think going from 16 to 8 bits won't matter.

@geiseri the formats there are only the ones the client supports. Forcing a specific format if the server has that one disabled will result in no audio at all.

@akallabeth: Why was this issue closed? Is there a fix for it and if yes, where do I find it? Tried the current master version and can confirm that the issue is still persisting.

@aszlig Ok, after going through this issue, I've opened #4309 as a feature request (which covers the issue you have with microphone redirection)

@akallabeth: I'm not sure this is really related to just codec support, because after setting the latency to 100ms in audio_pulse_open and using the tsched=0 workaround mentioned in https://github.com/FreeRDP/FreeRDP/issues/3872#issuecomment-288398788, the input still has some artifacts but now voice input is at least recognizable.


Bug and style

Maybe try to merge this first.

We have a profile photo uploading problem with DropDown Button. So I changed to normal Button.


Options to remove array elements and reindex the array

Fixes https://github.com/rhalff/dot-object/issues/32
This feature does not change current expectations.

You can now do Dot.pick('some.arrayItems[1]', obj, true, true) and it will delete some.arrayItems[1] as well as reindex some.arrayItems array.
Added tests for deleting array elements and reindexing the object
Added tests for picking and deleting together

Hi, looks good.
Do you think adding a delete method makes sense which wraps DotObject.prototype.remove = function(path, obj, reindexRemove)?
It will have reindexRemove set to true by default.
Would allow for .delete(path, obj) instead of .remove(path, obj, true)
delete is a reserved keyword, however there is also Map.delete() and I'm running out of possible method names.

I like the idea, but this might be confusing when there is a del method which is an alias for remove rather than delete.

ok, I'll merge it and add a delete method anyway with a note/warning in the docs.
Thanks for the pull request.


setAutoRepeat false not working, keeps repeating

Example reproduction code:
setLottieDrawable(
  AXrLottieDrawable.fromJson(jsonString, cacheKey)
      .setSize(decodeWidth, decodeHeight)
      .setAutoRepeat(false)
      .setSpeed(speed) // speed is set to 1.0f
      .build()
);

playAnimation();

Observation vs. Expectation
I expect the animation to only play once, and then stop. However, the animation keeps repeating.

I have the same problem

Hi @fourlastor and @hannojg,
Auto repeat is based on AXrLottieImageView's autoRepeat property.
This code should work:
lottieView.setAutoRepeat(false);

lottieView.setLottieDrawable(
  AXrLottieDrawable.fromJson(jsonString, cacheKey)
      .setSize(decodeWidth, decodeHeight)
      .build()
);

lottieView.playAnimation();

let me know if it didn't. Thanks

Fixed on v1.4.0


Enhance the pyodata executable to parse local metadata and specify policies

Every time we get an error report with metadata, we have to create a custom code to get the real error.
This PR should allow us to use the helper pyodata and avoid the need to write code.

Codecov Report

Merging #70 into master will increase coverage by 0.01%.
The diff coverage is 100%.

@@            Coverage Diff             @@
##           master      #70      +/-   ##
==========================================
+ Coverage   91.95%   91.97%   +0.01%
==========================================
  Files           6        6
  Lines        2400     2405       +5
==========================================
+ Hits         2207     2212       +5
  Misses        193      193

Impacted Files
Coverage Δ

pyodata/client.py
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 9dc3ad3...15db561. Read the comment docs.

Pushed:
b399f8a bin: enable specifying error policies
547fc1e bin: add the parameter disabling HTTP bootstrapping
9fec3f2 bin: support reading metadata from local file
5a61f80 client: add the possibility to provide static metadata
22cdfb2 build: add make targets for reporting coverage


ES6 Formatting help

Hey there,
I wish to know if there is a way to format the code when is written in ES6, the classic error throws me when I use the import syntax. If there is another sublime format code plugin please don't doubt in pass me.
Thanks

Please have a look at #33


Removes the s2i flag from the create file

What type of PR is this?
/kind cleanup
What does this PR do / why we need it:
It removes the s2i flag from the create command.
Which issue(s) this PR fixes:
Fixes part of #4932
Fixes #5012
PR acceptance criteria:

[ ] Unit test

[ ] Integration test

[ ] Documentation

[ ] I have read the test guidelines

How to test changes / Special notes to the reviewer:

Tests should pass.

Ginkgo ran 1 suite in 1m27.490986988s
Test Suite Passed
+ '[' true ']'
+ exit -1
{"component":"entrypoint","error":"wrapped process failed: exit status 255","file":"prow/entrypoint/run.go:80","func":"k8s.io/test-infra/prow/entrypoint.Options.Run","level":"error","msg":"Error executing test process","severity":"error","time":"2021-08-27T07:49:19Z"}
error: failed to execute wrapped command: exit status 255
INFO[2021-08-27T07:49:29Z] Step integration-e2e-integration-e2e-steps failed after 37m50s.
INFO[2021-08-27T07:49:29Z] Running step integration-e2e-gather-aws-console.
INFO[2021-08-27T07:49:49Z] Step integration-e2e-gather-aws-console succeeded after 20s.
INFO[2021-08-27T07:49:49Z] Running step integration-e2e-gather-must-gather.
INFO[2021-08-27T07:52:39Z] Step integration-e2e-gather-must-gather succeeded after 2m50s.
INFO[2021-08-27T07:52:39Z] Running step integration-e2e-gather-extra.
INFO[2021-08-27T07:54:29Z] Step integration-e2e-gather-extra succeeded after 1m50s.
INFO[2021-08-27T07:54:29Z] Running step integration-e2e-gather-audit-logs.
INFO[2021-08-27T07:55:09Z] Step integration-e2e-gather-audit-logs succeeded after 40s.
INFO[2021-08-27T07:55:09Z] Running step integration-e2e-ipi-deprovision-deprovision.
INFO[2021-08-27T07:59:59Z] Step integration-e2e-ipi-deprovision-deprovision succeeded after 4m50s.
INFO[2021-08-27T07:59:59Z] Releasing leases for test integration-e2e
INFO[2021-08-27T07:59:59Z] Ran for 1h32m54s
ERRO[2021-08-27T07:59:59Z] Some steps failed:
ERRO[2021-08-27T07:59:59Z]
  * could not run steps: step integration-e2e failed: "integration-e2e" test steps failed: "integration-e2e" pod "integration-e2e-integration-e2e-steps" failed: the pod ci-op-1rxvnxi0/integration-e2e-integration-e2e-steps failed after 37m41s (failed containers: test): ContainerFailed one or more containers exited

Container test exited with code 1, reason Error

/retest

@mik-dass Few Most of the scenarios in generic test file does not seem to be generic. For Ex https://github.com/openshift/odo/pull/5023/files?file-filters[]=.adoc&file-filters[]=.go&file-filters[]=.sh#diff-bb9e5f2fbc1b8edb31629193e23d6ec67b8963e0b730d9cb1f89305cd561a266L60-L130
Could you please fit those scenarios into respective file or remove if already exists

@mik-dass Few Most of the scenarios in generic test file does not seem to be generic. For Ex https://github.com/openshift/odo/pull/5023/files?file-filters[]=.adoc&file-filters[]=.go&file-filters[]=.sh#diff-bb9e5f2fbc1b8edb31629193e23d6ec67b8963e0b730d9cb1f89305cd561a266L60-L130
Could you please fit those scenarios into respective file or remove if already exists

This should be done in a separate PR as this PR is already huge.

/approve
/refresh

/lgtm


Interface improviments

Improviments:

[x] DateTime picker with a range option (start time cannot overlap with end time)
[x] Search actions that start with the words, not that is strict equal
[ ] Search by resource type
[x] Search by Criticity
[x] Pagination

search by Criticity was already done


Rust: Version is not updated in dependant Cargo Workspace crate

TL;DR
I have a project with four crates, two of which are handled by release-please-action. The second crate depends on the first, but its Cargo.toml does not get updated in release-please PR's.
Expected behavior
No response
Observed behavior
No response
Action YAML
name: release-please
jobs:
  release-please:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        crate:
          - plex-api
          - plex-cli
    steps:
      - uses: google-github-actions/release-please-action@v3
        id: release
        with:
          path: "crates/${{ matrix.crate }}"
          package-name: ${{ matrix.crate }}
          release-type: rust
          monorepo-tags: true
          bump-minor-pre-major: true
          bump-patch-for-minor-pre-major: true
          changelog-types: |
            [{"type":"feat","section":"Features","hidden":false},
             {"type":"fix","section":"Bug Fixes","hidden":false},
             {"type":"chore","section":"Miscellaneous","hidden":false},
             {"type":"test","section":"Tests","hidden":true},
             {"type":"ci","section":"Continuous Integration","hidden":true},
             {"type":"docs","section":"Documentation","hidden":true},
             {"type":"refactor","section":"Code Refactoring","hidden":true},
             {"type":"style","section":"Styles","hidden":true}]

Log output
https://github.com/andrey-yantsen/plex-api.rs/runs/5167378607?check_suite_focus=true

https://pipelines.actions.githubusercontent.com/ELtVuWvm260Hc4dTvb6JM2rBUkUpxOGSn7BeoerUGNq8wS17uG/_apis/pipelines/1/runs/914/signedlogcontent/2?urlExpires=2022-02-12T12%3A44%3A28.4586251Z&urlSigningMethod=HMACV1&urlSignature=uxQGhY1p%2BOcmZ9mG7MsU7Uw8EQhTzujbzvI2oKUUQSc%3D

Additional information
Here's the latest PR: andrey-yantsen/plex-api.rs#267. I expect the following line to be updated in the PR, but it doesn't: https://github.com/andrey-yantsen/plex-api.rs/blob/5ac0c2df33957c68d2d07487028c14cf6d2ce71e/crates/plex-cli/Cargo.toml#L13.
At the moment, I tend to think that I just missed some configuration steps. Still, it looks like release-please does not recognize the repo as a workspace — because Cargo.lock in the root of the repo should also be updated for the release, but again — it doesn't (here: https://github.com/andrey-yantsen/plex-api.rs/blob/5ac0c2df33957c68d2d07487028c14cf6d2ce71e/Cargo.lock#L1290).

I'm guessing that I see this behaviour because of the set path parameter. I've configured the action as suggested here: https://github.com/google-github-actions/release-please-action/issues/180#issuecomment-756631682, but unfortunately, it doesn't work as expected for me.

You are going to want to look into a "manifest" release configuration for handling multiple components. https://github.com/googleapis/release-please/blob/main/docs/manifest-releaser.md


Haplotype from assembled reads

Hi,
is there any way to generate haplotype from assembled reads using cliqueSNV
Thanks

What do you mean? Just find consensus? There is an option for this, although even samtools can do it too. What kind or reads do you have, in what format, what haplotype do you expect to get? Some more info can help here

I have fastq reads from HiV study, my main goal is to generate haplotype and quasispecies
I am following these route:
A)standard

filter reads
map to reference
cliqueSNV(it works here :) )

B) assemble-based

filter reads
2)remove the host
3)assemble reads using spades
align to ref
detect haplotype or build a consensus sequence(does clique SNV works here??)

C) ref-free

filter reads
remove the host
detect haplotype(does cliqueSNV works here??)

CliqueSNV works only with sam and bam. So you have to align reads one way or another. SO 2) I'm not familiar with spades, but if you align to ref then it should work. 3) no, we don't work reference-free, reads have to be aligned

Thanks for quick reply, I do appreciate it
the issue is that since these assembled reads are low in number the read count for bam is very low like 3 reads total
when I try cliqueSNV on this bam it gives me this error
`CliqueSNV version: <IP_ADDRESS>
Settings: {-m=snv-illumina, -in=try.bam}
Reads number 3
SNV got 0 haplotypes
[]
CliqueSNV didn't find any haplotypes (too low coverage)
time,ms 383`

Ah, I see now.
The only thing is then to find consensus (it is described in README for "-m" parameter). CliqueSNV won't be able to do anything else with such input.

generally, do you recommend to remove duplicate reads from bam using picard markduplicate or samtools rmdup

I'm not sure about the nature of duplicate reads(didn't work closely with obtaining data from raw data), but I assume they may skew the frequencies of haplotypes if their distribution is not uniform. So they may be removed because of this reason

Thank you so much for the quick reply
Another question is what is the best way to annotate this generated haplotype?
my idea is degap generated haplotype, append with reference, and the multiple align and a dendrogram
do you have a better idea?

This question is out of my competence


nzSelect with nzMode="multiple" error listOfSelectedValue.filter is not a function

Reproduction link
https://stackblitz.com/edit/angular-1kn6s7-zxkgcv
Steps to reproduce
Run the link and open the console
What is expected?
Possibility to use multiple mode with reactive form
What is actually happening?
multiple mode does not work with reactive form

Environment
Info

ng-zorro-antd
13.1.1

Browser
chrome 99.0.4844.51

You're writing it wrong and check out the link below
https://stackblitz.com/edit/angular-1kn6s7-qynjsv

@chenc041 thank you!

@chenc041 thank you!

You're welcome. You can close this issue if you're okay

You're writing it wrong and check out the link below
https://stackblitz.com/edit/angular-1kn6s7-qynjsv

Hi, the stackblitz sample provided does not open. Could you please check it?

You're writing it wrong and check out the link below
https://stackblitz.com/edit/angular-1kn6s7-qynjsv

Hi, the stackblitz sample provided does not open. Could you please check it?

I have the same problem, the page at this link doen't exist

You're writing it wrong and check out the link below
https://stackblitz.com/edit/angular-1kn6s7-qynjsv

Hi, the stackblitz sample provided does not open. Could you please check it?

Sorry, the sample has been deleted when finishing stackbiz
Please check the new sample https://stackblitz.com/edit/angular-1kn6s7-zvmhjq?file=src%2Fapp%2Fapp.component.ts

You're writing it wrong and check out the link below
https://stackblitz.com/edit/angular-1kn6s7-qynjsv

Hi, the stackblitz sample provided does not open. Could you please check it?

I have the same problem, the page at this link doen't exist

Sorry, the sample has been deleted when finishing stackbiz
Please check the new sample https://stackblitz.com/edit/angular-1kn6s7-zvmhjq?file=src%2Fapp%2Fapp.component.ts

Thanks a lot @chenc041

Thanks a lot @chenc041

You're welcome


i18n: add missing translations Fr death_report

Again problem regex. I hope my fix is correct. If no, help me plz.

Does that match what you see in game? The error is that these two translations should be the same:

https://github.com/quisquous/cactbot/blob/8eda2b4a4bca3bf3048a17d5449cf3ae1158a303/ui/raidboss/data/04-sb/trial/byakko-ex.ts#L242
https://github.com/quisquous/cactbot/blob/8eda2b4a4bca3bf3048a17d5449cf3ae1158a303/ui/raidboss/data/04-sb/trial/byakko-ex.ts#L347

Good ^^. It's ok for merged.