Qmix Performance Inconsistency

Hi, I'm currently running experiments on the scenario '3s5z' and '1c3s5z' and find there is a significant performance difference from the Qmix paper. (my experiment result is roughly 15% to 20% better than the paper regarding test won rate). So just wondering if there is any difference with the environment setting or algorithm change.
Thank you so much.

The results that you get using this repository should coincide with the results from our SMAC paper. Although our earlier QMIX paper features similar maps, many things have been slightly changed since the release of the PyMARL framework and SMAC benchmark, such as agent observations, the structure of the maps, parameters of the mixing network of QMIX, etc.


Add playoff % to standings and/or team class

Sport
Football
Summary
On the standings page, there is now a predicted percentage to make the playoffs. I would like to access this in the Team class or whatever you feel is appropriate.

https://fantasy.espn.com/apis/v3/games/FFL/seasons/2021/segments/0/leagues/{LEAGUE_ID_HERE}/?view=mStandings
stored in playoffPct

Thanks, I was just looking at that. I see ESPN now provides draftDayProjectedRank and currentProjectedRank that could be good team information.

There are also some interesting projections from a simulation in there, like projected wins and losses
"currentSimulationResults": {
        "divisionWinPct": 0.0,
        "modeRecord": {
            "gamesBack": 0.0,
            "losses": 11,
            "percentage": 0.0,
            "pointsAgainst": 0.0,
            "pointsFor": 0.0,
            "streakLength": 0,
            "streakType": "NONE",
            "ties": 0,
            "wins": 3
        },
        "playoffPct": 0.019,
        "rank": 10
    },


"will be" language should be converted to MUST, SHOULD, or MAY

Examples:

All endpoints will be prefixed by the API version and the repository name:
One or more errors will be returned in the following format:
The code field will be a unique identifier, all caps with underscores by convention.
A 404 Not Found response will be returned if the image is unknown to the registry.

Examples like the last one have impeded progress on the conformance tests, since most of the error codes are defined in terms of "will be" language, and we're unsure how much is simply conventional vs required by the spec.

I think this issue is a duplicate of #27? Please comment if this isn't covered by that issue.


fix(versions): upversion and fix deps range

Upversion ngx-base and fabric8-analytics-dependency-editor to the latest that include fixes for peer dependencies.
Moving all dependencies and peerDependencies to use the ^ range. As a library, as long as the dependencies installed by the application meet the API needs of the library, the library should not force a particular version on the application (assuming semantic versioning is used). If doing so the application must package multiple versions of the library which is not ideal. By using the ^, the library accepts any API compatible version and the application can instead bundle a single version of the library.
@edewit If there's a specific reason this was changed back to not use ^ please share.

-1 on using version ranges. Here is an article explaining why: https://www.lucidchart.com/techblog/2017/03/15/package-management-stop-using-version-ranges/

if it reduces conflicts and makes that no 2 versions of the same packages are included in the final build then I'm :+1:

@edewit what about build reproducibility? Version ranges will introduce flaky behavior no?

@gastaldi
Semantic versioning solves API compatibility but not patch breakages (and of course only if semantic versioning is followed correctly). So yes there's potential for breakages in accepting what should be a compatible version.
The package-lock.json, as I understand (I'm no expert), will ensure that for local installs the versions are the same. So your tests will continue to run against the same versions as specified in package.json even if the library is updated.
As a consumer of the library, here's where versions of a dependency can differ from what the library specified as the base compatible version of the dependency. If the library specifies a direct version requirement, then the consuming app must package that version. Using lodash as an example. If 10 micro libs all specified direct versions of lodash and all were behind on their package versions (worst case none were the same), we could end up with 10 different versions of lodash being packaged. I believe not even npm dedupe would fix this.
How else can a consumer library avoid micro library bloating unless libs used ranges to allow the consuming library to manage the libraries it wants to package?

@christianvogt I see, I am no NPM expert and unfortunately I don´t have a concrete answer to your question, but if I can build the component locally and have the same behavior as in another environment using the same version, I don´t mind having it 😄

@gastaldi I believe a consistent local build is achieved with package-lock.json. I'm certainly open to future discussions on the topic to learn and adjust our environments to the best solution.
@edewit do you mind approving and merging? I don't have the ability to add reviewers for whatever reason.

[test]


Unexpected input, duplicated characters.

[x ] I am on the latest Hyper.app version
[x ] I have searched the issues of this repo and believe that this is not a duplicate

OS version and name:
Hyper.app version:  3.0.2
Link of a Gist with the contents of your .hyper.js:
Relevant information from devtools (CMD+ALT+I on macOS, CTRL+SHIFT+I elsewhere):
The issue is reproducible in vanilla Hyper.app: yes

Issue
When trying to type ~/ on a portuguese keyboard laybout
~ : is done with by just pressing ~^ key
/ : is done with SHIFT + 7
Expected was to output :

~/

the output is :

//

also everytime I try to use "~" it just duplicates the next character, for example if I do "~" and then "t" the output is "tt".
I can avoid this issue only if I use space between inputs, example: "~", "SPACE", then "/" I the output is "~/" of course "~" shows up first.
I'm using git for windows but the same happens if I use default values aka Vanilla.

Same here, i just type "ls" and got "lls"


Add userId parameter to signOutUser

Parity with C#
Description
It was found that the TypeScript method signOutUser() differed from the C# implementation of signOutUserAsync(). The TypeScript version expects less parameters, not letting the user pass a userId.
TypeScript
public async signOutUser(context: TurnContext, connectionName: string): Promise<void>
C#
public virtual async Task SignOutUserAsync(ITurnContext turnContext, string connectionName = null, string userId = null, CancellationToken cancellationToken = default(CancellationToken))
Specific Changes

add userId optional parameter to the method
add handling for when userId is null

Pull Request Test Coverage Report for Build #2013

0 of 1 (0.0%)  changed or added relevant line in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-0.03%) to 87.407%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

libraries/botbuilder/src/botFrameworkAdapter.ts
0
1
0.0%

Totals

Change from base Build #2012:
-0.03%

Covered Lines:
3004

Relevant Lines:
3306

💛  - Coveralls

Hi @Aliandi,
Fork merges are disabled that's why the build is failing. Chris has grant you write permissions to the repo.
Could you please push your changes in a new branch from this repo?
Thanks!

Replaced by #837


Document can be previewed and published without adding project to Sidekick

Description
Preview, Publish, Copy URL buttons are active on action bar when project is not added to Sidekick
To Reproduce
Steps to reproduce the behavior:

Install extension
Go to project folder
Click on extension button to show action bar
Select document

Expected behavior
Preview, Publish, Copy URL buttons are not available when project is not added to Sidekick
Screenshots

This is a feature ("transient mode"). If you want to prevent this from happening, you need to enable authentication for authors.


constructor cleanup

Nothing spectacular: It appears the parser's constructor carried more code than really necessary.

Can you make sure PRs are able to be auto merged? I'm on vacation in Asia at the moment and have limited access to command line.

Heh, it's been mergeable when I created it. I'll take a look at it right away …

k, should work now.


feat: support ValueNotifier for rebuild.

Support ValueNotifiers for UI rebuild
Reference: https://api.flutter.dev/flutter/widgets/ValueListenableBuilder-class.html

You read my mind)

Hi! Would you add tests for these builders? You can check the same cases which checked for ListenableState multi-source builders.


Create itoa, memset, movmem Tests -- Update desc movmem Majorbbs.cs

Updated Description of movmem and renamed (was movemem) -- it was referencing ordinal 410 which is not implemented yet (memmove)

Changed itoa ordinal to return the point and asserted on NotEqual 0

Fixed memset to return value, updated documentation


Compatibility with EKS 1.21 and token service account expiry

What happened:
After our EKS was upgraded to 1.21, we saw annotations like the following appear in api server audit logs in AWS, for service accounts that Splunk Connect pods are using:
subject: system:serviceaccount:<namespace here>:<sa name here>, seconds after warning threshold: 3989

This is due to changes in token expiry in K8s 1.21 as described here:
https://docs.aws.amazon.com/eks/latest/userguide/service-accounts.html#identify-pods-using-stale-tokens
It would appear that there is 90d grace period, after which tokens will be rejected.
It looks like the Splunk Connect agents needs to use a later client SDK version, or is there a workaround?
What you expected to happen:
More recent k8s client sdk was used in 1.4.11 Splunk Connect used so that the tokens wouldn't get flagged. At some kube version when AWS will change to the default 1h tokens, the pods will get errors from api server after an hour (unless they are restarted earlier, as that would refresh the token I think as well).
How to reproduce it (as minimally and precisely as possible):
Install or upgrade EKS to 1.21 and check EKS cluster api server audit logs with this query:
fields @timestamp
| filter @logStream like /kube-apiserver-audit/
| filter @message like /seconds after warning threshold/
| parse @message "subject: *, seconds after warning threshold:*\"" as subject, elapsedtime

based on: https://docs.aws.amazon.com/eks/latest/userguide/service-accounts.html#identify-pods-using-stale-tokens
Anything else we need to know?:
Environment:

Kubernetes version (use kubectl version): 1.21
Ruby version (use ruby --version):
OS (e.g: cat /etc/os-release):
Splunk version:
Splunk Connect for Kubernetes helm chart version: 1.14.11
Others:

@harshit-splunk actually I would think this is a bug, as it can lead to issue after 90d, for clusters where nodes don't cycle often. Can you pls change priority of this?

I can see this warning also on OpenShift 4.8 (K8s 1.21). Can't find documentation about the "hard expiry" time on OpenShift.
Feature docs:

BoundServiceAccountTokenVolume: Migrate ServiceAccount volumes to use a
projected volume consisting of a ServiceAccountTokenVolumeProjection. Cluster
admins can use metric serviceaccount_stale_tokens_total to monitor workloads
that are depending on the extended tokens. If there are no such workloads, turn
off extended tokens by starting kube-apiserver with flag --service-account-extend-token-expiration=false.
Check Bound Service Account Tokens for more details.

in  Bound Service Account Tokens: Safe Rollout of Time-bound Token I read:

These extended tokens would not expire and continue to be accepted within one year.

@vinzent  I think EKS has made them valid only for 90d, but yeah I think vanilla k8s and other distros stick with the default 1y.

This is a major issue for us and will result in us having to look at alternative technologies. We've raised it with our splunk account manager, so hopefully it can be fixed quickly.

We have prioritised this request and will be working on this ASAP. Meanwhile, you can probably use service-account-token-volume-projection where you can provide expirationSeconds for token. I haven't tested it yet.

@harshit-splunk but will the application i.e. fluentd reload the token periodically? i.e. what's the point if I configure to refresh it by kubelet each hour, if the fluentd process will use the token it read during startup?

the token is already changed by k8s periodically. Only the user of the token needs to reload it from disk. IMHO "vanilla" fluentd doesnt use the k8s api. it's the plugins that use it. Like kubernetes_metadata_filter.
There is already an issue open about that: https://github.com/fabric8io/fluent-plugin-kubernetes_metadata_filter/issues/323
maybe SCK uses more k8s api things for object/metrics components.

This is also an issue for us, especially after AWS has started sending e-mails reminding about it...
I can confirm that running the 1.4.15 version of the chart still produces log messages about tokens that are too old.

Also an issue here. Any ETA regarding when a new chart version will be released to solve this problem?

Also an issue here. Any ETA regarding when a new chart version will be released to solve this problem?

Hi @tomsucho, @vinzent, I have updated each plugin so that it will read the bearer token file on each request. I have pushed images to docker hub.
hvaghani/fluentd-hec:1.2.13-refresh-token
hvaghani/kube-objects:1.1.12-refresh-token
hvaghani/k8s-metrics:1.1.12-refresh-token
hvaghani/k8s-metrics-aggr:1.1.12-refresh-token

Can you test if it is working? I have tested it on EKS, haven't received any audit logs for warning threshold so far.

@harshit-splunk probably tomorrow I can test the hvaghani/fluentd-hec image. The other components we don't use.

@harshit-splunk  pods with hvaghani/fluentd-hec:1.2.13-refresh-token are running 3 hours now. I don't see any annotations.authentication.k8s.io/stale-token in audit logs on OpenShift 4.8. According pod definition the token expires after 3607 seconds (expirationSeconds: 3607). I would expect that without refreshed token the stale-token audit annotation would already be back.

Not stale

Sorry to be impatient; but is it possible to get an update on this fix being released to a new version of splunk/fluentd-hec?

@boatmisser I have created https://github.com/ManageIQ/kubeclient/pull/566 and waiting for it to be merged. Meanwhile you can use hvaghani/fluentd-hec:1.2.13-refresh-token image. It is created from https://github.com/splunk/fluent-plugin-splunk-hec/pull/248

Support for refreshing the token is added in SCK 1.5.0 release. Closing the issue.


Feasibility of conditional module federation at development time?

One challenge with module federation in the docs says:

"It complicates deployment - each micro-frontend/container has to be deployed and available to the clients (usually via Internet)"

https://re-pack.netlify.app/docs/module-federation#challenges
I wonder if it would be possible to avoid this deployment requirement by only doing module federation at development time? So you get the benefits of running a local dev server for the module in development, but in production, the bundled federated modules that would normally be hosted on a CDN would be included with the app (ie not require a network request to load).
I see two possible approaches to accomplishing this:

Structure federated modules to also be packages and align the exposes/exports and shared/peerDependencies. Then have a conditional environment babel compilation step that either imports the package export as the entrypoint or a React.lazy(() => Federated.importModule(...)). Where I see that getting complicated is federated modules that reference other federated modules...

Another way could be to take the bundled JS files for a federated module intended to be stored on a CDN and store them in the ipa/apk similar to local chunks and have some custom script resolver logic that loads from file system in production instead of a devserverurl. I have not tried that yet and not sure if it would work

Is this an idea worth investigating more if my team pushes back against having to manage CDN servers and additional deployment steps?

One of the main benefits of using Repack and Module Federation is the ability to load Modules dynamically. You'll lose that if you choose to embed Federated Modules into the Host/Super App. Using Module Federation in development ONLY doesn't bring much value, at least to me, and is not something I would recommend. It is, however, possible.
I implemented a similar approach to the second point from the original question. It was only used as a temporary workaround while the CDN/Catalog Server was being developed in parallel and it was stopped being used after the CDN implementation was finished. The approach was a little bit more tricky than what you've described but it was mainly because of the complex brownfield nature of the project.
Off the top of my head that's what we did:

compile bundles of the desired features/mini apps
embed them in the Host App following platform specifc way, just like any other native asset (font etc). For iOS you have to copy them and add to specific targets for example.
configure ScriptManager resolver so that it will use devserver for development and local files in Prod. You'll have to get the filesystem URL of the scripts and pass them instead of CDN urls to the ScriptManager resolver. You can try using getFileSystemURL, but depending on your setup it might not work. In my case I had to implement a custom Native Module responsible for finding the proper files URLs of the bundles, but also set the absolute flag to true in the resolver configuration.

Given all these steps you'll still have to have some kind of "release" process and, when a Federated Module gets updated – compile and copy the bundles (similar process to uploading the bundles do CDN) and then go through normal App Store review (that would not be necessary in normal, remote Module Federaton approach because Over-The-Air updates are possible).

Wouldn't using experimental lazy bundling from Metro answer your case then? It's available in this release:  https://github.com/facebook/metro/releases/tag/v0.76.3 which is included with the latest v0.72 RC of React Native

Hi @sschottler, @rplotkin
I've recently created a quick POC of bundled federated modules from our super-app-showcase which you can explore on this branch:
local-module-federation-poc
This POC includes building production bundles of the federated modules in the monorepo and then bundling them just as local chunks into the app contents. To get it to work, before running the host app you need to prebuild the mini-apps using yarn bundle:ios in the root of the repository, and then build the app for iOS as usual. There is a Xcode script attached to help with moving the assets from their respective build directories into the app contents.
Note that instead of using Script.getFilesystemURL we are using custom one to handle the scenario of container bundles. Also, as some mini-apps are exposing the same component and end up producing bundles that have the exact same names, I needed to modify chunkFilename in every federated module webpack.config to include the name of that module to differentiate them from each other.

FYI when it comes to module federation during development time and dynamic switching. This is something that Zephyr Cloud will enable.

Hi @sschottler, @rplotkin
I've recently created a quick POC of bundled federated modules from our super-app-showcase which you can explore on this branch:
local-module-federation-poc
This POC includes building production bundles of the federated modules in the monorepo and then bundling them just as local chunks into the app contents. To get it to work, before running the host app you need to prebuild the mini-apps using yarn bundle:ios in the root of the repository, and then build the app for iOS as usual. There is a Xcode script attached to help with moving the assets from their respective build directories into the app contents.
Note that instead of using Script.getFilesystemURL we are using custom one to handle the scenario of container bundles. Also, as some mini-apps are exposing the same component and end up producing bundles that have the exact same names, I needed to modify chunkFilename in every federated module webpack.config to include the name of that module to differentiate them from each other.

@jbroma is it possible for us to use this approach combined with remote apps?
ex: we include remote bundles with the application when we release it to app store (this will avoid initial network hit or allow us to serve application packaged bundle when network latency is identified), then we allow pushing updates to those bundles. (similar to how codepush works for main bundle)


scs: 3.2.3 -> 3.2.7

Description of changes
Changelog: https://github.com/cvxgrp/scs/releases/tag/3.2.7
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[ ] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

Result of nixpkgs-review pr 337025 run on x86_64-linux 1

1 package built:

scs

Result of nixpkgs-review pr 337025 run on aarch64-linux 1

1 package built:

scs

Result of nixpkgs-review pr 337025 run on aarch64-darwin 1

1 package built:

scs

Result of nixpkgs-review pr 337025 run on x86_64-darwin 1

1 package built:

scs

Result of nixpkgs-review pr 337025 run on aarch64-linux 1

2 packages marked as broken and skipped:

python311Packages.bambi
python311Packages.bambi.dist

25 packages built:

python311Packages.blackjax
python311Packages.blackjax.dist
python311Packages.cvxpy
python311Packages.cvxpy.dist
python311Packages.jaxopt
python311Packages.jaxopt.dist
python311Packages.qpsolvers
python311Packages.qpsolvers.dist
python311Packages.qutip
python311Packages.qutip.dist
python311Packages.scs
python311Packages.scs.dist
python312Packages.blackjax
python312Packages.blackjax.dist
python312Packages.cvxpy
python312Packages.cvxpy.dist
python312Packages.jaxopt
python312Packages.jaxopt.dist
python312Packages.qpsolvers
python312Packages.qpsolvers.dist
python312Packages.qutip
python312Packages.qutip.dist
python312Packages.scs
python312Packages.scs.dist
scs

Result of nixpkgs-review pr 337025 run on x86_64-linux 1

2 packages marked as broken and skipped:

python311Packages.bambi
python311Packages.bambi.dist

25 packages built:

python311Packages.blackjax
python311Packages.blackjax.dist
python311Packages.cvxpy
python311Packages.cvxpy.dist
python311Packages.jaxopt
python311Packages.jaxopt.dist
python311Packages.qpsolvers
python311Packages.qpsolvers.dist
python311Packages.qutip
python311Packages.qutip.dist
python311Packages.scs
python311Packages.scs.dist
python312Packages.blackjax
python312Packages.blackjax.dist
python312Packages.cvxpy
python312Packages.cvxpy.dist
python312Packages.jaxopt
python312Packages.jaxopt.dist
python312Packages.qpsolvers
python312Packages.qpsolvers.dist
python312Packages.qutip
python312Packages.qutip.dist
python312Packages.scs
python312Packages.scs.dist
scs

Result of nixpkgs-review pr 337025 run on aarch64-darwin 1

2 packages marked as broken and skipped:

python311Packages.bambi
python311Packages.bambi.dist

25 packages built:

python311Packages.blackjax
python311Packages.blackjax.dist
python311Packages.cvxpy
python311Packages.cvxpy.dist
python311Packages.jaxopt
python311Packages.jaxopt.dist
python311Packages.qpsolvers
python311Packages.qpsolvers.dist
python311Packages.qutip
python311Packages.qutip.dist
python311Packages.scs
python311Packages.scs.dist
python312Packages.blackjax
python312Packages.blackjax.dist
python312Packages.cvxpy
python312Packages.cvxpy.dist
python312Packages.jaxopt
python312Packages.jaxopt.dist
python312Packages.qpsolvers
python312Packages.qpsolvers.dist
python312Packages.qutip
python312Packages.qutip.dist
python312Packages.scs
python312Packages.scs.dist
scs

Result of nixpkgs-review pr 337025 run on x86_64-darwin 1

10 packages marked as broken and skipped:

python311Packages.bambi
python311Packages.bambi.dist
python311Packages.blackjax
python311Packages.blackjax.dist
python311Packages.jaxopt
python311Packages.jaxopt.dist
python312Packages.blackjax
python312Packages.blackjax.dist
python312Packages.jaxopt
python312Packages.jaxopt.dist

17 packages built:

python311Packages.cvxpy
python311Packages.cvxpy.dist
python311Packages.qpsolvers
python311Packages.qpsolvers.dist
python311Packages.qutip
python311Packages.qutip.dist
python311Packages.scs
python311Packages.scs.dist
python312Packages.cvxpy
python312Packages.cvxpy.dist
python312Packages.qpsolvers
python312Packages.qpsolvers.dist
python312Packages.qutip
python312Packages.qutip.dist
python312Packages.scs
python312Packages.scs.dist
scs


ERROR: db migrate throws === Dangerous operation detected #strong_migrations ===

Describe the bug
db migrate throws Standard error: === Dangerous operation detected #strong_migrations ===
To Reproduce
Steps to reproduce the behavior:

Complete all the steps of local environment setup.
Run rails db:migrate

Expected behavior
DB migrations should have completed successfully.
Device Information [optional]:

OS: [e.g. iOS] Fedora 37
Browser [e.g. chrome, safari] Chrome
Version [e.g. 22]

Logs
# previous successful message logs
==<PHONE_NUMBER>1332 AddMigrationColumn: migrated (0.0025s) ======================

==<PHONE_NUMBER>0508 AddCascadePropertyToForumAssociatedFk: migrating ============
-- remove_foreign_key(:forum_posts, :users)
   -> 0.0043s
-- remove_foreign_key(:forum_threads, :users)
   -> 0.0029s
rake aborted!
StandardError: An error has occurred, this and all later migrations canceled:

=== Dangerous operation detected #strong_migrations ===

Adding a foreign key blocks writes on both tables. Instead,
add the foreign key without validating existing rows,
then validate them in a separate migration.

class AddCascadePropertyToForumAssociatedFk < ActiveRecord::Migration[7.0]
  def change
    add_foreign_key :forum_posts, :users, on_delete: :cascade, validate: false
  end
end

class ValidateAddCascadePropertyToForumAssociatedFk < ActiveRecord::Migration[7.0]
  def change
    validate_foreign_key :forum_posts, :users
  end
end

/home/danger-ahead/Documents/open_source/CircuitVerse/db/migrate/20221110160508_add_cascade_property_to_forum_associated_fk.rb:6:in `change'
<internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
<internal:/usr/share/rubygems/rubygems/core_ext/kernel_require.rb>:85:in `require'
-e:1:in `<main>'
Tasks: TOP => db:migrate
(See full trace by running task with --trace)

Are you working on this issue? (Yes/No) Willing to work

Fixed in https://github.com/CircuitVerse/CircuitVerse/commit/d4c0c2e1b37aaf64f4e470b81eb16e494177e86f


Add dependabot.yml

Helps keep GitHub Actions up-to-date

closing as it looks like renovate bot is dealing with it instead


Missing entries in the table of content

The content of this page was lacking some entries as described in https://github.com/MicrosoftDocs/windows-itpro-docs/issues/10026
It was fixed by https://github.com/MicrosoftDocs/windows-itpro-docs/pull/10225
However, the new content was not added to the table of contents https://github.com/MicrosoftDocs/windows-itpro-docs/blob/9c545726726211c813844676e80fa1bf5fbe99ef/windows/security/identity-protection/access-control/special-identities.md?plain=1#L38-L86
For example, but not limited to, "Attested Key Property" was added but it's missing from the table of contents list.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 486e8a12-8230-6ae2-264f-73c76b266682
Version Independent ID: 2f47f94f-b416-beac-c75d-8908e4000e66
Content: Special Identities (Windows 10) - Windows security
Content Source: windows/security/identity-protection/access-control/special-identities.md
Product: m365-security
Technology: windows-sec
GitHub Login: @Dansimp
Microsoft Alias: dansimp

@cnotin thanks for submitting this feedback! We've added the missing links and closed this issue.


[BUG] -  cannot appear as a descendant of

NextUI Version
2.6.3
Describe the bug
[<a> cannot appear as a descendant of <a>.](client.js:85 Warning: validateDOMNesting(...): <a> cannot appear as a descendant of <a>.)

Your Example Website or App
No response
Steps to Reproduce the Bug or Issue
as={Link} to the DropdownItem
Expected behavior
a tag shouldn't be added to another a tag
Screenshots or Videos
No response
Operating System Version
macOS
Browser
Chrome

introduced from https://github.com/nextui-org/nextui/pull/4168


endian: support apple endian.h

Apple uses a different include path for endian.h.
Signed-off-by: Paul Spooren<EMAIL_ADDRESS>

Please follow up on #52


Consider a tag factory to apply options to templates

As discussed in #179
// tag is probably a bad name
import {tag} from 'lit-html/lit-html.js';

const html = tag({stripWhitespace: true});
render(html`

whitespace is stripped

`,
  document.body);

Tags will need their own template cache if they change how templates are prepared.

We have a tag factory in 2.0 with the static module. That pattern can be applied to other rendering variations. We should track whitespace separately.


Segmentation model C++ API's based inference output different from Python tensors

System information

Have I written custom code:No
OS Platform and Distribution (e.g., Linux Ubuntu 18.04):  Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device:
TensorFlow installed from (source or binary): source
TensorFlow version (use command below): v2.3.0
Python version: 3.6.9
Bazel version (if compiling from source): 3.1.0
GCC/Compiler version (if compiling from source): 7.5.0
CUDA/cuDNN version: 10.1-cudnn7
GPU model and memory:

Describe the current behavior
We've trained a Segmentation model using Keras with Tensorflow backend. Now, We are trying to infer by using the TF C++ interface.
I used the below code snippet to convert the .h5 model to a .pb file.
keras_model_path="keras_model.h5"
model = tf.keras.models.load_model(keras_model_path,  compile=False)
model.summary()
tf.saved_model.save(model, "~/tf2_cpp/")

Our python pipeline,
model = tf.keras.models.load_model(keras_model_path,  compile=False)
img = cv2.imread("test.jpg")
resized_img = cv2.resize(img,(224, 224)).reshape([1, 224, 224,3]).astype(np.float32)
output = model.predict(resized_img) #7, 224, 244, 2
mask = tf.argmax(output[0], axis=3) #1, 244, 244

our C++ pipeline,
// Pre Processing
cv::Mat img = cv::imread("test.jpg")
cv::Size s(224,224);
cv::resize(img, img, s, cv::INTER_LINEAR);
tensorflow::Tensor input_tensor(tensorflow::DT_FLOAT, tensorflow::TensorShape({1, height, width, depth}));
float *p = input_tensor.flat<float>().data();
cv::Mat fakeMat(224, 224, CV_32FC3, p);
img.convertTo(fakeMat, CV_32FC3);

// Prediction
const string input_node = "serving_default_input_4:0";
// std::vector<std::pair<string, tensorflow::Tensor>> inputs_data  = {{input_node, input_tensor_mapped}};
std::vector<string> output_nodes = {{"StatefulPartitionedCall:3",
    "StatefulPartitionedCall:2",
    "StatefulPartitionedCall:5",
    "StatefulPartitionedCall:1",
    "StatefulPartitionedCall:4",
    "StatefulPartitionedCall:0",
    "StatefulPartitionedCall:6"}};

std::vector<Tensor> predictions;

this->bundle.GetSession()->Run({{input_node, input_tensor}}, output_nodes, {}, &predictions);

//Post-processing
auto argOps = tensorflow::ops::ArgMax(transposeScope, predictions[1], 3);
std::vector<Tensor> temOutputs;
TF_CHECK_OK(sess.Run({argOps}, &temOutputs));

The tensor value is different when I compare the result between Python and C++. And, C++ argmax API returning zero for all the tensor as well.
Describe the expected behavior
The output expected to be same in both Python and C++. Maybe with some precision changes.

@nullbyte91 ,
In order to expedite the trouble-shooting process, could you please provide a complete code to reproduce the issue.Also
could you please update TensorFlow to the latest stable version v.2.7 and let us know if you are facing the same error. Thanks!

ArgMax doesnt work, I suppose it has a bug, it returns a list of zero for the tensor


WebGPU: asynchronous pipeline creation is very slow

Investigate why asynchronous pipeline creation is extremely slow in WebGPU

@MikhailGorobets Verify that post-processing works in the browser.

Async creation seems to be working fine in the browser.
Our shader processing pipeline (HLSL->WGSL) also seems to be taking quite a lot of time. Using shader cache should likely help.


Freeze while loading datas

Hello, the following code result in a few seconds complete freeze until the result come in which mean that javascript is overloaded and getting totally fucked for a little while, how is that even possible ?)
import { Fetcher, Route } from '@uniswap/sdk'
        const netId = await web3.eth.net.getId();
        const PTE = await Fetcher.fetchTokenData(netId, web3.utils.toChecksumAddress(Env().PEET_CONTRACT_ADDR));
        const USDT = await Fetcher.fetchTokenData(netId, web3.utils.toChecksumAddress(Env().USDT_CONTRACT_ADDR));

        const pair = await Fetcher.fetchPairData(PTE, USDT);
        const route = new Route([pair], USDT)

Fetcher has been removed from this repository


about GigaNVS Dataset

Thank you very much for your great work, can you make the GigaNVS Dataset public so it can be used as a reference for me to make a dataset for this model

Hi @lhc991025,
Thanks for your interests in our work. Our dataset will be made public available at www.gigavision.ai in conjunction with the upcoming challenges.


When a player win/loses a round in ModeVs it should prompt player to take action

When in ModeVS and a player win or loses a round.

[x] A player can quit match or continue to next round
[ ] In multiplayer same options, though it will countdown to automatically start if not chosen

The Player can now choose to continue or not. However restarting throws an error after some time with replay saving

you can just press ok and play further but its disturbing the player


why is there not getter for fields in SingleOwnerAccount ?

I have a function that call a specific smart contract, I want to pass a SingleOwnerAccount as argument.
fn call_contract(
	account: SingleOwnerAccount<SequencerGatewayProvider, LocalWallet>
) {
      let contract_address = <some_address>
      let result = account
		.execute(&[Call {
			to: contract_address,
			selector: get_selector_from_name("mint").unwrap(),
			calldata: vec![
				account.address(),
				FieldElement::from_dec_str("1000000000000000000000").unwrap(),
				FieldElement::ZERO,
			],
		}])
		.send()
		.await
		.unwrap();
}

I want to use miner_account.address() as the first argument of the calldata.
Why is there no getters implemented onto  SingleOwnerAccount ?
Is it by design ? Or should I do a PR adding those getter methods ?

You're right we should have that on the Account trait.
BTW you can use the selector! and felt! macros to avoid the unwrap() calls :)

@tdelabro A new method address() has been added.


Update stream implementation

This PR aims to clean up the stream implementation by using the modern stream.pipeline() method instead of readable.pipe().
The change is not only cosmetic. This should reduce the chance of memory leaks and other problems, as pipeline() automatically forwards errors and destroys the streams if they error. It also has special handling for legacy streams, which we didn't have previously. So this should improve compatibility with the rest of the ecosystem. Although, it does require an upgrade to recent versions of Node, as pipeline() was introduced in Node 10 and the legacy compatibility was introduced in Node 14.

Do we have to target Node.js 14 though, it still works on Node.js 12, just not with legacy streams.

Do we have to target Node.js 14 though, it still works on Node.js 12, just not with legacy streams.

Yeah, on second thought, I agree. Node 12 users can still benefit from this. Updated CI and engines to >=12.


getEditsForFileRename: Combine edits from all projects

TypeScript Version:  master
If the moved file is in multiple projects, currently we just select the first project. We could combine from multiple projects, though there may be conflicting edits if the projects have different settings such as path mapping.
My suggestion would be to take all the edits from the first project (arbitrarily), and take edits in subsequent projects only if they are for files that don't already have edits.

Fixed by #25052


Amazon Cloud Front - image urls

Hi,
I'm using click & open tracking on postal mail server but i'm getting image urls from aws cloud front. The domain is  https://d178r3dmgae5qs.cloudfront.net.
But im trying to add tracking domains to postal panel, only mail server domains can be selected.

How can i solve this problem ?
Thanks,
Murat

Click tracking rewrites the links that can be clicked in the email so the domain needs to be one pointing at the Postal server.
If your images are hosted from CloudFront, then those URLs should be written into the message before it gets to Postal.


adds support for parameter set builders

adds support for methods parameter set builder

See the snippet difference https://github.com/microsoftgraph/microsoft-graph-docs/compare/snippet-generation/35511...snippet-generation/35512
Related
https://github.com/microsoftgraph/MSGraph-SDK-Code-Generator/pull/368
https://github.com/microsoftgraph/msgraph-sdk-java/pull/602

Snippet diff linked in the PR description doesn't have any diff.

Snippet diff linked in the PR description doesn't have any diff.

@zengin right, sorry this diff was super outdated and I forgot to provide an updated one. There you go https://github.com/microsoftgraph/microsoft-graph-docs/compare/snippet-generation/36189..snippet-generation/36190

@zengin right, sorry this diff was super outdated and I forgot to provide an updated one. There you go https://github.com/microsoftgraph/microsoft-graph-docs/compare/snippet-generation/36189..snippet-generation/36190


Why do we create a fork?

I'm not sure this is clear from the current material (feel free to disagree :) ).
In my mind there are two reasons:

We cannot edit the central repository (clearly explained in the material)

but for this, we could just clone and make our changes

The maintainer needs something to pull from. They cannot pull from my laptop if it's closed.
The cloud provider is a trusted place to pull from and allows review before pulling.

Good points! Another motivation for forking would be to keep the central repo tidier.


newrelic broke our bot

https://github.com/probot/stale/pull/185/files#diff-168726dbe96b3ce427e7fedce31bb0bcR1 adds require('newrelic'), but we self-host and don't have newrelic, so now our bot doesn't start.
https://github.com/probot/stale/blob/b8e5343941d00ba1f0902ae58bb10cdfa1399f47/index.js#L1

actually it is a dependency of probot-stale, so it's not this package. will continue hunting, sorry!

how do we turn off newrelic if we self-host?

@sohymg I have addressed this in https://github.com/probot/stale/pull/198. Hopefully that will work for you!


Build OGV burn page

Leading up to the OGV burn on October 10th, we'll want to promote a landing page that shows a countdown, how many tokens have gone unclaimed and will be burned, as well as with some other stats and content.
‣ Design in Figma (coming soon)

I made a quick page on the governance dApp for this to tide us over: https://governance.ousd.com/burn
@rolandpo There might be some useful math for you here: https://github.com/OriginProtocol/ousd-governance/blob/0b4313155c289633ecc4894de4bdb455e0bc5b9b/client/pages/burn.tsx#L57


Fix spelling issue in ISSUE_TEMPLATE.md

Proposed changes
Just a spelling issue, nothing big
Types of changes

[x] Bug fix (non-breaking change which fixes an issue).
[ ] New feature (non-breaking change which adds functionality).
[ ] Breaking change (fix or feature that would cause existing functionality to change).

Checklist:

[x] I have read the CONTRIBUTING document.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

Already got it as well as a few other things in #5325.

Oops, alright! Closing!


[BUG] V0.9.11 Some 2 wheeled vehicle cannot get bounding box.

Platform: Ubuntu 16.04
CARLA version: 0.9.11
Script: PythonAPI/examples/client_bounding_boxes.py

Same here. This is only happening to some cyclists and motorcycles.

Same here. This is only happening to some cyclists and motorcycles.

Hi, I will check. Thanks for your report.

Hi, I will check. Thanks for your report.

From what I observed, it is not that the bounding boxes for some 2-wheel vehicles are missing. Instead, the boxes are in the wrong position because some boxes appear in places without any object in my visualization. Also, this situation does not happen to all instances of 2-wheel vehicles, some are still good with correct boxes.
For those instances with wrong boxes, they seem to be easily identified. Those instances all have a width of 0 (vehicle.bounding_box.extent.y). So my current solution is simply to remove vehicles with a width of 0 after spawning several hundreds of vehicles, which removes about half of the 2-wheel vehicles. The rest are working perfectly.

From what I observed, it is not that the bounding boxes for some 2-wheel vehicles are missing. Instead, the boxes are in the wrong position because some boxes appear in places without any object in my visualization. Also, this situation does not happen to all instances of 2-wheel vehicles, some are still good with correct boxes.
For those instances with wrong boxes, they seem to be easily identified. Those instances all have a width of 0 (vehicle.bounding_box.extent.y). So my current solution is simply to remove vehicles with a width of 0 after spawning several hundreds of vehicles, which removes about half of the 2-wheel vehicles. The rest are working perfectly.

I believe this is related to https://github.com/carla-simulator/carla/issues/3670 as described by @xinshuoweng. The problematic vehicles (0 y-extent) are only ever two-wheelers (as described in 3670) and end up with incorrect bounding box locations.

I am also facing this problem.

With Carla 0.9.12 I still see this problem

@XGodina , is there any new with this issue?

Faced the same problem with 2-wheel vehicles. Are there any updates?

Why close this issue? it is not solved yet

I have had similar problems with the bounding boxes, and this is my wrapper function to correct for any misbehaving bounding boxes.
`
def get_bounding_box(actor: Actor, min_extent=0.5):
"""
Some actors like motorbikes have a zero width bounding box. This is a fix for these issues.
"""
# Fixing bug related to Carla 9.11 onwards where the bounding box location is wrongly registered for some 2-wheeled vehicles
# https://github.com/carla-simulator/carla/issues/3801
buggy_bbox = (bbox.extent.x * bbox.extent.y * bbox.extent.z == 0)
if buggy_bbox:
# print(f"Buggy bounding box found for {actor}")
bbox.location = carla.Location(0, 0, max(bbox.extent.z, min_extent))
# Fixing bug related to Carla 9.11 onwards where some bounding boxes have 0 extent
# https://github.com/carla-simulator/carla/issues/3670
if bbox.extent.x < min_extent:
bbox.extent.x = min_extent
if bbox.extent.y < min_extent:
bbox.extent.y = min_extent
if bbox.extent.z < min_extent:
bbox.extent.z = min_extent
return bbox

`

Yes exactly - this is because the bounding box location is not set properly. I’m just setting it to x=0 and y=0 here. This seems to work since the bounding box location is defined relative to the actor coordinate frame.

Through trial and error found a good enough solution.
The thing is, that the location and extent of some models are just seem to be switched places. So we just have to check for bug, as @shuishida suggested and then switch the data:
`
def get_bounding_box(actor: Actor, min_extent=0.5):
"""
Some actors like motorbikes have a zero width bounding box. This is a fix to this issue.
"""
if not hasattr(actor, "bounding_box"):
# print(f"Bounding box for type {actor.type_id} not implemented. Returning default.")
return carla.BoundingBox(carla.Location(0, 0, min_extent),
carla.Vector3D(x=min_extent, y=min_extent, z=min_extent))
bbox = actor.bounding_box
# Fixing bugs related to Carla 9.11 onwards where the bounding box location and extent is wrongly registered for some 2-wheeled vehicles
# https://github.com/carla-simulator/carla/issues/3801
# https://github.com/carla-simulator/carla/issues/3670
buggy_bbox = (bbox.extent.x * bbox.extent.y * bbox.extent.z == 0)
if buggy_bbox:
    # print(f"Buggy bounding box found for {actor}")
    loc = carla.Location(bbox.extent)
    bbox.extent = carla.Vector3D(bbox.location)
    bbox.location = loc
return bbox

`

Hey, if anyone still sees this thread, I have created a fix for my carla-KITTI framework.
`bicycle_fix_dict = {
"vehicle.vespa.zx125": {
"ext": [0.8, 0.45],
"sensor_refpoint": [0.6,0.45],
},
"vehicle.bh.crossbike": {
"ext": [0.8, 0.45],
"sensor_refpoint": [0.6, 0.45],
},
"vehicle.gazelle.omafiets": {
"ext": [0.75, 0.3],
"sensor_refpoint": [0.9, 0.13],
},
"vehicle.yamaha.yzf": {
"ext": [0.9, 0.45],
"sensor_refpoint": [0.5, 0.4],
},
"vehicle.kawasaki.ninja": {
"ext": [0.9, 0.35],
"sensor_refpoint": [0.65,0.4],
},
"vehicle.harley-davidson.low_rider": {
"ext": [0.9, 0.35],
"sensor_refpoint": [0.65,0.4],
},
"vehicle.diamondback.century": {
"ext": [0.7, 0.25],
"sensor_refpoint": [0.7,0.2],
},
}
if obj_type == "Bicycle" or obj_type == "Motorcycle":
    log.debug("new ext: %s", ext)
    if agent.type_id in bicycle_fix_dict:
        ext.x = bicycle_fix_dict[agent.type_id]["ext"][0]
        ext.y = bicycle_fix_dict[agent.type_id]["ext"][1]

        sensor_refpoint[0] += bicycle_fix_dict[agent.type_id]["sensor_refpoint"][0]
        sensor_refpoint[1] += bicycle_fix_dict[agent.type_id]["sensor_refpoint"][1]`

You can ignore the sensor_refpoint and instead manipulate the location in the same way.
I hopy this fix can help anyone that has the same problem.
This works in carla 0.9.14 :)


2.15.2 breaks upstart on Docker 1.8

Cookbook version
2.15.2
Chef-client version
2.19.36
Platform Details
Ubuntu 14.04.1 LTS
Docker Version
1.8.2
Scenario:
Unable to start Docker
Steps to Reproduce:
"service docker start" hangs as Docker can't start.  This also blocks convergence.
Expected Result:
Docker should start
Actual Result:
Docker doesn't start
This appears to be a breaking change between 2.6.1 and 2.15.2.  In 2.15.2 the --raw-logs flag was added to the daemon flags in /etc/init/docker.conf:
root@default-ubuntu-1404:~# grep raw.logs /etc/ -ir
/etc/init/docker.conf:    exec "$DOCKER" daemon $DOCKER_OPTS --raw-logs

This is not compatible with the older version of Docker, which is still supported per the readme (or at least 1.8.3).  We are pinned back to the older Docker version due to blocking Docker bugs.  It would be nice if this breaking change could be resolved in the same major version.  (I won't complain about dropping the older versions, but please bump major, and I think it's late for that.)  Thanks!

Did #870 resolve this for you?

closing  for now. please reopen if this is still an issue with the latest version.


Docs issue: lack of installation instructions

I'm currently trying to install the emacs lsp-grammarly package along with the language server (grammarly-languageserver on npm? ).
The docs are very unclear about what someone would need to do to get it running.

lsp-grammarly offers auto installation, so you don't need to install the language server manually. Or you can install globally:
npm i -g @emacs-grammarly/grammarly-languageserver

Let me know if it works!

FYI, the documentation is in the upstream package lsp-mode; see https://emacs-lsp.github.io/lsp-mode/page/lsp-grammarly/.
Feel free to open PRs for improvements! TBH, my documentation and English aren't very good (it's funny and reasonable why I have integrated Grammarly into Emacs)! 😅


Updating AndroidView layer tree cause 100% crash on some brand device

Hello,I met a crash when I use AndroidView.When I first time create AndroidView(full screen), Everything works well. But when I try to create another AndroidView, adding them into a stack  and changing their layout(The new one is under the old one，this time the native method 'create' will recall twice because of the widget life cycle), it result in 100% crash.
this problem only occur in some device. I met twice: VIVO NEX A and NUBIA NX595J.
Here is the crash stack and log.
I/CatcherGestureDetector(14536): DecorView onAttached
I/CatcherGestureDetector(14536): window type filtered
I/View    (14536): updateSystemUiVisibility, blackBackground = true, getVisibility() = 0, getWindowVisibility() = 0, viewBounds = Rect(0, 0 - 0, 0), windowBounds = Rect(0, 0 - 0, 0), view = TextureView:android.view.TextureView{c150802 V.ED..... ......ID 0,0-0,0}
I/View    (14536): updateSystemUiVisibility, blackBackground = true, getVisibility() = 0, getWindowVisibility() = 0, viewBounds = Rect(0, 0 - 0, 0), windowBounds = Rect(0, 0 - 0, 0), view = TextureView:android.view.TextureView{c150802 V.ED..... ......ID 0,0-0,0}
I/View    (14536): updateSystemUiVisibility, blackBackground = true, getVisibility() = 0, getWindowVisibility() = 0, viewBounds = Rect(0, 0 - 1080, 2009), windowBounds = Rect(0, 0 - 1080, 2009), view = TextureView:android.view.TextureView{c150802 V.ED..... ......ID 0,0-1080,2009}
I/DecorView(14536): onConfigurationChanged mBackgroundPadding = Rect(0, 0 - 0, 0), mFramePadding = Rect(0, 0 - 0, 0), pkg = com.zego.flutter.videotalk
I/CatcherGestureDetector(14536): DecorView onAttached
I/CatcherGestureDetector(14536): window type filtered
I/View    (14536): updateSystemUiVisibility, blackBackground = true, getVisibility() = 0, getWindowVisibility() = 0, viewBounds = Rect(0, 0 - 0, 0), windowBounds = Rect(0, 0 - 0, 0), view = TextureView:android.view.TextureView{f13513 V.ED..... ......ID 0,0-0,0}
I/View    (14536): updateSystemUiVisibility, blackBackground = true, getVisibility() = 0, getWindowVisibility() = 0, viewBounds = Rect(0, 0 - 0, 0), windowBounds = Rect(0, 0 - 0, 0), view = TextureView:android.view.TextureView{f13513 V.ED..... ......ID 0,0-0,0}
I/View    (14536): updateSystemUiVisibility, blackBackground = true, getVisibility() = 0, getWindowVisibility() = 0, viewBounds = Rect(0, 0 - 351, 498), windowBounds = Rect(0, 0 - 351, 498), view = TextureView:android.view.TextureView{f13513 V.ED..... ......ID 0,0-351,498}

D/AndroidRuntime(14536): Shutting down VM
E/AndroidRuntime(14536): FATAL EXCEPTION: main
E/AndroidRuntime(14536): Process: com.zego.flutter.videotalk, PID: 14536
E/AndroidRuntime(14536): java.lang.NullPointerException: Attempt to invoke virtual method 'int android.view.Display.getDisplayId()' on a null object reference
E/AndroidRuntime(14536): 	at android.view.ViewRootImpl$1.onDisplayChanged(ViewRootImpl.java:1124)
E/AndroidRuntime(14536): 	at android.hardware.display.DisplayManagerGlobal$DisplayListenerDelegate.handleMessage(DisplayManagerGlobal.java:555)
E/AndroidRuntime(14536): 	at android.os.Handler.dispatchMessage(Handler.java:106)
E/AndroidRuntime(14536): 	at android.os.Looper.loop(Looper.java:192)
E/AndroidRuntime(14536): 	at android.app.ActivityThread.main(ActivityThread.java:6809)
E/AndroidRuntime(14536): 	at java.lang.reflect.Method.invoke(Native Method)
E/AndroidRuntime(14536): 	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
E/AndroidRuntime(14536): 	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:817)

Here is the doctor output
lizhanpengZEGOde-MacBook-Pro:flutter_videotalk lizhanpeng$ flutter doctor -v
[✓] Flutter (Channel dev, v1.1.7, on Mac OS X 10.14 18A391, locale zh-Hans-CN)
    • Flutter version 1.1.7 at /Users/lizhanpeng/flutter
    • Framework revision 0d5ee0560b (2 days ago), 2019-01-08 09:24:27 -0800
    • Engine revision 7112b72cc2
    • Dart version 2.1.1 (build 2.1.1-dev.0.1 ec86471ccc)

[✓] Android toolchain - develop for Android devices (Android SDK version 28.0.3)
    • Android SDK at /Users/lizhanpeng/Library/Android/sdk
    • Android NDK at /Users/lizhanpeng/Library/Android/sdk/ndk-bundle
    • Platform android-28, build-tools 28.0.3
    • ANDROID_HOME = /Users/lizhanpeng/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jre/jdk/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1024-b01)
    • All Android licenses accepted.

[!] iOS toolchain - develop for iOS devices (Xcode 10.1)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Xcode 10.1, Build version 10B61
    ✗ Verify that all connected devices have been paired with this computer in Xcode.
      If all devices have been paired, libimobiledevice and ideviceinstaller may require updating.
      To update with Brew, run:
        brew update
        brew uninstall --ignore-dependencies libimobiledevice
        brew uninstall --ignore-dependencies usbmuxd
        brew install --HEAD usbmuxd
        brew unlink usbmuxd
        brew link usbmuxd
        brew install --HEAD libimobiledevice
        brew install ideviceinstaller
    • ios-deploy 2.0.0
    • CocoaPods version 1.5.3

[✓] Android Studio (version 3.1)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin version 28.0.1
    • Dart plugin version 173.4700
    • Java version OpenJDK Runtime Environment (build 1.8.0_152-release-1024-b01)

[!] IntelliJ IDEA Ultimate Edition (version 2018.2.2)
    • IntelliJ at /Applications/IntelliJ IDEA.app
    ✗ Flutter plugin not installed; this adds Flutter specific functionality.
    ✗ Dart plugin not installed; this adds Dart specific functionality.
    • For information about installing plugins, see
      https://flutter.io/intellij-setup/#installing-the-plugins

[✓] VS Code (version 1.30.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 2.21.1

[✓] Connected device (1 available)
    • vivo NEX A • 9cb87f39 • android-arm64 • Android 8.1.0 (API 27)

Did anyone resolve this issue? We are encountering it on many devices and keeps our main functionality of the Google map broken...
Please if someone has any idea how to fix it even locally would be also a thing

Hello, same issue using OnePlus, Xiaomi with google maps - java.lang.NullPointerException: Attempt to invoke virtual method 'int android.view.Display.getDisplayId()' on a null object reference

I also encountered this problem. My phone model is 360 N6 Pro.

Any update? Please take action on this

confirmed on Huawei P20 Lite Android 8.0.0

Also happening on the Firebase Test Lab's Pixel 2 API 26 (Android 8.0) device, it's also happening on 11 devices, all using Android 8.1 (API 27)
According to the google play report, it happened in these devices
Moto G (5th Gen) (cedric)
Moto G (5S) Plus (sanders_nt)
ZenFone Max Pro (M1) (ASUS_X00T_2)
C5 (C014)
moto e5 play (pettyl)
Moto G Plus (5th Gen) (potter_nt)
Moto G (5S) (montana)
B2017G (tulip)
ZenFone Max Shot (ZB634KL) (ASUS_A001D_2)
Moto G(4) Plus (athene_f)
ZenFone Max Plus M2 (ZB634KL) (ASUS_A001D_1)

Same issue on Pixel 3 Android 9... any update on a fix. Blocker for us as maps is a major part of the app.

This mostly happens, when I rapidly call setState.

I made a custom AndroidView and the first time that I navigate to the page where it is, it doesn't crash, but when I try to navigate for the second time to the same page the custom view make the app crash with the error reported in this issue. I tested it on P20 lite with Android 8. After I tried on an Honor 8 with Android 8 and everything works fine everytime.

Hope this help. In our app we found similar issue. Maybe 60% time the app will crash when we get in/out the map page. It looks like the issue is related to the Hero animation. After removing the Hero animation we didn't encounter the issue any more.

This mostly happens, when I rapidly call setState. Occurs on the admob_flutter plugin.

This is exactly the same thing for me.

@melbit-lukeshan @nemoryoliver @TheDOme6
In our case we do not setState and do not have Hero animations and it crashes constantly even with the latest flutter update v1.5.4-hotfix.2. Will anyone from the Google team ever look at this? How can you have such a great Google product like Flutter that cannot use another Google product like Google Maps, with which people want to build apps upon.
@zoechi

Having this issue with Moto G (5S) Plus right now. The crashes appear random.

This mostly happens, when I rapidly call setState. Occurs on the admob_flutter plugin.

Rapidly?

Tested it by calling setState every 0.5 seconds

Found the same issue in my app, in my case there are two likely suspects:

A call to setState throwing an error, due to me calling the method when I shouldn't have.
The Background Geolocation library.

In the first case, I think it might me the cause because the error was thrown at the same time as the app crashed, and solving it appears to have fixed this issue. As for the second case, it was the library itself the one that threw the error from within my app, rather than my own code, or so it seemed. It's quite tricky for me to pinpoint the error, as it's very difficult to reproduce.

Even i have to hide google maps in my app due to this issue. I hope team fixes it soon. There's really no way out for now.

who can fix this issue?

@zoechi I hope flutter develop team will fix this issue as soon as possible.

@qypengzai do you have a reproducible example?

also result in crash.

with exactly the same error output?

@zoechi Any update on this?...
@qypengzai  Trying google maps plugin crashes with the same exception.

Same error here, when i re-enter fast on some page with maps in Xiomi or Moto G

who has information on this issue?

My google_map still crashes, hopefully there is an explanation for this issue

me too

I encountered the same crash using google_maps_flutter.
An error occurred when go to the second google_map page from a screen with a keyboard.
I fixed it to navigate to the google_map page after hide the keyboard.
(Phone model is Xperia XZ3 SO-01L)

Hi, this crash occurs in our apps also, but only on newer, taller phones like xiaomi 9.
I found that if the height/width ratio is bigger crash happens
crash:
return Container(
  width: MediaQuery.of(context).size.width,
  height: MediaQuery.of(context).size.width * 1.8,
  child: GoogleMap(
	  initialCameraPosition: CameraPosition(
		  target: const LatLng(50.0, 19.0), zoom: 3.0)),
);

doesn't crash:
return Container(
  width: MediaQuery.of(context).size.width,
  height: MediaQuery.of(context).size.width * 1.5,
  child: GoogleMap(
	  initialCameraPosition: CameraPosition(
		  target: const LatLng(50.0, 19.0), zoom: 3.0)),
);

have you find the solution?

I'm also getting a similar error from multiple users from Xiaomi
Fatal Exception: java.lang.NullPointerException: Attempt to invoke virtual method 'int android.view.Display.getDisplayId()' on a null object reference
       at android.view.ViewRootImpl$1.onDisplayChanged(ViewRootImpl.java:1145)
       at android.hardware.display.DisplayManagerGlobal$DisplayListenerDelegate.handleMessage(DisplayManagerGlobal.java:493)
       at android.os.Handler.dispatchMessage(Handler.java:106)
       at android.os.Looper.loop(Looper.java:171)
       at android.app.ActivityThread.main(ActivityThread.java:6656)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:547)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:873)

Another similar kind of error which I'm getting:
Fatal Exception: java.lang.NullPointerException: Attempt to invoke virtual method 'android.view.DisplayAdjustments android.view.Display.getDisplayAdjustments()' on a null object reference
       at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:1761)
       at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1505)
       at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:7354)
       at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1092)
       at android.view.Choreographer.doCallbacks(Choreographer.java:888)
       at android.view.Choreographer.doFrame(Choreographer.java:819)
       at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1078)
       at android.os.Handler.handleCallback(Handler.java:873)
       at android.os.Handler.dispatchMessage(Handler.java:99)
       at android.os.Looper.loop(Looper.java:201)
       at android.app.ActivityThread.main(ActivityThread.java:6810)
       at java.lang.reflect.Method.invoke(Method.java)
       at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:547)
       at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:873)

on many production devices. We only use https://github.com/sarbagyastha/youtube_player_flutter for embedding a few YouTube videos and some logs of Crashylytics points to users opening it before the crash, but not many data points to confirm anything. Was unable to reproduce the bug on my local Android/iOS device and emulator.

me too

me too

@mklim is there any chance this work can be started anytime soon? this one plus https://github.com/flutter/flutter/issues/38063 are the two ones causing most of our crashes

Unfortunately I don't think it likely that I'm going to be able to work on this in the near term future, so I'm taking off my assignment and resetting the milestone. Sorry about that everyone. :disappointed:
For what it's worth for anyone else looking into this I had started investigating this briefly already, and I was blocked on not being able to get reproduction of the crash. I had tried with some of the devices mentioned in this report, but not all of them. I haven't tried the most recent code sample posted here either.

@mklim I have demo to reproduce this crash, 100%.
When WebView is playing videos, when you open the keyboard, the crash happen immediately.
You can find my demo in https://github.com/singun-lxd/flutter_webview_crash, or another simple demo below.
Models:
Huawei Y9 2019, Android 9
Huawei nova 3i, Android 9
Stack:
2020-03-04 13:36:08.938 3901-3901/com.demo.flutter_app E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.demo.flutter_app, PID: 3901
    java.lang.NullPointerException: Attempt to invoke virtual method 'android.view.DisplayAdjustments android.view.Display.getDisplayAdjustments()' on a null object reference
        at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:1900)
        at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1636)
        at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:7946)
        at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1092)
        at android.view.Choreographer.doCallbacks(Choreographer.java:893)
        at android.view.Choreographer.doFrame(Choreographer.java:812)
        at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1078)
        at android.os.Handler.handleCallback(Handler.java:907)
        at android.os.Handler.dispatchMessage(Handler.java:105)
        at android.os.Looper.loop(Looper.java:216)
        at android.app.ActivityThread.main(ActivityThread.java:7625)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:524)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:987)

Demo code:
import 'package:flutter/material.dart';
import 'package:webview_flutter/webview_flutter.dart';

void main() => runApp(MyApp());

class MyApp extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: MyHomePage(title: 'Flutter WebView Crash'),
    );
  }
}

class MyHomePage extends StatelessWidget {
  MyHomePage({Key key, this.title}) : super(key: key);

  final String title;

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(title),
      ),
      body: Column(
        children: <Widget>[
          Text("Please tap the text field below to trigger the crash"),
          TextField(),
          Expanded(
            child: WebView(
              initialUrl: "https://www.youtube.com/watch?v=mlBIpwtNxxE",
              javascriptMode: JavascriptMode.unrestricted,
            ),
          ),
        ],
      ),
    );
  }
}

Dependeccies:
webview_flutter: 0.3.19+8

Got a crash report for nubia 563j with OS 7.1.1. Our app has a google map view on the homepage.

I found this issues with the google_maps_flutter plugin. My workaround was just to wrap it in a FutureBuilder with a 500ms delay.
              return FutureBuilder(
                  future:
                      Future.delayed(Duration(milliseconds: 500), () => true),
                  builder: (ctx, future) {
                    if (future.hasData) {
                      return Container(
                        child: GoogleMap(
                          mapToolbarEnabled: false,
                          myLocationButtonEnabled: false,
                          markers: snapshot.data,
                          mapType: MapType.normal,
                          initialCameraPosition: camPos,
                          onCameraMove: _homeBloc.streams.listenToMapMovement,
                          onMapCreated: _homeBloc.registerMapController,
                        ),
                      );
                    } else {
                      return Center(
                        child: SpinKitCubeGrid(
                          color: Colors.white70,
                        ),
                      );
                    }
                  })

Any solution for this. I am using the online video call library in that also same error is coming

Repro-ed on Huawei Nova 4 (VCE-AL00), thanks @singun-lxd for the straightforward repro!

I've submit a PR(flutter/engine/pull/17611) to Flutter Engine repo.
To fix it currently without having to wait for PR acceptance and flutter release, I use AOP to add some code to the flutter engine class(very simply, with only a couple of lines of code).
Here is the PATCH file: https://raw.githubusercontent.com/dotcink/flutter_webview_crash/fix-crash/fix.patch
It's been tested with @singun-lxd 's repro project, as in https://github.com/dotcink/flutter_webview_crash/tree/fix-crash.

To fix it currently without having to wait for PR acceptance and flutter release, I use AOP to add some code to the flutter engine class(very simply, with only a couple of lines of code).

@dotcink Can you provide instructions on using AOP to test this out? I'm not familiar with it, and I'd like to test if this clears out #53598 as well.

To fix it currently without having to wait for PR acceptance and flutter release, I use AOP to add some code to the flutter engine class(very simply, with only a couple of lines of code).

@dotcink Can you provide instructions on using AOP to test this out? I'm not familiar with it, and I'd like to test if this clears out #53598 as well.

You can see the diff at https://github.com/dotcink/flutter_webview_crash/commit/0bd6d2ac46022accfaee6b6663c2c5f6aa4ae9d8
Or just try to apply the patch https://raw.githubusercontent.com/dotcink/flutter_webview_crash/fix-crash/fix.patch
It's very simple. No more than 20 lines of code.

This should be fixed by https://github.com/flutter/engine/pull/17897

Is there any chance the fix could land in a hotfix for 1.12.13?
This is a number one crash in our app's reports and number one complaint by our users with Xiaomi phones.

The changes have landed in the engine. Once it rolls into the framework (should happen in next few hours), it would be helpful if someone could verify it fixes the crash in their specific case.
Once verified, I can see if this is hotfixable. I will submit it to the hotfix list.

[...]it would be helpful if someone could verify it fixes the crash in their specific case.

Confirmed, it fixes the crash in singun-lxd's Feb 24 comment, as well as the one documented at #53598. Tested on both the DragonTouch M7 tablet, and the Minix NEO T5 Android TV.

How can we tell if this fix, (or specifically flutter/engine#17897) is merged into dev or beta or a hotfix right now? The changelogs are not super helpful.

@qimingweng it's a little involved. There's a guide on the "Where's my commit?" page on the Wiki for tracking specific commits like this.
In this case it's rolled into the framework in flutter/flutter@e0c63cd35e15e407a80dc44281cc392535fcce25, v1.18.0-8.0.pre. which you can then see on the SDK releases page has currently rolled into the dev but no other channel.

Oh wow OK. @mklim I think I follow but the instructions are a bit involved haha. Thanks for pointing it out, really appreciated it.
For everyone else, I can verify that this (the dev channel) did fix the crash on google_maps_flutter on a Nexus 5X device on Android 8.1.

Specifically @GaryQian can confirm this fixes one symptom of the issue and would appreciate it if we could hotfix this.

@qimingweng I will see what I can do!

Hey, just for record's sake, we upgraded to Flutter 1.17.0 and the issue seems to have gone away on our Nexus 5X for Google Maps. However, I also went through the process from this link: https://github.com/flutter/flutter/wiki/Where's-my-Commit%3F#finding-the-framework-commit-that-contains-engine-commit-x and could not necessarily confirm that this fix has rolled into 1.17.0 (no hotfix suffix).
For whatever it's worth.

I have tested this issue with the dev channel. This issue is resolved in the dev channel.

We noticed there is a new version of flutter, 1.17.1 out May 13. Do those tags indicate that this issue is resolved in this version?

@qimingweng yes, it was mentioned in the release notes: https://github.com/flutter/flutter/wiki/Hotfixes-to-the-Stable-Channel#flutter-117-changes

For what it's worth, I can confirm this has fixed our issue in stable-1.17.1.


Update opencensus, guava, and Truth

Looks like something is incompatible here (probably truth 1.0)

I made some changes to fix the build. @vam-google can you take a look?


Suggestion: Ability to disable commands & the discord status

Self-explanatory by the title, the ability to disable the Discord Bot Commands & The Discord Commands would be amazing. Majority of the stuff my community uses in terms of bots is custom and having txadmin's discord status and commands tends to interfere with that stuff

You can either: A) entirely disable/remove the bot, ot b) make the prefix so incredibly long and hard to guess that nobody would get it.

Then simply remove your token from the settings page.

As above, you can:

Disable the bot;
Change prefix;
Or even just remove his permission to speak in any channel that is not the server status one.


Aalen-Johansen Variance

For large sample sizes, the AalenJohansenFitter has some trouble estimating the variances. While working on a tutorial for competing risks, I noticed some unexpected behavior for confidence intervals of the Aalen-Johansen estimator.
This does not have to do with the formula, but rather it becomes a problem for large samples. Somewhere in the variance formulation (in _bounds), one element gets close to zero (or something similar). This near-zero value is causing the variance calculations to blow up. Everything works exactly as expected when n ~ 1000, but when I increase the sample size closer to n ~ 10,000 is when problems start occurring.
Below is where the variance calculation takes place
            sf["part1"] = ((F_t - sf["Ft"]) ** 2) * (
                sf["observed"] / (sf["at_risk"] * (sf["at_risk"] - sf["observed"]))
            )
            sf["part2"] = (
                ((sf["lagS"]) ** 2)
                * sf[self.label_cmprisk]
                * ((sf["at_risk"] - sf[self.label_cmprisk]))
                / (sf["at_risk"] ** 3)
            )
            sf["part3"] = (F_t - sf["Ft"]) * sf["lagS"] * (sf[self.label_cmprisk] / (sf["at_risk"] ** 2))
            variance = (np.sum(sf["part1"])) + (np.sum(sf["part2"])) - 2 * (np.sum(sf["part3"]))

My guess is that something is going wrong with (sf["observed"] / (sf["at_risk"] * (sf["at_risk"] - sf["observed"])) or  (sf[self.label_cmprisk] / (sf["at_risk"] ** 2)), but I haven't tracked down this problem yet. The reason I target these two parts as responsible is because they will get close to zero for large sample sizes, especially the second option
When writing the original, I missed this because I tested it on sample sizes below 2,000
For replication, you can use the following data generator
def simulate_competing_risk(large_n):
    if large_n:
        n_idu = 1143
        n_non = 6039
    else:
        n_idu = 114
        n_non = 603

    df_idu = pd.DataFrame()
    df_idu['t_death'] = np.random.weibull(0.8, size=n_idu) * 10
    df_idu['t_aids'] = np.random.weibull(1, size=n_idu) * 10
    df_idu['idu'] = 1

    df_non = pd.DataFrame()
    df_non['t_death'] = np.random.weibull(1.1, size=n_non) * 10
    df_non['t_aids'] = np.random.weibull(1.1, size=n_non) * 10
    df_non['idu'] = 0

    df = pd.concat([df_idu, df_non], ignore_index=True)
    df['t_censor'] = np.random.weibull(1.3, size=n_idu + n_non) * 3
    df['t_censor'] = np.where(df['t_censor'] > 5, 5, df['t_censor'])  # Admin censoring at t=5

    df['t'] = np.min(df[['t_death', 't_aids', 't_censor']], axis=1)
    df['event'] = np.where(df['t'] == df['t_censor'], 0, np.nan)
    df['event'] = np.where(df['t'] == df['t_aids'], 1, df['event'])
    df['event'] = np.where(df['t'] == df['t_death'], 2, df['event'])
    df['t'] = np.round(df['t'], 5)
    df['id'] = df.index
    return df[['id', 'event', 't']]

When large_n=True the variance explodes. Also, I know that it is somewhere before the SE transformation, since AalenJohansenFitter.variance is blown up. It has to be in the part 1 / part 2 / part 3 calculations somewhere

Thanks for the detailed bug report 👍
Just thinking quickly, it may be an overflow/under problem. A solution might be to "flatten" the divisions and multiplications. Ex:
            sf["part2"] = (
                ((sf["lagS"]) ** 2)
                * sf[self.label_cmprisk]
                * ((sf["at_risk"] - sf[self.label_cmprisk]))
                / (sf["at_risk"] ** 3)

to
sf["part2"] = (
    sf["lagS"] ** 2 / sf["at_risk"]
    * sf[self.label_cmprisk] / sf["at_risk"]
    * (sf["at_risk"] - sf[self.label_cmprisk]) * sf["at_risk"]
)

Good call! Spacing out the terms seems to work when I tested it. I will create a PR with the two issues I raised today
@CamDavidsonPilon One more item since I am working on this: Aalen-Johansen variance is slow since it needs to go over every event time. When I use IPW, I have to use non-parametric bootstrapping to get the confidence intervals. Since AalenJohansenFitter calculates the variance for each individual resample, it takes more computation time than necessary (since these variances play no role in the bootstrap). Would you mind if I add something like calculate_variance to init? The default would be True and calculate the variance. If set to False it would skip over these calculations to save computation time during bootstrapping

right right, that makes sense. Yea, I think that additional kwarg is fine 👍
Thanks for being a good code owner!


Introduce formPost action for HTTP client

Description:
Currently when setting form data directly to client action as a string, the content type is set as text/plain. Since posting data to a form is a common requirement, it's better if we could introduce a client action called "formPost" and set header as application/x-www-form-urlencoded
var clientResponse = clientEP->formPost("/", "name=abc&age=12");

Steps to reproduce:
Affected Versions:
OS, DB, other environment details and versions:
Related Issues (optional):

Suggested Labels (optional):

Suggested Assignees (optional):

Closing the issue as it contradicts the current client action consistency.


MB-7432 Rename fetchMTOUpdatesFast to listMoves

Description
This PR renames the fetchMTOUpdatesFast endpoint to listMoves since this will eventually become the new default fetch endpoint for the Prime API.
Setup
Populate your database and run the server:
make db_dev_e2e_populate server_run

Then, using Postman, hit the new listMoves endpoint at /moves to retrieve Prime-available moves.
Alternatively, use the Prime API Client to get the list in the terminal:
rm -rf bin/prime-api-client
make bin/prime-api-client
prime-api-client list-moves --insecure | jq

Messages

:book:
:link: MB-7432

  Generated by :no_entry_sign: dangerJS against 84090f0d7745b565a134a729181d1b770472c41c


Disable or change Stepper Animation

Hi, How can i change or disable Stepper animation. Please instruct me.

@proxact
Hi, I will add a new property ContentAnimationsEnabled to enable (true) or disable (false) the animations of the stepper.


Optimize unnecessary column copy for HashAgg

What problem does this PR solve?
Issue Number: close #8891
Problem Summary:
When there are group by key in select item(a.k.a. first_row), tiflash have extra agg func, which cause unnecessary copy from HashMap to Column.
What is changed and how it works?
Basic idea:

Optimization-1 (with collation):
What: For group by keys that with collation, there will be first_row/any agg func for it to keep original data. So no need to copy these keys from HashMap, instead just a pointer to reference its corresponding first_row/any result is enough.
How:

Detect if there is a first_row agg func in the select item.
If so, ignore any agg func.  If not, add the 'any' agg func.
Also, set key_from_agg_func to indicate that this key is equivalent to first_row/any agg func, which can avoid copying this key from the HashMap in subsequent operations. (check DAGExpressionAnalyzer::buildAggGroupBy)
If all keys are included in first_row/any (which is rare, but still can happens), will skip copy keys(template argument skip_serialize_key is true)

Optimization-2(no collation)
What: When SQL query has agg func like: first_row(group_by_key_col), and that group_by_key_col has no collation. Then we can eliminate the first_row agg func, just use a pointer to reference the group by key is enough.
Results

25% improvement
workload:

20M rows. More rows a different, it means we have very high NDV.
3 varchar columns, 3 decimal columns, 3 int columns (that means Aggregator will use HashMethodSerialized)

before:

after:

Check List
Tests

[x] Unit test
[ ] Integration test
[ ] Manual test (add detailed scripts or steps below)
[ ] No code

Side effects

[ ] Performance regression: Consumes more CPU
[ ] Performance regression: Consumes more Memory
[ ] Breaking backward compatibility

Documentation

[ ] Affects user behaviors
[ ] Contains syntax changes
[ ] Contains variable changes
[ ] Contains experimental features
[ ] Changes MySQL compatibility

Release note

None

/run-all-tests

/test all

/test all

/test pull-integration-tes

/test pull-integration-test

/test pull-unit-test

/hold

comment /unhold to merge pr~
@guo-shaoge

/test pull-integration-test


KAFKA-9436: New Kafka Connect SMT for plainText => Struct(or Map)

New SMT

plain text => struct(map)
regex group condition with ordered key name
compatible with single plain text input and struct field input plain text

sample1
"<IP_ADDRESS> - - [08/Aug/2019:18:15:29 +0900] \"OPTIONS /api/v1/service_config HTTP/1.1\" 200 - 101989 \"http://local.test.com/\" \"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/75.0.3770.142 Safari/537.36\""
SMT connect config with regular expression below can easily transform a plain text to struct (or map) data.

"transforms": "TimestampTopic, RegexTransform",
"transforms.RegexTransform.type": "org.apache.kafka.connect.transforms.ToStructByRegexTransform$Value",

"transforms.RegexTransform.struct.field": "message",
"transforms.RegexTransform.regex": "^([\\d.]+) (\\S+) (\\S+) \\[([\\w:/]+\\s[+\\-]\\d{4})\\] \"(GET|POST|OPTIONS|HEAD|PUT|DELETE|PATCH) (.+?) (.+?)\" (\\d{3}) ([0-9|-]+) ([0-9|-]+) \"([^\"]+)\" \"([^\"]+)\""

"transforms.RegexTransform.mapping": "IP,RemoteUser,AuthedRemoteUser,DateTime,Method,Request,Protocol,Response,BytesSent,Ms:NUMBER,Referrer,UserAgent"

sample2
{
   "code" : "dev_kafka_pc001_1580372261372"
   ,"recode1" : "a"
   ,"recode2" : "b"
}

"transforms": "RegexTransform",
"transforms.RegexTransform.type": "org.apache.kafka.connect.transforms.ToStructByRegexTransform$Value",

"transforms.RegexTransform.struct.field": "message",
"transforms.RegexTransform.regex": "^(.{3,4})_(.*)_(pc|mw|ios|and)([0-9]{3})_([0-9]{13})" "transforms.RegexTransform.mapping": "env,serviceId,device,sequence,datetime:TIMEMILLIS"

@whsoul thanks for the contribution. Can you please create a KAFKA Jira issue with the needs-kip label, since adding a new SMT will require a KIP. Thanks!

@rhauch thanks rhauch
I already create JIRA issue (https://issues.apache.org/jira/browse/KAFKA-9436)
but forgot label "needs-kip"... but now added
Thanks~

The issue is unresolved.  Do we think this PR can be merged once the checks pass?

@amerixan, this issue cannot be merged because adding an SMT requires a KIP, and that KIP hasn't been created, discussed, and approved.
However, I'll start a developer list discussion thread to discuss whether we want to add any new SMTs to Apache Kafka. The AK project implements no real connectors (just the example file source and sink connectors), and that was an explicit decision when Connect was added to the AK project with KIP-26. We've learned that it's good for AK to include some very simple and general SMTs, but whether something is general or simple enough is a grey area. Plus, AK releases are already complicated, and adding more SMTs means these can only be fixed or improved with a whole AK time-boxed release or infrequent patch release. So even though we already have a few SMT KIPs that have not yet been approved, I will likely advocate that we not add more SMTs to AK and instead suggest folks create and maintain SMTs through outside projects -- the same as we expect for Connectors.

I am confused by this statement: "that KIP hasn't been created"  KIP 678?
https://cwiki.apache.org/confluence/display/KAFKA/KIP+678%3A+New+Kafka+Connect+SMT+for+plainText+%3D>+Struct(or+Map)+with+Regex
As for the suggestion of not adding any more SMT, I am just a lowly user who is grateful for everyone's time developing this, but I can tell you that as far as third party SMT's go, I haven't found anything to provides this functionality.  Additionally, looking for third party SMT's has proven painful.  If it is decided not to create any more, I would hope that some kind of directory to other solutions is created.
Use case for this SMT:
I currently have an application that can generate a JSON object, but for some stupid reason it has an unconfigurable behavior where it puts a timestamp in front of the object breaking any ability to work with it as a normal JSON object.  This SMT would resolve that by allowing an avenue to make it a proper object again.
Again, all work on this is much appreciated.  I hope this SMT is able to continue.  Meanwhile, I am no coder, but I am going to try to see what I can do about piecing things together and trying to use this as my current resolution to the above issue in its current state.

it puts a timestamp in front of the object

If that's the case, I don't think this specific transform will help as it'd make a Struct wrapper around the extracted json object within the string.
My recommendation instead is to define your own subclass of StringDeserializer that substrings the message to remove the date prefix

@OneCricketeer
I was reading the above solution as the following:
transforms.RegexTransform.regex": <regex for timestamp and then wildcard to $ (end for existing json object)

"transforms.RegexTransform.mapping": "timestamp,existingjson"
struct {"timestamp": ,"existingjson":{json object here}}
My recommendation instead is to define your own subclass of StringDeserializer that substrings the message to remove the date prefix
I have no idea how or where to begin with that, but will see if I can't find something.  Thank you for the recommendation.

Well, Kafka record have their own timestamps since version 0.10.2, so they shouldn't need to be part of the record string...

no idea how or where to begin with that

Assuming Java, then
public class MyDeserializer extends StringDeserializer {
  private int dateLength = "YYYYmmdd".length; // for example
  @Override
  public void deserialize(String topic, byte[] data) {
    String s = super.deserialize(data);
    return s.substring(dateLength);
 }
}

In the consumer config, set value.deserializer to MyDeserializer.class
In your poll loop, accept strings and use a JSON library to further process the record

In the consumer config, set value.deserializer to MyDeserializer.class
In your poll loop, accept strings and use a JSON library to further process the record
I'm utilizing Kafka Connectors.  After doing some review, looks like the solution would be called via value.converter rather than value.deserializer
Once again, thank you @OneCricketeer

The class provided needs to be part of consumer code and must be value.deserializer not the Connect API since it's not a converter. You said you're consuming strings, not JSON, which would imply you're using StringConverter already, rather than JSONConverter or something else

You said you're consuming strings
While I am consuming strings initially, it is the intention to have it as proper JSON before it arrives in Kafka.  The full solution is:
app (generate sha1 for indexId as part of the json object generated to prevent duplicate ingests) -> kafka connector (use transform to pull indexId into a header)  -> kafka -> camel k elastic sink solution (works with ES7 without issues) that pushes it into Elastic which needs JSON and indexId

Again, I am no developer.  Just looking for an easy solution.  The above is completely existing except the solution to take out the timestamp or assign it to a JSON object and process proper json objects.

a custom value.converter may work.

https://www.confluent.io/blog/kafka-connect-single-message-transformation-tutorial-with-examples/

a custom value.converter may work.

Or a transform

Correct.  To be clear, this was the first thought before you suggesting deserializer. This SMT is the only transform I found in existence that I thought would work for the issue.  Since developing a solution from scratch might prove to be a bit much for me (not a developer by trade), I figured this would be the best path.

suggest using grok patterns

I knew I'd seen this somewhere before, but finally found it again
https://github.com/streamthoughts/kafka-connect-transform-grok

suggest using grok patterns

I knew I'd seen this somewhere before, but finally found it again
https://github.com/streamthoughts/kafka-connect-transform-grok
So I was able to build the solution presented here, but after some hair pulling, I did end up using the grok pattern solution since it performed better overall to include being able to parse curly brackets correctly.  I did note that the grok SMT had not been updated in over a year.  Hopefully it is just because they consider it stable rather than it becoming abandonware.

Regardless, thank you for sharing @OneCricketeer   It was very much appreciated.
I definitely appreciated the workflow as well.  The grok solution provides a method where depending on the message you may need to declare it as a string using the Cast SMT, but you are then able to directly work with the message to form the struct and appropriate fields.  With the solution presented here, I had to use Cast to declare the message and string, then apply HoistField to make a simple struct for all of the message field, then I could use this SMT to try and work with the struct itself.  The grok method was definitely more efficient and the inclusion of the usual + additional patterns was a very nice addition.
The note I would add that was not immediately obvious to me was that while I could place this SMT into the Kafka plugins directly, I received very odd errors with the Grok SMT.  It took me far longer than I care to admit to realize that the Grok SMT and the 2 supporting jar files needed to be placed into their own subfolder within the Kafka plugins directly in order for the jar files to be loaded properly.


DateTime Input component with auto convert local time to server time zoned datetime

Describe what feature you'd like. Pseudo-code, mockups, or screenshots of similar solutions are encouraged!
Would be nice to have a DateTime Input component
That  auto convert local date-time (from browser)  to time-zoned date-time (for sever) and vise versa.
Generally database save date-time  with time zone and it need to be converted to local date-time before we can bind it to HTML date-time input.
I have this solution , but would be nice to bundle a better implementation with skeleton
https://github.com/skeletonlabs/skeleton/issues/new/choose
What type of pull request would this be?
New Feature
Any links to similar examples or other references we should review?
No response

Hey @xmlking, I've finally had a chance to look through some older issues in our backlog and saw this. I can see the appeal to your request, but for now we're pretty keen to stick to native-only inputs whenever possible. If you're looking for a good way to manipulate date/times I recommend this library:
https://date-fns.org/
Additionally, there is a proposal in place for a new date/time API for browsers called Temporal. I believe it's still in the early phases, but probably worth keeping an eye on:
https://tc39.es/proposal-temporal/docs/
Beyond that, we do hope to one day introduce date, time, and date/time pickers to Skeleton, however much like Data Tables this is a very involved request. Whole open source projects are dedicated to creating these features, so the scope may be beyond what we can handle right now.
But thank you for your suggestion and sorry we couldn't be more help!


Get existing outreach efforts regarding clearing drains

From Jean

Jean Walsh quote:

They are cleaned by DPW and SF PUC. We encourage residents to clean them too,
but there is no official program.

Closing this issue.


[BUG] Jupyter cannot block and logs are sent to /dev/null

Describe the bug
When working on the RAPIDS Helm Chart we have a pod which will be running Jupyter as a daemon. Ideally this pod would simply run Jupyter as the main blocking process.
The current config for the runtime image is for Jupyter to be run during the entrypoint with nohup and all stdout/stderr is sent to /dev/null.
The default command is bash which just exits if the container is not running in interactive mode.
In order for us to block on the Jupyter process we currently have to override the entrypoint and command with a new one which runs Jupyter in the foreground. This results in duplication of the Jupyter startup command in the helm chart. Alternatively we could instead tail the nohup output as the command in order to block, but this isn't possible due to all output going to /dev/null.
Steps/Code to reproduce bug
docker run --rm 0.14-cuda10.1-runtime-ubuntu18.04-py3.7

Container just exits.
Expected behavior
I would expect to be able to run the container with Jupyter as the foreground process.
A few ways this could be achieved:

Pipe the output to somewhere like /var/log/jupyter.log so that we can override the command as tail -f /var/log/jupyter.log
Run Jupyter in the foreground by default without nohup and output redirection
Split the start_jupyter.sh script into a foreground and background version, allowing us to override the entrypoint/command to the foreground version.

Environment details (please complete the following information):
Any
Additional context
NA
cc @quasiben

Our current workaround is to override the entry point and command (via the command: option in Kubernetes) and the args.
We are basically running the command from start_jupyter.sh without nohup and backgrounding. But this is not ideal as any changes to start_jupyter.sh wouldn't be applied in the helm chart without a manual sync.
https://github.com/rapidsai/helm-chart/blob/48da21c0b37d73d4cf3b367f13b658bea4e3853c/rapidsai/values.yaml#L93-L96

once the new nightlies are built, you can use the following docker run flag to enable jupyter-lab to run in the foreground: -e JUPYTER_FG=true

Thanks @ajschmidt8

Sorry to say I'm seeing this error with the change.
/.run_in_rapids: line 12: /dev/tty: No such device or address


changle babel preset from es2015 to env

Firefox last 2
Chrome last 2
Samsung internet
Carmel?

Carmel is the old version, now it's called Oculus browser which is based on chromium 57 currently I think.