feat: update readme/add owners

Signed-off-by: Abdulla Mulail Mohamed<EMAIL_ADDRESS>

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by:
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

/assign @zacaaron

/assign @maciej-szlosarczyk

LGTM label has been added.  Git tree hash: 62f6e82e0dca7ce95ff69d82eef1206a1c55981d

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: zacaaron
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[APPROVALNOTIFIER] This PR is NOT APPROVED
This pull-request has been approved by: maciej-szlosarczyk, zacaaron
The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment


[FEATURE] Officer Dashboard Updates

Describe the feature

Rank - Add To Dashboard Could we get rank added here?

Could we change Badge to Callsign on here somehow?
Since Callsign is what people need to know the most not badge number.

Additionally, I actually would love to know if I can remove the Badge sections entirely from the CAD. Because we don't use it. We basically use the Callsigns as their one and only identifier.
Thank you!
Additional Context
No response

For now I'll have to keep the badge-numbers since I don't want to bloat the Features tab. I'll have to figure out how I want it to look, or figure out a better design pattern for the features 😅


Error with early open keyword in Get Started with F# in Visual Studio Code

In the "Get Started with F# in Visual Studio Code" article, the use of namespace ClassLibraryDemo, "below the open System declaration" causes System to be red-underlined, along with the problem: "Only '#' compiler directives may occur prior to the first 'namespace' declaration F# Compiler (530)".
Is there an error in the tutorial, or perhaps my setup is wrong?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 9b07eeb2-d274-35da-78d4-b7d78c255590
Version Independent ID: bf145620-7f23-eb4d-376d-f739562d9a79
Content: Get Started with F# in Visual Studio Code
Content Source: docs/fsharp/get-started/get-started-vscode.md
Product: dotnet-fsharp
GitHub Login: @cartermp
Microsoft Alias: phcart

Nothing wrong with your setup @pkeir. This example fails because whoever added it to the page didn't run it like they suggest you do! I'm a newbie and here on my holiday having a first look at F# and got the same issue but you can see from the error and a quick Google that the namespace is (probably) irrelevant at this stage of just running a first f# cmd line app built from a single file of F#. Not even sure the module is necessary - the example can be made to work without that also. Take out the namespace declaration and you should be good to go.
Namespacing is no doubt important to using F# for a real project but you can worry about that if you get beyond this page. I suspect that the last edit date of this page (12/23/2018) says something about how popular F# was in 2019. Its that time of year again and your comment was only three weeks ago...
Merry Christmas!

@spavett Thanks a bunch Had the same errors! 🚀


Query exporting only 2ˆ28 documents (data have more) even after setting query limit to 2ˆ29

Description
We're working with a CouchDB database that holds about 272 million records. When attempting to retrieve all documents using a curl command and writing to a file, we noticed that only 268435456 records were returned, despite the database having more entries.
The curl command in question:
curl -k -X GET "https://$(cat credential_couchdb)@{host}/{db}/_all_docs?include_docs=true" > all_docs
This led us to suspect a default query limit issue, which aligns with discussions in this GitHub pull request.
To resolve this, we updated the config file to set both partition_query_limit and query_limit to 536870912 (2^29).
"query_server_config":{"partition_query_limit":"536870912","query_limit":"536870912"}
Despite this change and the offset field from the output file displaying the correct amount of total records, the output still holds 268435456 records.
Expected Behaviour
We expect the number of records in the output file to match the offset field value, not to be stuck in this default value count.
Additional Context
The CouchDB instance in question is integrated as a state database within a Hyperledger Fabric setup.
The database is updated once a day.
@lucasmation

https://github.com/apache/couchdb/blob/aca60ddf147811547b11cf5e8fd10bcfbe3c053e/src/couch_mrview/include/couch_mrview.hrl#L57
You may need to paginate.


Ckeditor integration?

As this is an alternative to the SonataMediaBundle, I wonder if there's a way use this bundle with Ckeditor.

As adding images to a WYSIWYG editor adds hardcoded urls and non-responsive images I don't really feel a need to support this. Feel free to discuss how this could be solved in a proper way.


UVI Request

--- UVI JSON ---
{
  "vendor_name": "Linux",
  "product_name": "Kernel",
  "product_version": "versions from v4.9.273 to before v4.9.276",
  "vulnerability_type": "unspecified",
  "affected_component": "unspecified",
  "attack_vector": "unspecified",
  "impact": "unspecified",
  "credit": "",
  "references": [
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=8958181c1663e24a13434448e7d6b96b5d04900a",
    "https://git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=d2f0b960d07e52bb664471b4de0ed8b08c636b3a"
  ],
  "extended_references": [
    {
      "type": "commit",
      "value": "8958181c1663e24a13434448e7d6b96b5d04900a",
      "note": "introduced"
    },
    {
      "type": "commit",
      "value": "d2f0b960d07e52bb664471b4de0ed8b08c636b3a",
      "note": "fixed"
    }
  ],
  "reporter": "joshbressers",
  "reporter_id": 1692786,
  "notes": "",
  "description": "scsi: core: Fix bad pointer dereference when ehandler kthread is invalid\n\nThis is an automated ID intended to aid in discovery of potential security vulnerabilities. The actual impact and attack plausibility have not yet been proven.\nThis ID is fixed in Linux Kernel version v4.9.276 by commit d2f0b960d07e52bb664471b4de0ed8b08c636b3a, it was introduced in version v4.9.273 by commit 8958181c1663e24a13434448e7d6b96b5d04900a. For more details please see the references link."
}
--- UVI JSON ---

This issue has been assigned UVI-2021-1001458


fix __dict__ on Python 3.9 with limited API

Found in CI trying to merge #4194
It looks like we knew about this in test cases already; when I originally implemented #[pyclass(dict)] for abi3 in #1342 I set these test cases to be ignored.
The problem is that PyObject_GenericGetDict doesn't exist in the limited API until Python 3.10. In this PR, instead of omitting the attribute we build a basic implementation of PyObject_GenericGetDict which works for our need.

Thanks for the review, I'll try to address all points tonight 👍


CORS issue

Just fyi - seems there's a CORS issue with the streamlit app:
Error message from the console (for landing page URL  https://h2-network.streamlit.app/) below:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://share.streamlit.io/-/auth/app?redirect_uri=https%3A%2F%2Fh2-network.streamlit.app%2Fapi%2Fv1%2Fuser. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing). Status code: 303.

Thanks for letting me know. I just got to this, and the app was already running again.


Vxlan vnet route

sonic-sairedis: support VxLAN vnet route

Reference: https://github.com/sonic-net/SONiC/blob/69ad163ad79c8ac1d9e9fa77d0a3d0648067f2f1/doc/vxlan/Vxlan_hld.md

Introducing SaiObjectDB to manage SAI object relationship.

Implementing VxLAN encap and decap using VPP data plane.
Signed-off-by<EMAIL_ADDRESS>

Is there an overview of VPP in SONIC (i.e. HLD) ? Could you add schema to illustrate both the example (Baremetal to VM connection) and how VPP is handling the VXLAN traffic? Is there a major difference in how VPP is handling VXLAN comparing to how it is now in SONIC ?

Thanks for the quick feedback. Here is the VPP HLD: SONICVPP-HLD.md.
I don't see major difference in vxlan handling between vpp and sonic to support vnet peering.
Also, can you clarify what schema are you expecting?

I meant some kind of schema that illustrate VM and Baremetal connectivity.

I meant some kind of schema that illustrate VM and Baremetal connectivity. Thank you for the VPP HLD, it answers a lot of questions.

That's great. I updated docs/HLD/vxlan-vnet.md with a diagram showing VM1 and BM1 connectivity via vxlan tunnel. Please check if that helps.


Add Elmo primitive

Elmo primitive uses tensorflow and is a NLP primitive, making it ideal for this repo

Fixed in https://github.com/alteryx/nlp_primitives/pull/64


alacritty: 2018-08-30 -> 0.2.0

Based on popular demand I decided to upgraded alacritty from my own fork.
We currently also have a version in staging that works without but we
cannot easily backport this one because it relies on some changes in the rust
build infrastructure
related to #46807
Motivation for this change
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[x] Fits CONTRIBUTING.md.

@GrahamcOfBorg build alacritty

Failure on aarch64-linux (full log)
Attempted: alacritty
Partial log (click to expand)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ conflicting implementation for `u8`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0119`.
error: Could not compile `winit`.
warning: build failed, waiting for other jobs to finish...
error: build failed
builder for '/nix/store/1v56bsxjpcfml2xy9kgxnbh5yjlsad66-alacritty-unstable-0.2.0.drv' failed with exit code 101
error: build of '/nix/store/1v56bsxjpcfml2xy9kgxnbh5yjlsad66-alacritty-unstable-0.2.0.drv' failed

Success on x86_64-darwin (full log)
Attempted: alacritty
Partial log (click to expand)

installing
post-installation fixup
gzipping man pages under /nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0/share/man/
strip is /nix/store/4w56qihlrddav67p7d1vy5qkyayaqw11-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0/bin
patching script interpreter paths in /nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0
strip is /nix/store/4w56qihlrddav67p7d1vy5qkyayaqw11-cctools-binutils-darwin/bin/strip
patching script interpreter paths in /nix/store/f313dhn205a3k2hj171bfkhflrsli58z-alacritty-unstable-0.2.0-terminfo
/nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0

Success on x86_64-linux (full log)
Attempted: alacritty
Partial log (click to expand)
post-installation fixup
gzipping man pages under /nix/store/lyxwdfcx73cz2aszpmbf98mnbizlca06-alacritty-unstable-0.2.0/share/man/
strip is /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/strip
stripping (with command strip and flags -S) in /nix/store/lyxwdfcx73cz2aszpmbf98mnbizlca06-alacritty-unstable-0.2.0/bin
patching script interpreter paths in /nix/store/lyxwdfcx73cz2aszpmbf98mnbizlca06-alacritty-unstable-0.2.0
checking for references to /build in /nix/store/lyxwdfcx73cz2aszpmbf98mnbizlca06-alacritty-unstable-0.2.0...
strip is /nix/store/h0lbngpv6ln56hjj59i6l77vxq25flbz-binutils-2.30/bin/strip
patching script interpreter paths in /nix/store/f7iws1iglww8qdqy8lcvbrfvkfifad7w-alacritty-unstable-0.2.0-terminfo
checking for references to /build in /nix/store/f7iws1iglww8qdqy8lcvbrfvkfifad7w-alacritty-unstable-0.2.0-terminfo...
/nix/store/lyxwdfcx73cz2aszpmbf98mnbizlca06-alacritty-unstable-0.2.0

Failure on aarch64-linux (full log)
Attempted: alacritty
Partial log (click to expand)
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ conflicting implementation for `u8`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0119`.
error: Could not compile `winit`.
warning: build failed, waiting for other jobs to finish...
error: build failed
builder for '/nix/store/1v56bsxjpcfml2xy9kgxnbh5yjlsad66-alacritty-unstable-0.2.0.drv' failed with exit code 101
error: build of '/nix/store/1v56bsxjpcfml2xy9kgxnbh5yjlsad66-alacritty-unstable-0.2.0.drv' failed

the aarch64 dependency is probably fixed upstream but not yet released: https://github.com/tomaka/winit/pull/586

Success on x86_64-darwin (full log)
Attempted: alacritty
Partial log (click to expand)

installing
post-installation fixup
gzipping man pages under /nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0/share/man/
strip is /nix/store/4w56qihlrddav67p7d1vy5qkyayaqw11-cctools-binutils-darwin/bin/strip
stripping (with command strip and flags -S) in /nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0/bin
patching script interpreter paths in /nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0
strip is /nix/store/4w56qihlrddav67p7d1vy5qkyayaqw11-cctools-binutils-darwin/bin/strip
patching script interpreter paths in /nix/store/f313dhn205a3k2hj171bfkhflrsli58z-alacritty-unstable-0.2.0-terminfo
/nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0

No attempt on aarch64-linux (full log)
The following builds were skipped because they don't evaluate on aarch64-linux: alacritty
Partial log (click to expand)

a) For `nixos-rebuild` you can set
  { nixpkgs.config.allowUnsupportedSystem = true; }
in configuration.nix to override this.

b) For `nix-env`, `nix-build`, `nix-shell` or any other Nix command you can add
  { allowUnsupportedSystem = true; }
to ~/.config/nixpkgs/config.nix.

Success on x86_64-darwin (full log)
Attempted: alacritty
Partial log (click to expand)
/nix/store/vl1sy5md36hmhwr4my6m7v7hjm1b2v3i-alacritty-unstable-0.2.0


clippy: unused_enumerate_index

Problem
some changes for bumping Rust version: https://github.com/anza-xyz/agave/pull/1309
https://rust-lang.github.io/rust-clippy/master/index.html#/enumerate
Summary of Changes
fix it!

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 81.6%. Comparing base (186befe) to head (be3706f).
Report is 1 commits behind head on master.

Additional details and impacted files
@@            Coverage Diff            @@
##           master    #1316     +/-   ##
=========================================
- Coverage    81.6%    81.6%   -0.1%
=========================================
  Files         867      867
  Lines      368767   368764      -3
=========================================
- Hits       300967   300962      -5
- Misses      67800    67802      +2


Allow documents to be created with a fixed ID and initial schema

For context, I'm playing around with automerge for a side-project, and I need to store per-user data with a defined schema. The easiest way to accomplish this seems to be a combination of the trick in the cookbook with a fixed documentId. This seems like a common pattern, so I think some way of accomplishing it should be upstreamed. Here's what I've been using locally.

This looks like a sound implementation to me. I am hesitant to review this alone as I know that there has been some thought put in to this before I joined.
Can I ask what the reasoning behind the fixed DocumentId is? Is the idea to have something which could be made unique to each user to avoid a clash? Just for my own interest.
@pvh what do you think?

No, you should never reuse document IDs. When you merge changes from
another user, the root object (and everything below it) will conflict and
you will only have one document or the other (perhaps even a frankenstein
of both!) after a merge. This is why there is no affordance to provide a
document ID.
If you really want to reuse the document, you could create a document
during initialization and then distribute it as part of the application's
bundle, but I would personally probably not do that.

@pvh This PR handles the conflict problem by hard-coding the actor ID and timestamp for the initial change—as described in the guide—and using a stable iteration order over the object that defines the initial schema (instead of initializing each field by hand).
I understand that Cambria solves this in a more robust way, but I would like to use the sync protocol in automerge-repo and as I explained above I need a globally unique document with a fixed schema. Is there a better way to accomplish this?

Hey, I'm back from my vacation and I'd like to talk more about this. I suspect the best approach is just to save a starter document and load it into the repo on first boot... but maybe that wouldn't work for reasons I don't understand? Let's discuss live outside of this issue -- can you ping me on the Slack?

I'm clearing out the backlog of PRs here and while I agree with the problem statement here, I don't think this is the way I'd like to support it in core. (Though it's certainly fine to do if it works for your app!) I'm going to close this one but if you swing back around to this stuff let me know and I'd be happy to iterate on the problem.


Fix integration test service logging

Recently I am getting the following error when running integration tests within the service logs, which prevents me from seeing most of the logs:
ERROR StatusLogger No log4j2 configuration file found. Using default configuration: logging only errors to the console.

This PR sets the log4j configuration explicitly for the integration test services.

TeamCity failure seems unrelated.


Respect iskeyword and Add Support for Dictionary Based Completion (opt.dictionary and opt.thesaurus)

Essentially the title. iskeyword is obvious in its usefulness, but the reason for dictionary/thesaurus completion is that I use this plugin for more than just coding and having constant auto completion when I set things properly and load suggestions from my dictionaries and thesaurus is very useful.

Respect iskeyword

Please see this.
https://github.com/hrsh7th/nvim-cmp/issues/808#issuecomment-1050879607

Add Support for Dictionary Based Completion

You should use it.
https://github.com/uga-rosa/cmp-dictionary


Different contrastive loss from the paper

Hi, thanks for the great work!
However, I found that the contrastive loss here seems to be different from Eqn. (3) in the paper. According to the paper, the denominator should be 1 specific positive pair plus all the negative pairs. But in the current implementation, the denominator will be all the positive pairs (excluding identical matching) plus all the negative pairs.

Hi, thanks for your interest. In short, we only compute the sum of neg_logits but not exp_logits .
More specifically, in L121, exp_logits has a shape of NxM and neg_logits has a shape of Nx1, where each element in exp_logits denotes the logit of each sample pair, while each element in neg_logits denotes the sum of logits of all negative pairs of one anchor, i.e., neg_logits = neg_logits.sum(1, keepdim=True).  In this way, their sum exp_logits + neg_logits actually adds each anchor's sum of negative logits to each specific sample pair.
Please let me know if you have further concerns.

Hi, thanks for your interest. In short, we only compute the sum of neg_logits but not exp_logits .
More specifically, in L121, exp_logits has a shape of NxM and neg_logits has a shape of Nx1, where each element in exp_logits denotes the logit of each sample pair, while each element in neg_logits denotes the sum of logits of all negative pairs of one anchor, i.e., neg_logits = neg_logits.sum(1, keepdim=True). In this way, their sum exp_logits + neg_logits actually adds each anchor's sum of negative logits to each specific sample pair.
Please let me know if you have further concerns.

Thanks for your reply. However, I am still confused with line (L121)[https://github.com/tfzhou/ContrastiveSeg/blob/9a381b9799c16d81e18d8f9f25ab509b93fb56de/lib/loss/loss_contrast.py#L121].
#why use:
log_prob = logits - torch.log(exp_logits + neg_logits)
# instead of:
log_prob = exp_logits  - torch.log(exp_logits + neg_logits)

In Eq (3): I think logits  here should be the exp(i*i_+ /t) in the numerator， which should be  exp_logits .
Thanks for you time.

Hi, thanks for your interest. In short, we only compute the sum of neg_logits but not exp_logits .
More specifically, in L121, exp_logits has a shape of NxM and neg_logits has a shape of Nx1, where each element in exp_logits denotes the logit of each sample pair, while each element in neg_logits denotes the sum of logits of all negative pairs of one anchor, i.e., neg_logits = neg_logits.sum(1, keepdim=True). In this way, their sum exp_logits + neg_logits actually adds each anchor's sum of negative logits to each specific sample pair.
Please let me know if you have further concerns.

Thanks for your reply. However, I am still confused with line (L121)[https://github.com/tfzhou/ContrastiveSeg/blob/9a381b9799c16d81e18d8f9f25ab509b93fb56de/lib/loss/loss_contrast.py#L121].
#why use:
log_prob = logits - torch.log(exp_logits + neg_logits)
# instead of:
log_prob = exp_logits  - torch.log(exp_logits + neg_logits)

In Eq (3): I think logits  here should be the exp(i*i_+ /t) in the numerator， which should be exp_logits . Thanks for you time.

log exp(a) = a


Cryptography: Add build dependencies lookup

Blackduck scan failure details
Summary: 0 violations, 0 files pending approval, 1 file pending identification.


Remove deprecated method call

This PR removes the Decimal deprecation warning.

It looks like compare has a different API between 1.6 and 2.0 so I had to add this conversion method. It can be removed if Ecto bumps Decimal dependency to 2.0.

Hi, @pawurb
There's an issue for that: https://github.com/elixir-ecto/ecto/issues/3470
Ecto is still using Decimal.cmp/2 so it doesn't force anyone to update to decimal 2.0 for now.
There were other pull requests doing the same thing as yours: #3431, #3425,

https://github.com/elixir-ecto/ecto/pull/3425#issuecomment-699818342 but updating the dependencies is what will generate the warning now. cmp is deprecated in 2.0. My PR removes the warning for both versions without forcing anyone to update or change dependencies.

Oh, I see! Sorry for the confusion. LGTM!

:green_heart: :blue_heart: :purple_heart: :yellow_heart: :heart:


API Move PreviewLink logic into File dataobject

@flamerohr can I suggest moving the preview link back into DataObject? The failover behaviour isn't going to work the way you're expecting it to, but this version should be fine. :)

This makes sense to me, I don't understand why Travis is failing, so I'll leave that to you to check out if that's alright @tractorcow :D


Trying to display heat map of dataset

I am trying to draw a heat map of a dataset on a map following the example on the NYC taxi.
My data read looks as follows
data_sources = pd.read_csv("https://s3.amazonaws.com/fordgobike-data/2017-fordgobike-tripdata.csv")

Next I seek to aggregate and plot
url = 'https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg'
tile_opts  = dict(width=1000,height=600,xaxis=None,yaxis=None,bgcolor='black',show_grid=False)
map_tiles  = gv.WMTS(url).opts(style=dict(alpha=0.5), plot=tile_opts)
points     = hv.Points(df, ['start_station_longitude', 'start_station_latitude'])
bike_trips = datashade(points, x_sampling=1, y_sampling=1, cmap=fire, width=1000, height=600)

points * bike_trips

this gives me the following error
WARNING:root:dynamic_operation: Exception raised in callable 'dynamic_operation' of type 'function'.
Invoked as dynamic_operation(height=600, scale=1.0, width=1000, x_range=None, y_range=None)
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/IPython/core/formatters.py in __call__(self, obj, include, exclude)
    968
    969             if method is not None:
--> 970                 return method(include=include, exclude=exclude)
    971             return None
    972         else:

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/dimension.py in _repr_mimebundle_(self, include, exclude)
   1229         combined and returned.
   1230         """
-> 1231         return Store.render(self)
   1232
   1233

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/options.py in render(cls, obj)
   1287         data, metadata = {}, {}
   1288         for hook in hooks:
-> 1289             ret = hook(obj)
   1290             if ret is None:
   1291                 continue

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in pprint_display(obj)
    278     if not ip.display_formatter.formatters['text/plain'].pprint:
    279         return None
--> 280     return display(obj, raw_output=True)
    281
    282

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in display(obj, raw_output, **kwargs)
    254     elif isinstance(obj, (HoloMap, DynamicMap)):
    255         with option_state(obj):
--> 256             output = map_display(obj)
    257     elif isinstance(obj, Plot):
    258         output = render(obj)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in wrapped(element)
    140         try:
    141             max_frames = OutputSettings.options['max_frames']
--> 142             mimebundle = fn(element, max_frames=max_frames)
    143             if mimebundle is None:
    144                 return {}, {}

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in map_display(vmap, max_frames)
    208         return None
    209
--> 210     return render(vmap)
    211
    212

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/ipython/display_hooks.py in render(obj, **kwargs)
     63         renderer = renderer.instance(fig='png')
     64
---> 65     return renderer.components(obj, **kwargs)
     66
     67

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/bokeh/renderer.py in components(self, obj, fmt, comm, **kwargs)
    257         # Bokeh has to handle comms directly in <0.12.15
    258         comm = False if bokeh_version < '0.12.15' else comm
--> 259         return super(BokehRenderer, self).components(obj,fmt, comm, **kwargs)
    260
    261

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/renderer.py in components(self, obj, fmt, comm, **kwargs)
    319             plot = obj
    320         else:
--> 321             plot, fmt = self._validate(obj, fmt)
    322
    323         widget_id = None

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/renderer.py in _validate(self, obj, fmt, **kwargs)
    218         if isinstance(obj, tuple(self.widgets.values())):
    219             return obj, 'html'
--> 220         plot = self.get_plot(obj, renderer=self, **kwargs)
    221
    222         fig_formats = self.mode_formats['fig'][self.mode]

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/bokeh/renderer.py in get_plot(self_or_cls, obj, doc, renderer)
    150             doc = Document() if self_or_cls.notebook_context else curdoc()
    151         doc.theme = self_or_cls.theme
--> 152         plot = super(BokehRenderer, self_or_cls).get_plot(obj, renderer)
    153         plot.document = doc
    154         return plot

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/renderer.py in get_plot(self_or_cls, obj, renderer)
    185         """
    186         # Initialize DynamicMaps with first data item
--> 187         initialize_dynamic(obj)
    188
    189         if not isinstance(obj, Plot):

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/plotting/util.py in initialize_dynamic(obj)
    242             continue
    243         if not len(dmap):
--> 244             dmap[dmap._initial_key()]
    245
    246

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in __getitem__(self, key)
   1123         # Not a cross product and nothing cached so compute element.
   1124         if cache is not None: return cache
-> 1125         val = self._execute_callback(*tuple_key)
   1126         if data_slice:
   1127             val = self._dataslice(val, data_slice)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in _execute_callback(self, *args)
    909
    910         with dynamicmap_memoization(self.callback, self.streams):
--> 911             retval = self.callback(*args, **kwargs)
    912         return self._style(retval)
    913

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in __call__(self, *args, **kwargs)
    540         kwarg_hash = kwargs.pop('memoization_hash', ())
    541         (self.args, self.kwargs) = (args, kwargs)
--> 542         if not args and not kwargs: return self.callable()
    543         inputs = [i for i in self.inputs if isinstance(i, DynamicMap)]
    544         streams = []

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/overlay.py in dynamic_mul(*args, **kwargs)
     27             from .spaces import Callable
     28             def dynamic_mul(*args, **kwargs):
---> 29                 element = other[args]
     30                 return self * element
     31             callback = Callable(dynamic_mul, inputs=[self, other])

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in __getitem__(self, key)
   1123         # Not a cross product and nothing cached so compute element.
   1124         if cache is not None: return cache
-> 1125         val = self._execute_callback(*tuple_key)
   1126         if data_slice:
   1127             val = self._dataslice(val, data_slice)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in _execute_callback(self, *args)
    909
    910         with dynamicmap_memoization(self.callback, self.streams):
--> 911             retval = self.callback(*args, **kwargs)
    912         return self._style(retval)
    913

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/spaces.py in __call__(self, *args, **kwargs)
    570
    571         try:
--> 572             ret = self.callable(*args, **kwargs)
    573         except KeyError:
    574             # KeyError is caught separately because it is used to signal

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/util/__init__.py in dynamic_operation(*key, **kwargs)
    431                 self.p.kwargs.update(kwargs)
    432                 obj = map_obj[key] if isinstance(map_obj, HoloMap) else map_obj
--> 433                 return self._process(obj, key)
    434         else:
    435             def dynamic_operation(*key, **kwargs):

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/util/__init__.py in _process(self, element, key)
    417             kwargs = {k: v for k, v in self.p.kwargs.items()
    418                       if k in self.p.operation.params()}
--> 419             return self.p.operation.process_element(element, key, **kwargs)
    420         else:
    421             return self.p.operation(element, **self.p.kwargs)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/operation.py in process_element(self, element, key, **params)
    141         """
    142         self.p = param.ParamOverrides(self, params)
--> 143         return self._apply(element, key)
    144
    145

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/operation.py in _apply(self, element, key)
    119         for hook in self._preprocess_hooks:
    120             kwargs.update(hook(self, element))
--> 121         ret = self._process(element, key)
    122         for hook in self._postprocess_hooks:
    123             ret = hook(self, ret, **kwargs)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/operation/datashader.py in _process(self, element, key)
    911
    912     def _process(self, element, key=None):
--> 913         agg = rasterize._process(self, element, key)
    914         shaded = shade._process(self, agg, key)
    915         return shaded

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/operation/datashader.py in _process(self, element, key)
    742             op = transform.instance(**op_params)
    743             op._precomputed = self._precomputed
--> 744             element = element.map(op, predicate)
    745             self._precomputed = op._precomputed
    746         return element

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/dimension.py in map(self, map_fn, specs, clone)
    692             return deep_mapped
    693         else:
--> 694             return map_fn(self) if applies else self
    695
    696

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/operation.py in __call__(self, element, **params)
    161                                 operation=self, kwargs=params)
    162         elif isinstance(element, ViewableElement):
--> 163             processed = self._apply(element)
    164         elif isinstance(element, DynamicMap):
    165             if any((not d.values) for d in element.kdims):

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/operation.py in _apply(self, element, key)
    119         for hook in self._preprocess_hooks:
    120             kwargs.update(hook(self, element))
--> 121         ret = self._process(element, key)
    122         for hook in self._postprocess_hooks:
    123             ret = hook(self, ret, **kwargs)

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/operation/datashader.py in _process(self, element, key)
    460             # Replacing x and y coordinates to avoid numerical precision issues
    461             eldata = agg if ds_version > '0.5.0' else (xs, ys, agg.data)
--> 462             return self.p.element_type(eldata, **params)
    463         else:
    464             layers = {}

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/element/raster.py in __init__(self, data, kdims, vdims, bounds, extents, xdensity, ydensity, rtol, **params)
    280         xdensity = xdensity if xdensity else compute_density(l, r, dim1, self._time_unit)
    281         ydensity = ydensity if ydensity else compute_density(b, t, dim2, self._time_unit)
--> 282         SheetCoordinateSystem.__init__(self, bounds, xdensity, ydensity)
    283         self._validate(data_bounds, supplied_bounds)
    284

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/sheetcoords.py in __init__(self, bounds, xdensity, ydensity)
    164         self.lbrt = np.array(bounds.lbrt())
    165
--> 166         r1,r2,c1,c2 = Slice._boundsspec2slicespec(self.lbrt,self)
    167         self.__shape = (r2-r1,c2-c1)
    168

/usr/local/anaconda3/envs/bike-demand-forecasting/lib/python3.6/site-packages/holoviews/core/sheetcoords.py in _boundsspec2slicespec(boundsspec, scs)
    516         b_m,r_m = scs.sheet2matrix(r,b)
    517
--> 518         l_idx = int(np.ceil(l_m-0.5))
    519         t_idx = int(np.ceil(t_m-0.5))
    520         # CBENHANCEMENT: Python 2.6's math.trunc()?

ValueError: cannot convert float NaN to integer

For what's worth, trying to plot the dots on a map using
(gv.WMTS('http://c.tile.openstreetmap.org/{Z}/{X}/{Y}.png') * points)

gives the following plot

My environment.yml looks as follows
name: example
channels:
 - bokeh
 - conda-forge
 - ioam

dependencies:
 - attrs
 - beautifulsoup4
 - bokeh
 - cartopy
 - colorcet
 - conda-forge::graphviz
 - conda-forge::pytest
 - conda-forge::pytest-benchmark
 - conda-forge::python-graphviz
 - dask>=0.15.4
 - datashader
 - dill
 - distributed
 - fastparquet
 - flake8
 - geoviews
 - ioam::holoviews>=1.8.3
 - ipython
 - iris
 - jupyter
 - jupyter_dashboards
 - krb5
 - matplotlib
 - nbconvert
 - nbformat
 - networkx>=2.0
 - numba
 - numpy
 - pandas
 - param>=1.5.1
 - paramnb
 - pyproj
 - pytables
 - python-snappy
 - python=3.6
 - rasterio
 - requests
 - scikit-image
 - scipy
 - shapely
 - snappy
 - statsmodels
 - tblib
 - xarray
 - yaml
 - xgboost=0.71
 - autopep8
 - jupyter_contrib_nbextensions
 - pip:
   - cachey
   - streamz==0.2.0
   - webargs

Any suggestions how I could generate the heat map?

It works if you remove x_sampling=1 and y_sampling=1.  A value of 1 was appropriate for the Web Mercator Taxi data (where the 1 is in meters), but for this lon/lat data not previously projected into Web Mercator, "1" is one degree on the globe.  All of the data appears to fall into one degree on the globe, so enforcing that as a minimum sampling size effectively give a single-pixel plot, which gives no valid data and ends up with that obscure and impressively long traceback above. I think we already have an issue or PR to make that case not be an error, but for the meantime just fix those settings.
You also presumably didn't mean to overlay bike_trips on points, as they are both the same set of points; I assume you meant map_tiles.  The result:
import holoviews as hv, geoviews as gv, pandas as pd
from holoviews.operation.datashader import datashade
from colorcet import fire
hv.extension("bokeh")

df = pd.read_csv("https://s3.amazonaws.com/fordgobike-data/2017-fordgobike-tripdata.csv")
url = 'https://server.arcgisonline.com/ArcGIS/rest/services/World_Imagery/MapServer/tile/{Z}/{Y}/{X}.jpg'
tile_opts  = dict(width=1000,height=600,xaxis=None,yaxis=None,bgcolor='black',show_grid=False, alpha=0.3)
map_tiles  = gv.WMTS(url).options(**tile_opts)
points     = gv.Points(df, ['start_station_longitude', 'start_station_latitude'])
bike_trips = datashade(points, cmap=fire, width=1000, height=600)

map_tiles * bike_trips

Maybe not the most exciting Datashader plot ever, but it seems to work...

Thanks for helping resolve this. Could I help the project by adding more meaningful error messages to help people debug?

Yes, absolutely, let's leave this issue open to address that.

Right, though I think @philippjfr and I agreed earlier this shouldn't be an error at the HoloViews level, and probably not at the Datashader level either.  It should just render the same value across the entire image.

Right, though I think @philippjfr and I agreed earlier this shouldn't be an error at the HoloViews level, and probably not at the Datashader level either. It should just render the same value across the entire image.

Correct, I think this can probably be handled but separately a single pixel Image should provide a good error message when when xdensity and ydensity aren't supplied.


CLI: PicklingError: logger cannot be pickled

🐛 Bug
self = <Logger jsonargparse_null_logger (NOTSET)>

    def __reduce__(self):
        # In general, only the root logger will not be accessible via its name.
        # However, the root logger's class has its own __reduce__ method.
        if getLogger(self.name) is not self:
            import pickle
>           raise pickle.PicklingError('logger cannot be pickled')
E           _pickle.PicklingError: logger cannot be pickled

../../../../.pyenv/versions/3.9.1/lib/python3.9/logging/__init__.py:1727: PicklingError

To Reproduce
def test_bug(tmpdir):
    with mock.patch('sys.argv', ['any.py']):
        cli = LightningCLI(
            BoringModel,
            trainer_defaults={'default_root_dir': str(tmpdir), 'accelerator': 'ddp_cpu', 'max_steps': 1}
        )

Expected behavior
Runs
Environment
jsonargparse==3.14.0

cc: @mauvilsa

The issue goes away with a minor change:
def test_bug(tmpdir):
    with mock.patch('sys.argv', ['any.py']):
        cli = LightningCLI(
            BoringModel,
            save_config_callback=None,
            trainer_defaults={'default_root_dir': str(tmpdir), 'accelerator': 'ddp_cpu', 'max_steps': 1}
        )

The problem is that SaveConfigCallback includes the parser which is needed to save the config. Making an ArgumentParser picklable in general I think is a daunting task. However, there is no point in having SaveConfigCallback in all processes of a distributed job. I am not sure how the distributed works so right now I am unable to say how it could be fixed. But I guess this could be solved by having something like a rank zero callback meaning that this callback would be removed before any forking.


Cherry-pick changes to ghc-head

Hi!  As of today, patches for the ghc-head branch must be done on the Haddock mirror on GitLab, at https://gitlab.haskell.org/ghc/haddock.


Exception with optimization_stage_3

Using CUDA 11.1, pytorch 1.8.1 and Deepspeed 0.3.14.
Model trains with FP16 and optimization_stage 2, but fails with optimization_stage 3 with the following exception:
b'    model_engine.backward(loss)\n'
b'  File "/opt/miniconda/lib/python3.7/site-packages/deepspeed/runtime/engine.py", line 997, in backward\n'
b'    self.optimizer.backward(loss)\n'
b'  File "/opt/miniconda/lib/python3.7/site-packages/deepspeed/runtime/zero/stage3.py", line 2555, in backward\n'
b'    self.loss_scaler.backward(loss.float(), retain_graph=retain_graph)\n'
b'  File "/opt/miniconda/lib/python3.7/site-packages/deepspeed/runtime/fp16/loss_scaler.py", line 53, in backward\n'
b'    scaled_loss.backward(retain_graph=retain_graph)\n'
b'  File "/opt/miniconda/lib/python3.7/site-packages/torch/tensor.py", line 245, in backward\n'
b'    torch.autograd.backward(self, gradient, retain_graph, create_graph, inputs=inputs)\n'
b'  File "/opt/miniconda/lib/python3.7/site-packages/torch/autograd/init.py", line 147, in backward\n'
b'    allow_unreachable=True, accumulate_grad=True)  # allow_unreachable flag\n'
b'  File "/opt/miniconda/lib/python3.7/site-packages/torch/autograd/function.py", line 89, in apply\n'
b'    return self._forward_cls.backward(self, *args)  # type: ignore\n'
b'  File "/opt/miniconda/lib/python3.7/site-packages/deepspeed/runtime/zero/linear.py", line 85, in backward\n'
b'    grad_weight = grad_output.t().matmul(input)\n'
b'RuntimeError: expected scalar type Half but found Float\n'

@fcampagnexandr, thanks for using DeepSpeed and reporting this issue. Can you please provide more details on how to repro?

@fcampagnexandr, are you still seeing this issue?

Yes, I we are seeing the issue. This is an issue we encounter with a pytorch implementation of the perceiver model architecture. Will try to come up with a simple code for reproduction, but no promise, it's quite busy at my end.

That is understandable. Please share at your convenience. Thanks so much.

Here's the code to reproduce: https://github.com/fac2003/repro_deepspeed_multi_modality_perceiver_stage_3/blob/main/DeepSpeedMultiModalityPerceiver.ipynb
Same code works if you switch optimization_stage to 2 from 3.

I checked the repro code against Deepspeed <IP_ADDRESS> (latest at this time) and found the issue no longer occurs in this release. Seems fixed.

@fac2003, thanks for sharing your experience wth 0.3.15. What is strange is that I am able  to repro the issue on 0.3.15, so I am curious why we have different observations :). I traced the problem to lack of support for autocast in ZeRO 3, and I just created a PR. Can you please share your ds_report again?

This is strange indeed. Here's the report with 0.3.15 on colab:
DeepSpeed C++/CUDA extension op report
NOTE: Ops not installed will be just-in-time (JIT) compiled at
runtime if needed. Op compatibility means that your system
meet the required dependencies to JIT install the op.
JIT compiled ops requires ninja
ninja .................. [OKAY]
op name ................ installed .. compatible
cpu_adam ............... [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
[WARNING]  sparse_attn requires one of the following commands '['llvm-config', 'llvm-config-9']', but it does not exist!
[WARNING]  sparse_attn requires CUDA version 10.1+, does not currently support >=11 or <10.1
sparse_attn ............ [NO] ....... [NO]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
utils .................. [NO] ....... [OKAY]
[WARNING]  async_io requires the libraries: ['libaio-dev'] but are missing.
async_io ............... [NO] ....... [NO]
DeepSpeed general environment info:
torch install path ............... ['/usr/local/lib/python3.7/dist-packages/torch']
torch version .................... 1.7.1+cu110
torch cuda version ............... 11.0
nvcc version ..................... 11.0
deepspeed install path ........... ['/usr/local/lib/python3.7/dist-packages/deepspeed']
deepspeed info ................... 0.3.15, unknown, unknown
deepspeed wheel compiled w. ...... torch 1.8, cuda 10.1

@fac2003, I think I may have figured out the mystery. But I need your help to be sure. Can you please wrap your model creation in deepspeed.zero.Init() context? Such as below?
with deepspeed.zero.Init(
):
    model = MultiModalityWithTextPerceiver(
        modalities=(video_modality, image_modality),
        depth=2,  # depth of net, combined with num_latent_blocks_per_layer to produce full Perceiver
        num_latents=12,
        # number of latents, or induced set points, or centroids. different papers giving it different names

Does this repro the problem?

Excellent! Thanks for the confirmation. What happened is that ZeRO 3 has an optimized linear layer that does not support amp autocasting. This linear layer was previous enabled by default when using ZeRO 3, which is why you originally ran into the issue. Now, a recent change made deepspeed.zero.Init() context a requirement to get this ZeRO 3 linear layer, which is why you no longer saw it. Regardless, we need this PR to add support for amp autocast. Thanks so much.
On a separate note, to get the full benefit of ZeRO 3 and ZeRO-Infinity, do need to wrap your model with deepspeed.zero.Init().

@fac2003, the fix is now merged can you please verify so this issue can be closed?

Cloning master and building from source fixed the initial error, but I am seeing a new one with stage 3 on backward:
Pasting here the ds_report output and training run output from repro steps running in colab.
DeepSpeed C++/CUDA extension op report
NOTE: Ops not installed will be just-in-time (JIT) compiled at
runtime if needed. Op compatibility means that your system
meet the required dependencies to JIT install the op.
JIT compiled ops requires ninja
ninja .................. [OKAY]
op name ................ installed .. compatible
cpu_adam ............... [NO] ....... [OKAY]
fused_adam ............. [NO] ....... [OKAY]
fused_lamb ............. [NO] ....... [OKAY]
[WARNING]  sparse_attn requires one of the following commands '['llvm-config', 'llvm-config-9']', but it does not exist!
[WARNING]  sparse_attn requires CUDA version 10.1+, does not currently support >=11 or <10.1
sparse_attn ............ [NO] ....... [NO]
transformer ............ [NO] ....... [OKAY]
stochastic_transformer . [NO] ....... [OKAY]
utils .................. [NO] ....... [OKAY]
[WARNING]  async_io requires the libraries: ['libaio-dev'] but are missing.
async_io ............... [NO] ....... [NO]
DeepSpeed general environment info:
torch install path ............... ['/usr/local/lib/python3.7/dist-packages/torch']
torch version .................... 1.7.1+cu110
torch cuda version ............... 11.0
nvcc version ..................... 11.0
deepspeed install path ........... ['/usr/local/lib/python3.7/dist-packages/deepspeed']
deepspeed info ................... 0.3.15+03d24fe, 03d24fe, master
deepspeed wheel compiled w. ...... torch 1.7, cuda 11.0
[2021-04-24 16:16:23,518] [INFO] [distributed.py:37:init_distributed] Not using the DeepSpeed or torch.distributed launchers, attempting to detect MPI environment...
[2021-04-24 16:16:23,965] [INFO] [distributed.py:89:mpi_discovery] Discovered MPI settings of world_rank=0, local_rank=0, world_size=1, master_addr=<IP_ADDRESS>, master_port=29500
[2021-04-24 16:16:23,966] [INFO] [distributed.py:47:init_distributed] Initializing torch distributed with backend: nccl
nn.functional.linear has been overridden with a more memory efficient version. This will persist unless manually reset.
[2021-04-24 16:16:27,160] [INFO] [logging.py:60:log_dist] [Rank 0] DeepSpeed info: version=0.3.15+03d24fe, git-hash=03d24fe, git-branch=master
[2021-04-24 16:16:27,162] [WARNING] [config.py:78:_sanity_check] DeepSpeedConfig: cpu_offload is deprecated. Please use offload_optimizer.
[2021-04-24 16:16:27,163] [WARNING] [config.py:78:_sanity_check] DeepSpeedConfig: cpu_offload_params is deprecated. Please use offload_param.
[2021-04-24 16:16:27,174] [INFO] [engine.py:80:_initialize_parameter_parallel_groups] data_parallel_size: 1, parameter_parallel_size: 1
Using /root/.cache/torch_extensions as PyTorch extensions root...
Creating extension directory /root/.cache/torch_extensions/fused_adam...
Detected CUDA files, patching ldflags
Emitting ninja build file /root/.cache/torch_extensions/fused_adam/build.ninja...
Building extension module fused_adam...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
Loading extension module fused_adam...
Time to load fused_adam op: 21.77649974822998 seconds
[2021-04-24 16:16:49,666] [INFO] [engine.py:616:_configure_optimizer] Using DeepSpeed Optimizer param name adam as basic optimizer
[2021-04-24 16:16:49,667] [INFO] [engine.py:620:_configure_optimizer] DeepSpeed Basic Optimizer = FusedAdam
Checking ZeRO support for optimizer=FusedAdam type=<class 'deepspeed.ops.adam.fused_adam.FusedAdam'>
[2021-04-24 16:16:49,673] [INFO] [logging.py:60:log_dist] [Rank 0] Creating fp16 ZeRO stage 3 optimizer
Initializing ZeRO Stage 3
[2021-04-24 16:16:49,746] [INFO] [utils.py:583:see_memory_usage] Stage 3 initialize beginning
[2021-04-24 16:16:49,749] [INFO] [utils.py:588:see_memory_usage] MA 0.0 GB         Max_MA 0.0 GB         CA 0.0 GB         Max_CA 0 GB
[2021-04-24 16:16:49,753] [INFO] [utils.py:593:see_memory_usage] CPU Virtual Memory:  used = 2.66 GB, percent = 20.9%
[2021-04-24 16:16:49,756] [INFO] [stage3.py:624:init] Reduce bucket size 300000
[2021-04-24 16:16:49,758] [INFO] [stage3.py:625:init] Allgather bucket size 20000
Using /root/.cache/torch_extensions as PyTorch extensions root...
Creating extension directory /root/.cache/torch_extensions/utils...
/usr/local/lib/python3.7/dist-packages/torch/cuda/memory.py:346: FutureWarning: torch.cuda.memory_cached has been renamed to torch.cuda.memory_reserved
FutureWarning)
/usr/local/lib/python3.7/dist-packages/torch/cuda/memory.py:354: FutureWarning: torch.cuda.max_memory_cached has been renamed to torch.cuda.max_memory_reserved
FutureWarning)
Emitting ninja build file /root/.cache/torch_extensions/utils/build.ninja...
Building extension module utils...
Allowing ninja to set a default number of workers... (overridable by setting the environment variable MAX_JOBS=N)
Loading extension module utils...
Time to load utils op: 12.045355319976807 seconds
[2021-04-24 16:17:01,878] [INFO] [utils.py:583:see_memory_usage] Before creating fp16 partitions
[2021-04-24 16:17:01,880] [INFO] [utils.py:588:see_memory_usage] MA 0.0 GB         Max_MA 0.0 GB         CA 0.0 GB         Max_CA 0 GB
[2021-04-24 16:17:01,883] [INFO] [utils.py:593:see_memory_usage] CPU Virtual Memory:  used = 2.66 GB, percent = 20.9%
[2021-04-24 16:17:01,885] [INFO] [stage3.py:39:print_rank_0] fp16 group 0 has 1 subgroups
[2021-04-24 16:17:01,901] [INFO] [stage3.py:39:print_rank_0] Swappable FP32 Partitions: count=0 size= 0.00 GB
[2021-04-24 16:17:01,902] [INFO] [stage3.py:39:print_rank_0] In-Memory FP32 Partitions: count=1 size= 0.00 GB
[2021-04-24 16:17:01,905] [INFO] [stage3.py:819:init] optimizer state initialized
[2021-04-24 16:17:01,906] [INFO] [stage3.py:39:print_rank_0] Largest partitioned param numel = 377914
[2021-04-24 16:17:01,992] [INFO] [utils.py:583:see_memory_usage] After initializing ZeRO optimizer
[2021-04-24 16:17:01,994] [INFO] [utils.py:588:see_memory_usage] MA 0.01 GB         Max_MA 0.01 GB         CA 0.02 GB         Max_CA 0 GB
[2021-04-24 16:17:01,996] [INFO] [utils.py:593:see_memory_usage] CPU Virtual Memory:  used = 2.66 GB, percent = 20.9%
[2021-04-24 16:17:01,999] [INFO] [logging.py:60:log_dist] [Rank 0] DeepSpeed Final Optimizer = adam
[2021-04-24 16:17:02,000] [INFO] [engine.py:451:_configure_lr_scheduler] DeepSpeed using configured LR scheduler = WarmupLR
[2021-04-24 16:17:02,002] [INFO] [logging.py:60:log_dist] [Rank 0] DeepSpeed LR Scheduler = <deepspeed.runtime.lr_schedules.WarmupLR object at 0x7fbba0557410>
[2021-04-24 16:17:02,004] [INFO] [logging.py:60:log_dist] [Rank 0] step=0, skipped=0, lr=[0.001], mom=[[0.8, 0.999]]
[2021-04-24 16:17:02,005] [INFO] [config.py:743:print] DeepSpeedEngine configuration:
[2021-04-24 16:17:02,007] [INFO] [config.py:747:print]   activation_checkpointing_config  {
"partition_activations": false,
"contiguous_memory_optimization": false,
"cpu_checkpointing": false,
"number_checkpoints": null,
"synchronize_checkpoint_boundary": false,
"profile": false
}
[2021-04-24 16:17:02,009] [INFO] [config.py:747:print]   aio_config ................... {'block_size': 1048576, 'queue_depth': 8, 'thread_count': 1, 'single_submit': False, 'overlap_events': True}
[2021-04-24 16:17:02,011] [INFO] [config.py:747:print]   allreduce_always_fp32 ........ False
[2021-04-24 16:17:02,012] [INFO] [config.py:747:print]   amp_enabled .................. False
[2021-04-24 16:17:02,013] [INFO] [config.py:747:print]   amp_params ................... False
[2021-04-24 16:17:02,015] [INFO] [config.py:747:print]   checkpoint_tag_validation_enabled  True
[2021-04-24 16:17:02,016] [INFO] [config.py:747:print]   checkpoint_tag_validation_fail  False
[2021-04-24 16:17:02,018] [INFO] [config.py:747:print]   disable_allgather ............ False
[2021-04-24 16:17:02,019] [INFO] [config.py:747:print]   dump_state ................... False
[2021-04-24 16:17:02,021] [INFO] [config.py:747:print]   dynamic_loss_scale_args ...... {'init_scale':<PHONE_NUMBER>, 'scale_window': 1000, 'delayed_shift': 2, 'min_scale': 1}
[2021-04-24 16:17:02,023] [INFO] [config.py:747:print]   elasticity_enabled ........... False
[2021-04-24 16:17:02,024] [INFO] [config.py:747:print]   flops_profiler_config ........ {
"enabled": false,
"profile_step": 1,
"module_depth": -1,
"top_modules": 3,
"detailed": true
}
[2021-04-24 16:17:02,025] [INFO] [config.py:747:print]   fp16_enabled ................. True
[2021-04-24 16:17:02,027] [INFO] [config.py:747:print]   global_rank .................. 0
[2021-04-24 16:17:02,028] [INFO] [config.py:747:print]   gradient_accumulation_steps .. 1
[2021-04-24 16:17:02,030] [INFO] [config.py:747:print]   gradient_clipping ............ 0.0
[2021-04-24 16:17:02,031] [INFO] [config.py:747:print]   gradient_predivide_factor .... 1.0
[2021-04-24 16:17:02,033] [INFO] [config.py:747:print]   initial_dynamic_scale ........<PHONE_NUMBER>
[2021-04-24 16:17:02,035] [INFO] [config.py:747:print]   loss_scale ................... 1
[2021-04-24 16:17:02,037] [INFO] [config.py:747:print]   memory_breakdown ............. False
[2021-04-24 16:17:02,038] [INFO] [config.py:747:print]   optimizer_legacy_fusion ...... False
[2021-04-24 16:17:02,039] [INFO] [config.py:747:print]   optimizer_name ............... adam
[2021-04-24 16:17:02,041] [INFO] [config.py:747:print]   optimizer_params ............. {'lr': 0.001, 'betas': [0.8, 0.999], 'eps': 1e-08, 'weight_decay': 3e-07}
[2021-04-24 16:17:02,044] [INFO] [config.py:747:print]   pipeline ..................... {'stages': 'auto', 'partition': 'best', 'seed_layers': False, 'activation_checkpoint_interval': 0}
[2021-04-24 16:17:02,046] [INFO] [config.py:747:print]   pld_enabled .................. False
[2021-04-24 16:17:02,050] [INFO] [config.py:747:print]   pld_params ................... False
[2021-04-24 16:17:02,051] [INFO] [config.py:747:print]   prescale_gradients ........... False
[2021-04-24 16:17:02,053] [INFO] [config.py:747:print]   scheduler_name ............... WarmupLR
[2021-04-24 16:17:02,055] [INFO] [config.py:747:print]   scheduler_params ............. {'warmup_min_lr': 0, 'warmup_max_lr': 0.001, 'warmup_num_steps': 1000}
[2021-04-24 16:17:02,057] [INFO] [config.py:747:print]   sparse_attention ............. None
[2021-04-24 16:17:02,059] [INFO] [config.py:747:print]   sparse_gradients_enabled ..... False
[2021-04-24 16:17:02,062] [INFO] [config.py:747:print]   steps_per_print .............. 2000
[2021-04-24 16:17:02,063] [INFO] [config.py:747:print]   tensorboard_enabled .......... False
[2021-04-24 16:17:02,065] [INFO] [config.py:747:print]   tensorboard_job_name ......... DeepSpeedJobName
[2021-04-24 16:17:02,066] [INFO] [config.py:747:print]   tensorboard_output_path ......
[2021-04-24 16:17:02,070] [INFO] [config.py:747:print]   train_batch_size ............. 3
[2021-04-24 16:17:02,072] [INFO] [config.py:747:print]   train_micro_batch_size_per_gpu  3
[2021-04-24 16:17:02,077] [INFO] [config.py:747:print]   wall_clock_breakdown ......... False
[2021-04-24 16:17:02,086] [INFO] [config.py:747:print]   world_size ................... 1
/usr/local/lib/python3.7/dist-packages/torch/cuda/memory.py:346: FutureWarning: torch.cuda.memory_cached has been renamed to torch.cuda.memory_reserved
FutureWarning)
/usr/local/lib/python3.7/dist-packages/torch/cuda/memory.py:354: FutureWarning: torch.cuda.max_memory_cached has been renamed to torch.cuda.max_memory_reserved
FutureWarning)
[2021-04-24 16:17:02,097] [INFO] [config.py:747:print]   zero_allow_untested_optimizer  False
[2021-04-24 16:17:02,099] [INFO] [config.py:747:print]   zero_config .................. {
"stage": 3,
"contiguous_gradients": false,
"reduce_scatter": false,
"reduce_bucket_size": 3.000000e+05,
"allgather_partitions": true,
"allgather_bucket_size": 5.000000e+08,
"overlap_comm": false,
"load_from_fp32_weights": true,
"elastic_checkpoint": true,
"offload_param": null,
"offload_optimizer": null,
"sub_group_size": 1.000000e+06,
"prefetch_bucket_size": 2.000000e+04,
"param_persistence_threshold": 1.000000e+04,
"max_live_parameters": 6.000000e+05,
"max_reuse_distance": 1.000000e+07,
"gather_fp16_weights_on_model_save": false
}
[2021-04-24 16:17:02,100] [INFO] [config.py:747:print]   zero_enabled ................. True
[2021-04-24 16:17:02,101] [INFO] [config.py:747:print]   zero_optimization_stage ...... 3
[2021-04-24 16:17:02,103] [INFO] [config.py:754:print]   json = {
"train_batch_size": 3,
"steps_per_print": 2.000000e+03,
"optimizer": {
"type": "Adam",
"params": {
"lr": 0.001,
"betas": [0.8, 0.999],
"eps": 1e-08,
"weight_decay": 3e-07
}
},
"fp16": {
"enabled": true,
"loss_scale": 1,
"initial_scale_power": 32,
"loss_scale_window": 1000,
"hysteresis": 2,
"min_loss_scale": 1
},
"scheduler": {
"type": "WarmupLR",
"params": {
"warmup_min_lr": 0,
"warmup_max_lr": 0.001,
"warmup_num_steps": 1000
}
},
"wall_clock_breakdown": false,
"zero_optimization": {
"stage": 3,
"cpu_offload": false,
"cpu_offload_params": false,
"overlap_comm": false,
"contiguous_gradients": false,
"stage3_max_live_parameters": 6.000000e+05,
"stage3_max_reuse_distance": 1.000000e+07,
"stage3_prefetch_bucket_size": 2.000000e+04,
"stage3_param_persistence_threshold": 1.000000e+04,
"reduce_bucket_size": 3.000000e+05,
"sub_group_size": 1.000000e+06
}
}
Using /root/.cache/torch_extensions as PyTorch extensions root...
No modifications detected for re-loaded extension module utils, skipping build step...
Loading extension module utils...
Time to load utils op: 0.0007159709930419922 seconds
RuntimeError                              Traceback (most recent call last)
 in ()
124       loss = outputs.mean()
125
--> 126       model_engine.backward(loss)
127       model_engine.step()
128 print("DONE")
7 frames
/usr/local/lib/python3.7/dist-packages/deepspeed/runtime/zero/linear.py in backward(ctx, grad_output)
89         if ctx.needs_input_grad[0]:
90             #print(f"Computing grad input weight {weight.shape} grad_output {grad_output.shape}")
---> 91             grad_input = grad_output.matmul(weight)
92             #print(f"Computed grad input {grad_input.shape}")
93         if ctx.needs_input_grad[1]:
RuntimeError: mat1 dim 1 must match mat2 dim 0

Here's the updated repro notebook installing from DeepSpeed master and wrapping model in zero init:
https://github.com/fac2003/repro_deepspeed_multi_modality_perceiver_stage_3/blob/main/DeepSpeedMultiModalityPerceiver-from-master.ipynb?short_path=52c54dd

@fac2003, thanks for reporting this new issue. Can you please open a new issue report for this?

Why not continue with this one until the repro works and the issue is fully fixed? The title still matches the problem, there is an exception when  turning on stage 3 for this model. I'd rather not have to provide the repro in yet another issue. Would that be OK with you?

There are number of benefits to restricting an issue report to one issue or bug, rather than adding multiple, even related, bugs into a report.

This guide on best practices for GitHub issues gives a number of reasons, including a principle of One issue per issue.

Besides enabling collaboration of the reporter and fixer of an issue, the issue report also serves as an archive or historical record for future users who want to quickly know whether their issue is similar to old ones or completely. It is a lot easier for those users to filter issue reports if issue reports are shorter, simpler, and describes one error message or stack trace, as much as possible.

It is also much cleaner if one PR is associated with one issue report because it is makes much easier to find regressions and port fixes to other branches.

Regarding the title matching the problem, do note that the title is overly broad since many kinds of Python bugs manifest as exceptions in the interpreter. And so this is not a good reason to associate every bug triggers a Python interpreter exception under the same issue report, since dozens of bugs would qualify.

@tjruwase It is the same issue from my point of view because the repro code has not changed, neither has the description, we are now hitting a new exception further in the repro code, so this is progress, but the issue is not resoled. The fix PR you developed perhaps has only addressed the forward.
I can open another issue if it's easier to track with PRs at your end, but I would not close this one until the repro does work. Thanks for your help so far.

As requested, I created a new issue here: https://github.com/microsoft/DeepSpeed/issues/1006

@fac2003, I greatly appreciate the help. And yes, I don't expect you to close this issue until all issues are resolved to your satisfaction.


Allow point labels location based on their axis value

Here is syntax and data to generate the chart.
point x(LEVEL) y(POSITION) key(ID) label(LABEL) style('label-location:top;fill-opacity:1; ')
ID,LEVEL,POSITION,LABEL,FieldName,ImportanceValue,NodeType
0,A,-7,BP=HIGH,BP,0.2,1
1,A,-6,BP=LOW,BP,0.2,1
2,A,-5,BP=NORMAL,BP,0.2,1
3,A,-4,Cholesterol=HIGH,Cholesterol,0.4,1
4,A,-3,Cholesterol=NORMAL,Cholesterol,0.4,1
5,A,-2,Drug=drugA,Drug,0.1,1
6,A,-1,Drug=drugB,Drug,0.1,1
7,A,0,Drug=drugC,Drug,0.1,1
8,A,1,Drug=drugX,Drug,0.1,1
9,A,2,Drug=drugY,Drug,0.1,1
10,A,3,K,K,0.15,1
11,A,4,Na,Na,0.05,1
12,A,5,Sex=F,Sex,0.13,1
13,A,6,Sex=M,Sex,0.15,1
-1,A,7,Bias,Bias,0,0
14,B,-7,H(1:1),Hidden,1.0,1
15,B,-3.5,H(1:2),Hidden,1.0,1
16,B,0,H(1:3),Hidden,1.0,1
-2,B,3.5,Bias,Hidden,1.0,0
17,C,0,Age,Age,1.0,1
Right now, all the point labels are in the same location, left, top, ....
We are required to have something like this:

At the moment, the CSS style feature is likely the one we would use to make this work; you would have two different label styles and uses a css aesthetic to choose one for each. Currently the css aesthetic does not apply to the labels for points, and I think there is an oder of execution issue, but it's definitely doable. I'll add it as a feature.

New plan! Use two label statements for two fields and given the text a location corresponding to where you want them to be. By making the labels missing where you don't need then, it shows the effect desired
An example is x(x) y(y) key(#row) label(L1:top, L2:right) axes(none) + data('raw:a,b;1,7;1,8;2,8;7,9') edge key(a, b) for the data set:
X,Y,L1,L2
1,-2,Red,
1,-1,Yellow,
1,0,Green,
1,1,Blue,
1,2,Purple,
2,-1,,Alpha
2,0,,Beta
2,1,,Delta
3,-0.5,TRUE,
3,0.5,FALSE,


bug in the program

version 2.5.18
Uncaught TypeError: Cannot read property '0' of undefined
jquery.datetimepicker.full.js:1087

I also found the same issue. This happens to me when I try to scroll months or years lists using the scroll handle, or when I try to select an item from the mentioned lists. I think this happens due to the fact that the touches property of the event object is accessed despite the event is an instance of MouseEvent instead of TouchEvent. I solved it by simply checking the event type:
1085 .on('touchstart mousedown.xdsoft', function (event) {
1086    if(event instanceof TouchEvent){
1086        this.touchMoved = false;
1087        this.touchStartPosition = event.originalEvent.touches[0]
1088    }
1088    event.stopPropagation();
1089    event.preventDefault();
1090 })

Tested on Google Chrome  64.0.3282.186 and Mozilla Firefox 58.0.2

Duplicate of #636


Unable to install homeassistant: No space left on device

Добрый день!
Установка производится на полностью чистый, только что прошитый шлюз. Ни какаих дополнительных пакетов после прошивки не ставилось.
Лог установки во вложении.
install.log

Maybe you don't have space left on your device?

Maybe you don't have space left on your device?

This is true, but it is assumed that the script can install HA without external storage. It may be possible to change the installation script

256 MB is likely min.  I will try with something having 1GB.

I just installed it on openwrt 23.5 and used the automated script.  After the done was displayed, telling me that the installation was complete, it took additional 301 MB of space. I think its fair to assume that 500 MB should be made a minimum.
I could not find luci zigbee tooks in my System > software > update


use auto closeable on connection

Summary by CodeRabbit

New Features

Enhanced JDBC connection and statement management with new wrapper classes.

Refactor

Improved database connection handling for better stability and performance.

Chores

Added concurrency configuration to workflows for better job control and cancellation.

@coderabbitai review

https://github.com/coderabbitai review


Apparent incompatibility with Ionic

First, thanks for this awesome component!
Does anyone have experience using this with Ionic? Without altering the code as described below, I am unable to alter the cropping box, as no mouse events  seem to propagate to the component.
Steps to reproduce:
Install blank ionic starter app (using Ionic command line tool):
ionic start clipTest blank
add the js library for clipping into index.html
insert the demo code from the home page for angular-img-cropper
(attached HTML file)
add the example controller code to the app.js file (attached)
Archive.zip
dragging on the clip box will not do anything.
angular-img-cropper sets up event watches on various events of the form
mousemove.angular-img-cropper
and so on. If I edit angular-img-cropper.js to change all the event names from .angular-img-cropper to  (e.g.
.on('mouseup.angular-img-cropper', this.onMouseUp.bind(this))
becomes
.on('mouseup', this.onMouseUp.bind(this))
it seems to work fine. But I don't know whether that has any unanticipated side effects. My hypothesis is that ionic is somehow taking over event processing and so the event targets aren't as expected.

very old, closing


scikit-nni - A helper tool to tune parameters of scikit-learn pipelines (for classification) using NNI

Hi,
This is not an issue.
First, thanks for this awesome tool. Really great work. I played with it for last 2 days and it works really well.
I was tuning some classification algorithms and end up creating a generic tool and package that I call scikit-nni.
Please find the repository here - https://github.com/ksachdeva/scikit-nni.
It essentially generates the NNI configuration files and also has a generic scikit pipeline builder that takes the specification (from YAML file). This way users of NNI do not have to write the code for the scikit pipelines. I have multiple examples and a decemt documentation in the repository.
Would appreciate any feedback you may have.
Thanks again for NNI.
Regards
Kapil

Hello @ksachdeva  This work is interesting! It's a good example of using NNI to build a tool~

@ksachdeva thanks for your work, we will add a reference to your repo soon.

Hi @QuanluZhang , I also wrote an article about Microsoft NNI and it got published few hours ago.
Title- Find thy hyper-parameters for scikit-learn pipelines using Microsoft NNI
Here is the link to the post -
https://towardsdatascience.com/find-thy-hyper-parameters-for-scikit-learn-pipelines-using-microsoft-nni-f1015b1224c1
Here is the friend link to the post to avoid the membership limitations of medium -
https://medium.com/p/find-thy-hyper-parameters-for-scikit-learn-pipelines-using-microsoft-nni-f1015b1224c1?source=email-d95181aaafe2--writer.postDistributed&sk=b894354f22ea47cc9c1e9605d131ce82
Thanks for your great work with Microsoft NNI.
Regards
Kapil

@ksachdeva thanks again for your contribution, we have reference the repo and article in pr #1754.


src/index.ts Cannot be found with by webpack dev server

Hi, I really like this package but recently I ran into a problem in a TypeScript project:
Hot reloading does not work well. I don't have this problem for other packages like typeorm which also provides it's own type definitions.
17:13:28 webpack.1    | (Emitted value instead of an instance of Error) Cannot find source file '../../src/index.ts': Error: Can't resolve '../../src/index.ts' in '/Users/me/project/node_modules/graphql-request/dist/src'
17:13:28 webpack.1    |  @ ./client/components/orgs.tsx 2:0-42 23:8-15
17:13:28 webpack.1    |  @ ./client/components/client.tsx
17:13:28 webpack.1    |  @ ./client/index.tsx
17:13:28 webpack.1    |  @ multi (webpack)-dev-server/client?http://localhost:8081 webpack/hot/dev-server babel-polyfill ./client/index.tsx
17:13:28 webpack.1    |

For some reason it's traversing 2 directories up and then failing to find that file.

I found the same issue and the answer here:
https://github.com/apollographql/react-apollo/issues/597#issuecomment-306091791
To quote:
"In my webpack config I have the source-maps-loader but didn't exclude node_modules and this seemed to be causing the issue. Adding the exclude fixed it for me."
And the config should be:
      {
        enforce: 'pre',
        test: /\.js$/,
        loader: 'source-map-loader',
        exclude: [/node_modules/, /build/, /__test__/],
      },

I am unable to reproduce this issue in the latest version of WebPack, so I will assume this issue is fixed. If someone is still experiencing issues let us know.


Able to login in using ActiveDirectory (windows authentication)?

At first glance it doesn't look like Falcon supports connecting to MS Sql Server using activedirectory authentication, only an actual sql login--is this correct?

Judging by this PR https://github.com/tediousjs/tedious/pull/497 it looks like this is possible (although only on Windows machines and using a native module).
I'm going to mark this issue as an enhancement, but I feel hesitant to mark this issue as community because:

it requires the native module https://www.npmjs.com/package/sspi-client (the PR above mentions that this is a native module for windows only; if this is the case, we would need to introduce a window-only backend; this is doable, but we better tidy up our webpack.configs before going that way).
I can't find the documentation to enable windows authentication.


Validate login session in cypress tests

Describe the change
This PR ensures that the login session is validated as active in every Cypress test scenario.
Bonus:

Only a single, generic Before statement is required for the entire Cypress test suite.

Steps to test the PR
Verify that the CI tests pass
Issue reference
Fixes https://github.com/kiali/openshift-servicemesh-plugin/issues/392

I ran the OSSMC Cypress tests twice (just to be sure, as this issue occurs occasionally), and all tests passed successfully:
https://master-jenkins-csb-servicemesh.apps.ocp-c1.prod.psi.redhat.com/job/kiali/job/main-pipelines/job/upstream-istio-pipeline/2262/
https://master-jenkins-csb-servicemesh.apps.ocp-c1.prod.psi.redhat.com/job/kiali/job/main-pipelines/job/upstream-istio-pipeline/2263/


[Feature Request] Load exrc files like .nvim.lua on Project loading

Problem
Say one wants to open ~/proj1/ as a project.
The exrc option is enabled.
Structure of proj1:
proj1
├── .nvim.lua
├── index.qmd
└── post1.qmd

.nvim.lua enables the project specific configurations.

They can nvim ~/proj1 to open the project.
Neovim starts up in ~/proj1.
As a result .nvim.lua configuration file is run.
Or they may start neovim as nvim.
Then switch to the project by running Telescope neovim-project discover command.
But the .nvim.lua project configuration isn't run.

This difference makes method 2 inferior for opening projects.
Expected behavior
The benefit of method 2 of opening projects:

We can create a general Neovim application shortcut.
We can switch between projects in the same session, and load the corresponding configurations.

Method 2 should also invoke and get the benefits of .nvim.lua exrc configuration file.
Note
As specified here here, the main reason this isn't implemented yet is security. And since in neovim-project the projects are listed manually, it should be safe to allow .nvim.lua to run.

user's lua files execution is beyond the scope of this project. The concept of security and how such a plugin should work is quite subjective.  I even tried to write my own plugin and used it for a while. The main problem is that among them there is no one that will suit everyone and become standard.
You can use autocmd to implement this feature in your dotfiles or in the separate plugin.
local augroup = vim.api.nvim_create_augroup("user_cmds", { clear = true })
vim.api.nvim_create_autocmd("User", {
  pattern = { "SessionLoadPost" },
  group = augroup,
  desc = "Run lua file after loading session",
  callback = function()
    -- ...todo
  end,
})

Thanks for the tip


DEV: switch to poetry for dev/packaging, fix #14

add pyproject.toml
add poetry.lock
remove setup.py

Merging even though this failed because it's an issue with how the Travis CI is set up, not with the PR itself.
About to switch CI to GitHub actions anyway


Enable access to the pre-redesign VM detail page

With incremental releases planned (after each agile sprint), not all
of the previous functionality will be available

Enable navigation to the pre-redesign / legacy VM detail page from
the current VM details page

This allows access to the old Console, Disks, Nics and Snapshots
functionality

Solves #718

@sjd78 Did you tested it? Because I got this errors:

@bond95 - I did test it.  I saw those errors, but they are in code that was untouched and the page still works so I didn't care.  They may not even be new errors (these come from the PropType check which is hidden in production code).
I really don't want to take any more time working on this.

@bond95 , @gregsheremeta - I just tested the details page from tags/1.4.0-3 and the same errors get reported on the dev server.

@sjd78 the problem not in console errors by itself, but in that they cause blank page.

the problem not in console errors by itself, but in that they cause blank page.

for posterity, we determined this wasn't a real issue


Log out user / Log in another user still old user visible

Current behavior
When I logout in the capybara theme and login with another user. I am still logged in with the old user.
Expected behavior
That user is switched to the new user.
Steps to reproduce the issue
Log in with user A
Logout user A
Login with user B
Profile is user A
Can you handle fixing this bug by yourself?

[ ] YES
[x ] NO

Environment details

Browser: Chrome
OS: macos
Code Version: 1.12.1 (Capybara)

Additional information

Hi @davevh, When I tried to see this issue, Whenever I was changing the user, My user was getting updated. Can you check it once more and it would be great if you can provide any Video reference in which the bug is visible.


How to exclude a file in the dirs configuration folder?

Clear and concise description of the problem
How to exclude a file in the dirs configuration folder?
Suggested solution
autoImport({
    imports: ['vue', 'pinia'],
    dirs: [
      resolve(__dirname, 'src/apis')
    ],
    exclude: [/node_modules/, /\.git/], // how to exclude a file in src/apis/
    dts: resolve(__dirname, 'src/auto-imports.d.ts')
  }),

Alternative
No response
Additional context
No response
Validations

[X] Follow our Code of Conduct
[X] Read the Contributing Guide.
[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

dirs option use minimatch under the hood. In minimatch use negative pattern prefix(!) to exclude files;
autoImport({
  imports: ['vue', 'pinia'],
  dirs: [
    resolve(__dirname, 'src/apis'),
    "!src/api/excludeFile.ts", // You don't have to use `path.resolve`
    // `!${resolve(__dirname, 'src/api/excludeFile.ts')}` // but if you prefer to use `path.resolve` you can use it like this
  ],
  exclude: [/node_modules/, /\.git/], // how to exclude a file in src/apis/
  dts: resolve(__dirname, 'src/auto-imports.d.ts')
}),

@antfu Excuse me.I have this question too. Can auto import from node_modules?

@lgzhang yes you can. unplugin-auto-importhas multiple presets for the most used npm packages. If these are not fit for you. You can still import npm packages via theimports` field in the vite plugin options. If you still have problems, let me know which package you want to use via auto-import.


[WIP] Support for upcoming SPT 3.10, using EFT <IP_ADDRESS>757

Opening a new PR, given SPT 3.10 is now using <IP_ADDRESS>757

这个是中文的吗？

3.10 BE is on <IP_ADDRESS>420 now

3.10 BE is on <IP_ADDRESS>420 now

Yes thanks I'm going to have a look

Closing in favor of https://github.com/sailro/EscapeFromTarkov-Trainer/pull/590


Fix limits creation

Description
Changes proposed in this pull request:

...
...
...

Related issue(s)

Resolved #1315

/test pre-busola-integration-test-k3s-cluster


Finally caught up with your blog posts!

... and found some typos along the way. But Pycharm spell checker found a lot more, credit to it!

Merci beaucoup !


Allow install from package

Allows you to install the Ubuntu or Debian packages if desired but continues to default to source for most up to date version.

LGTM, thanks!


"Corbu Vechi" is listed in the drop-down menu

Description:
"Corbu Vechi" is listed in the drop-down menu alongside its corresponding county and commune, according to the law
Precondition:
The website is up an running.

Step 1
Type "Corbu Vechi" in the search bar.

Expected results
The location is listed in the drop-down menu as "Corbu Vechi, Brăila (Măxineni)".


Python version for Conda environments without Python contains incorrect information

Create a conda env without Python
The version information returned is invalid {"major":-1,"minor":-1,"patch":-1}
See below

This causes issues in Jupuyter extesnion as we display messages based on the version of the python environment, and -1 isn't a valid version number.
Ideally it should be undefined
I would setting the whole version property to be undefined as -1 is not a valid version.
Or have a seperate type for Non-Python environments. Such as conda environments that do not contain Python, as thats definitely possible, after all conda is not just for python, one can create environments for Java, Node.js, etc with Conda

Looks like an issue at our end.

I guess the bigger question is how do we identify Conda Environments that are not valid Python envionrments.
The current properties have no way to clearly convey this.
Hence the suggestion to have a seprate type that describes this, and that type could have a single property or the like to indicate its not a python environment.

executable.uri is undefined for such environments, will go with setting the whole version property as undefined. A single property won't do as there are other details which are relevant.


Support Consumers Set Custom Retry Delay

Master Issue: #6448
Motivation
For many online business systems, various exceptions usually occur in business logic processing, so the message needs to be re-consumed, but users hope that this delay time can be controlled flexibly. The current user's processing method is usually to send this message to a special retry topic, because production can specify any delay, so consumers subscribe the business topic and retry topic at the same time. I think this logic can be supported by pulsar itself, making it easier for users to use, and it looks like this is a very common requirement.
Modifications
This change can be supported on the client side,  need to add a set of interfaces to org.apache.pulsar.client.api.Consumer
void reconsumeLater(Message<?> message, long delayTime, TimeUnit unit) throws PulsarClientException;
CompletableFuture<Void> reconsumeLaterAsync(Message<?> message, long delayTime, TimeUnit unit);
CompletableFuture<Void> reconsumeLaterAsync(Messages<?> messages, int delayLevel);

DeadLetterPolicy add retry topic
public class DeadLetterPolicy {

    /**
     * Maximum number of times that a message will be redelivered before being sent to the dead letter queue.
     */
    private int maxRedeliverCount;

    /**
     * Name of the retry topic where the failing messages will be sent.
     */
    private String retryLetterTopic;

    /**
     * Name of the dead topic where the failing messages will be sent.
     */
    private String deadLetterTopic;

}

org.apache.pulsar.client.impl.ConsumerImpl add a retry producer
  private volatile Producer<T> deadLetterProducer;

  private volatile Producer<T> retryLetterProducer;

Can specify whether to enable retry when creating a consumer，default unenable
    @Override
    public ConsumerBuilder<T> enableRetry(boolean retryEnable) {
        conf.setRetryEnable(retryEnable);
        return this;
    }

@liudezhi2098 Thanks for the great work. Would you please help add some tests for it?

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

/pulsarbot run-failure-checks

Doc has been added https://github.com/apache/pulsar/pull/7151

@liudezhi2098 Hi, could you please comment on my question above?


docs: Update start.md

What this PR does:
Add java-sdk uds demo doc
Which issue(s) this PR fixes:

Special notes for your reviewer:
Does this PR introduce a user-facing change?:

/cc @wenxuwan

/lgtm
/approve
/merge
@ChloroplastYu thanks for your contribution


Stop working for smarttags device since app's update

Hello,
Since app's update, there is a new tab for "devices" and "things". Smarttags 2 went in "things" tab.
Since that update, I can't find my smarttags in HA.
Thank you for this addon ! 🥇

Hi,
have you tried removing and readding the integration? It's indeed an issue, that devices disappear after a few days or weeks. I don't think it's related to the new app design, because this integration get's it's data from the smartthings find website which did not change recently.

Hi,
There is an update to smartthings APP, after updating, I can't re install the "Smartthings find" HA integration.
Tried to add it using a tablet where I haven't updated the smartthings app, and it worked

I don't understand. This integration has nothing to do with the smartthings app.
Also, it doesn't make a difference on which device you setup the integration in HA.
Can you please be a bit more specific?


Don't use Object.runtimeType.toString() in release mode

This operation is non-trivial and should not be used as excessively as it is used right now.

Closed via #122


Patient can see new proposed time on Dash board but it is not shown when patient clicks on see details

See details about appointment does not have proposed new time by doctor but it do ask for confirmation...  But new proposed time is visible on dash board
But it do show right time after confirming on detail page.

please close if this is resolved

closed


intro to ops guide (#2387)

btw, I reckon we should move this guide down the menu order somewhat. At the moment it rather disrupts the flow from "Installing Weave Net" to "Using Weave Net" (the latter being the place where the user actually sees Weave Net doing something useful), since it sits between those too. I suggest dropping it to just above the FAQ, or at the very least to below the three "Integrating ..." items.

I've had a stab at reworking this for technical accuracy:
This operational guide is intended to give you an overview of how to
operate and manage a Weave Network in production. In consists of three
parts:

* A glossary of concepts with which you will need to be familiar
* Detailed instructions for safely bootstrapping, growing and
  shrinking Weave networks in a number of different deployment
  scenarios:
    * An interactive installation, suitable for evaluation and
      development
    * A uniformly configured cluster with a fixed number of initial
      nodes, suitable for automated provisioning but requiring manual
      intervention for resizing
    * A heterogenous cluster comprising fixed and autoscaling
      components, suitable for a base load with automated
      scale-out/scale-in
    * A uniformly configured cluster with dynamic nodes, suitable for
      automated provisioning and resizing
* A list of common administrative tasks, such as configuring Weave Net
  to start on boot, upgrading clusters, cleaning up peers and
  reclaiming IP address space


[Feature Request]: Option to change a transform's label background color

What would you like to happen?
Migration: https://issues.apache.org/jira/browse/HOP-3856
Please add an option to change a transform's label background color / set it to transparent / change its opacity.
A hop's symbol is barely visible if two transforms are very close to each other. Also, I find that the GUI appears clearer and less cluttered when the label is "transparent" (compare hop vs. spoon).
Issue Priority
Priority: 3
Issue Component
Component: Hop Gui

With the new ui to mimic a hyperlink #3335, this request is obsolete.


Fix bug where len is compared to less than 0

Change Overview
len() will never equate to less than zero.
Pull request type
Please check the type of change your PR introduces:

[ ] :construction: Work in Progress
[ ] :rainbow: Refactoring (no functional changes, no api changes)
[ ] :hamster: Trivial/Minor
[x] :bug: Bugfix
[ ] :sunflower: Feature
[ ] :world_map: Documentation
[ ] :robot: Test

Issues

fixes #1683

Test Plan

[ ] :muscle: Manual
[ ] :zap: Unit test
[ ] :green_heart: E2E

Hey @bathina2,
Thanks for fixing this. Could you please resolve the conflicts?


fontspec 2.9b breaks microtype \textls

Description
Since upgrading to v2.9b in current TeXlive, microtype's \textls does not work anymore for fonts with spaces(?).
Check/indicate

[x] Relevant for LuaTeX
[x] Issue tracker has been searched for similar issues?

Minimal example demonstrating the issue
\documentclass{article}
\usepackage{unicode-math}

% \setmainfont[Ligatures={TeX}]{Carlito} % works
\setmainfont[Ligatures={TeX}]{Libertinus Serif} % does not work
\usepackage[]{microtype}

\begin{document}

Kaffe\textls{kanne}

\end{document}

Further details
error message:
luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: Font "kernfactor=0.1;Libertinus Serif" not found

rolling back to v2.9a, the spacing works and no error message appears.
Not sure if this is a fontspec or a microtype issue, but the CHANGES file told me to write if there are problems with font names with spaces… ;-)

I would say while the change is from fontspec it should nevertheless be corrected by microtype. As there can now be a space in the font name, tex.fontname(font.current()) gives back a quoted font name:
 "name:Libertinus Serif:mode=node;script=latn;language=dflt;+tlig;"

but the function add_ls in microtype.lua doesn't expect that.

@u-fischer, thank you, I will tell them.

(to be) fixed in microtype

It's still there "luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: Font "kernfactor=0.1;Noto Sans at 10.95pt" not found.C:/Miktex/tex/luatex/luaotfload/fontloader-basics-gen.lua:156: bad argument #2
to 'find_file' (invalid option '95pt')."

@LyonsDo it is there, true. It it is going to be fixed in microtype and tracked there, see link above (there it is reported as "fixed in dev" so probably distributed soon). [[The "closed as completed" is a bit misleading, I agree, but that is something that happens more or less automatically if close is being used)

No problem. Just saying :-)
On Thu, 4 Jul 2024 at 15:56, Frank Mittelbach @.***>
wrote:

@LyonsDo https://github.com/LyonsDo it is there, true. It it is going
to be fixed in microtype and tracked there, see link above (there it is
reported as "fixed in dev" so probably distributed soon). [[The "closed as
completed" is a bit misleading, I agree, but that is something that happens
more or less automatically if close is being used)
—
Reply to this email directly, view it on GitHub
https://github.com/latex3/fontspec/issues/509#issuecomment-2209060872,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACXO43VR7DAFEITWQXMRMYDZKVIAPAVCNFSM6AAAAABG5APL3WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBZGA3DAOBXGI
.
You are receiving this because you were mentioned.Message ID:
@.***>


Create a report abuse system

If the recipe doesn't sound vegan, show a button to report abuse.

I would like to help here. Are you accepting contributors?
If yes
Can you provide more details for this feature like:

How you'd like to receive reports (email, a database table and a cron to notify, etc.)
Where to place the report button (I was thinking about placing it beside the like button)
Which information to put on the report


Make the MockWebServer tests more Android compatible.

There are a few cases in OkHttp where API calls don't exist on
Android, but this is one that we can work around with any loss
of understanding.
The same test also behaves differently on Android due to
differences between the OpenJDK and Android's
HttpURLConnection implementation.
See issue #1824.

Neat!


need support: unable to reproduce satisfactory training results

Hi Firas,
I really appreciate your great work on this benchmark, it saves us a lot of time in our research project. Currently, I am trying to reproduce the walking and running examples demonstrated by the gif in Readme. However, I was not able to get a satisfactory result using the examples/imitation_learning training pipeline for HumanoidTorque4Ages environment (my models failed even after I tried different configurations). I only able to get a good result with the HumanoidTorque.walk environment. I trained the model with arms and feet moving due to the requirements of my project, I also tried using only box_feet for the HumanoidTorque4Ages but was not successful. Have been working on the training for almost a month, if you could provide the working pretrained model for HumanoidTorque4Ages that will be super helpful, would appreciate any support/help, thanks a lot!

Hi @leyongloh,
sorry for my late reply and thanks for using LocoMuJoCo!
HumanoidTorque4Ages is a challenging environment, as it is randomly changing the humanoid. I have a different project in which I actually use this environment to train an agent suitable across different humanoid sizes (check it out here). But this was without arms. The gif in the readme just shows the motion capture, not a trained policy. Due to time constraints, I did not have time to solve this environment with arms. So, while I can not directly provide you with an agent, I am more than willing to help you solving this env!  Drop me an email (click on "contact me" here), maybe we can have a meeting.