[CELEBORN-1519] Do not update estimated partition size if it is unchanged

What changes were proposed in this pull request?
We will not update the estimated partition size if it is unchanged.
Why are the changes needed?
Celeborn currently triggers an workerinfo update even-though the estimated partition size is not changed. This leads to unnecessary logging and redundant worker info update operations.
Example log -
[master-partition-size-updater] WARN org.apache.celeborn.service.deploy.master.clustermeta.AbstractMetaManager - Celeborn cluster estimated partition size changed from 64.0 MiB to 64.0 MiB

Does this PR introduce any user-facing change?
NO
How was this patch tested?
Existing UT's

Thanks, merged to main


Musnad Ahmad with gradings

After jami kamil is released, so these things can be done together

Dear Fawaz,
Does that mean you are gonna have the Musnad Ahmad sometime soon as the 8th book of your previous API work?

salam alaykum haluk, I hope you are doing good.
Inshallah I will add musnad ahmad together with jami kamil.
Currently jami kami translations are not released,  it will release next year (2023). So it will take time

salam alaykum haluk, I hope you are doing good.
Inshallah I will add musnad ahmad together with jami kamil.
Currently jami kami translations are not released,  it will release next year (2023). So it will take time

It can be found here in Turkish.
http://www.normalift.com/kitap/files/oth/a.say/HADİS-G.htm

salam alaykum haluk, I hope you are doing good.
Inshallah I will add musnad ahmad together with jami kamil.
Currently jami kami translations are not released,  it will release next year (2023). So it will take time

Musnad Ahmad can be found here in Turkish. http://www.normalift.com/kitap/files/oth/a.say/HADİS-G.htm

Inshallah, I will add it together with jami kamil


authenticatorConfig is ignored

Current Behavior
I manually configured keycloak, including the setting in this screen

which is important for my stuff to work. Then I exported the config via kc.sh export and got the config file in JSON format. This file contains the following section, where I assume the above screen is stored
  "authenticatorConfig": [
    {
      "alias": "keycloak-us-oidc",
      "config": {
        "defaultProvider": "keycloak-us-oidc"
      },
      "id": "d863d33c-de7a-4649-a005-54d6b424f664"
    }
  ],

but when this configuration is applied during the next deployment, somehow that configuration seems to be missing.

Expected Behavior
I expect the exported authenticatorConfig configuration applied again to lead to the exact same resulting configuration (in the UI) again.
Steps To Reproduce
No response
Environment

Keycloak Version: docker.io/bitnami/keycloak:24.0.4-debian-12-r1
keycloak-config-cli Version: docker.io/bitnami/keycloak-config-cli:5.12.0-debian-12-r5
Java Version: OpenJDK Runtime Environment Temurin-21.0.3+9 (build 21.0.3+9-LTS) (as of the config-cli image)

Anything else?
No response

Hi @MBA-aix,
the tool works without IDs, you just have to put the alias (keycloak-us-oidc) into the field authenticatorFlow.
Please let us know if that worked.
Best Regards
Jonas

Hi @jonasvoelcker,
first and foremost, that did the trick! I removed the id fields and everything works as expected.
Just one more question: Is your statement about IDs universal, i.e. there should not be any such "id": "<uuid>" properties in the config export or is this only true for the authenticatorConfig part?
Thanks in advance!

Hey @MBA-aix,
whereever references are used (e.g. authFlowOverrides) it uses the names instead of the IDs as they are generated by Keycloak and they are not equal through the different stages.
You can easily delete every id-row in your export, you just need to reference without the IDs though.

Documentation will be added to the project!


oriendb rest api

hey, just wanted you guys to know that I'm working on a rest driver
https://github.com/gusnips/node-oriendb-rest-api
it's working fine so far, no bugs
it also doesn`t depend on node, works in the browser or any browser-like enviroments/webviews, such as electron or react-native
I'm planning to change it to work on 3.0 as soon as it's stable
Just wanted to let you know, in case it interests you

Hi @gusnips
Thank you very much for letting us know, it's definitely interesting.
Please let us know how it proceeds and if you need help on it.
Thanks
Luigi


backdrop: make sure users can't scroll behind the backdrop with all overlays

I confirm this is a real issue in iOS safari, affecting alerts, popover and actionsheet.

This was an issue in v1 as well: #4793. The fix for v2 might be similar?

@Manduro thanks for linking to that! :)

@brandyscarney @adamdbradley my fix: https://github.com/driftyco/ionic/pull/6687


Update 01-upgrading-to-v1.6.md

adding SL doc placeholder for 1.6 beta launch
Resolves #3385
Resolves #3503

do we still need this @matthewshaver ? or can i close?


User Stories for Interface / Feature Design and Documentation

As discussed in the rust arrow chat, this is a place to chart a way forward by collecting examples of both what people do in other libraries and what they want to do but can't do easily with current tools. In addition to clarifying Python interface requirements, I hope it provides fodder for lower-level functions, encourages knowledgeable folks to explain how to do things (which can then get documented), and clarifies what is important to what types of people and why. There's a daunting amount of work, but the opportunity and potential are tremendous.
For contributors, I'd like to structure this roughly as follows:

If it's your first post in this thread, a brief description of your background and the types of work you've done/do.
Examples of operations on data you think are important. Broadly of the form "I can do X in library Y with code sample Z". If it's something that you like, consider saying why. If you don't like it, explain how it could be easier or clearer.
Examples of things you frequently do but have had to implement yourself and think should be considered for core operations.
OPTIONALLY: Thoughts you have on what's important to a good data frame API in a dynamic language.

OK, now it's my turn to start.
My background is in statistics, machine learning, and data science. Most of what I do is focused on modeling and analyzing data, though I've done a good bit of pipeline and data processing/cleaning too. As such, I place a premium on in-memory interactive work (notebooks, rmarkdown, etc.). Partly because of this, I think a lot of tools mistake verboseness for clarity, and striving for conciseness often helps readability rather than hurting it if done correctly.
I have the most experience with R's data.table but I've also used dplyr, polars, pandas, and Julia. So here's sampling of a few things I would want in a dataframe library in no particular order.

Here's an example in data.table that shows features I think are both good and bad.

> dt1 = data.table(t = 1:5, v = 5:1)
> dt2 = data.table(start = c(1, 4), end = c(3, 10), x = c("a", "b"))
> dt1[dt2, x := i.x, on = .(t >= start, t < end)]
> dt1
   t v    x
1: 1 5    a
2: 2 4    a
3: 3 3 <NA>
4: 4 2    b
5: 5 1    b

First, I find non-equi joins, especially range joins incredibly useful. They're common in SQL but a lot of dataframe libraries don't have them. data.table also makes it easy to update in place with the := operator, which can be used to create new columns as well as update existing ones. As I understand it, arrow strives for immutability, but at the same time, it won't make copies of the whole frame if it doesn't have to, so maybe this is less of an issue. However, I do like the idea of using a join like this to explicitly tag/annotate another table.

Reshaping data. These functions transform data between "wide" to "long" (sometimes known as "tidy") formats. Sometimes they go by cast/melt or pivot and there's even a simple transpose function in a lot of packages. It doesn't seem common in database-world, but to me reshaping in-memory data is important for a lot of use cases.

Rolling groupbys. Both Pandas and Polars have pretty good support for creating overlapping groups and aggregating over them. These are commonly used for time series analysis. I also think the ability to define groups not just by a number of rows, but by a potentially variable-width lookback (like, at most 1 month before the current date) is useful. Polars does a pretty good job at this, and I think Pandas might too.

Alright, that's a few to get started and this is long enough as it is. I'm looking forward to seeing what everyone thinks is important, their thoughts on good DataFrame API design, and what is and isn't currently possible in DataFusion.

Hello,
My name is Cody and I'm a Technical Product Manager at Voltron Data. My background is in electrical engineering and data science. For nearly a decade I've used Python and gone through a journey of using pandas, PySpark, Dask, and more recently being interested in DataFusion/Polars. While I've never had a real data engineering or data science job, I've done work in those areas and worked with many engineers in them. I've written tons of examples with these frameworks.
Full disclose: I primarily work on and Voltron Data is invested in the success of Ibis.
My worry here is that you and the DataFusion community are going to go through the struggles that every new Python dataframe library does, and start facing the same type of questions -- is it groupby or group_by? to_csv or write_csv? The list goes on. I've seen the fragmentation of the Python data community over the years and would be far more excited to work on a standard API that supports many backends (Ibis) than bringing another Python dataframe library to the table.
The good news, if going in this direction, is that a lot of the work is done! DataFusion in Ibis is fairly functional (thanks to lots of community contributions), though there still is a ways to go before it's up to par with the DuckDB backend. Here's a quick example notebook: https://gist.github.com/lostmygithubaccount/73414943a71ffb471605c3132eda0727
We'd love to have more collaboration on Ibis for the DataFusion backend if that'd be an interesting direction to you and others. Ibis was created by Wes McKinney (creator of pandas) and taken an opinionated stance on most issues I suspect you'll face with a new dataframe library. Plus, it takes heavy inspiration from R and other previous tools! Let us know if this would be interesting to you.

Hi Cody! Thanks for your interest in this. I've seen a little bit of Ibis and it looks interesting. I'm also not sure improving Ibis support and making a better "native" API are conflicting goals.

My worry here is that you and the DataFusion community are going to go through the struggles that every new Python dataframe library does, and start facing the same type of questions -- is it groupby or group_by? to_csv or write_csv? The list goes on. I've seen the fragmentation of the Python data community over the years and would be far more excited to work on a standard API that supports many backends (Ibis) than bringing another Python dataframe library to the table.

These differences, at least as you've described them here, seems more like a mild annoyance than a struggle to me. As long as there's reasonable documentation, I've never found slightly different names to be nearly as big a barrier as identical or similarly named things behaving differently, or differing capabilities across libraries.

We'd love to have more collaboration on Ibis for the DataFusion backend if that'd be an interesting direction to you and others. Ibis was created by Wes McKinney (creator of pandas) and taken an opinionated stance on most issues I suspect you'll face with a new dataframe library. Plus, it takes heavy inspiration from R and other previous tools! Let us know if this would be interesting to you.

I'm not opposed to this at all, especially if Ibis can provide a consistent API while still exposing the full power of each underlying library. At some point, though, it seems like you'll encounter differences that preclude a uniform interface or require a specialized API for a unique feature Ibis doesn't support. However, I can see the appeal if you have people who occasionally use a large number of backends or are trying to build something that can interact with multiple systems. So I'd be surprised if there weren't value to both a native interface that exposed all of a tool's power and a universal interface since they seem to be solving different problems. If I'm wrong about Ibis' goals and capabilities, please do correct me though.

It's already integrated with visualization frameworks (Altair, Plotly, Streamlit -- any that support the __dataframe__ protocol natively, and any others through to_pandas()) and ML frameworks (scikit-learn, XGBoost, more in this area coming soon).

Glad that you brought this up. What's the relationship between Ibis and the Python dataframe standards protocol?

What's the relationship between Ibis and the Python dataframe standards protocol?

Ibis is one of many voices at that particular table.  We have robust support for the dataframe interchange protocol (__dataframe__) and we're interested in adopting the dataframe standard compat stuff, the __dataframe_consortium_standard__ -- although at the moment, the latter is designed with an eye towards eager computation and it isn't clear how it will play with lazy / deferred computation.

I'm also not sure improving Ibis support and making a better "native" API are conflicting goals.

I don't disagree, but I think they are duplicative efforts. My overall concern here is that the DataFusion project will spent a lot of time and energy on re-hashing a lot of these discussions about what a great Python dataframe library should be and this is bad for the ecosystem overall.
Ibis's primary innovation is portability, decoupling the dataframe API from the execution engine. While this might not be a big deal for an individual developer who can learn pandas & Polars & Snowpark & PySpark APIs, it represents a major source of siloing and duplication of efforts across the ecosystem. I've heard many cases of data scientists "throwing pandas code overall the wall" for data/ML engineers to rewrite in PySpark. With the Ibis projects (and other open-source projects) we (the Voltron Data we) are hoping to create a more modular and composable ecosystem like this:

I do think it's totally valid for DataFusion to pursue its own Python dataframe API, I just think it would be better to spend effort improving DataFusion as an execution engine and leveraging all the work already done in Ibis for the user-facing API!
Very fair concern on how Ibis achieves what it claims -- we're actually in the process of moving our documentation to Quarto and I'd love your feedback on the "Why Ibis?" page and "Backends concept page" that explains it -- in short, SQL dialects and tooling for them (sqlalchemy, sqlglot, etc.) are close enough and manageable enough with the scalable API Ibis exposes. For the most part, Python expressions are compiled to SQL.


fix XSS issues to prevent users from getting "hacked"

i dont know why this wasn't fixed already. i use this extension a lot and i felt unsafe that XSS worked. this pull request makes the extension sanitize the comment text before putting it in html. i don't believe there are any ways to bypass it since it sanitizes everything that's not hexadecimal.
have a nice day!

i see, i didn't think about that and just thought this would do from the one test i did.

0001-Fix-XSS.patch
Perhaps this patch is better. Doesn't remove languages and emoji, but removes the XSS.

looks the same even with my patch, are you sure that it actually does break languages or emojis? doesn't seem like it does.

Sorry, I'm a dumbass, I misread the code as stripping away anything that wasn't a-z or 0-9.
I still think we should try to implement DOMPurify as the list of potential XSS payloads is huge, but your patch is a good first step.

i think you're very smart and you just simply overlooked something.
i also think that DOMPurify might be better, but i believe it's currently impossible to bypass turning every non alpahnumeric character into a html entity

Thanks for your input!
I think DOMPurify is probability the go to solution.
I'll take a look later.

i mean, dompurify does stuff like converting

into stuff like

what we really need is to not have any html tags at all and my code does that, however i think dompurify has settings and stuff that can just sanitize every illegal char into a entity.
i never really messed with it so i don't know but if it's better then it's better i guess, but after doing a bit more tests i think my patch is adequate enough i haven't been able to bypass it.

what we really need is to not have any html tags at all and my code does that, however i think dompurify has settings and stuff that can just sanitize every illegal char into a entity.

I share the same opinion, and after surveying DOMPurify, it doesn't achieve the desired effect.
Your version seems to work just fine. 👍


Implement aliases Property for Cfgu Declaration

Description
Introduce an aliases property to the Cfgu declaration. This property, represented as an array of strings, will enable users to reference a single configuration by multiple alias names. When performing an upsert, users can utilize any of the aliases. Meanwhile, during an eval, all associated aliases will return the same value.
{
  ...
  "MY_EMAIL": {
    "type": "Email",
    "aliases": ["myEmail", "EMAIL", ...]
  }
  ...
}

Motivation
To enhance the flexibility and usability of the Cfgu declaration by allowing multiple reference points for a single configuration.
Additional context

Can u assign me this issue . Thank you

@rannn505
could you please assign me this issue.

Hi @Aman1919, I'll clarify the requirements for you:
Schema validations
The inclusion of this new Cfgu property needs to include the following suite of validations:

Aliases must not conflict with other schema keys or aliases belonging to other schema keys.
Aliases must be of type array.
An alias cannot be an empty string.
An alias must conform to the same validations as config keys (no reserved words, invalid characters, etc...). The existing key validations can be reused for this.

The parse method of the ConfigSchema class should handle this.
Examples:
Valid schema
{
  ...
  "MY_EMAIL": {
    "type": "Email",
    "aliases": ["myEmail", "EMAIL"]
  }
  ...
}

Invalid schema - alias conflicting with another key:
{
  ...
  "MY_EMAIL": {
    "type": "Email",
    "aliases": ["myEmail", "EMAIL"]
  }
 "EMAIL": {
    "type": "Email"
  }
  ...
}

Invalid schema - alias conflicting with another alias:
{
  ...
  "MY_EMAIL": {
    "type": "Email",
    "aliases": ["myEmail", "EMAIL"]
  }
 "SUPPORT_EMAIL": {
    "type": "Email",
    "aliases": ["EMAIL"]
  }
  ...
}

Upsert
When performing an upsert, any of the aliases of a specific key can be used to perform the upsert. The upsert command should identify the key that the alias correlates to and then complete the upsert using the original key. The upsert command needs to validate that only a single alias can be provided per key AND that the key cannot be provided together with the alias, this is to keep the upsert deterministic.
Given this schema:
{
  ...
  "MY_EMAIL": {
    "type": "Email",
    "aliases": ["myEmail", "EMAIL"]
  }
  ...
}

Upserting with only one of MY_EMAIL, myEmail, or EMAIL is valid.
Upserting with more than one of any of the above is invalid.

Eval
When performing an eval, the aliases of any specific key should also be present in the eval result with their value corresponding to the value of their original key.
Given this schema:
{
  ...
  "MY_EMAIL": {
    "type": "Email",
    "aliases": ["myEmail", "EMAIL"]
  }
  ...
}

And given that the evaluated value of MY_EMAIL is SOME_VALUE, the resulting eval should be:
{
  ...
  "MY_EMAIL": "SOME_VALUE",
  "myEmail": "SOME_VALUE",
  "EMAIL": "SOME_VALUE",
  ...
}

You can also take a look at an excellent PR that implemented the options Cfgu to draw inspiration for implementation details.
Hope this clarifies everything. Let me know if you need any further clarification.

@RichardAkman Could u review the pr I changed the ts file first . If this commit is ok then I'll update other files

Can you please assign me the issue?


Spec: \x escaped used in double-quoted string

At  https://github.com/cuelang/cue/blob/master/doc/ref/spec.md#string-and-byte-sequence-literals the spec contains a few examples, which contain \x byte escape codes in double-quoted strings. This is invalid syntax, as far as I understand.
cue eval says invalid string: invalid syntax:.
"\xff\u00FF"
"\xe6\x97\xa5\xe6\x9c\xac\xe8\xaa\x9e"  // the explicit UTF-8 bytes

This issue has been migrated to https://github.com/cue-lang/cue/issues/793.
For more details about CUE's migration to a new home, please see https://github.com/cue-lang/cue/issues/1078.


Error using google/UL2 model

The model:
google/ul2
The Hardware:
2x RTX Titan
AMD Ryzen 9 5900X 12-Core Processor
64Gb RAM
The Environment:
Python 3.9.13
Pytorch 1.12.0+cu102
NVIDIA-SMI 495.29.05    Driver Version: 495.29.05    CUDA Version: 11.5
Code used:
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM
from parallelformers import parallelize
import torch

tokenizer = AutoTokenizer.from_pretrained("google/ul2")
model = AutoModelForSeq2SeqLM.from_pretrained("google/ul2")

parallelize(model, num_gpus=2, fp16=True, verbose='detail')

input_string = "[S2S] Mr. Dursley was the director of a firm called Grunnings, which made drills. He was a big, solid man with a bald head. Mrs. Dursley was thin and blonde and more than the usual amount of neck, which came in very useful as she spent so much of her time craning over garden fences, spying on the neighbours. The Dursleys had a small son called Dudley and in their opinion there was no finer boy anywhere <extra_id_0>"

inputs = tokenizer(input_string, return_tensors="pt")

outputs = model.generate(**inputs, max_length=200)

print(tokenizer.decode(outputs[0]))

Error Message:
$ python test.py
/home/******/miniconda3/envs/ul2/lib/python3.9/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 16 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '
Bus error (core dumped)

Is this something I can fix? I would love to use this large model, as it's near SOTA on everything :)

@dnhkng Sorry for delay. Did you solve this problem?

I gave up, as I found that the model was trained on TPU (brainfloat16), which is not supported on the RTX Titan.

That sounds bad. I am sorry for not being able to help.

getting same error Bus error (core dumped)
/usr/local/lib/python3.7/multiprocessing/semaphore_tracker.py:144: UserWarning: semaphore_tracker: There appear to be 16 leaked semaphores to clean up at shutdown
len(cache))

Hello @tahercoolguy Did you able to figure out the issue? I am also getting same exception while running it in docker.

@hyunwoongko any pointer to fix this issue?


Adding a new partition mode for querier

Fixes #22
Adds a new concept when creating a Querier so that we don't need to expose the the isClosedForPartition (and the new resetForPartition).

Coverage increased (+0.002%) to 99.567% when pulling c853211e629f620f9ea0408457365f7e9b3c2892 on akshaisarma:partition-mode into 1a9a0d78de3a7b263aa463779481db9d04415ad8 on yahoo:master.

@hbxie Done


[webview_flutter] Speculative getTitle fix

Attempts to fix the getTitle flake by running JS before accessing the
title, to force the page to be fully processed by the native code.
No CHANGELOG change: Description would be the same as something that is
already in the NEXT section.
Speculative fix for https://github.com/flutter/flutter/issues/94117
Pre-launch Checklist

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I read and followed the relevant style guides and ran the auto-formatter. (Unlike the flutter/flutter repo, the flutter/plugins repo does use dart format.)
[x] I signed the CLA.
[x] The title of the PR starts with the name of the plugin surrounded by square brackets, e.g. [shared_preferences]
[x] I listed at least one issue that this PR fixes in the description above.
[x] I updated pubspec.yaml with an appropriate new version according to the pub versioning philosophy, or this PR is exempt from version changes.
[ ] I updated CHANGELOG.md to add a description of the change, following repository CHANGELOG style.
[x] I updated/added relevant documentation (doc comments with ///).
[x] I added new tests to check the change I am making, or this PR is test-exempt.
[x] All existing and new tests are passing.

I'll retry the tests a few times before landing.

Passed once; trying again...

Twice...

Three times (and that's each of the four runs—{master,stable}x{app-facing,impl}—each time). We may have a winner here.

Could also revert the other speculative fix if it didn't work. Up to you though.

I'm going to give a shot at reverting the other fix in this branch (that'll give us yet another run of the test)

Merged on red to heal the tree.


Tags not properly updated to 1.21

As part of 1.21, Mojang changed the directories for various tags to use singular names instead of plural.
Your mod uses tags in one or more of the old directories that was changed! This means those tag files will no longer load in 1.21 unless you update the directories!
The affected directories and their new counterparts are:

tags/blocks -> tags/block
tags/items -> tags/item
tags/entity_types -> tags/entity_type
tags/fluids -> tags/fluid
tags/game_events -> tags/game_event

Please ensure that any of these that you are using is updated to the new name in all namespaces (including your own, minecraft, and c).

fixed in latest. thanks.


Update README.md

Removed option to manually add the Lovelace reference, as it's no longer supported by HA.

Thanks, didn't now the YAML method isn't supported anymore.

Great! This is hereby my first contribution to a Gitub project ever :-)


Basic Uploading of Siafiles and a homepage with all files listed

Basic setup. Subject to change.

ok, I went pretty hard on the comments, but most of them are minor stuff. It seems like we should separate the sunfish code into its own library and then call it from a very minimal main package.

In general we try to avoid exposed panics in code that will be running frequently. The consensus package is a lot more strict about this because it is very fragile crypto code, but Sunfish does not need to be so carefully crafted.
It's probably sufficient to log an error and return a '500' response to the caller, as opposed to panicking.

None of this code has docstrings, every function and struct (and for the most part, constant and global variables too) needs a docstring.

Can you minify skeleton.css?

Looks like SearchFile and DeleteFile are still not implemented

I have docstrings for the handlers. Will run back through and add more. SearchFile and Delete are not implemented. Looking into efficient ways to search the DB. Might start with just the tags. Otherwise it will be expensive.
Plan on replacing panics. Talked with Luke about that.

when you add more commits, can you also add a comment to  the PR, that way it shows up in our notifications

Ok. I have "merged" this pull request by creating a staging branch and moving it over to the staging branch. There are still a few comments left, so when you get to them you should pull request them into the staging branch.
We're going to have 2 siafiles urls. The first is the production url (siafiles.com) and the second is the staging url (staging.siafiles.com). I'm not sure what the database interaction is going to be like, but it makes sense that the staging site would have a recent backup database, such that it's more or less a realistic environment.
All pull requests will be done into the staging branch, and there will be git hooks and scripts that automatically deploy the changes to the staging website. If we decide that the staging branch is stable and makes us happy, we'll advertise it to the users and have them play around for a while. If nothing breaks, we'll move it to master.
Then more git hooks and scripts will automatically deploy the changes to the production website.


Feature request: Variable substitution on --output-recursive for copying outputs to {job_id} in bucket

Currently you can specify --output-recursive OUTPUT_PATH=gs://bucket/path to have your job write output files and subdirectories to ${OUTPUT_PATH} and these files will be copied to the specified bucket/path. But it would be very useful to be able to automatically set the path in the bucket to the job_id when calling dsub.
This is already available for --logging, where you can format the filenames of the logfiles using variable substitution and by default the files are tied to the job_id.
Can we have something like --output-recursive OUTPUT_PATH=gs://bucket/{job_id}?

Hi @sereeena!
Thanks for the suggestion. As you indicated, this seems like a very natural extension of the capabilities provided for the logging path.
Is the job_id the specific field you are interested in? Just want to make sure that the feature request, when implemented, would fulfill your use case.
Thanks!

Yes, I thought it would be useful to automatically keep outputs generated
by each job, linked to job_id.
Use case is more for debugging your pipeline, if you were running it
multiple times with the same input for example, you could easily keep the
outputs across each run.
This is what I was doing, but then I did think that in practice, in
production, you probably would be creating a new bucket, putting your input
file in there and then using that bucket as the output.
So I'm not sure if this is enough reason to implement it? But since logging
does it, I thought it might be easier to add..
Thanks
On Tue, 5 Apr 2022 at 03:09, Matt Bookman @.***> wrote:

Hi @sereeena https://github.com/sereeena!
Thanks for the suggestion. As you indicated, this seems like a very
natural extension of the capabilities provided for the logging path.
Is the job_id the specific field you are interested in? Just want to make
sure that the feature request, when implemented, would fulfill your use
case.
Thanks!
—
Reply to this email directly, view it on GitHub
https://github.com/DataBiosphere/dsub/issues/237#issuecomment-1087833230,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ADHDT57DHTGXQ45E6ZGZ7RLVDMSORANCNFSM5SHMXI3A
.
You are receiving this because you were mentioned.Message ID:
@.***>


METRON-1756: REST tests should use Embedded LDAP in metron-security

Contributor Comments
This PR switches authentication from JDBC-based to LDAP in metron-rest.
Changes Included

Added test dependencies for metron-ui-security and Apache Directory to metron-rest
Removed JDBC authentication (the TestSecurityConfig class)
Updated the metron-rest integration test to set the EMBEDDED_LDAP_PROFILE
Added additional users to the LDAP schema as required by metron-rest tests
Suppressed Apache directory log warnings when running metron-rest tests

Testing
All metron-rest unit and integration tests should continue to pass.
Other Considerations
After switching the metron-rest tests to use LDAP through metron-ui-security, I noticed a significant number of LDAP-related log warnings when running the integration tests.  As far as I can tell they are caused by missing LDAP attributes and similar issues that we probably don't care about.  I can also see the argument for wanting these logs enabled to make it easier for future contributors to find and diagnose problems when working on these tests (although it's easy enough to update the log4j.properties file if needed).  Thoughts?
Pull Request Checklist
Thank you for submitting a contribution to Apache Metron.
Please refer to our Development Guidelines for the complete guide to follow for contributions.
Please refer also to our Build Verification Guidelines for complete smoke testing guides.
In order to streamline the review of the contribution we ask you follow these guidelines and ask you to double check the following:
For all changes:

[ ] Is there a JIRA ticket associated with this PR? If not one needs to be created at Metron Jira.
[ ] Does your PR title start with METRON-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
[ ] Has your PR been rebased against the latest commit within the target branch (typically master)?

For code changes:

[x] Have you included steps to reproduce the behavior or problem that is being changed or addressed?

[x] Have you included steps or a guide to how the change may be verified and tested manually?

[x] Have you ensured that the full suite of tests and checks have been executed in the root metron folder via:
mvn -q clean integration-test install && dev-utilities/build-utils/verify_licenses.sh

[x] Have you written or updated unit tests and or integration tests to verify your changes?

[x] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?

[ ] Have you verified the basic functionality of the build by building and running locally with Vagrant full-dev environment or the equivalent?

For documentation related changes:

[ ] Have you ensured that format looks appropriate for the output in which it is rendered by building and verifying the site-book? If not then run the following commands and the verify changes via site-book/target/site/index.html:
cd site-book
mvn site

Note:
Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.
It is also recommended that travis-ci is set up for your personal repository such that your branches are built there before submitting a pull request.

+1 Looks good, thanks


Massive performance regression when 3D physics are used. 90 percent FPS drop.

Godot version:
3.2 Dev Calinou Hash 08081b3f6
OS/device including version:
Win10 64 bit, Ryzen 7 2700X CPU, 32 gigs RAM, Nvidia 1060 6GB GPU.
Issue description:
The big Bullet performance bug is back, the FPS in many cases will cap at 6 or 7 regardless of level complexity and optimization. The library was updated recently, so that commit could be the culprit.
Steps to reproduce:
It's the same project (but in .7z form since it's grown a bit since), Press shift ~ in the hub to see the issue. This was running at 60 FPS with 3.1.1 official.
Minimal reproduction project:
http://www.mediafire.com/file/lxm7dbcsgz227a8/Collect_The_Gems.7z/file

Okay, closing this until further notice, it's apparently not related to Bullet but something with the project itself. I'm not sure what though. Will reopen if it's a genuine problem.


Alex Bell’s srcset parser

https://github.com/baloneysandwiches/parse-srcset/issues/4

We still looking to get this in for 3.0 team?

Closing in favor of #322 actually.


History Network: Options to optimize for lower latency tranfers

In the history network we have three main pieces of content

Headers with proofs
Block Bodies
Receipts

Headers can be proven canonical by utilizing the proof included with them, on the other hand block bodies and receipts to validate these pieces of content we need to fetch the header with proof.
Currently when validiting the canonicalness of block bodies and receipts we request the header with proof sequencially after recieving the respective block bodies or receipts. This causes a delay to when users can access this data. The reason we choose this route originally was because it was the quicker way to get a live product, but moving into launching looking into these paths seems more beneficitial.
Solution 1
Request the header at the same time you request the block bodies or receipts.
Solution 2
The header with proof is very small almost always less then 1200 bytes why not include it with block bodies and receipts. This lowers the amount of uTP connections we have to open minimizing congestion

todo add pros and cons

Nice write-up!  I am thinking about a similar issue related to the state network as verifying an account/storage slot needs recursively lookup/verify the trie nodes (suppose an early version of Portal maintains a trie_hash => trie_node mapping).
One question is about the latency of the request that you observed?


swiftlint disable all

Hello
I added a parameter --swiftlint-disable-all for those who use Swiftlint and don't want to see warnings in the generated files

Thank you @happynik!


Support url with subfolder

Hi,
If app is deployed in subfolder of domain name (i.e. http://example.com/velos-paris), index cannot load asset (index look at http://example.com/_next ...).
When adding config with basePath, build automatically add prefix to links (https://nextjs.org/docs/api-reference/next.config.js/basepath).
Thomas

Better to have @Tristramg to verify that he can still deploy in production with this fix before merging that

Sorry for taking so long before replying…
It seems to work and breaking nothing if the variable is not set :+1:


Revise api

As discussed here
The main idea is that instead of passing an sqla_object, query, and columns, only pass the query and the columns. And instead of passing the columns by name, one can pass any SQLAlchemy column like expression, like func.upper(User.name).
An example:
Without datatables, a query can look like this:
query = self.session.query(
        User.id,
        User.name,
        User.birthday,
        func.date_part('year', func.age(User.birthday)),
    ).filter(User.id > 10)

result = query.all()

Currently, with sqlalchemy datatables there is no way to calculate the age on the fly using sql statements. In my implementation, you have to split the query and the columns and pass those to DataTables:
query = self.session.query().filter(User.id > 10)
columns = [
        ColumnDT(User.id),
        ColumnDT(User.name),
        ColumnDT(User.birthday),
        ColumnDT(func.date_part('year', func.age(User.birthday))),
        ]
rowTable = DataTables(request, query, columns)

If you check the code, you can see it is much cleaner, as dealing with joins, properties etc are made by sqlachemy. The result of the query is determined operating on the sqlalchemy expressions.
TODO:

add more tests
scrutinise new api
fix django and pyramid examples
update documentation
reimplement regex

Coverage decreased (-16.8%) to 70.37% when pulling fc18edde02a1c8c1dc499a12256647a8ebb1a88b on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage decreased (-16.8%) to 70.37% when pulling fc18edde02a1c8c1dc499a12256647a8ebb1a88b on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage decreased (-16.8%) to 70.37% when pulling 81377579be9377fdd276215ef2b48466f39b4c13 on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage decreased (-9.4%) to 77.714% when pulling fe0b85cefaef2338f2445921f714cb1d8832548e on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage decreased (-9.4%) to 77.714% when pulling fe0b85cefaef2338f2445921f714cb1d8832548e on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

As this became active again, there are few open issues and pull request which can be also looked right away. The one which I encountered were in regex/search while throws exception when seared for integer as the replace method don't exist

Coverage increased (+2.6%) to 89.773% when pulling 34cac09de518e3be51a7338ad95c348e9a7dbc33 on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage increased (+2.6%) to 89.773% when pulling 34cac09de518e3be51a7338ad95c348e9a7dbc33 on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage increased (+2.6%) to 89.773% when pulling c1329f0400b3b6b8c59efc6962d25b59429a300d on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage increased (+2.6%) to 89.773% when pulling c1329f0400b3b6b8c59efc6962d25b59429a300d on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

@devcorn, could you add some tests to illustrate the issue? I already put in code to cast values to string before doing a global search. For per column searches, using regexes on numerical values just feels wrong. I added a search_method='numeric' option which allows for searches as >=10 to return values greater than or equal to 10.
Alternatively, you can use the yadcf plugin in combination with search_method='yadcf_range_number_slider' to provide sliders for searching numerical data.

Coverage increased (+4.1%) to 91.257% when pulling 0f337c8a90c1a8fd83843854530b9f63e71f623f on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage increased (+4.1%) to 91.257% when pulling 0f337c8a90c1a8fd83843854530b9f63e71f623f on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage increased (+4.1%) to 91.257% when pulling 9e00b10c350b1e86c0d272b63c1900df934fa85d on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage increased (+4.1%) to 91.257% when pulling 9e00b10c350b1e86c0d272b63c1900df934fa85d on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage increased (+4.1%) to 91.257% when pulling 9e00b10c350b1e86c0d272b63c1900df934fa85d on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

Coverage increased (+4.1%) to 91.257% when pulling 69e00f12fd3833b6d1992031eea0757debc6f2fb on tdamsma:revise-api into 096ace624a3ca68f2d4aad6721547777ed38a98e on Pegase745:master.

@Pegase745 Even though there are some todo's left, I think the PR is ready to be merged. Perhaps you can have look

@tdamsma Great job indeed. I'll take a deeper look tonight if you won't mind and merge the whole thing :+1:

@tdamsma done. Could you just update the README's example or at least an example here if you have the time please ?


Fix Google Font link for Courier Prime

Removed bad link to Inconsolata and updated with the correct one

thanks!


Build is failing

[x] Review the documentation: https://github.com/oblador/react-native-vector-icons
[x] Search for existing issues (including closed issues): https://github.com/oblador/react-native-vector-icons/issues

Description
Fresh react native installation, followed instructions from http://react-native-material-
design.github.io/installation
I did click on the link which leads to this repository and added the
project.ext.vectoricons = [
iconFontNames: [ 'MaterialIcons.ttf', 'EvilIcons.ttf' ] // Name of the font files you want to
apply from: "../../node_modules/react-native-vector-icons/fonts.gradle"`
to the start of the file android/app/build.gradle
Now my build fails with the following:
error: bundling failed: SyntaxError: C:\Users\michael\projects\ReactNativeTest\node_modules\react-native\scripts\node_modules\react-native-material-design\node_modules\react-native-vector-icons\lib\icon-button.js: Rest element must be last element (60:16)
58 |        iconStyle,
59 |        children,

60        |...props,
|              ^
61 |            } = this.props;
62 |
63 |       let iconProps = pick(props, Object.keys(Text.propTypes), 'style', 'name', 'size', 'color');
at Object.raise (C:\Users\michael\projects\ReactNativeTest\node_modules@babel\parser\lib\index.js:6344:17)

Odd, because it is the last element. Are you using any custom babel plugins? Does this happen on a fresh project?

I just tried to reproduce this problem with a fresh project.
Vector icons alone works.
https://github.com/xotahal/react-native-material-ui which uses vector icons as a dependency also works.
http://react-native-material-design.github.io which uses vector icons as a dependency and which I initially used does not work, it produces the above error

I too get the same issue while using react-native-material-ui which installs react-native-material-design as a dependency and react-native-vector-icons as a dependency. The exact error goes like this:
node_modules\react-native-vector-icons\lib\icon-button.js: Line 60, Column 16: Transformation Error:
REST element must be last element

Seems this is an issue specific to the RNMD library. One suspect thing is that RNVI is not hoisted, so maybe the RNMD ships their .babelrc in the production bundle and that affects how babel transforms RNVI since it's not hoisted. I'd try to use yarn why react-native-vector-icons to see if you have conflicting dependencies, and maybe add a forced resolution.

Doesn't seem like a conflict though. Here's what I got from yarn why:
`=> Found<EMAIL_ADDRESS>info Reasons this module exists

"react-native-material-design" depends on it
Hoisted from "react-native-material-design#react-native-vector-icons"
Done in 16.67s.`

Well, this path looks really strange:
C:\Users\michael\projects\ReactNativeTest\node_modules\react-native\scripts\node_modules\react-native-material-design\node_modules\react-native-vector-icons\lib\icon-button.js

For one I don't understand why you have a node_modules folder in react-native/scripts. Have you tried nuking node_modules and running yarn again?

Your version is extremely old too, latest is 6.4.2

I tried npm update react-native-vector-icons but that gave me no response at all. Sorry if this was a beginner question. I have just started exploring react-native

For anyone else finding this issue, I had a similar problem with React and Docker Cloud. I noticed that there was a comma after my rest element, removed it, and it worked fine after that.
From
style, ..custom,}) to style, ..custom})


While updating Now CLI to 12.0.0-canary.80, npm fails at the postinstall step

I tried to install the new 12.0.0-canary.80 build of Now CLI, and the installation failed at this step:
$ npm install -g now@canary
/Users/realtable/.nvm/versions/io.js/v3.3.1/bin/now -> /Users/realtable/.nvm/versions/io.js/v3.3.1/lib/node_modules/now/download/dist/now

><EMAIL_ADDRESS>postinstall /Users/realtable/.nvm/versions/io.js/v3.3.1/lib/node_modules/now
> node download/install.js

/Users/realtable/.nvm/versions/io.js/v3.3.1/lib/node_modules/now/download/dist/download.js:8960
module.exports = (fp, opts) => {
                            ^^

SyntaxError: Unexpected token =>
    at exports.runInThisContext (vm.js:53:16)
    at Module._compile (module.js:413:25)
    at Object.Module._extensions..js (module.js:448:10)
    at Module.load (module.js:355:32)
    at Function.Module._load (module.js:310:12)
    at Module.require (module.js:365:17)
    at require (module.js:384:17)
    at Object.<anonymous> (/Users/realtable/.nvm/versions/io.js/v3.3.1/lib/node_modules/now/download/install.js:16:1)
    at Module._compile (module.js:430:26)
    at Object.Module._extensions..js (module.js:448:10)
npm ERR! Darwin 16.7.0
npm ERR! argv "/Users/realtable/.nvm/versions/io.js/v3.3.1/bin/iojs" "/Users/realtable/.nvm/versions/io.js/v3.3.1/bin/npm" "install" "-g" "now@canary"
npm ERR! node v3.3.1
npm ERR! npm  v2.14.3
npm ERR! code ELIFECYCLE

npm ERR<EMAIL_ADDRESS>postinstall: `node download/install.js`
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>postinstall script 'node download/install.js'.
npm ERR! This is most likely a problem with the now package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     node download/install.js
npm ERR! You can get their info via:
npm ERR!     npm owner ls now
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /Users/realtable/Documents/Other/hitsuji/npm-debug.log

The npm-debug.log is at this Gist:
https://gist.github.com/realtable/6526f3a8cf8ee2d93315bf9ff0276153
Does anyone know how to fix this?

As it turns out, I got it working using node.js instead of io.js.


Updating to 2.4.5 from 2.3.4 DeprecationWarning: Tapable.plugin

Version
v2.4.5
Steps to reproduce
I have updated my app from 2.3.4 to 2.4.5 version of nuxt. And I am taking this error when dev server is building nuxt.client.js.
Complete error text is:
(node:61302) DeprecationWarning: Tapable.plugin is deprecated. Use new API on .hooks instead
It resumes building and completes with a compilation warning which is:
chunk pages/product/_id.pages/profile/_id [extract-css-chunks-plugin]

Conflicting order between:
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-1-2!./node_modules/vue-loader/lib/loaders/stylePostLoader.js!./node_modules/postcss-loader/src??ref--9-oneOf-1-3!./node_modules/sass-loader/lib/loader.js??ref--9-oneOf-1-4!./node_modules/vue-loader/lib??vue-loader-options!./components/cards/SearchResultProductCard.vue?vue&type=style&index=0&id=08d7c683&scoped=true&lang=scss&
 * css ./node_modules/css-loader/dist/cjs.js??ref--9-oneOf-1-2!./node_modules/vue-loader/lib/loaders/stylePostLoader.js!./node_modules/postcss-loader/src??ref--9-oneOf-1-3!./node_modules/sass-loader/lib/loader.js??ref--9-oneOf-1-4!./node_modules/vue-loader/lib??vue-loader-options!./components/modals/ProfileNewListModal.vue?vue&type=style&index=0&id=41f29e2c&scoped=true&lang=scss&

After build complete when i enter localhost:3000 i get ReferenceError: regeneratorRuntime is not defined.
What is expected ?
Expected to build and compile without any errors or warnings.
Additional comments?
I have updated my plugin names and nuxt.config file as in the official nuxt documentation suggest before update.
I have created a new nuxt app with yarn create nuxt-app <app_name> command and did not edit anything but still getting the same DeprecationWarning error and when i check the browser after, got this error:
NuxtServerError: Cannot find module 'pages/index.js' from '...app/folder/nuxt25-test'

This bug report is available on Nuxt community (#c8754)

Try npm-check-updates
https://www.npmjs.com/package/npm-check-updates

Okay. you are mixing up three errors here.

(node:61302) DeprecationWarning: Tapable.plugin is deprecated. Use new API on .hooks instead

Have you tried googling it? nuxt DeprecationWarning will lead the correct result 🤷♂
https://github.com/nuxt-community/pwa-module/issues/120

chunk pages/product/_id.pages/profile/_id [extract-css-chunks-plugin] Conflicting order between

Again, google. https://github.com/webpack-contrib/mini-css-extract-plugin/issues/250#issuecomment-415345126

NuxtServerError: Cannot find module 'pages/index.js' from '...app/folder/nuxt25-test'

#5132

experiencing the same issue in my Nuxt project
<EMAIL_ADDRESS>dependences<EMAIL_ADDRESS>but<EMAIL_ADDRESS>has error, like this issues-->extract-css-chunks-webpack-plugin, issues-->Nuxt
so, you can set<EMAIL_ADDRESS>dependences<EMAIL_ADDRESS>i use yarn, so i soloved it by change yarn.lock dependences<EMAIL_ADDRESS>


As a developer, I want to ensure the integration process is seamless and doesn't introduce any conflicts or issues with the existing codebase.

Checking for any conflicts in the codebase is in progress.

codebase conflicts have been resolved


Improve definition of trailing vs leading comments

This PR changes the definition of trailing comment to be a comment that occurs after a non-whitespace token and before a newline. All other comments are now leading comments.
Previously, a trailing comment was one that occurred after a non-whitespace token and before another token, or before a newline. All other comments were leading comments, but this set was very hard to characterize.
Some examples:
/*1*/ x /*2 */, /*3*/ y /*4*/ // 5
// 6
z
// 7

Leading comment of 'x'
Leading comment of ','
Leading comment of 'y'
Trailing comment of 'y'
Trailing comment of 'y'
Leading comment of 'z'
Leading comment of end-of-file

I needed this work for the new-improved JSDoc parser that I'm working on -- comments like /*3*/ need to be leading comments, never trailing ones. Unfortunately, this led to a lot of noise in the old emitter. But the baselines are mostly improved — they pull in a lot of previously-missed comments. @rbuckton or @yuit maybe you have some suggestions about the emitter code?

This is no longer needed for the improved jsdoc parser, so I'm closing it.


/workspace/container/docker-shopware/public is not shared from OS X

On OSX (EL Capitain) with Docker 1.13.0 and docker-compose 1.10.0  i'll get this error with "docker-compose up --build":

ERROR: for data  Cannot start service data: Mounts denied: ces for more info.
.
/workspace/container/docker-shopware/public
is not shared from OS X and is not known to Docker.

Is there any chance to use this container on OSX/MacOS or at least to adjust this path in .env or somewhat else?

We had huge performance issues with this docker-setup (due to NFS shares) - so we lately switched to a vagrant-based setup which is way faster. Please have a look at shopware-vagrant as we might discontinue this project.

Thx for clarifying. We've ended up also with shopware-vagrant.
However, you should mark this repo as "discontinued" :)


Extend cosmwasm-schema to be able to annotate the resultant JsonSchemas with Parameter/ReturnType/State tags. (Or similar)

Hi, This both a proposal and a question.
The problem: i'm trying to generate typescript bindings for cosmwasm contracts to speed up the development with our js package. In the ideal world, query and execute methods along with types would be transpiled directly from rust contract to typescript and that'd be the end of story. Looking at this bit currently.
In the short term though it'd be really nice to have the clarity as to which generated .json type corresponds to a method's parameter and which -- to its return type.
Currently
In terms of the cw-template and in the most basic case this would work as follows:
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq, JsonSchema)]
#[serde(rename_all = "snake_case")]
pub enum QueryMsg {
    GetCount {},
}

yields the following schema
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "QueryMsg",
  "anyOf": [
    {
      "type": "object",
      "required": [
        "get_count"
      ],
      "properties": {
        "get_count": {
          "type": "object"
        }
      },
      "additionalProperties": false
    }
  ]
}

Desirable
With "param/return type method tags" or whatever this might instead look like the following:
#[derive(Serialize, Deserialize, Clone, Debug, PartialEq, JsonSchema)]
#[serde(rename_all = "snake_case")]
#[ XXX (method=query, "parameter")]             // <---- tag this (maybe something inside schemars or serde)?
pub enum QueryMsg {
    GetCount {},
    GetOwner{}
}

yields the following schema
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "title": "QueryMsg",
  "myCustomTag": {"typeSet": "query", "role":"parameter"},               // <--- now i know this type belongs to the query set
  "anyOf": [
    {
      "type": "object",
      "required": [
        "get_count"
      ],
      "properties": {
        "get_count": {
          "type": "object"
        }
      },
      "additionalProperties": false
    }
  ]
}

This is just one possible implementation, probably with lots of edge cases down the road -- i'd really love to hear what you guys think to this end. Perhaps this is better done as a custom schemars trait or a macro or a serde injection...
Implementation details aside, i think it'd be useful to have this additional reference/link between the schema files because the space of things that can be reconstructed (in typescript or other langs) from them right now is limited by its lack.
In general, i'd be very happy to contribute to any transpilation effort here. I'm mainly involved with the Terra dev ecosystem, but i wonder if this can speed up the development cycles for any cosmos-based network and their sdks.
Seeing as @webmaster128  wrote this bit -- would like to refer to your expertise on this. Please advise!

Here is something from JsonSchema spec.

A few other things that can maybe be tied into this:

producing schemas for the Error types and tying them into the same overarching structure ( not yet clear what that is ).
deriving a custom Response schema for each method with its particular set of SubMsgs: Attributes,CosmosMsgs, Events etc. I'm still very much making sense of how it is put together, but it seems like all those sub-messages have JsonSchema already #[derived.

In particular:

* deriving a custom [Response](https://github.com/CosmWasm/cosmwasm/blob/main/packages/std/src/results/response.rs) schema for each method with its particular set of `SubMsgs`: `Attribute`s,`CosmosMsg`s, `Event`s etc.

I think this can be done as a procedural macro.
I.e. for a given cosmwasm contract method, say try_modify_state_x which returns Response with 2 attributes, the macro would document the arguments and ab object with 2 fields in schema/try_modify_state_x.json. I'll try to work out the assumption that both the args and the response's sub-messages are serializable.
Again, motivation being: it seems like a lot of data returned by a given contract that is integrations-relevant is Response decorated with a particular set of sub messages. So, as far as all of the sub-messages for a particular Response are serializable (have the correct traits derived) the method can readily provide the JSONSchema for what it returns to be used in SDKs and off-chain integrations etc.

hey @rtviii by chance are you using https://github.com/pyramation/cosmwasm-typescript-gen — if so, I think what you may be looking for (as a temporary solution) is what we're doing here https://github.com/public-awesome/stargaze-contracts/blob/main/contracts/sg721/examples/schema.rs#L20-L39

@pyramation love your work with ast's & macros in that package and postgres(unrelated). been meaning to take a look at the cosmwasm-typescript-gen for over a week now ever since somebody on the team brought it up. thanks a ton for putting in the work.

@uint can you confirm if the core of this request is done in https://github.com/CosmWasm/cosmwasm/pull/1339? If yes, please close. Follow-up tickets can be created for anything that is left on the wishlist.

@uint can you confirm if the core of this request is done in #1339? If yes, please close. Follow-up tickets can be created for anything that is left on the wishlist.

I think so, although we're doing it quite differently from what OP suggested.
From that wishlist, we don't (yet!) have a way of describing possible events a contract might emit, but it's going to happen.
For reference:

PRs #1332 and #1339 are relevant.
Issue #1338 is relevant.
An example of the format can be found here - note the root fields like query or instantiate are always the same (as opposed to arbitrary filenames enforced only by convention). Also note there's a responses field that maps every query to its response type.

I think we can close this and let @rtviii reopen if he doesn't feel we addressed his problem.

Awesome work!

amazing! 👏🏻👏🏻👏🏻


Adding boat goals

[ ] adding back button with link
[ ] aligning text font
[ ] adding two goals: problem with center alignment

I massaged a little bit the list to make it left aligned. @leewujung I will merge it for now.


Added Privates Gymnasium Marienstatt

Added my School
Website: https://www.gymnasium-marienstatt.de

Looks right

@Kohlibri01 First stage of verification passed, official domain for Privates Gymnasium Marienstatt is gymnasium-marienstatt.de as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@Kohlibri01 Thank you for your patience while waiting for a review and processing your request. Pull request merged.


Default value lookup

I'm going to drop this pr just to clarify what was exactly the change I needed for my case.
#60

This looks very reasonable to me

@gbtb I'd be good with pulling this in for the next version

Great! I undraft PR then.


OWD project: Organise Web APIs

Ideally Web APIs should be categorised. This will benefit visibility for users, but also help drive how we organise the standalone demos MDN has. Currently the majority of 'github' demos within the MDN org are Web API ones and they could do with an update & better architecture. However this proposal is the first part to categorisation, which is to start by moving interface docs under their relevant Web API.
For example:

Web Audio API (Parent folder)

AudioNode (Interface)

context
numberofinputs
...
connect()
disconnect()

There's a couple of caveats - some things just don't fit in this structure (like fetch) and possibly can be left out. Also probably worth a discussion is tutorial docs, which are usually under the main API folder.
One possible solution is:

Web Audio API (Parent folder)

Interfaces
Tutorials

It might be worth considering tutorials which cover more than one API - should these be a part of learn?
Effort
This should be just the first step to categorising Web APIs. However there would be a lot of moving of docs, so it's not a small effort
Dependancies
No changes to the platform should be needed
Community
One approach could be to raise an issue for each API detailing which docs need to be moved. This might encourage community help
Momentum
There's another proposal to create a searchable page for APIs. This is really the platform grounding to begin something like that
Enabling learners
Eventually a better way to discover APIs - however this is not covered by this first part
Enabling professionals
As above
Underrepresented topics / ethical web
N/A
Operational necessities
N/A
Addressing needs of the Web industry
Eventually be able to categorise Web APIs and mirror that with the external examples we have. Thus making it easier for users to discover both the docs & code examples.

You implied, but didn't say explicitly that this will change URLs. Does it?
What's the plan for keeping the URLs recorded in BCD in sync?

This was really a proposal which I want, but haven't thought too much about the details :)
In regards to bcd, this is data held in the frontmatter anyway, so there shouldn't need to be any change.

The MDN content team will be moving repos about in Q2 - originally I wanted to reorg the API docs to drive that work, but I've reversed my decision. Once we've tidied up the repositories containing code examples I'll come back to this.
Tl:dr; not a priority at this time and something the content team might take on moving forward 👍

I saw this yesterday and realised I had just the same thought the other day, so I'm writing it up here, to add a bit of detail and justifications to your proposal.

Problems
There are three problems with the Web/API docs that this proposal tries to address.
(For background on Web/API sidebars I wrote this: https://discourse.mozilla.org/t/defaultapisidebar-apiref-and-groupdata/40210 a long while ago.)
Web/API sidebars are complicated
The sidebar system for Web/API involves lots of moving parts and things to remember, and things very often go wrong.
You have to:

remember which sidebar macro to use (DefaultAPISidebar versus APIRef)
pass the correct argument to the sidebar macro
have exactly the right data in GroupData.json
have pages in the right places under top-level pages
have all the right tags (now obsoleted by page types)

In practice many people get some of this wrong, and the sidebar is then broken to some extent.
Web/API sidebars are too fragmentary
As said above, we have 2 main sidebar types (DefaultAPISidebar and APIRef). DefaultAPISidebar is for API overview pages and guide pages, while APIRef is for everything else, including interface pages. The system will produce:

a different sidebar for every API (including the overview page and guide pages)
a different sidebar for every interface (including the interface itself and its members)

Since we have about 100 APIs, and about 1000 interfaces, this means we have about 1100 different sidebars in Web/API.
For example, if I'm learning about Fetch, I might start at the main page (https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API, sidebar 1), click on Request (https://developer.mozilla.org/en-US/docs/Web/API/Request, sidebar 2), then on the link to Headers, (https://developer.mozilla.org/en-US/docs/Web/API/Headers, sidebar 3). That's three sidebars in three closely related pages, in quite a small API. I think it would be better if there were a single sidebar that stayed the same as I navigated around Fetch.
There are too many directories at the top level
This is really for maintainers, but it matters still. Currently all API overview pages and all interface pages are at the top level, under web/api: https://github.com/mdn/content/tree/main/files/en-us/web/api . This makes it really hard to work with, and hard for maintainers to see the relations between interfaces.
Solution
What if we did this:

under Web/API, have only "API Overview" (aka Specifications) pages, like https://developer.mozilla.org/en-US/docs/Web/API/Fetch_API
under each of these, have 2 subdirectories, "Guides" and "Interfaces"
under Guides, have the guides for this API
under Interfaces, have a page for each interface, and under there, pages for properties, methods, events, as we have now.

Like:
Web/API/
Fetch_API/
index.md
/Guides
/Using_the_fetch_api
index.md
/etc
index.md
/Interfaces
/Request
/index.md
/json
index.md
/etc
index.md
/etc
index.md
...so we have like 100 pages or so directly under Web/API, one for each API. And under there (almost) all the pages for that API.
Then sidebars are just built like: if you are at a page under Web/API, find your top-level directory (e.g. Fetch_API) and construct a sidebar by iterating through the pages under it. Every page in an API gets the same sidebar, there's no more DefaultAPISidebar/APIRef distinction. I think this is a good thing!
Two problems I can see:

guides want to appear in a curated order, not alphabetised. For this I think we should steal the concept of "weight" from Hugo and other SSGs. So you have a "weight" front matter item, and items with less weight appear higher in the list.
items which belong to different interfaces, like https://developer.mozilla.org/en-US/docs/Web/API/Navigator/getBattery. For this, if we can't get that info out of webref, we could just list the things in the API page front matter. I think items like this should appear in both sidebars, but when you are actually on the page, the sidebar should be from the API that contains the interface that owns the item.

So for instance, https://developer.mozilla.org/en-US/docs/Web/API/Battery_Status_API should list Navigator/getBattery(), but https://developer.mozilla.org/en-US/docs/Web/API/Navigator/getBattery should show the sidebar for the API that defines Navigator. Maybe. There are other options here too though, and maybe more problems?
This seems so simple I can't believe it would work. No GroupData, only one sidebar, no more getting the right argument to the sidebar, no more 1100 pages at the top level. So what's wrong with it?
One problem as @teoli2003 pointed out is that it basically destroys domxref, which depends on this very flat hierarchy. But I think we should stop using domxref
anyway.

I think this is a good plan.
I'm trying to think if there are any edge cases that wouldn't fit into this, but I'm coming up blank. If there are any, it would be good to deal with them now rather than waiting to stumble into them.
You should also have a plan for redirects from the existing structure. People have been linking to the existing structure for more than a decade. That's a lot of links to break in one go.

Thanks for the write up @wbamberg :) this was pretty much what I was thinking - I do think there could be a possibility something wouldn't fit with this organisation, but I can't think of it off the top of my head without looking.
It's a toughie with domxref - am i just used to it and when actually markdown links are (almost) as easy... I would forego the macro for this better organisation, especially when it improves other things such as sidebars.
It is a lot of moving... if we do plan to go ahead it would be good to let translated content know too I think

Is there really no way to make domxref work? This certainly improves things for readers. But if article locations are less predictable (and I think they are) we're going to be playing whack-a-mole with broken links.

Is there really no way to make domxref work?

It could be made to work in theory. AFAIK there's a global namespace for interfaces, so we know that {{domxref("Request")}} would only map to /API/Fetch_API/Interfaces/Request, and there can't be /API/Some_Other_API/Interfaces/Request.
But the implementation would be more complicated.
I'm not really a fan of our xref macros and would replace them all with Markdown links but I think I'm in a minority here.

I do think there could be a possibility something wouldn't fit with this organisation

One issue would be globals, which we recently moved out of interfaces and into the top level, like https://developer.mozilla.org/en-US/docs/Web/API/fetch. But I've started to think this might have been a mistake, because there isn't a single global scope, there are several: window scope, worker scope, service worker scope, etc. And we don't represent that at the moment.

Ideally Web APIs should be categorised. This will benefit visibility for users, but also help drive how we organise the standalone demos MDN has.

tl;dr I don't think categories/groups are as important to our audience. The ability to find objects, events, methods, properties, is important, however.
One of the strengths of MDN (vs the specifications, for example) is that MDN merges specifications into a single place. As a web developer, I love this. I'm not so much interested in how the Web API surface is grouped, I don't need to know API modules or groups necessarily. I like the docs, because don't need to browse 500+ specifications (or groups) and collect for myself from various places what the whole API surface has to offer me. It is an abstraction that is often not relevant to me. If I do DOM stuff, I interact with various objects, events, methods, properties that are presented to me. I don't actively choose to only interact with DOM events, or with the HTML DOM, I interact with all of this and I need to see how it plays together with more objects that I stumble upon. If I see a new object that is unknown to me, I don't need to know where it belongs to. I will browse the docs at the familiar URL and then that page fills me in on the broader context of this object and guides me to concepts and the API group this object belongs to. So, to me the categorization is for creating context for the reader, but the category is a helper for that, it doesn't come first necessarily.
tl;dr grouping is hard, group assignments change. quite a few groups is groupdata.json make no sense or overlap.
Coming up with groups that make sense for web developers isn't trivial and the groups might change over time. Some things move from one spec to another and if we tight APIs more strictly to groups, we will sure end up moving things around more as they make more sense with new specs the web platform adds.
There is also often overlap, or you have to decide how small you make a group. WebXR has like 20 specs, we would surely not create 20 groups there. You can't say one group per spec, I don't think. How do we group? is this an editorial choice?
And what about Document, Window, navigator? They are like infrastructure interfaces extended by many API groups. Would you want to fragment the docs of these interfaces by group? /API/Fetch_API/Window/fetch and /API/XHR_API/Window/xhr
If I take a few random specs, I can either group broadly or quite granular. It's not clear to me how I would do it:
Fetch
BackgroundFetch
Request
Response
Push
Worker
SharedWorker
ServiceWorker
Canvas
WebGL
WebGL2
WebGPU
WebXR
DOM core
DOM events
HTML DOM
I also don't like that the api/ namespace is polluted with basically everything. If I would re-do it, I would try to organize it by things. Like api/interfaces/*. Sure, that would be a huge folder still, but then it is clear what goes there (defined by WebIDL).
Conceptual material, like a guide that explains how 10 interfaces and a property on window grouped together make you get a job done around network stuff (fetch API, XHR), can live somewhere else like api/networking/bla, or even in the learning area or so.
And sidebars are helpers to let you find context, too. And alphabetical list of all interfaces isn't useful, of course. In sidebars you also want to present relevant related interfaces, events, etc.
The current sidebar implementation with hand-maintained groups does a decent job, but of course it is hard to maintain and has all the problems that Will outlines.

The sidebar system for Web/API involves lots of moving parts and things to remember, and things very often go wrong.

True and I'm afraid that we bring these problem to the page structures itself if we basically introduce similar grouping ideas there.
So, I think, I'd prefer organizing API pages by type (per WebIDL), have guide material separately, and have a sidebar implementation that can work with a guaranteed structure (by type) and additional input (e.g. from webref) to generate useful links to related interfaces, properties, etc.

I also don't like that the api/ namespace is polluted with basically everything. If I would re-do it, I would try to organize it by things. Like api/interfaces/*. Sure, that would be a huge folder still, but then it is clear what goes there (defined by WebIDL). Conceptual material, like a guide that explains how 10 interfaces and a property on window grouped together make you get a job done around network stuff (fetch API, XHR), can live somewhere else like api/networking/bla, or even in the learning area or so.

Note that we could move the overview pages (xxxy_API) from Web/API to Web/API/Overview to do 80% of such a cleaning (there would be only a few global properties/methods in it, and a few dictionaries.

I've been considering the comments here for a while now and I think this requires more discussion before moving forward

Closing this as not planned (for the moment). Maybe we can talk about this at the proposed Mozilla Content Workshop (tentatively planned in February 2024), but I think we can create a new OWD project if we actually identify a project for OWD out of that workshop.
Also, there is now a idea of feature groups with the web-features project (which OWD contributes to, see https://github.com/openwebdocs/project/issues/169)


Fix format of the Python example of OpenFeature in README

Just a simple doc fix

Oh good catch @ChihweiLHBird.
Thanks a lot for your contribution.


feat: Add support for Google Login

This pull request introduces Google Login to limit access to the tech-radar. The login is pinned to extendaretail.com. A new backend server has been introduced to validate JWTs on the server-side. The application validates that the token is a valid Google token and that the hosted domain in the JWT is extendaretail.com.
With this change, we can run the radar on a publicly available URL as is the case with Cloud Run.

Related to #83.


Monitoring: Send error objects to Rollbar separately so it can serialize

JS properties in Error objects are non-enumerable by default, so {error, moreInfo} means Rollbar serializes the error as {}.  Update the way we call to keep these separate params.

selfie


Integration of files bundle

Use case

User creates a record. The response contains link in json serialization to bucket /records/1/files/.
User uploads a file to the bucket via PUT (binary stream)
User can now view the files in the API:

Records serialization
Bucket endpoint

User can now view files in the UI:

Record landing page should list files
Landing pages should include a preview.

Tasks (incomplete):

[x] Bucket should be created when the record is created (internally there should be a nice error message when a location is missing)
[x] Record serializers needs to be changed to include files (from _files) as well as a link to the bucket in links. (https://github.com/inveniosoftware/cookiecutter-invenio-instance/issues/172)
[ ] Example documentation (for inveniosoftware.org and invenio quickstart needs to be created) (already defined in https://github.com/inveniosoftware/invenio/issues/3968)
[x] Templates for records landing page must be created, that list the files and includes a previewer. (already defined in #175, #173)
[x] Default location has to be created during setup (#165)
[x] JSONSchema and ES Mappings have to support _files property and probably a _bucket property. (blocked by https://github.com/inveniosoftware/cookiecutter-invenio-instance/issues/177, already defined in https://github.com/inveniosoftware/cookiecutter-invenio-instance/issues/174)

The third bullet is done. I think it can be closed.


requirements.txt looks skinny

It's great that exact versions are provided, but are you sure we have everything there? I am missing jupyter for instance.

Not urgent, but something to check. ádám, you'll be in charge...

pip freeze might be of help

this is done in https://github.com/gabors-data-analysis/da_case_studies/commit/608a500b165b34636463a00140ec8b47d67face3
pipreqsnb helped, as pip freeze checks all my installed packages


Tensorflow Quantiser - Issue with Input shape

Hi, I am trying to use the vai tensorflow quantiser to quantise a custom model consisting of fully connected layers. The model has multiple input nodes whose shapes are 2 dimensional (like ?,400 : ?,100 : ?,100).
I have tried using both tensorflow and tensorflow2 quantisers.

vai_q_tensorflow issue: Accepts only 4 dimensional inputs
vai_q_tensorflow2 issue: Accepts 2 dimensional input but only accepts a single input node

I have ensured that I am using the updated versions of all tools in Vitis AI. I am using the vai1.2 version from Vitis AI 1.4. I would like to know if there is any way to quantise a model which has multiple 2-dimensional input nodes which are passed on to dense layers consisting of 2D weight matrices, like in the example mentioned above.
Issue #57 refers to the input shape issue, but I am unable to pass on 2D inputs in the latest version as well.
Thanks!

Hi @kavyab11 ,
the dpu input node must be 4 dimensional,  so maybe you can add some expand, squeeze or reshape op to operate the input tensor or tensor before enter "input_node".

Closing since no activity for more than 1 year, please open a new issue if you still have any questions, thanks.


Exception during deserialization

Acknowledgement

[X] I have read Getting-Started and FAQ

🐛 Describe the bug
I tried to create my own voicebank, but when installing in openutau, a deserialization error appeared, everything seems to be correct with utau files
Explains how to reproduce the bug
https://github.com/user-attachments/assets/00d05757-db86-416c-8cb9-83213565901b

Install OpenUtau from aur package openutau-bin
Open application
Tools > Install singer
Navigate to directory containing compressed file

OS & Version
Windows 10, 64 bit
Logs
2024-12-20 15:29:16.539 +03:00 [INF] Logging initialized.
2024-12-20 15:29:16.643 +03:00 [INF] Microsoft Windows NT 10.0.19045.0
2024-12-20 15:29:16.644 +03:00 [INF] Microsoft Windows 10.0.19045 X64 X64
2024-12-20 15:29:16.645 +03:00 [INF] OpenUtau v0.1.529.0 win10-x64
2024-12-20 15:29:16.645 +03:00 [INF] Data path = D:\игры\OpenUtau
2024-12-20 15:29:16.645 +03:00 [INF] Cache path = D:\игры\OpenUtau\Cache
2024-12-20 15:29:16.896 +03:00 [INF] Initializing application.
2024-12-20 15:29:16.969 +03:00 [INF] Initializing culture.
2024-12-20 15:29:17.020 +03:00 [INF] Initialized culture.
2024-12-20 15:29:17.020 +03:00 [INF] Initializing theme.
2024-12-20 15:29:17.028 +03:00 [INF] Initialized theme.
2024-12-20 15:29:17.028 +03:00 [INF] Initializing OpenUtau.
2024-12-20 15:29:17.033 +03:00 [INF] Searching singers.
2024-12-20 15:29:17.067 +03:00 [INF] Search all plugins: 00:00:00.0327661
2024-12-20 15:29:17.067 +03:00 [INF] Search all legacy plugins: 00:00:00.0002523
2024-12-20 15:29:17.069 +03:00 [INF] Initialized OpenUtau.
2024-12-20 15:29:17.069 +03:00 [INF] Initializing audio.
2024-12-20 15:29:17.069 +03:00 [INF] ReleaseSourceTemp D:\игры\OpenUtau\Cache
2024-12-20 15:29:17.198 +03:00 [INF] Initialized audio.
2024-12-20 15:29:17.198 +03:00 [INF] Initialized application.
2024-12-20 15:29:17.198 +03:00 [INF] Framework initialization completed.
2024-12-20 15:29:17.245 +03:00 [INF] Creating main window.
2024-12-20 15:29:17.411 +03:00 [INF] Initialized main window component.
2024-12-20 15:29:17.514 +03:00 [INF] Main window checking Update.
2024-12-20 15:29:17.515 +03:00 [INF] Created main window.
2024-12-20 15:29:18.645 +03:00 [INF] Checking update at: https://github.com/stakira/OpenUtau/releases/tag/build/0.1.529
2024-12-20 15:29:18.645 +03:00 [INF] Checking appcast: https://github.com/stakira/OpenUtau/releases/download/build/0.1.529/appcast.win-x64.xml
2024-12-20 15:29:19.918 +03:00 [INF] Search all singers: 00:00:02.8854471
2024-12-20 15:29:19.989 +03:00 [INF] Publish notification Load project
2024-12-20 15:29:28.370 +03:00 [ERR] Failed to install singer D:\Загружалки wwwwwwwwwwwww\鶏音えま.zip
(Line: 1, Col: 1, Idx: 0) - (Line: 3, Col: 15, Idx: 40): Exception during deserialization

What files are there in the voicebank zip? Does it contain a character.yaml? if it contains, please upload it here.

鶏音えま.zip


Parse Error

Receive an error when using yaml2json to parse the following. Online parsers work fine. Cannot find any similar issues online. I also tried a yaml file that had the strings quoted but got the same issue.
YAML:

---
  foo: bar
  baz:
    - qux
    - quxx
  corge: null
  grault: 1
  garply: true
  waldo: "false"
  fred: undefined

CLI
$ yaml2json ./example.yml

CONSOLE ERROR
/Users/user/.nvm/v0.10.29/lib/node_modules/yaml2json/node_modules/yaml/lib/yaml.js:179
  throw new Error(msg + ', ' + context(this.peek()[1].input))
                                                  ^
TypeError: Cannot read property '1' of undefined
    at Parser.expect (/Users/user/.nvm/v0.10.29/lib/node_modules/yaml2json/node_modules/yaml/lib/yaml.js:179:51)
    at Parser.parseDoc (/Users/user/.nvm/v0.10.29/lib/node_modules/yaml2json/node_modules/yaml/lib/yaml.js:266:8)
    at Parser.parse (/Users/user/.nvm/v0.10.29/lib/node_modules/yaml2json/node_modules/yaml/lib/yaml.js:232:19)
    at Object.exports.eval (/Users/user/.nvm/v0.10.29/lib/node_modules/yaml2json/node_modules/yaml/lib/yaml.js:384:46)
    at handleInput (/Users/ben/.nvm/v0.10.29/lib/node_modules/yaml2json/index.js:31:17)
    at /Users/user/.nvm/v0.10.29/lib/node_modules/yaml2json/index.js:56:9
    at fs.js:271:14
    at Object.oncomplete (fs.js:107:15)

A coworker (a Windows user) found that yaml files using CR/LF line endings cause yaml2json to fail spectacularly. I'm a bit nonplused that line endings actually have any effect on the conversion. Or does YAML require Unix line terminators?


Force refresh of IntelliSense markings after adding/disposing "addExtraLibs"

Suppose I have a running Monaco editor.  On some external event (button press, etc.), I want to refresh the default libraries that I reference
monaco.languages.typescript.typescriptDefaults.addExtraLib(response.data, response.url);

Likewise, I may want to dispose of existing previously-added libraries, only adding the new libs.
Right now, when I dispose any s, the Editor does not get immediately updated.  For example, if I had object XYZ defined already, then dispose and add new libraries, the code editor makes the XYZ object look as "any" on mouse-over, but nothing else.  Only on touching the editor (e.g.: adding a new line) does the IntelliSense re-scan get triggered, and now I see the new squiggle lines.
I tried calling this._monacoEditor.render(); after adding the libraries, but it doesn't seem to make any difference.
monaco-editor npm version: 0.6.1
Browser:  Chrome
OS:  Windows

Perhaps the DiagnostcsAdapter should loop over all models again in case of options changed.

We now dispose and recompute all diagnostics on change of any of the default options


🌐 Add Japanese translation for docs/ja/docs/deployment/https.md

This PR translates deployment/https.md in Japanese.
To all reviewers. This section is only separated from #1948 (with few changes), but please review it.

Thank you @sh0nk !
I accepted your suggestions. And fixed some weird translation.
could you check it again?

Thank you @sh0nk !
I accepted your suggestions. And fixed some weird translation.
could you check it again?

📝 Docs preview for commit 846636fcfad5e666cbac22bbe12ccd57b49b3de7 at: https://649a13ba443f2f103c861546--fastapi.netlify.app

It seems this was handled in https://github.com/tiangolo/fastapi/pull/10298, so I'm gonna close this one. Thank you! ☕


Catalina 10.15 Beta

Catalina 10.15 Beta
my log id: 7b2a4a4dd1e53a0c9ff2cb1c

Catalina 10.15 Beta 19A536g
my log id: 271bad7a6f7e3cbe7ca25e1c

Catalina support will be arriving in the next release, hopefully out next week sometime.

Keybase 4.4.0 (out today) has better file system support for Catalina now.  Please let us know if you have more problems with it.  Note that the /keybase mount point is no longer viable on fresh installs of Catalina due to macOS namespace restrictions, so now we mount at /Volumes/Keybase if /keybase does not already exist.

Works for me now, thanks!
On Tue, Sep 3, 2019 at 6:16 PM Jeremy Stribling<EMAIL_ADDRESS>wrote:

Keybase 4.4.0 (out today) has better file system support for Catalina now.
Please let us know if you have more problems with it. Note that the
/keybase mount point is no longer viable on fresh installs of Catalina
due to macOS namespace restrictions, so now we mount at /Volumes/Keybase
if /keybase does not already exist.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/keybase/client/issues/18833?email_source=notifications&email_token=AAHZVOX2FUDFEVVM4VSRJWDQH3O45A5CNFSM4IKJUEDKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD5ZXYWI#issuecomment-527662169,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAHZVOX26EQ2V3DIPD6ZPRLQH3O45ANCNFSM4IKJUEDA
.

--
"Edward G Prentice"<EMAIL_ADDRESS>My Public
https://keys.mailvelope.com/pks/lookup?op=get&search=0xCB496336B351E045
PGP Key


[WIP] Display current votes as GitHub status

Todo:

[x] Optimise loops (it runs through some PRs twice now)
[x] Add hours left in status description?
[x] Add votes count?

Any help/input is appreciated.
See https://developer.github.com/v3/repos/statuses/ for API reference
Demo here: https://github.com/hongaar/chaos/pull/1
PR overview:

PR detail:

Will this post duplicate comments?

It doesn't seem to post comments.

@rudehn No, it posts only a new PR status, which can contain a description:

With latest update:

looking good @hongaar !

Closing this in favour of #148


The Error of encryption algorithms in different goroutine？

When I put many Encryption Algorithm to goroutines, I get the following error. So how to use multi-threading to Encrypt data without using a Mutex?
plainSlice := make([]*ckks.Plaintext, 0)
cipherSlice := make([]*ckks.Ciphertext, 0)
var wg sync.WaitGroup
wg.Add(5)
for i := 0; i < 5; i++ {
	var cipher *ckks.Ciphertext
	cipherSlice = append(cipherSlice, cipher)
	text := encoder.EncodeSlotsNew([]float64{55}, params.MaxLevel(), params.DefaultScale(), params.LogSlots())
	plainSlice = append(plainSlice, text)
}
for i := 0; i < 5; i++ {
	go func(num int) {
		cipherSlice[num] = encryptor.EncryptNew(plainSlice[num])
		wg.Done()
	}(i)
}
wg.Wait()
for i := 0; i < 5; i++ {
	result := decryptor.DecryptNew(cipherSlice[i])
	output := encoder.DecodeSlots(result, params.LogSlots())
	fmt.Println(output[0])
}

Lattigo is not thread safe. You have to call .ShallowCopy on elements like Encryptor or Evaluator to create a threadsafe copy of the target element, that will re-allocate its internal buffers. Then for each go routine after the first one you use a different shallow copy.

Lattigo is not thread safe. You have to call .ShallowCopy() on elements like Encryptor or Evaluator to create a threadsafe copy of the target element, that will re-allocate its internal buffers. Then for each go routine after the first one you use a different shallow copy.

Ok, thankyou! I will have a try.


build: release on pypi

After a successful release, the package should be published to pypi.org to make it available to users.

please consider #20 first

we should also adpat the readme regarding additional chapter to easy to use installation


fix(4.x): migrate from optimist to yargs

Follow-up on #1662, added a few unit test on top of the original PR.
closes #1658
closes #1661
Note that there are not yet tests for all command line options. I know too little about handlebars yet (it's just a sub-sub-dependency of a package I use). If someone has good suggestions on what makes sense to test, fire away, I will update the PR.
Since I use npm audit to fail my build to not get vulnerable packages get released to production, I'm very interested into getting this merged.
Tested:

[x] -f
[ ] --map
[x] -a
[ ] -c
[ ] -h
[ ] -k
[ ] -o
[ ] -m
[x] -n
[x] -s
[ ] -N
[ ] -i
[ ] -r
[ ] -p
[ ] -d
[ ] -e
[ ] -b
[ ] -v
[ ] --help

I recommend looking at yargs unit tests and maybe mimicking what is done there to test handlebars cli options.
@fabb, happy to help on this, can I get permissions to your fork?

Seems like the unit testing structure in place will make this some what difficult.
Will try to leverage the current unit testing structure to get this done.

@aorinevo I invited you to this repo, feel free to commit to this branch, thanks for offering! I‘ll probably only get to continue on this in 2 days. I‘ll then have a look at what you‘ve done in the mean time, and at the yargs tests as you suggested. I guess for coordination it should be enough when we write here in this PR when we start and stop making changes, what do you think?

I think that makes sense.  I'll update with comments accordingly.

Tracking testing progress on my end via:

[ ]  -f
[ ]  --map
[ ]  -a
[ ]  -c
[ ]  -h
[ ]  -k
[ ]  -o
[ ]  -m
[ ]  -n
[ ]  -s
[ ]  -N
[ ]  -i
[ ]  -r
[ ]  -p
[ ]  -d
[ ]  -e
[ ]  -b
[ ]  -v
[ ]  --help

@fabb, please give me a heads up before merging any changes from your end on this branch.  I'd like to squash my commits into one. It's easier if there are no interwoven commits from other contributors.

@fabb, unfortunately I squashed your commits down with mine.
If you have those commits locally, I can add them back. Just push them to a branch called faab-mty-commits and I'll take it from there.

Added back @AviVahl's commits.

Going to pause here for a bit.
Added unit tests for 8 options.  Might pick this up again later.

@fabb, unfortunately I squashed your commits down with mine.
If you have those commits locally, I can add them back. Just push them to a branch called faab-mty-commits and I'll take it from there.

I'm not here for fame, so it's fine with me ;-)

@fabb and @aorinevo Thanks so much for taking this on. I'll be supporting this PR through to release.
If anyone out there regularly uses the handlebars CLI and can verify this PR is working for them, that would be fantastic.
Otherwise I'll give this a spin once you're both happy it's done.

Thanks @ErisDS, I’ll try to get this wrapped up tonight :)

@fabb, picking this backup now. Let me know if you have any WIP.

@ErisDS, I'd like to recommend skipping a unit test for -b as use of BOMS is not required or recommended (see https://en.wikipedia.org/wiki/Byte_order_mark).
Thoughts?

@ErisDS, assuming we skip -b, this PR is ready for review.

Added last remaining unit test for -b.
Unit test is valid as it runs against a file that includes a BOM (see image below).

(Disclaimer: This is just something I tried in the volunteer spirit, while we wait for a proper review. I am not fluent with JS projects, so take this with a grain of salt/feel free to ignore if it is not helpful. Apologies in advance if that is the case.)
Should these tests pass with the old optimist-based code dropped in place of the new yargs code? I tried to do this (new tests, old code/dependencies) and it failed, but it may have been a mistake on my end.
(My Travis runs can be viewed here: https://travis-ci.com/github/DeeDeeG/handlebars.js/branches)

@DeeDeeG, I'm rather curious myself why that's the case.  I'm going to take a look.

@DeeDeeG, still looking for the root cause but worth noting that all the tests pass with the exception of the unit test for --help when I use optimist.

Found it :)
Optimist puts the --help option at the bottom of the menu while yargs places it at the top.

Oh wow, I thought there would be some leftovers for me when I continue today in the evening 😅
Thanks for the work you put in!

LGTM!
i think the --help test is a bit of overkill, but it can easily be removed if it breaks too often in future PRs.

@DeeDeeG your suggestion was really useful - great idea thanks :)
@fabb and @aorinevo seems like this is ready to go? I'll take it for a spin in a bit.

@ErisDS, yep, all done :)

Manually merged via b440c38886f62bd1801b3bb2da78f1f341c2b121, d79212a662d2aae106449338a28717352ec65ce6 and 80c4516fdad5d8fcb6e24faca51db97bd6cc94c5

Thanks everyone!

Fancy :)
Thank you @ErisDS, @fabb, @AviVahl, and @nknapp!

Thanks all for taking this across the finishing line.
@ErisDS any chance this could be tagged and published? :)

By the way... I can confirm that, with optimist plunked in instead of yargs, all these tests (other than the --help test, as discussed above) are passing.
Here is a green/passing CI run (with optimist instead of yargs, and the --help test disabled): https://travis-ci.com/github/DeeDeeG/handlebars.js/builds/157285103

I have been working on the release since I merged the changes 😉
Unfortunately, I've hit several road blocks. The latest one is that the sourcemap tests here result in the file source.map.amd.txt being written to. The content doesn't change but the timestamp on the file does.
The release process runs git diff-index --name-only HEAD -- which then fails and blocks the release.
I'm looking to see if I can find a fix, but I'm also wondering why that test is there twice. Is it a typo?

@ErisDS looking into that release process snag: I noticed just running the command the test runs will also write to that sourcemap file. (Minimal-ish reproducer for the issue).
handlebars -i '
1
' -a -m -N test --map './spec/artifacts/source.map.amd.txt'
(Updates the modification date for ./spec/artifacts/source.map.amd.txt)
I'm not sure if there's a way to really test/run the feature without updating [the modification date of] the file.
A quick workaround would be:
git checkout HEAD -- spec/artifacts/source.map.amd.txt
(Freshly writes the file out to disk from the git index, apparently resetting git's expectation of the modification date to "now" in the process)
Kinda hacky, but it works? (To be run after the test and before git diff-index --name-only HEAD --)

Should probably tell the cli test to output to a temp file at os.tmpdir() and delete it afterwards (instead of trying reset a git-tracked file).
Could use fs.mkdtempSync() to get a unique folder inside temp for each run.

I don't think the test itself writes anything, other than via running handlebars.
I may be wrong, but I think handlebars is writing to a file even though there are no changes to be made. Perhaps handlebars needs to skip writing out the sourcemap file to disk if there are no changes to be made.

In other projects, I use a folder "test-tmp/test-name" to write files generated by the tests. The advantage is that the files are easier to find and inspect if a test fails. "test-tmp" is part of gitignore and the subfolder is deleted and recreated before the test runs.
It may indeed be the case that all tests rib twice on release, but I am not sure. I think the unit tests do, for different environments.

I may be wrong, but I think handlebars is writing to a file even though there are no changes to be made. Perhaps handlebars needs to skip writing out the sourcemap file to disk if there are no changes to be made.

I can confirm that that is indeed what handlebars does.

@ErisDS, happy to take this on.  I think I see a fix for this with a low-level of effort; however, I don't want to duplicate effort if you're already on this.
Let me know :)

The fix is to use unlink.  That will remove the file after the test has been run.
@ErisDS, I'll prepare a PR for this.

For your reviewing please :) https://github.com/wycats/handlebars.js/pull/1667 (cc @ErisDS)

@DeeDeeG have cherry picked your commit removing the duplicate
I believe Nils' suggestion is the best approach for a fix here.

I've used the tmp dir approach to fix this locally and am going to push it to unblock the release.

Thank you everyone for your efforts. This was a productive and cordial experience, and I was glad to be a part of it. Open-source can be pretty cool sometimes.
Hope everyone is safe and well out there.

Hey everyone 👋  The release of 4.7.4 is now fully complete, including the various packages and updating the handlebars website.
Thank you to everyone who helped get this over the line, really appreciate it ❤


Moar stuff

Coverage increased (+3.0%) to 34.757% when pulling bc813cc3fa97d6729ef78367bd8996f235991927 on moar-stuff into f7836a2006d5170590c34e75ffa72119f31bcd3d on master.

Coverage increased (+3.0%) to 34.757% when pulling f929e145c43a44fcde591230badcd67a147928b8 on moar-stuff into f7836a2006d5170590c34e75ffa72119f31bcd3d on master.


docs: fix links for options subpages

Current: https://kulshekhar.github.io/ts-jest/docs/getting-started/options/options/compiler
New: https://kulshekhar.github.io/ts-jest/docs/getting-started/options/compiler

Summary

Hyperlinks for options subpages in the table have an extra options/ in the path.
Test plan

Does this PR introduce a breaking change?

[ ] Yes
[X] No

Other information

hmm ... looks like broken links?

I think something else was interfering with the link generation. Will investigate further and reopen if I figure it out.