Collected ENCODE and RNAseq issues

Here are some RNAseq related issues from ENCODE people like @jahilton , @bonitalam , @carrieadavis  , @siiraa , venkat . They are in varying stages of completeness.

#877
#872
#864
#863
#852
#851
#850
#849
#848
#847
#831
#665
#650

I think there was also some talk about the potential of combinatorial/highly pre-composed terms like polyA-selected single cell RNAseq. See #995 and #930

We will close old (650, 665 etc.) items barring further feedback.

How important are these issues now?


What does this part of code mean?

https://github.com/saber-nyan/kernel-modules-hook/blob/a43cd6bca2bbe65440f544e5f08d53b96c870903/linux-modules-cleanup.service#L6
Sorry, I am bash newbie. I am confused about the if statement and $$ in the script above. Can you try to explain more of it?

@craftyguy
Seems like bug, thanks!
And I probably broke hook with last commit... shit.

@craftyguy
Maybe I do not quite get it, but in the file linux-modules-cleanup.service there is no || pacman -Qou "$${i}";, there is only || pacman -Qo "$${i}";...

@saber-nyan hmm, you're right. I don't know how that 'u' got in there in my copy/paste.. You can probably close this :) (and sorry for the confusion!)

You're welcome~


PHAR version requires "_HumbugBox" prefixed implementation of ProviderInterface

Bug Report

Q
A

BC Break
no

Version
2.9.0

Summary
It seems that the packaging to a PHAR file does some monkey business with the namespace of the project. The provider has to be an instance of _HumbugBoxaa7bf30d75ed\Phly\KeepAChangelog\Provider\ProviderInterface.
Current behaviour
When releasing a version with keep-a-changelog version:release 1.2.3 an error is raised: The provider as currently configured is incapable of performing a release.
Problem is in the config file the provider class is Phly\KeepAChangelog\Provider\GitHub whereas the PHAR wants an implementation of _HumbugBoxaa7bf30d75ed\Phly\KeepAChangelog\Provider\ProviderInterface.
How to reproduce
Download the PHAR file, create a global config with keep-a-changelog config:create -g, set the GitHub token and try to release a version. The error above is raised. When changing the provider to _HumbugBoxaa7bf30d75ed\Phly\KeepAChangelog\Provider\GitHub it works as expected.
Expected behaviour
The _HumbugBoxaa7bf30d75ed namespace shouldn't be there.
It seems every packaging creates a random string after the prefix _HumbugBox. I also tested it with version 2.8.1. The namespace is _HumbugBoxdcb0919ddeeb there.

@heiglandreas Have you run into this? Any ideas on how to fix it?

After a bit of digging I think that the compactor does this for PHAR code isolation.
But I'm not sure if this is required or can be worked around.
Funny sidenote: Apparently, box itself has this "issue" in their PHAR files. I played around with it a bit and because of an error message in box compile I executed box compile -v and got this:
PHP Fatal error:  Uncaught Error: Call to undefined function _HumbugBox48cf944fc33a\Symfony\Component\Console\get_debug_type() in phar:///home/neo/bin/box/vendor/symfony/console/Application.php:523
Stack trace:
#0 phar:///home/neo/bin/box/vendor/symfony/console/Application.php(516): _HumbugBox48cf944fc33a\Symfony\Component\Console\Application->doActuallyRenderThrowable()
#1 phar:///home/neo/bin/box/vendor/symfony/console/Application.php(491): _HumbugBox48cf944fc33a\Symfony\Component\Console\Application->doRenderThrowable()
#2 phar:///home/neo/bin/box/vendor/symfony/console/Application.php(84): _HumbugBox48cf944fc33a\Symfony\Component\Console\Application->renderThrowable()
#3 phar:///home/neo/bin/box/vendor/symfony/console/Application.php(104): _HumbugBox48cf944fc33a\Symfony\Component\Console\Application->_HumbugBox48cf944fc33a\Symfony\Component\Console\{closure}()
#4 phar:///home/neo/bin/box/bin/box(41): _HumbugBox48cf944fc33a\Symfony\Component\Console\Application->run()
#5 /home/neo/bin/box(18): require('phar:///home/ne...')
 in phar:///home/neo/bin/box/vendor/symfony/console/Application.php on line 523

😄
So, maybe it's a feature, not a bug?

I just discovered the same regarding code isolation, @rieschl .
However, I tried locally building the phar without the code isolation compactor, and it appears to work fine. As such, I'm going to create a patch and do a release around that, as that version appears to fix the issues in this and #82.
Can you do me a favor, and do the following please?

Clone the phly/keep-a-changelog repo
Run composer buildphar
Use the build/keep-a-changelog.phar file in the project you're using that has a ProviderInterface implementation present

and then report to me if there are still issues. I don't think there will be, but I am definitely curious.

Oh, important note: remove line three of the box.json.dist file (the one referencing "compactors"), and then remove the trailing comma from the preceding line BEFORE running composer buildphar.

That is actually a feature. It is a so called "scoped PHAR" (I blogged about that). The issue is that the scope-namespace needs to be added to the config file...
I'll see whether that can automagically added...

We probably need to whitelist the Providers as described in the PHP-Scoper docs...

It is a so called "scoped PHAR"

Right, I also discovered that in the meantime 😄

We probably need to whitelist the Providers as described in the PHP-Scoper docs...

I played around a bit with that. Whitelisting the Interface or the implementation doesn't solve the problem. It's probably also necessary if we want to keep the scoped PHAR. Whitelisting scopes the class but also adds a class_alias for the original FQCN.
The problem is the scoping of src/Provider/ProviderSpec.php. It is changed from:
    public function isComplete(): bool
    {
        return $this->className
            && class_exists($this->className)
            && in_array(ProviderInterface::class, class_implements($this->className), true);
    }

to
    public function isComplete(): bool
    {
        return $this->className && class_exists($this->className) && in_array(
                \_HumbugBoxa35b2ab0d726\Phly\KeepAChangelog\Provider\ProviderInterface::class,
                class_implements($this->className),
                true
            );
    }

Because of that the check fails if the _HumbugBox prefix isn't in the config file. Whitelisting the file (files-whitelist) also doesn't work because then the Class isn't found anymore. I could think of some tricking the scoper so it's not rewritten (e.g. put the namespace parts into an array and implode('\\', $parts) it.)

Can you do me a favor, and do the following please?
* Clone the phly/keep-a-changelog repo
* Run `composer buildphar`
* Use the `build/keep-a-changelog.phar` file in the project you're using that has a `ProviderInterface` implementation present

Yes, removing the scoper solves that issue and also #82. But I'm not the one to decide to remove the scoping of the PHAR file. Probably there won't be an issue if we remove the scoping because the PHAR doesn't interact with other classes of the project it is used in. But I haven't used box-project or php-scoper before so I could be wrong. I'd be happy to create a PR if you want.

@heiglandreas So, I see three separate issues, then:

Some users may want to extend the functionality of keep-a-changelog by supplying custom providers. I'd argue that for those users, we can probably stipulate that this functionality only works with a global or local composer install, and/or a clone-based install.

Our configuration uses class names to indicate which provider should be used. If scoping changes those names and/or changes inheritance, this can cause an inability to use configuration when using the PHAR.

When generating configuration, I would expect that doing so from the PHAR file would be identical. However, per #82, it looks like there's a value being injected before the first section token, which thus makes the configuration unusable. (I've verified this with the 2.9.0 phar file).

Locally, I checked to see if I could resolve either issue, in a variety of ways.
First,  removed the scoping compactor entirely, and all problems were resolved.
Second, I tried adding a scoper.inc.php that whitelisted the provider namespace, using the string Phly\KeepAChangelog\Provider\*. This left us in the same state as having if we just had the scoper compactor without configuration.
Third, I tried whitelisting the various classes in that same namespace individually. This resolved the second issue, but not the third.
Looking into the third, I tried whitelisting the Phly\KeepAChangelog\ConfigCommand\CreateLocalConfigListener class. This did not work. At that point, I extracted the phar locally so I could look at files. Of interest was the aforementioned class, and its sibling, CreateGlobalConfigListener. Each has a TEMPLATE constant they declare, with roughly the same contents (the second file also provides entries for tokens for each of the github and gitlab providers). However, the CreateGlobalConfigListener class does not have the generated unique Humbug namespace present; only the CreateLocalConfigListener class does.
As an experiment, I tried changing CreateLocalConfigListener::getConfigTemplate() to return a nowdoc, instead of the class constant. This, too, got injected.
The scoper documentation indicates that certain strings that reference classes can be rewritten, but it seems arbitrary that one and not the other is not rewritten in this case. This makes me think #82 may actually be a bug with either the Box project or the PHP-Scoper project.
I'm going to continue looking into this, but if either of you have ideas, please share!


feat: binding for Node.JS

Node.JS binding for Drive
CI for Node.JS
Add Rust deps security audit
Release CI
Cancel previous CI runs

based on our discussions it seemed like you did this, but if not, please see https://github.com/dashevo/rs-drive/commit/6bbb97455cd5ac044582986956463ab644f14706

And for you as well!


doc: Update README.md

Update old links

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


Trace context is included multiple times when Thrift messages involves sub-structures

Because the trace context information is written in writeFieldStop(), it will be included at the end of each Thrift structure that gets encoded on the protocol. This is problematic when the request includes collections of other Thrift structures, as the trace context information will be added many many times at the end of each "object", adding a considerable amount of overhead to the network exchanges.
That being said, I'm not sure there is another way to do this given the hooks given to us by the TProtocol interface, unless there is somehow a way to use writeMessageEnd()?

Thanks for 0.0.7, looks like it does the trick! Still have to do some more thorough testing but it looks good so far!


Include information about distros

Adding a simple note to inform about linux distro supported to run script

@joubertredrat thank you!


Support for Google cloud storage is missing in the diagram

Issue: Support for Google cloud storage is missing in the diagram
Space Cloud supports Google cloud storage but in the below logo only AWS is mentioned this is misleading.

Please add Google cloud storage also in the diagram.
Unable to find source of the diagram to fix and raise pull request

Thanks for pointing it out @rmkanda!
@Jayesh333 let's get this updated!

Yess! Also I think we need to swap the positions of databases and custom functions, since databases are the core of Space Cloud.

@Jayesh333 do we have an eta on this?

Yes. I have a couple of other ideas as well to improve the readme like showing a graphic of graphql query in the readme itself for which I have created a separate issue as well. We can batch all those graphics changes together.
So let's do it in v0.15?

Sounds good

Fixed


OpenAI On Your Data skill ignores custom metadata field

- [x] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

setup bring your own data with the Azure Playground to create the indexes, indexer, skill... based on a blob storage
add a custom field with the same name as the blob storage files meta data field in both new created indexes
modify the created skill by adding the needed custom field
the preprocessing-jobs api (used in the skill) does ignore the custom field and does not create it in the resulting chunk.json

Expected/desired behavior
both indexes that are created with the OpenAI Playground should contain the custom metadata field from the file in the blob storage.

Is this the same as #251?

Is this the same as #251?

no its not releated

@reechamishra  to investigate if this is related to this repo ?

@JakobStadlhuber Before we pick this item to work on this , can you confirm this issue is related to the code with this particular accelerator "Chat with your data" ?

@JakobStadlhuber - I am closing this as the description talks about "On Your Data" and the OpenAI playground which is not this repository. Please re-open if I have closed incorrectly


Applications that require training never marked "completed"

Applications for facilities that require training currently only get as far as the application being accepted. This means that they are unable to start a new application because an accepted application is still classed as an open application.
Applications that require training and are marked as "accepted" need to have the functionality to mark as "training completed", and therefore mark the application as complete.

Perhaps this should be an automated check, especially for Pilot Rating as they do not need to be ratified.
I.E. once an account has a P2 qual added, they should be marked as completed.
I will discuss with Barrie the exact circumstances that people should and should not be allowed to create new applications.


[fix] externalize middlewares.js when using adapter-node's entryPoint

Fixes #2481, hopefully without breaking anything else. This is almost the same as my suggested solution in my comment on that issue - except it also includes an override for the path in the onResolve hook to avoid having import { assetsMiddleware, prerenderedMiddleware, kitMiddleware } from "../build/middlewares.js"; in the final file.
Before submitting the PR, please make sure you do the following

[x] It's really useful if your PR references an issue where it is discussed ahead of time. In many cases, features are absent for a reason. For large changes, please create an RFC: https://github.com/sveltejs/rfcs
[x] This message body should clearly illustrate what problems it solves.
[ ] Ideally, include a test that fails without this PR but passes with it.

Tests

[x] Run the tests with pnpm test and lint the project with pnpm lint and pnpm check

Changesets

[x] If your PR makes a change that should be noted in one or more packages' changelogs, generate a changeset by running pnpx changeset and following the prompts. All changesets should be patch until SvelteKit 1.0

And we would then tell esbuild to treat that specially when resolving it, and instead point it to .svelte-kit/node/middlewares.js? That's possible. It would probably be tidier, but would it be too magical? That would be a separate discussion, though, and we should fix what we have in the meantime. I think I'll open an issue shortly so we can chat about that.

Hm. I'm having second thoughts somewhat. IIRC part of the reason for the middlewares.js thing - even before there was entryPoint- was that we wanted a way to expose middleware that could then be used as part of an entirely different build process afterwards. Does this sound familiar? We could add a nice alias for ../build/middlewares.js later as a non-breaking change, but I don't think we should stop producing that as its own build artifact if we want to give people the option to consume that however they want.

And we would then tell esbuild to treat that specially when resolving it, and instead point it to .svelte-kit/node/middlewares.js?

I was thinking it could be just a regular import from a regular node module. We might have to refactor slightly to pass in render

we wanted a way to expose middleware that could then be used as part of an entirely different build process afterwards. Does this sound familiar?

Folks wanted to be able to create a custom server and this was the solution that came out of it. I don't think anyone would miss the separate build artifact as long as they can still build a custom server

Oops. This made middlewares.js no longer external when you're not using entryPoint, because the path to the imported file is different - it's in .svelte-kit/node/middlewares.js for regular builds. There's only an import from build/middlewares.js in custom entryPoint builds where you're pointing to that file. I'll submit a PR shortly.


Bufexplorer MRU sort doesn't work properly if using restore_session.vim

I'm using a really simple plugin called restore_session that automatically calls mkview when saving a buffer or exiting vim, and then calls silent loadview when entering the buffer. This allows you to return to the exact previous cursor location of a file when you re-enter vim, along with restoring some other things like folds. However, using this plugin breaks the MRU sort in BufExplorer, because of this line:
autocmd SessionLoadPost * call s:Reset()

Every time loadview gets called (which happens whenever you enter a buffer), Reset() is called, which re-initializes the MRU list. I'm able to workaround the issue by commenting out the above line. However, I can see from the changelog that the line  in question was originally added to fix a bug related to MRU sort after loading sessions. Oddly, it introduces a bug for me when loading sessions, but I'm assuming there's some other use-case that breaks without this line. I'm not sure how/if you can fix both problems at the same time, but I'll let you figure that out :)

I was having problems related to bufexplorer handling of the BufEnter event and was able to work around it with the eventignore option, you might be able to do something similar.  Simply putting set eventignore=SessionLoadPost in your .vimrc might fix things, although it may break other stuff.


knife vcenter --help vs knife vsphere --help behave differently

Description
knife vcenter --help raises a fatal error with helper text to install instruction but knife vsphere --help seems missing instruction.
eg.

knife vcenter --help

FATAL: Cannot find subcommand for: 'vcenter --help'
Use `chef gem install knife-vcenter` to install the plugin into ChefDK

knife vsphere --help

FATAL: Cannot find subcommand for: 'vsphere --help'
Available subcommands: (for details, knife SUB-COMMAND --help)

** BOOTSTRAP COMMANDS **
knife bootstrap [PROTOCOL://][USER@]FQDN (options)

** CLIENT COMMANDS **
knife client bulk delete REGEX (options)
knife client create CLIENTNAME (options)
knife client delete [CLIENT [CLIENT]] (options)
knife client edit CLIENT (options)
knife client key create CLIENT (options)
knife client key delete CLIENT KEYNAME (options)
knife client key edit CLIENT KEYNAME (options)
knife client key list CLIENT (options)
knife client key show CLIENT KEYNAME (options)
knife client list (options)
knife client reregister CLIENT (options)
knife client show CLIENT (options)

Replication Case

Install chef as a gem.
Execute knife vcenter --help and  knife vsphere --help

Fixed in #8899


Module 'leaflet' resolves to an untyped module which cannot be augmented ^v2.8.0

After upgrading my application from Angular 10 to Angular 11 and updating @geoman-io/leaflet-geoman-free from 2.7.0 to 2.9.0 I get the following error when building it with webpack:
Error: node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:3:16 - error TS2665: Invalid module name in augmentation. Module 'leaflet' resolves to an untyped module at 'C:/***/node_modules/leaflet/dist/leaflet-src.js', which cannot be augmented.

3 declare module 'leaflet' {
                 ~~~~~~~~~
node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:79:24 - error TS2304: Cannot find name 'ControlPosition'.

79             position?: ControlPosition;
                          ~~~~~~~~~~~~~~~
node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:136:37 - error TS2304: Cannot find name 'PathOptions'.

136             setPathOptions(options: PathOptions): void;

Versions:
"@angular/core": "^11.0.5",
"@angular/cli": "^11.0.5",
"leaflet": "^1.7.1",
"@geoman-io/leaflet-geoman-free": "^2.9.0",

If I revert back to "@geoman-io/leaflet-geoman-free": "2.7.0" everything works perfectly.
Please advise if this is a bug or a problem with something else.

In the release 2.8.0 we added Typescript typings #678. I think this is the cause of the problems, but @codeofsumit and I are both no typescript developer, so we don't know what is correct and how to fix this. Maybe you can upgrade the typings and make a PR.
Else you can try to remove the file leaflet-geoman.d.ts and changes from the PR #678 and look if it is now working.
It would be very nice to hear from you, when you resolved this

In the release 2.8.0 we added Typescript typings #678. I think this is the cause of the problems, but @codeofsumit and I are both no typescript developer, so we don't know what is correct and how to fix this. Maybe you can upgrade the typings and make a PR.
Else you can try to remove the file leaflet-geoman.d.ts and changes from the PR #678 and look if it is now working.
It would be very nice to hear from you, when you resolved this

After reading this post I moved the the first line in leaflet-geoman.d.ts below the module declaration like this:
declare module 'leaflet' {
    import * as L from 'leaflet';

    export interface GeomanLayerOptions {
        pmIgnore?: boolean;
    }
***

And I could compile my Angular application but then there were some other issues in the file regarding leaflet types.
I will clone this repository and see if I can fix the types. The issue is with Module Augmentation and building with webpack.

After reading this post I moved the the first line in leaflet-geoman.d.ts below the module declaration like this:
declare module 'leaflet' {
    import * as L from 'leaflet';

    export interface GeomanLayerOptions {
        pmIgnore?: boolean;
    }
***

And I could compile my Angular application but then there were some other issues in the file regarding leaflet types.
I will clone this repository and see if I can fix the types. The issue is with Module Augmentation and building with webpack.

@lukekroon have you found a solution to solve this?

@lukekroon have you found a solution to solve this?

I'm looking into this now, it seems like its only an issue with newer typescript versions.

I'm looking into this now, it seems like its only an issue with newer typescript versions.

@lukekroon if you keep import * as L from "leaflet"; at the top of the file, and try just these changes does that work on your end. I tested that with a newer typescript 4.x and my older typescript 3.x project I initially targeted.
@Falke-Design if this works for @lukekroon or if there isn't a response I think it's safe to do a PR. I forked master, should I open a PR against develop this time around?

@lukekroon if you keep import * as L from "leaflet"; at the top of the file, and try just these changes does that work on your end. I tested that with a newer typescript 4.x and my older typescript 3.x project I initially targeted.
@Falke-Design if this works for @lukekroon or if there isn't a response I think it's safe to do a PR. I forked master, should I open a PR against develop this time around?

So with that changes the angular project builds but still gives me this error:
Generating browser application bundles (phase: building)...
    Error: node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:3:16 - error TS2665: Invalid module name in augmentation. Module 'leaflet' resolves to an untyped module at 'C:/Users/lukek/Documents/CrownHill/Farming/farming-front-end/node_modules/leaflet/dist/leaflet-src.js', which cannot be augmented.
    3 declare module 'leaflet' {
                     ~~~~~~~~~
    node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:65:29 - error TS2304: Cannot find name 'PathOptions'.
    65             templineStyle?: PathOptions;
                                   ~~~~~~~~~~~
    node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:66:29 - error TS2304: Cannot find name 'PathOptions'.

    66             hintlineStyle?: PathOptions;
                                   ~~~~~~~~~~~
    node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:79:24 - error TS2304: Cannot find name 'ControlPosition'.
    79             position?: ControlPosition;
                              ~~~~~~~~~~~~~~~
    node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:136:37 - error TS2304: Cannot find name 'PathOptions'.
    136             setPathOptions(options: PathOptions): void;
                                            ~~~~~~~~~~~
√ Browser application bundle generation complete.

I have made some changes that allows me to build without errors, but I don't know if the types will still work after it:
I moved import * as L from 'leaflet'; inside the module declaration below declare module 'leaflet' {.
Then I renamed the declaration to declare module 'leaflet-new' {
And changed position?: ControlPosition; to position?: L.ControlPosition;
And changed setPathOptions(options: PathOptions): void; to setPathOptions(options: L.PathOptions): void;
Sorry for not making a commit, but if @ryan-morris can please test these changes in his repo it would be great !

So with that changes the angular project builds but still gives me this error:
Generating browser application bundles (phase: building)...
    Error: node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:3:16 - error TS2665: Invalid module name in augmentation. Module 'leaflet' resolves to an untyped module at 'C:/Users/lukek/Documents/CrownHill/Farming/farming-front-end/node_modules/leaflet/dist/leaflet-src.js', which cannot be augmented.
    3 declare module 'leaflet' {
                     ~~~~~~~~~
    node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:65:29 - error TS2304: Cannot find name 'PathOptions'.
    65             templineStyle?: PathOptions;
                                   ~~~~~~~~~~~
    node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:66:29 - error TS2304: Cannot find name 'PathOptions'.

    66             hintlineStyle?: PathOptions;
                                   ~~~~~~~~~~~
    node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:79:24 - error TS2304: Cannot find name 'ControlPosition'.
    79             position?: ControlPosition;
                              ~~~~~~~~~~~~~~~
    node_modules/@geoman-io/leaflet-geoman-free/dist/leaflet-geoman.d.ts:136:37 - error TS2304: Cannot find name 'PathOptions'.
    136             setPathOptions(options: PathOptions): void;
                                            ~~~~~~~~~~~
√ Browser application bundle generation complete.

I have made some changes that allows me to build without errors, but I don't know if the types will still work after it:
I moved import * as L from 'leaflet'; inside the module declaration below declare module 'leaflet' {.
Then I renamed the declaration to declare module 'leaflet-new' {
And changed position?: ControlPosition; to position?: L.ControlPosition;
And changed setPathOptions(options: PathOptions): void; to setPathOptions(options: L.PathOptions): void;
Sorry for not making a commit, but if @ryan-morris can please test these changes in his repo it would be great !

@lukekroon how are you doing your imports in other files? With the different module name just doing basic imports don't show up.
import * as L from 'leaflet';
import '@geoman-io/leaflet-geoman-free';

L.PM.initialize({ optIn: true }); // Property 'PM' does not exist on type 'typeof import(".../node_modules/@types/leaflet/index")'.ts(2339)

@lukekroon how are you doing your imports in other files? With the different module name just doing basic imports don't show up.
import * as L from 'leaflet';
import '@geoman-io/leaflet-geoman-free';

L.PM.initialize({ optIn: true }); // Property 'PM' does not exist on type 'typeof import(".../node_modules/@types/leaflet/index")'.ts(2339)

Hey guys,
can you take a look on: #783 and #747

@elliots @ocram1095 you both seem to be using the typescript definitions. Have you experienced this error or have any feedback on it?

I'm not currently seeing any error, but I'm also importing geoman direct from github while I'm using my fork so that may change things. I'm not super knowledgeable about it, but happy to test any potential changes/fixes.
(I'm using parcel-bundler 1.12.4, typescript 4.1.3, insiders vscode (for m1 support))

i resolve adding npm install --save @types/leaflet


maven repository does not work

Hi,There:
I tried couple time, but the maven repository does not work, I have tried
http://lightadmin.org/nexus/content/repositories/releases
and
http://lightadmin.org/nexus/content/repositories/snapshots
Is there anyone can help? thanks a lot.
Neo

Hi Neo,
Sorry for inconvenience - Fixed now.

Thx,I will try it and update later.
发自我的 iPhone

在 2015年7月20日，17:52，Maxim Kharchenko<EMAIL_ADDRESS>写道：
Hi Neo,
Sorry for inconvenience - Fixed now.
―
Reply to this email directly or view it on GitHub.


use querystring on homepage browsing

for example, browsing to https://clearlydefined.io/?license=MIT should have the license filter preselected.
Actions you take to filter or sort should be reflected in the querystring as well

This would be really helpful to allow searching through the available package versions; especially if this information could be routed to from stats page.

onFilter onSort etc can be found here: https://github.com/clearlydefined/website/blob/master/src/components/Navigation/Pages/PageBrowse/PageBrowse.js#L117


Fixes width of course header

allows shopping cart button to go right next course header
@stvstnfrd

Before:

After:

After with long course name:


Node lister

Few things about node lister.
Did not found a way to remove it without creating a node, escape key should hide it again?
Also to create it, using a key as well as button is generally nice (space key is common in many softwares).

You ca use Tab key ( a la Maya) to bring the node lister. Also, in 0.10 version you'll be able to click around the menu to close it.


As a developer, I want to create a setup that will feed data to the block producer dashboard, so the block producers have various information and control over the block production and delegations

Producer's dashboard

Dashboard proxy (rust app in the openmina node project):
A rust app in the middle of the mina archive node and the dashboard FE. To fully fill the required data to the producer dashboard FE, we should utilize the ocaml archive node.
The dashboard proxy should:

Fetch data from the archive node DB
Calculate VRF
Serve data to the FE

Lower priority:

Reward calculation
Payout tool

Some notes
I will place this app into the openmina node, mainly to reuse the VRF algorithm from the node.
Deploymnet
Also focus on the ease of deployment and scalability
Possible environments:

Docker compose (for local testing and simple demos)
Helm chart for k8s production (?)
Terraform (?)

Also we should test it with the most popular cloud solutions like aws, azure, etc...

Update (2.4.2024):
The proxy will require the producer's private key. To maximize security, we should at least implement the encryption mina uses to store the secret key files on disk (in an encrypted file).
Encryption TODO:

[x] Implement decryption
[ ] Implement encryption
[ ] Move to openmina

I've also been experimenting with the archive DB. I'm currently experimenting with diesel (ORM for the popular relation DBs in rust). The discussion online whether to use on ORM vs raw SQL queries is pretty wild, so I would like some input from you guys on this matter 😄

Update (8.4.2024):

Implemented the private key file encryption/decryption
Further analysed the archive node and it's DB, It lacks the Account indexes that is needed for VRF. The DB will only be used to check for produced/missed/orphaned blocks
Need to dump the epoch ledger from the node periodically to be able to calculate VRF

Update (15.4.2024):
Implemented:

node GraphQL polling for node status
archive access for produced blocks
an RPC server for FE
staking ledger dumping
VRF calculation for current epoch
docker compose setup (with archive capabilities)

Implementation of the underlying storage is in progress.

Update:
Deployed https://staging-devnet-openmina-block-producers-dashboard.minaprotocol.network/
Testing


Add NFT page add Listing Actions

Adds the support to load a single NFT on the page via ?mint= url query param
API Actions added:

List NFT
Delist NFT

Can you also make the buttons more uniform/same size?

So can you load a user's inventory + listed/non-listed NFTs now?

No there isnt a user inventory yet, this enables single view NFT page and linking to a nft/listing from an external source (my first thought is discord bot can link users to an active listing)

Can you also make the buttons more uniform/same size?

Yup, will do

Fixed buttons


Valid access_token but no identity

Hi Manfred,
I am currently trying to setup a Microsoft AAD authentication with OIDC component. I am doing trylogin() first with implicitFlow().
I am able to get valid access_token (with expiration) and use it to service API, however I am NOT able to get id token and identity claim. Both getIdentityClaims() function and getIdToken() function returns null. Could you please help take a look, whether this is normal? Thank you so much.
Following is my configuration.
this.oauthService.loginUrl = 'https://login.microsoftonline.com/"tennant_id"/oauth2/authorize';
this.oauthService.clientId = 'client AAD ID';
this.oauthService.resource = 'service URL';
this.oauthService.logoutUrl = 'https://login.microsoftonline.com/tennant_id/oauth2/logout';
this.oauthService.redirectUri = window.location.origin + '/';
this.oauthService.scope = 'openid';
this.oauthService.oidc = false;
this.oauthService.setStorage(sessionStorage);

Do you have any error in your console? I think the issue you might have is that 1) you're missing the issuer in the configuration and 2) you're missing the jwks as well, which you have to setup manually.
All this is because Azure AD B2C doesn't support CORS and can't get the information straight from .well-known/openid-configuration so you have to do it manually.
Head to https://login.microsoftonline.com/"tennant_id"/v2.0/.well-known/openid-configuration and, in that document :

get the issuer property that you'll want to add to this.oauthService.issuer.
get the jwks_uri property, follow the link in your browser and create a json object from that JSON file that you'll put in this.oauthService.jwks property.

After that, you should be good to go and it should (hopefully) work. I had success with that, but using Azure AD B2C, but it should be the same with "classic" AAD.

Hi @Gimly
Thank you for the replay. I do see the following error when I set this.oauthService.oidc = true
angular-oauth2-oidc.umd.js:1502 Error validating tokens
angular-oauth2-oidc.umd.js:1503 Wrong issuer: https://sts.windows.net/"tennant_id"/

Changing this.oauthService.oidc = false can let the error gone.
I tried for a day with your suggestion, but no luck. this.oauthService.issuer  property is straightforward to assign. But for  this.oauthService.jwks property, what I end up with is creating two new classes as below, and instantiate it with the data retrieved from jwks_uri. Then I passed the JWKArrays object to this.oauthService.jwks property. And I still Cannot get the identity claims. Is there anything wrong with my creating JSON object? Thank you so much.
export class JWKObject {
   kty: string;
   use: string;
   kid: string;
   x5t: string;
   n: string;
   e: string;
   x5c: Array<string>;
   issuer: string;
}

export class JWKArrays {
  keys: Array<JWKObject>;
  constructor(_keys: Array<JWKObject>) {
    this.keys = _keys;
  }
}

I don't have access to my code right now, but if I recall correctly, what I did was doing something like:
this.oauthService.jwks = {
 keys: [
     {
        kid: "X5eXk4xyojNFum1kl2Ytv8dlNP4-c57dO6QGTVBwaNk",
        nbf:<PHONE_NUMBER>,
        use: "sig",
        kty: "RSA",
        e: "AQAB",
        n: "tVKUtcx_n9rt5afY_2WFNvU6PlFMggCatsZ3l4RjKxH0jgdLq6CScb0P3ZGXYbPzXvmmLiWZizpb-h0qup5jznOvOr-Dhw9908584BSgC83YacjWNqEK3urxhyE2jWjwRm2N95WGgb5mzE5XmZIvkvyXnn7X8dvgFPF5QwIngGsDG8LyHuJWlaDhr_EPLMW4wHvH0zZCuRMARIJmmqiMy3VD4ftq4nS5s8vJL0pVSrkuNojtokp84AtkADCDU_BUhrc2sIgfnvZ03koCQRoZmWiHu86SuJZYkDFstVTVSR0hiXudFlfQ2rOhPlpObmku68lXw-7V-P7jwrQRFfQVXw"}
]}

You can try this and if still doesn't work for you, tell me, I'll check my code and give you more info.

Sorry, still no luck. I don't see any error message coming from loading jwks object.
Could you please help take a look and give me more info, thanks!
Here is my declaration:
this.oauthService.issuer = 'https://login.microsoftonline.com/"tennant_id"/v2.0';
this.oauthService.jwks = {
        "keys": [
          {
            "kty": "RSA",
            "use": "sig",
            "kid": "kid_code_1",
            "x5t": "x5t_code_1 same as kid_code_1",
            "n": "n_code_1",
            "e": "AQAB",
            "x5c": ["x5c_code_1"],
             "issuer": "https://login.microsoftonline.com/"tennand_id"/v2.0"
          },
          {
            "kty": "RSA",
            "use": "sig",
            "kid": "kid_code_2",
            "x5t": "x5t_code_2 same as kid_code_2",
            "n": "n_code_2",
            "e": "AQAB",
            "x5c": [ "x5c_code_2"],
             "issuer": "https://login.microsoftonline.com/"tennand_id"/v2.0"
          },
          {
            "kty": "RSA",
            "use": "sig",
            "kid": "kid_code_3",
            "x5t": "x5t_code_3 same as kid_code_3",
            "n": "n_code_3",
            "e": "AQAB",
            "x5c": ["x5c_code_3"],
            "issuer": "https://login.microsoftonline.com/"tennand_id"/v2.0"
          }
        ]
      };

@stepsma Could you try without the quotes in the properties names?
{
   keys:
    [{
       kty: "RSA",
        ...
     }]
}

Thanks a lot, let me give a further try on this.

I'm late to this party. sorry. Did it work?

Thank you for the help. It still didn't work for me actually, and I later switched to ADAL. Let me explore little more using OIDC, and close two of my issues for now, thanks.

@manfredsteyer, @stepsma I am having the same issue. Any luck with this. Any know resolution. I am using Angular 5.

@stepsmaI I tried this library https://blogs.msdn.microsoft.com/premier_developer/2017/04/26/using-adal-with-angular2/ but with angular 5 and no luck with this either.  Have you found a library that works for you.

@stuartjdavies I got it to work with MSAL.js https://github.com/AzureAD/microsoft-authentication-library-for-js. You can check out this repo for a working sample https://github.com/Gimly/NetCoreAngularAzureB2CMsal.

I guess I will need some time to provide an example using Azure AD as it seems to provide some challenges ...

For what it's worth I managed to get it to work with Azure. It was a massive PITA though. I was already using angular-oauth2-oidc for Okta and didn't want to use another library like ADAL.
@stepsma you have to have oidc = true to get identity claims. If you switch it off, angular-oauth2-oidc will not send the request for id_token!!!!!!!
and
this.oauthService.jwks = {       "keys": [         {           "kty": "RSA",           "use": "sig",           .....whatever     };
Hope it helps.
I am actually thinking this might be worth writing an article/doc about it....

@Greg5ki Have you written some article about using angular-oauth2-oidc with AAD authentication yet?. It helps me if you did. I'm thinking to use this library for authentication of our app. As you mentioned I also want to use the same library for Okta and AAD.
Thanks!!!

For what it's worth I managed to get it to work with Azure. It was a massive PITA though. I was already using angular-oauth2-oidc for Okta and didn't want to use another library like ADAL.
@stepsma you have to have oidc = true to get identity claims. If you switch it off, angular-oauth2-oidc will not send the request for id_token!!!!!!!
and
this.oauthService.jwks = { "keys": [ { "kty": "RSA", "use": "sig", .....whatever };
Hope it helps.
I am actually thinking this might be worth writing an article/doc about it....

i have tried that, but didn't work, it always makes the /certs call, and returning the cors problem.


Minimap's char width for full-width characters is incorrect.

VSCode Version: 1.10
OS Version: Windows 10 x64

Repro text:
asd = "擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦"                |
asd = "擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦"                |
asd = "擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦"                |
asd = "擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦"                |
asd = "擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦"                |
asd = "擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦擦"                |
asd = "あああああああああああああああああああああああああああああああああああああああ"                |
asd = "あああああああああああああああああああああああああああああああああああああああ"                |
asd = "あああああああああああああああああああああああああああああああああああああああ"                |
asd = "あああああああああああああああああああああああああああああああああああああああ"                |
asd = "あああああああああああああああああああああああああああああああああああああああ"                |
asd = "あああああああああああああああああああああああああああああああああああああああ"                |
asd = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"                |
asd = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"                |
asd = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"                |
asd = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"                |
asd = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"                |
asd = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"                |

After:


Upgrade Model-Binding to 6.0

Fixes #23763.
Internal Preview URL.

@Rick-Anderson Thanks for the suggestions. I often forget to check the width of the code snippets in the preview, but I'm working on that...


SecurityRequirement annotation

Q
A

Bug or feature request?
bug

Which Swagger-Core version?
2.0.0-rc4

Which Java version?
1.8.0_92

Which JAX-RS framework & version?
jersey 2.26

I'm using swagger annotations to generate openAPI schema. I have below class:
@OpenAPIDefinition(
     security = {@SecurityRequirement(name = "basicAuth")}
     info = @Info ( title = "Test", description = "", version = "1.0-SNAPSHOT"))
@SecurityScheme(name="basicAuth", type = SecuritySchemeType.HTTP, scheme = "basic")
public class Definition { }

While generating openAPI schema I'm receiving such output:
{
   "info" : {
      "title": "Test",
      "description": "",
      "version": "1.0-SNAPSHOT"
   },
   "security": [
       {
           "basicAuth": []
       }
    ],
    "components": {
        "securitySchemes": {
            "basicAuth": {
                 "type": "http",
                 "scheme": "basic",
                 "name": "basicAuth"
             }
        }
    }
}

Unfortunately I don't know why basicAuth securitySchemes definition contains also "name": "basicAuth". According to documentation:
https://github.com/OAI/OpenAPI-Specification/blob/master/versions/3.0.0.md#securitySchemeObject
For basicAuth we shouldn't give name. Could you please provide me an example how to achieve it?
Also according to documentation "name" should be pointing to header/query/cookie name. However after setting it to "Authorization" I'm either loosing name "basicAuth" (after changing the name both in @SecurityRequirement and @SecurityScheme) or I'm loosing security support completely (if names between @SecurityRequirement and @SecurityScheme differs).

Hi this is my method of java for creating security scheme object
private SecurityScheme securitySchemesItem() {         return new SecurityScheme().type(SecurityScheme.Type.HTTP).scheme("Bearer").description("Bearer Token : \n Default : Basic YWRtaW46YWRtaW4= (Use if you don't have any)").bearerFormat("Bearer").in(SecurityScheme.In.HEADER).name("Authorization");     }
issue is my security scheme object created like this
"securitySchemes": {
"basicAuth": {
"type": "HTTP",
"description": "Bearer Token : \n Default : Basic YWRtaW46YWRtaW4= (Use if you don't have any)",
"name": "Authorization",
"in": "HEADER",
"scheme": "Bearer",
"bearerFormat": "Bearer"
}
}
as the type value is in (capital) HTTP its showing me the authorization like this


Failure to install Qt with new 5.15.2 version on windows-latest

With the new QT version, our CI build seems to fail, see https://github.com/simulationcraft/simc-publish/runs/1501915984?check_suite_focus=true
CI setup:
runs-on: windows-latest
step:
      - name: Install Qt
        uses: jurplel/install-qt-action@v2
        with:
          modules: qtwebengine
          arch: win64_msvc2017_64

Error:
 Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\aqt\__main__.py", line 28, in <module>
    sys.exit(main())
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\aqt\__init__.py", line 38, in main
    return cli.run()
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\aqt\installer.py", line 373, in run
    return args.func(args)
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\aqt\installer.py", line 186, in run_install
    finisher(target_config, base_dir, self.logger)
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\aqt\installer.py", line 460, in finisher
    raise e
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\aqt\installer.py", line 445, in finisher
    with open(os.path.join(base_dir, qt_version, arch_dir, 'bin', 'qt.conf'), 'w') as f:
FileNotFoundError: [Errno 2] No such file or directory: 'D:\\a\\simc-publish/Qt\\5.15.2\\msvc2017_64\\bin\\qt.conf'
Error: The process 'python' failed with exit code 1

Workflow file:
https://github.com/simulationcraft/simc-publish/actions/runs/401795782/workflow#L120
Interestingly, it works on macos-latest :-)

Ok so the problem was that it was still using win64_msvc2017_64. Replacing this with win64_msvc2019_64 and win32_msvc2019 seems to help for Qt5.15.2. I was able to replicate this locally with aqtinstall, and get it working on my personal machine.
Now the CI build is stuck on some python problems, see e.g. https://github.com/simulationcraft/simc-publish/runs/1510148438?check_suite_focus=true
 python -m aqt install 5.15.2 windows desktop win32_msvc2019 -m qtwebengine -O D:\a\simc-publish/Qt
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\runpy.py", line 188, in _run_module_as_main
    mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\runpy.py", line 147, in _get_module_details
    return _get_module_details(pkg_main_name, error)
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\runpy.py", line 111, in _get_module_details
    __import__(pkg_name)
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\aqt\__init__.py", line 25, in <module>
    from aqt.installer import Cli
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\aqt\installer.py", line 36, in <module>
    import py7zr
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\py7zr\__init__.py", line 30, in <module>
    from py7zr.py7zr import ArchiveInfo, FileInfo, SevenZipFile, is_7zfile, pack_7zarchive, unpack_7zarchive
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\py7zr\py7zr.py", line 40, in <module>
    from py7zr.archiveinfo import Folder, Header, SignatureHeader
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\py7zr\archiveinfo.py", line 35, in <module>
    from py7zr.compressor import SevenZipCompressor, SevenZipDecompressor
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\py7zr\compressor.py", line 32, in <module>
    from Crypto.Cipher import AES
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\Crypto\Cipher\__init__.py", line 27, in <module>
    from Crypto.Cipher._mode_ecb import _create_ecb_cipher
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\Crypto\Cipher\_mode_ecb.py", line 35, in <module>
    raw_ecb_lib = load_pycryptodome_raw_lib("Crypto.Cipher._raw_ecb", """
  File "C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\Crypto\Util\_raw_api.py", line 308, in load_pycryptodome_raw_lib
    raise OSError("Cannot load native module '%s': %s" % (name, ", ".join(attempts)))
OSError: Cannot load native module 'Crypto.Cipher._raw_ecb': Trying '_raw_ecb.cp39-win_amd64.pyd': Could not find module 'C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\Crypto\Cipher\_raw_ecb.cp39-win_amd64.pyd' (or one of its dependencies). Try using the full path with constructor syntax., Trying '_raw_ecb.pyd': Could not find module 'C:\hostedtoolcache\windows\Python\3.9.0\x64\lib\site-packages\Crypto\Cipher\_raw_ecb.pyd' (or one of its dependencies). Try using the full path with constructor syntax.

I have it working now after adding
         setup-python: 'false'

I am not really sure what the intention is of setting up python manually with this action, but it seems to work with the provided python environment at least for the hosted windows-latest runner I am using.
Feel free to close this, or keep it to investigate why it is not running with the default setup-python: 'true' setting.

The intention of setting up python is for ubuntu-16.04, which doesnt have a correct version by default. Sorry for the delay in responding.


azure-search-documents 11.4.0b11 does not contain SearchIndexerIndexProjections class

Package Name: azure-search-documents
Package Version: 11.4.0b11
Operating System: windows
Python Version: 3.11.x

Describe the bug
azure-search-documents 11.4.0b11 does not contain  SearchIndexerIndexProjections class which is a new feature included in the 2023-10-01-preview API version.
To Reproduce
Expected behavior
Add SearchIndexerIndexProjections class. Here is REST API example:
"indexProjections": {         "selectors": [             {                 "targetIndexName": "fsunavala-vector-test",                 "parentKeyFieldName": "ParentKey",                 "sourceContext": "/document/pages/*",                 "mappings": [                     {                         "name": "page",                         "source": "/document/pages/*"                     },                     {                         "name": "vector",                         "source": "/document/pages/*/vector"                     }                 ]             }         ],         "parameters": {             "projectionMode": "skipIndexingParentDocuments"         }     }
Screenshots
If applicable, add screenshots to help explain your problem.
Additional context
Add any other context about the problem here.

@xiangyan99 @BevLoh @pallavit


Reworking the kernel tree to include flash attention and layernorm

(faster RMS norm).

Adding 3000 files? Maybe it's possible to isolate only the relevant bits?

Flash and layer norm depend on c10 and Aten and cutlass... Those files are highly interlinked unfortunately. If you know a way to shake useless files I'll take it. Otherwise we could have separate crates entirely, but right now it's convenient to have it in a single location I feel.

But does this mean that the final binary will include the c10 and aten bits? This would be likely to bloat it a lot no? I guess the other way would be to find alternative flash attention kernels like this triton one from their tutorials?

No no ,this still compiles down everything to PTX, it's just that the cuda kernels themselves depend on macro magic.

flash attention kernels like this triton one from their tutorials?

Still uses padding if I'm not mistaken.

Done


Add a Count function to query language

There should be a way to query the graph to see how many nodes exist without printing all of the values. This can currently be achieved with _size(g.V( ).ToArray( )), but it should be implement in cayley as something like g.V( ).count( ), but the count function should be applicable to any type of call. For example, calling g.V( ).Tag("source").Out("follows").count( ) on the testdata.nq graph should return the number of people each person follows.

+1
I am interested in counting the number of nodes without loading all the data in memory (it may be big)

:+1: I would very much like this feature too.

:+1:

:+1:

:+1:

Moved to feature requests on the forum: https://discourse.cayley.io/t/add-a-count-function-to-query-language/244

Implemented as g.V().Count() in v0.6


podman_tutorial.md typo: arguement -> argument

Looks like your commit's missing a signoff - can you please add one with git commit --amend -s and re-push?
Otherwise LGTM

/approve
/ok-to-test
bot, add author to whitelist

@mheon Done :+1: ; sign-off added

/lgtm


Only expand task targets if they exist

Fixes #3123

Bypassing requirements because the Windows tests are failing for unrelated reasons due to a recent image update on the runners.


ServiceInstances & ServiceBindings metrics missing

Dears
I've deployed cf_exporter in PCF. I see that the Space, ServiceInstance & ServiceBindings metrics are missing from the scraped metrics. Should I make any config change in the code to get the these metrics please ?
Regards.

Replacing username/pwd with client_id/secret has fixed this !
Thanks.


After update cannot create ManagedClusters with azure-cli local or cloudshell in GovRegions.

This is autogenerated. Please review and update as needed.
Describe the bug
Command Name
az aks create
Errors:
No registered resource provider found for location 'usgovarizona' and API version '2021-02-01' for type 'managedClusters'. The supported api-versions are '2017-08-31, 2018-03-31, 2019-02-01, 2019-04-01, 2019-06-01, 2019-08-01, 2019-10-01, 2019-11-01, 2020-01-01, 2020-02-01, 2020-03-01, 2020-04-01, 2020-06-01, 2020-07-01, 2020-09-01, 2020-11-01, 2020-12-01'. The supported locations are 'usgovvirginia, usgovarizona, usgovtexas'.

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az aks create --resource-group {} --name {} --node-count {} --node-vm-size {} --enable-addons {} --generate-ssh-keys

Expected Behavior
Before the update, ManagedCluster was created successfully.  After the update, get the documented error.
Environment Summary
Linux-4.15.0-1110-azure-x86_64-with-debian-10.2 (Cloud Shell)
Python 3.6.10
Installer: DEB

azure-cli 2.21.0

Extensions:
ai-examples 0.2.5
ssh 0.1.4

Additional Context
Local system is OSX Catalina, attempting to downgrade azure-cli version with homebrew extract but unsuccessful so far.

route to appropriate team

We updated to latest Azure CLI and all AKS functions in GovCloud are failing. Same error:
No registered resource provider found for location 'usgovvirginia' and API version '2021-02-01' for type 'managedClusters'.


Importing Mermaid diagram with markdown text syntax does not seem to support auto-wrap behavior

I'm trying to import a relatively simple Mermaid diagram into excalidraw:
---
title: Connect Four Gameplay
---
flowchart
	Start(["`Start`"])
	Initialize["`Initialize the game`"]
	NextToken["`Get the next player's token`"]
	DropToken["`Drop the token into the first unoccupied spot from the bottom of the column`"]
	IsWinningMove?{"`Did the current player make the winning move?`"}
	HaveOpenSpaces?{"`Are there open spaces left?`"}
	GameOverWinner["`The current player wins`"]
	GameOverDraw["`It's a draw`"]
	Stop(["`Stop`"])

	Start-->Initialize-->NextToken
	NextToken-->DropToken-->IsWinningMove?
	IsWinningMove?-->|Yes|GameOverWinner-->Stop
	IsWinningMove?-->|No|HaveOpenSpaces?
	HaveOpenSpaces?-->|No|GameOverDraw-->Stop
	HaveOpenSpaces?-->|Yes|NextToken

I'm using the markdown syntax for dealing with text because it also auto-wraps text, at least when I use it in Obsidian.
When I try to import this into excalidraw, it effectively kills any wrapping. Maybe there's another issue at play, but it seems like the text most nodes that have whitespace in its text get collapsed and the nodes themselves are rendered with just a character width worth of space.
https://ibb.co/kDp3F1k

It looks like even GitHub has problems rendering too, but not as bad as excalidraw.
This is what it looks like, correctly rendered in Obsidian


fix: Correctly report version. Fixes #2264

Checklist:

[x] Either (a) I've created an enhancement proposal and discussed it with the community, (b) this is a bug fix, or (c) this is a chore.
[x] The title of the PR is (a) conventional, (b) states what changed, and (c) suffixes the related issues number. E.g. "fix(controller): Updates such and such. Fixes #1234".
[x] I have written unit and/or e2e tests for my change. PRs without these are unlike to be merged.
[x] Optional. I've added My organization is added to the README.
[ ] I've signed the CLA and required builds are green.

Fixes #2264

This PR can be merged as soon as approved, but should be back-ported to v2.5.


How to filter/hide private files from beeing included?

Hi there,
I am using mkdocstrings to automatically generate my documentation and I have to say that this package is doing a great job! My problem is, that I have modules with many methods which brought me to the idea to split my module (here called b) into two submodules which are independent and some general functions into the submodule _b which are imported into the b.__init__.py to avoid lengthly init files. Using __all__ in the b.__init__.py I managed to show the methods in the documentation of b.
src/mypackage
├── __init__.py
├── a.py
└── b
    ├── __init__.py
    ├── _b.py
    ├── b1.py
    └── b2.py

And here my question. How do I prevent mkdocstrings from creating the documentation of _b.py? Similar to the option filters option I would like to filter also by file names, e.g. files with leading underscores.
I am using mkdocstrings with the python(experimental) handler.

Hello! How do you inject docs in your markdown pages? Can I see your ::: thing lines?


Memory leak for fragments

After PR https://github.com/amardeshbd/android-barebone/pull/19 merged, leakcanary is detecting more memory leaks. Look into it.

Now that viewpager 2 is used #23 should re-check if memory leak is still an issue.

No longer happing.


Q: Is this notifier only?

I have TCS intercom in my apartment and would like to use it for opening the door. Can this be achieved using your code, or can it  only to decode message when someone rings the bell from main entrance?
Anything changed on being able to use it via esphome?

My code is only for decoding the messages and sending them via MQTT.
For information on sending messages check out https://github.com/atc1441/TCSintercomArduino
I have not further tried to integrate with esphome since my MQTT approach works for me. :)

@stomko11 If you're still looking for a full solution with reading and writing to the bus with homeassistant, you could check out my project: https://github.com/peteh/doorman


Fix a performance degradation introduced by PR #183

As the Joiner code would still get executed, still wrap this in a conditional.

/cc @ge0ffrey

yes, better... thanks : )


compress true option fails

Hi,
I can see someone has put a lot of care in developing this library. I just want to report that the examples which have the compress: true option fail. It returns -10000 error
Also, I want to generate a pdf with a table. Is there any ready-made or approximate solution I can use ?
Thanks

Can you post here a snippet of the code that doesn't work?
Please write here your OS and V version that you are using :) tnx.

4.18.19-100.fc27.x86_64, V 0.1.29 62a692b
ERROR!: Compress Failed! error: -10000
The document is written ok when compress: false

import pdf
import os
fn main() {
    mut doc := pdf.Pdf{}

    doc.init()

    page_n := doc.create_page({
            format: 'A4',
            gen_content_obj: true,
            compress: true
    })

    mut page := &doc.page_list[page_n]
    page.user_unit = pdf.mm_unit

    mut fnt_params := pdf.Text_params{
            font_size    : 22.0
            font_name    : "Helvetica"
            s_color : {r:0,g:0,b:0}
            f_color : {r:0,g:0,b:0}
    }
    if !doc.use_base_font(fnt_params.font_name) {
            eprintln("ERROR: Font ${fnt_params.font_name} not available!")
            return
    }

    page.push_content( page.draw_base_text("Test string.", 10, 10, fnt_params))

    txt := doc.render()
    os.write_file_array('t.pdf', txt.buf)

}

What is you OS? Windows, linux, macos?

Linux Fedora Core 27

I'll test is Monday, at the present time I'm using and Ubuntu.
I'll let you know ASAP.

I've checked it, there is a problem with the -prod flag with the included zip library.
I'll check it better as soon I'll have some spare time, for now please excuse me.

It's not urgent at all. This happens without the -prod flag. Thank you

Still failing today, with latest V, without -prod flag.


does not work on 1.13.0

since some version updated

Sorry to hear that it's not working for you. Can you provide some more details on why it doesn't work?
The snippets will only work when the current editor tab language mode is set to Typescript or HTML (depending on which snippet you want to use)

sorry that I do it in *.vue file....
then how can I use the snippets in .vue?

The snippets are currently designed for Typescript files (.ts) and HTML files (.html) only.
It hasn't been tested in Vue files (*.vue) although should work the same. I will test it and let you know.

it works well in script with ts tag in .vue


Add setters for Gradle properties?

Follow up from https://github.com/apollographql/apollo-android/pull/3451:
We could add extra setters to make it easier to call the Gradle APIs:
apollo {
  // this works
  schemaFile.set(file("src/main/graphql/schema.graphqls"))
  // this could work too !
  schemaFile("src/main/graphql/schema.graphqls")
}

It's not 100% clear if this is going to help (by hiding the Property API) or make things more complex (as there are now multiple ways to do the same thing). This issue is here to collecting feedbacks before we commit to maintaining this.

Closing this one. I'm not sure what the Gradle best practices are these days but we don't get too many issues about our Gradle API so we shouldn't fix it if it ain't broken.


Feature request: More options for "move object with arrow key"

[X] I agree to follow the Code of Conduct that this project adheres to.
[X] I have searched the issue tracker for a feature request that matches the one I want to file, without success.

Is your feature request related to a problem? Please describe.
The arrow-key actions for selected options do not seem to be included in https://app.diagrams.net/shortcuts.svg - would it be possible to add them?
I wasn't able to find any more move options other than "arrow" and "shift+arrow". I would like to have an option for moving bigger distance (e.g., one large grid cell).
Many thanks!

That?

Hi @alderg - thank you - yes.
I see!

'scroll / move cell' wasn't obvious to me - it might be better to call 'move selection/object'
'move cell (grid size) or page' also isn't clear to me.

I would prefer

Cursor ... Scoll page view / Move selected object (pt)
Shift+cursor ... First/last/next/prev. page / Move selected object (grid box)

For Ctrl+cursor (without selection) the effect seems to be the same as for shift+cursor - what is the difference?
So the move by one grid box is the largest jump? I would prefer an option to be able to move objects by a 'large grid' spacing as well.


Module Federation shared deep dependency resolved incorrectly

Bug report

What is the current behavior?
When using Module Federation, sharing a dependency and your other dependencies are using it too but in different versions, it won't respect semver matching and use the same version for everything.
For example, even when it is explicitly defined to be shared with ^8.3.2 in the top-level, it matches the 3.4.0 version that another deep dependency consumes.
3.4.0 does not satisfy constraint ^8.3.2

If the current behavior is a bug, please provide the steps to reproduce.
Repo that reproduces the issue - https://github.com/tzachbon/mf-shared-deep-dependency
This example tries to reproduce the problem when sharing a dependency that exists in multiple versions across the node modules but resolves the request incorrectly.
In this example repo, I use uuid package.
└─┬<EMAIL_ADDRESS>-> ./app1
  ├─┬<EMAIL_ADDRESS>-> ./lib
  │ └──<EMAIL_ADDRESS>  ├──<EMAIL_ADDRESS>  └─┬<EMAIL_ADDRESS>    └─┬<EMAIL_ADDRESS>      └──<EMAIL_ADDRESS>deduped

This graph shows that the root uuid is 8.3.2 and the version under lib is 3.4.0.
lib uses the v3 method from uuid.
This method exists only in version 8.3.2.
By default, if we build this project, serve it and click on new uuid, everything crashes as expected:

But when we use Module Federation and share this package from app1 like this:
[
      new ModuleFederationPlugin({
      name: 'app1',
      shared: {
        lib: deps['lib'],
        uuid: deps['uuid'], // <-- resolve to ^8.3.2
        react: { singleton: true, requiredVersion: deps.react },
        'react-dom': { singleton: true, requiredVersion: deps['react-dom'] },
      },
    })
]

We can see that the runtime understands the difference between the packages:

But when we click on new uuid which crashed our app, it works (not the desired behavior)...

When we look inside the index file, we can see the moduleToHandlerMapping resolve it incorrectly:

And when we remove the uuid entry from the shared section in ModuleFederationPlugin we can see that the app behaves as expected:

For some reason, even when it is explicitly defined to be shared with ^8.3.2, it matches the 3.4.0 version that lib consumes.
What is the expected behavior?
I would expect that each dependency would get the correct package resolution according to the node resolution algorithm.
In our example -

top level (App) - 8.3.2
lib - 3.4.0

Other relevant information:
webpack version:  5.72.1
Node.js version: v16.15.0
NPM version: 8.5.5
Operating System: MacOS Monterey 12.4
Also asked in the Module Federation examples repo -
https://github.com/module-federation/module-federation-examples/issues/2033

Major issue for me as well.

This configuration:
shared: {
  uuid: "^8.3.2",
  // or the equivalent:
  uuid: { requiredVersion: "^8.3.2" },
}

will force the version to be ^8.3.2.
But it's not recommended to do that. You should prefer to let webpack automatically infer the requiredVersion from the nearest package.json (from import location).
shared: ["uuid"],
// or the equivalent:
shared: {
  uuid: {},
}

Note you can also mix array and object:
shared: [
  "lib",
  "uuid",
  {
    react: { singleton: true },
    'react-dom': { singleton: true },
  }
],

or use that:
shared: [
  lib: {},
  uuid: {},
  react: { singleton: true },
  'react-dom': { singleton: true },
},

I have the issue with my application. However if I follow @sokra 's suggestion to not provide the requiredVersion to the shared libs, I encounter the issue reported here:
https://github.com/webpack/webpack/issues/13457

I have the same issue.


Support for libadwaita?

Is there any chance to add support for libadwaita on Linux? Or would that go beyond the vision of this project?

I'd be happy to add support for libadwaita, the eventual aim is to have quite a few other backends other than Gtk, so libadwaita is well within the scope of this project. Someone else asked about this a while ago and I think they said that I would really only have to add one line to the project to enable libadwaita support on Linux? Would you also want certain libadwaita widgets to replace the Gtk ones? (I've never used libadwaita before, so I don't really know anything about it)

Adding basic Libadwaita support is basically using adw_application_new instead of gtk_application_new. This uses at least the Libadwaita stylesheet, so it looks more like the other GNOME apps. The other widgets are more complementary and help following the GNOME HIG.

Was about to add my take on it but @JCWasmx86 beat me to it :)

Adding basic Libadwaita support is basically using adw_application_new instead of gtk_application_new. This uses at least the Libadwaita stylesheet, so it looks more like the other GNOME apps. The other widgets are more complementary and help following the GNOME HIG.

Ok sounds good. For now I might make some way for people to select that they want to use adwaita on Linux, and just use that to decide how the application is created. Eventually I reckon I’ll probably make it a ‘separate’ backend option where the basic widgets are updated to use Adwaita’s widgets (if there are drop-in replacements). Any auxiliary adwaita widgets that don’t fit into the SwiftUI API can probably be vended as a separate target or package providing specific adwaita support for those who don’t want to be as cross platform (or are happy to put a bit more effort into being so).


Audit and make documentation for command KEYS consistent

The documentation of the command [KEYS](https://dicedb.io/commands/keys/) present in the docs could have become stale. We need your help to ensure that the documentation is complete, correct, and consistent.  Completing this issue will give you much-needed exposure to how the command is implemented and the different parameters it accepts. Hence, this is a great way to build a good understanding of the project and its functionality. While still making a significant contribution. Here's what you need to do:

Go through the documentation of the command and run all the examples, making sure they all work as expected
If the same command is present in Redis, then the expected output of the DiceDB command is the same as the Redis output
If the command is not in Redis, then as per your judgment, raise an issue with this template or fix the documentation and raise a pull request.
The documentation should contain the following sections in the following order

short and concise introduction paragraph about the command covering what it does
Syntax
Parameters: List all the parameters the command accepts, and ignore the section if there are no parameters
Return values: List all possible return values and under what condition
Behaviour: Describe the behavior of the command, what it does, how it does it, some internal specifics if any
Errors: List all possible errors the command can throw and under what condition
Examples: List all possible examples of the command and the expected output. Assume CLI implementation

Note: The title of the sections should be the exact strings mentioned above, like "Examples", "Return values", etc.
Note: The description of the command in Frontmatter of the command.md file is the first paragraph (introduction) of the command.
Go through the DiceDB/dice repository and explore the command implementation

to understand all possible return values
to understand all possible error values and conditions
to understand different parameters the command accepts

Please use the documentation of the SET command as the reference point and structure the
documentation of this command in a very similar way, including the following points

if you see any "Conclusion" section, remove it
the headers (h1, h2, h3) should be appropriately used
the CLI prompt should be <IP_ADDRESS>:7379> and not anything else
Use markdown tables for parameters and return values (ref SET command documentation)
wrap any command or paramter in backticks (`) to highlight them
the section header should be capitalize as used in the SET command documentation

If you find any inconsistencies, please fix the documentation and raise the PR.
The core idea of this exercise is to ensure that the documentation is consistent, correct, and complete.
Make sure you comment on the other issues you created (if any) as a comment on this issue and also any PR (if any) that you created.
Thank you for picking this up and contributing to the DiceDB. It means a ton.

Issue transferred from DiceDB/docs to DiceDB/dice: https://github.com/DiceDB/docs/issues/108
Will work on it. Please assign this to me.

Issue transferred from DiceDB/docs to DiceDB/dice: DiceDB/docs#108
Will work on it. Please assign this to me.

Assigned.

Closing, merged as part of https://github.com/DiceDB/dice/pull/852


strict option to fail on any QA notice

I'm not sure if I agree with this one. We often gets warnings like this:
 * QA Notice: Package triggers severe warnings which indicate that it
 *            may exhibit random runtime failures.

especially after compiler updates. I think what you're looking for is something more of this:
https://github.com/juippis/incus-gentoo-github-pullrequest-tester/blob/master/container/bin/gentoo_pkg_errors_and_qa_notices.sh after logging every pkg-testing-tool run with e.g. --extra-env-file 'test.conf' that saves the logs:
https://github.com/juippis/incus-gentoo-github-pullrequest-tester/blob/master/container/etc/portage/env/test.conf#L13
Sorry about this babbling, I hope you can understand what I'm after :)


parsing of PC_DISABLE_TUI should be consistent with -t flag

Defect
Currently all of the following will result in TUI being disabled:
PC_DISABLE_TUI=false process-compose
PC_DISABLE_TUI=true process-compose
PC_DISABLE_TUI=1 process-compose
PC_DISABLE_TUI=0 process-compose
PC_DISABLE_TUI=whatever process-compose
PC_DISABLE_TUI= process-compose

So the only way override PC_DISABLE_TUI once it's set is by un-setting the env var, or by adding -t flag.
By comparison, when using t flag following result in tui being disabled:
process-compose -t=0
process-compose -t=false

while following results in it being enabled:
process-compose -t=1
process-compose -t=true
process-compose -t

anything else (like t=whatever) is an error
Version of process-compose:
Version:        v0.51.4
Commit:         e3cc52e
Date (UTC):     2023-06-16T18:14:56Z

OS environment:
nixos
Steps or code to reproduce the issue:
runt
Expected result:
I would expect the parsing of PC_DISABLE_TUI to be consistent with parsing o -t
Btw. I think it's a bit confusing that  PC_DISABLE_TUI is reverse of -t. I would deprecate PC_DISABLE_TUI and add PC_TUI instead.
Actual result:
setting PC_DISABLE_TUI to any value is interpreted as true

one use case is to address a comment here
also, if PC_DISABLE_TUI=false worked as expected then in the linked code above, we could do
  ''export PC_DISABLE_TUI=''${PC_DISABLE_TUI-${builtins.toJSON (!config.tui)}}''

which would allow user to override the default value by setting PC_DISABLE_TUI before executing the wrapper

@F1bonacc1 what do you think about the above?

Hey @adrian-gierakowski,
I always considered it standard practice to use any value in an env variable acting as a boolean.
Having said that, I see how this can be an issue in a wrapped situation.
I would prefer not to break the existing behavior and remain backward compatible.
Will it work if I change the behavior to be:

From: env var defined
To: env var defined and length > 0

This way you will be able to reset it with something like PC_DISABLE_TUI=""

I always considered it standard practice to use any value in an env variable acting as a boolean

Could you please elaborate, I am not sure what you mean.
I believe that if an env var is provided as and alternative to a cli flag, then following the principle of least surprise, one should be able to use the same values with it as with the flag to achieve the same result.
Your proposed solution is fine for me, but personally I’d change it to support the same values as the flag, if not now then at least in the next major release.

Fixed in v1.5.0.
You can now set PC_DISABLE_TUI="" or PC_DISABLE_TUI="false"


Scratchpad improvments

Add an option to disable the client floating state when closing the scratchpad
Fix an issue where a scratchpad can't be toggled with dont_focus_before_close unless doing so from the first tag

Ping @Nooo37


Updating to add the new unordered_set implementation.

This should relatively significantly reduce memory usage as there is no additional storage being used for storing the keys to each of the items in the set.

May I suggest using std::unordered_set with heterogenous key lookup (is_transparent typedef on hasher and equality tester) if DPP is built for C++20?
Also, why is the actual set behind an unique_ptr?

this is apparently faster than an unordered map.
would love to see some profiling in real world use - not just filling a container with n items and comparing to map.
my concern is only that this hasn't been tested in production like unordered map is and we risk introducing some bugs in subtle places.

im closing this pr for now, i think its safer and more compatible and safer to stick with standard containers, especially as users can directly access the containers.


disable unattended-upgrade in docker-ce-20.10.20

fix by https://github.com/ssst0n3/docker_archive/commit/e698fe6b5c0e16ab3090b6f64eafeb4894f4e2ac


Added CustomQABot and CustomQABotAllFeatures

Addresses #3723
Proposed Changes

This is a sample bot to use Custom question answering feature in Language Service
Default Welcome Message can be configured from the .env file.

Testing

@gabog please merge.


Update from upstream again

should bring this branch back in sync with main again

gah this is confusing

I give up


elfmalloc: Handle thread-local storage on platforms without #[thread_local]

On platforms without the #[thread_local] attribute, thread-local storage (TLS) for an allocator is particularly tricky because most obvious implementations (including the one implemented in the standard library) require allocation to work, and do so in such a way that detecting what state a TLS value is in requires allocation, meaning that recursion (caused by an allocation call accessing TLS, triggering further allocation) cannot be detected. While it's possible that this will be fixed at some point in the future, it won't be fixed any time soon.
Inspired by this comment on a jemalloc issue, our best bet may be to register a handler that is called by pthread whenever a thread spawns, and use this opportunity to preemptively initialize TLS for that thread.

Replying to @davidtgoldblatt's comment:

@joshlf, the tsd struct is defined at https://github.com/jemalloc/jemalloc/blob/a315688be0f38188f16fe89ee1657c7f596f8cbb/src/tsd.c#L15, with the tls model set in the configure script https://github.com/jemalloc/jemalloc/blob/82d1a3fb318fb086cd4207ca03dbdd5b0e3bbb26/configure.ac#L725.
Basically, you just need to set the TLS model to be "initial-exec" (or "gnu2" if you feel more confident about the implementation issues there than we do). (Not sure how easy it is to do this with rust).
Happy to help more, but let's discuss on the the elfmalloc issue to avoid polluting this thread.

Awesome, thanks so much! I'll take a look at those.

So to clarify our problem, we already know what we want to be stored in TLS - we just aren't sure how to get that structure into TLS in all cases. Rust supports a #[thread_local] attribute, which corresponds to LLVM's TLS support (the GeneralDynamicTLSModel in particular, if I'm reading the PR correctly) which, as I understand it based on this LLVM documentation, maps to an ELF feature. On platforms on which that's supported, everything works fine.
However, on platforms on which it's not supported, the only mechanism that the standard library exposes uses allocation under the hood (pthread_setspecific/pthread_getspecific on POSIX, etc), which of course we can't use. So what I'm trying to figure out is whether there's a way - without support for the ELF TLS feature - to get TLS to work in a way that doesn't recursively depend on allocation or, if it does, allows this recursion to be easily detectable so that we can fall back to a global allocation if necessary.
If I'm reading your comment correctly, what you're describing is a solution to the first problem - that we solve with Rust's #[thread_local] attribute - but not to the second problem?

Oh, I see what you're saying. Yeah, the initial-exec TLS model addresses the issue that, if you're loading your TLS-containing object from a shared library, the memory from it will get lazily allocated, via malloc.
pthread_[g|s]etspecific should actually work for the most part. A pthread_key_t is (under glibc, and I assume most libcs) just an index into a per-thread data structure whose early bits are initialized with the thread. If your initialization code runs early enough (which you can probably ensure through a constructor attribute, or just the fact that malloc will probably be called very early in process startup), you should be able to reserve one of the early keys, which probably won't require dynamic allocation.
Some early multithreaded mallocs had a global hashmap (backed by a slow fallback allocator that doesn't use thread-local data, but that a thread needs to use only once, on its first allocator call) that mapped thread ids to the thread-local data for that thread. I suspect this approach would have unpleasantly large constant factors, though.
In the end, I don't think there's a good way of getting around all the annoying platform-specific stuff if you're trying to optimize for speed; I think jemalloc ends up with something like 4 substantially different TLS implementations. You might find some useful inspiration in https://github.com/jemalloc/jemalloc/blob/dev/include/jemalloc/internal/tsd.h and friends (e.g. tsd_generic.h, tsd_win.h, etc.) and https://github.com/jemalloc/jemalloc/blob/dev/src/tsd.c (you can ignore the TSD struct and the X-macro nonsense; the tsd_fetch implementation is the interesting part).
Note that TLS isn't the only reentrancy scenario you might need to worry about; on many platforms, "safe"-seeming standard library calls end up allocating (e.g. mutex locking, file reading, getting the CPU count, off the top of my head).

That's hugely helpful, thanks! I'll mull over that for a while :)

Update: On OS X, we're getting bitten by the same issue that rpmalloc is: https://github.com/rampantpixels/rpmalloc/issues/29. It seems to come from the fact that when using DYLD_INSERT_LIBRARIES, _tlv_bootstrap is implemented simply as abort().

pthread_[g|s]etspecific should actually work for the most part. A pthread_key_t is (under glibc, and I assume most libcs) just an index into a per-thread data structure whose early bits are initialized with the thread. If your initialization code runs early enough (which you can probably ensure through a constructor attribute, or just the fact that malloc will probably be called very early in process startup), you should be able to reserve one of the early keys, which probably won't require dynamic allocation.

@davidtgoldblatt Do you mean that if you get there late enough, pthread_create_key will allocate, or every thread spawn will require allocation to make space for the key you allocated with pthread_create_key?

My memory on this is a little hazy; you'd have to check the glibc (or whatever libc you're interested in) to be sure. Here's the way I remember glibc working:

a pthread key is just an integer, an index into a per-thread, growable control structure
That per-thread structure has SOME_CONSTANT_1 structs to contain the metadata you initialize in pthread_setspecific, an initially null pointer to a followup structure, and some counters and whatnot to maintain consistency.
All these structures share the same internal layout; if only one thread touches key 123, then all other threads have null metadata in slot 123 of the structure.
Every thread gets a copy of that control structure upon creation (created by start.S for the initial thread, or by the parent thread for other threads), living at the top of the stack.
When you've created SOME_CONSTANT_1 pthread keys, then, you can still allocate more. The first time you pthread_setspecific on that key index, the implementation will notice that the followup pointer in the control structure is null, allocate another one (using malloc), and use that to satisfy the allocation.

So pthread_key_create won't allocate (it just acquires a mutex and bumps a counter). It's the first time you touch the per-thread data in a thread that you get an allocation.
We get our pthread key in a library constructor at load time[1]. As far as I know, this is early enough that it hasn't caused problems for anyone.
[1] We actually do this even if TLS is available, to get us a place to put our per-thread shutdown hooks.

OK that makes perfect sense, thanks! We'll probably end up doing something similar because even if we fix this dylib issue (more on that in a second...), we'll still need to support platforms that don't support the #[thread_local] attribute.
I just discovered something interesting about that issue I linked to above: https://github.com/rampantpixels/rpmalloc/issues/29. It looks like the issue subsides if you only access TLS after the dynamic loading is complete. My hunch, given the stack trace pasted below, is that our malloc is getting called while the dynamic loader is still doing its thing, in turn using TLS, and TLS is not supported during dynamic loading. There's a hint at reduced TLS support in dynamic libraries here ("No fast thread local storage. pthread_getspecific(3) is currently the only option."), although it seems to imply that fast TLS is never supported, while in reality, I was able to get a test working that used TLS in a dynamic library, but after that library was fully loaded.

Stack trace
* thread #1: tid = 0x0000, 0x00007fff9364bf06 libsystem_kernel.dylib`__pthread_kill + 10, stop reason = signal SIGSTOP
  * frame #0: 0x00007fff9364bf06 libsystem_kernel.dylib`__pthread_kill + 10
    frame #1: 0x00007fff9868f4ec libsystem_pthread.dylib`pthread_kill + 90
    frame #2: 0x00007fff86d856df libsystem_c.dylib`abort + 129
    frame #3: 0x00007fff925a53e0 libdyld.dylib`_tlv_bootstrap + 9
    frame #4: 0x000000010d622e99 libelfc.dylib`elfmalloc::general::global::alloc(size=8) + 41 at general.rs:371
    frame #5: 0x000000010d5d4436 libelfc.dylib`elfmalloc::alloc_impl::{{impl}}::c_malloc(self=0x000000010d6658e5, size=8) + 70 at alloc_impl.rs:60
    frame #6: 0x000000010d5b9c47 libelfc.dylib`elfc::malloc(size=8) + 39 at <define_malloc macros>:4
    frame #7: 0x00007fff622861be dyld`operator new(unsigned long) + 30
    frame #8: 0x00007fff622736c5 dyld`std::__1::vector<char const* (*)(dyld_image_states, unsigned int, dyld_image_info const*), std::__1::allocator<char const* (*)(dyld_image_states, unsigned int, dyld_image_info const*)> >::insert(std::__1::__wrap_iter<char const* (* const*)(dyld_image_states, unsigned int, dyld_image_info const*)>, char const* (* const&)(dyld_image_states, unsigned int, dyld_image_info const*)) + 343
    frame #9: 0x00007fff6226e507 dyld`dyld::registerImageStateBatchChangeHandler(dyld_image_states, char const* (*)(dyld_image_states, unsigned int, dyld_image_info const*)) + 147
    frame #10: 0x00007fff925a489e libdyld.dylib`dyld_register_image_state_change_handler + 76
    frame #11: 0x00007fff925a465f libdyld.dylib`_dyld_initializer + 47
    frame #12: 0x00007fff9757c9fd libSystem.B.dylib`libSystem_initializer + 116
    frame #13: 0x00007fff6227d10b dyld`ImageLoaderMachO::doModInitFunctions(ImageLoader::LinkContext const&) + 265
    frame #14: 0x00007fff6227d284 dyld`ImageLoaderMachO::doInitialization(ImageLoader::LinkContext const&) + 40
    frame #15: 0x00007fff622798bd dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 305
    frame #16: 0x00007fff62279852 dyld`ImageLoader::recursiveInitialization(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 198
    frame #17: 0x00007fff62279743 dyld`ImageLoader::processInitializers(ImageLoader::LinkContext const&, unsigned int, ImageLoader::InitializerTimingList&, ImageLoader::UninitedUpwards&) + 127
    frame #18: 0x00007fff622799b3 dyld`ImageLoader::runInitializers(ImageLoader::LinkContext const&, ImageLoader::InitializerTimingList&) + 75
    frame #19: 0x00007fff6226c0ab dyld`dyld::initializeMainExecutable() + 138
    frame #20: 0x00007fff6226fd98 dyld`dyld::_main(macho_header const*, unsigned long, int, char const**, char const**, char const**, unsigned long*) + 3596
    frame #21: 0x00007fff6226b276 dyld`dyldbootstrap::start(macho_header const*, int, char const**, long, macho_header const*, unsigned long*) + 512
    frame #22: 0x00007fff6226b036 dyld`_dyld_start + 54

Have you guys run into anything similar with jemalloc?

Doesn't ring a bell, but most of the tricky TLS stuff was figured out before my time. I don't know the details of TLS on OS X (actually, my memory was that their compiler people had disabled it there because they wanted to come up with a new implementation without having to worry about backwards compatibility). I think we exclusively use pthreads for our thread-local data on OS X.

Gotcha. I actually managed to solve this with a surprisingly simple approach - I have a global bool that's initialized to false and then set to true in the library constructor. The TLS code checks that boolean and uses a global slow path if it's false (avoiding accessing TLS).


Feat/exec all

added support for the ExecAll call.

Good point.

@SimoneLazzaris I reworked the tests. There are some parts in typeconf.py, which are not fully covered.

LGTM

I did some cleaning up to make tests easier to maintain, @SimoneLazzaris feel free to comment or discard commits following up to this point if something seems off otherwise we can proceed merging

Thx @ufierro for the fixtures!


Not yet compatible with Maven parallel execution

When executing Maven with -T 1C:
[WARNING] *****************************************************************
[WARNING] * Your build is requesting parallel execution, but this         *
[WARNING] * project contains the following plugin(s) that have goals not  *
[WARNING] * marked as thread-safe to support parallel execution.          *
[WARNING] * While this /may/ work fine, please look for plugin updates    *
[WARNING] * and/or request plugins be made thread-safe.                   *
[WARNING] * If reporting an issue, report it against the plugin in        *
[WARNING] * question, not against Apache Maven.                           *
[WARNING] *****************************************************************
[WARNING] The following plugins are not marked as thread-safe in Guava: Google Core Libraries for Java:
[WARNING]   org.spdx:spdx-maven-plugin:0.7.3
[WARNING]
[WARNING] Enable debug to see precisely which goals are not marked as thread-safe.
[WARNING] *****************************************************************

Thanks @sgammon for reporting this.  The underlying SPDX libraries used by the maven plugin are thread-safe, however, we should review the plugin code itself to see if it is implemented in a thread safe manner.

@goneall For posterity's sake, the SPDX goal has not failed even once yet and it's part of my regular build (multi-threading on)

I found one potential issue - MavenToSpdxLicenseMapper is a singleton class which is not using any locking when creating the static instance - should be a relatively straightforward fix.  I'll create a PR.
Additional analysis following the advice in this stackoverflow article:

Check all static fields/variables in plugin/plugin code are not subject to threading problems.

Most are final simple types with the exception of the Logger, Pattern.compile (which claims to be thread-safe), static HashMap s which are initialized in static blocks, and the above referenced singleton class.

Check any plexus components.xml; if the components defined are singletons they need to be thread-safe.

None found

Check for presence of known tainted libraries.

The SPDX libraries are threadsafe and the other dependencies are all related to Maven - so I'm assuming they are threadsafe

Actually - after thinking about the design of MavenToSpdxLicenseMapper - I don't think there would be much impact to having 2  (or more) instances created if the window is hit.  That being said, I'll still create a PR to make it clear that the code is threadsafe.

Nice! If you need a tester downstream, I'm adding this plugin to Guava.


Update .node-version to latest LTS (v20)

No idea how template updates are handled.
I'll try with this PR and wait if anyone picks this up. If not, I'll try something else.

Thanks! Lets also update the Dockerfile (node:20-alpine should be good for both stages) and it might also make sense to specify the engine version in the package.json?
something like
  "engines": {
    "node": ">=20"
  }

Thanks again for picking this up, Chris!


deserialize Cow should be decoding into slices

Currently if I encode, for example, Cow<String>, and deserialize it back using deserialize (that is, passing a complete slice of bytes and not just a stream), I get a Cow::Owned back even though I own the entire buffer, and so could have a cheap allocation-free decoding into Cow::Borrowed.
Note that when decoding into explicit &str, this produces a reference as expected.

This issue would need to be raised with the serde repo itself since it controls how Cow is deserialized.


Diktat CLI doesn't accept folders as input patterns

Duplicates #1854

opens issue without a description 7 hours ago
opens another issue 6 hours ago
closes first as a duplicate
community gets spam
???
PROFIT

open issue without a description 7 hours ago
open another issue 6 hours ago
close first one as a duplicate
community gets spam
???
PROFIT

Yeap, sorry for noise. Will try to avoid creating extra issues


Updating parlour version and test case for sqoop import

Updating parlour version to support replace/removal of new line and control-A character from string columns during Sqoop import

Looks good, just one suggestion to add a comment for the change to the test. :+1:

@rowandavies Thanks for reviewing


Use fst module in morph_check app

This will also require some re-factoring of morph_check, and moving more code from the app to modules.

Implemented in #400.