conformance: Gateway Infrastructure Metadata

What type of PR is this?
/kind test
/area conformance
What this PR does / why we need it:
Include a conformance test and create a feature for this experimental functionality
Which issue(s) this PR fixes:
N/A
Does this PR introduce a user-facing change?:

NONE

/assign @arkodg @mlavacca @sunjayBhatia  # approvers
/cc @LiorLieberman @michaelbeaumont @xunzhuo  #reviewers

apologies - unsure why prow didn't assign - trying again without the # comment
/assign @arkodg @mlavacca @sunjayBhatia
/cc @LiorLieberman @michaelbeaumont

Conformance reviewers/approvers - can I get review please @arkodg @mlavacca @sunjayBhatia @michaelbeaumont @LiorLieberman @Xunzhuo

Sounds good everyone


Package tests broken on nightly

The package test runner is broken on 1.53.0-nightly8
Example output (from this set of commits):
$ curl -s https://raw.githubusercontent.com/atom/ci/master/build-package.sh | sh
Downloading latest Atom release on the nightly channel...
Using Atom version:
Atom    : 1.53.0-nightly8
Electron: 6.1.12
Chrome  : 76.0.3809.146
Node    : 12.4.0
Using APM version:
apm  2.5.2
npm  6.14.8
node 12.4.0 x64
atom 1.53.0-nightly8
python 2.7.17
git 2.24.1
Downloading package dependencies...
Installing locked modules ✓
Running specs...
The command "curl -s https://raw.githubusercontent.com/atom/ci/master/build-package.sh | sh" exited with 1.
Done. Your build exited with 1.

Note the specs exit with an issue, but do not print any errors. The same repo state tests fine on Atom stable and beta.
(Will also mention I was alerted of this because I have my packages on cron jobs running their specs regularly against each release channel of Atom. Everything was fine yesterday).

Likely related: I can no longer open Atom in dev mode on nightly without setting ATOM_DEV_RESOURCE_PATH. I get a blank window with this stack in the dev console:
index.js:129 Error: Cannot find module 'nanoid/non-secure'
Require stack:
- /Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/input.js
- /Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/parse.js
- /Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/lazy-result.js
- /Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/postcss.js
- /Applications/Atom Nightly.app/Contents/Resources/app.asar/src/style-manager.js
- /Applications/Atom Nightly.app/Contents/Resources/app.asar/src/atom-environment.js
- /Applications/Atom Nightly.app/Contents/Resources/app.asar/src/initialize-application-window.js
- /Applications/Atom Nightly.app/Contents/Resources/app.asar/static/index.html
    at Module._resolveFilename (internal/modules/cjs/loader.js:627)
    at Module._resolveFilename (/Applications/Atom Nightly.app/Contents/Resources/electron.asar/common/reset-search-paths.js:41)
    at Function.Module._resolveFilename (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/module-cache.js:336)
    at Function.Module._load (internal/modules/cjs/loader.js:531)
    at Module.require (internal/modules/cjs/loader.js:685)
    at require (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/native-compile-cache.js:68)
    at Object.<anonymous> (/Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/input.js:5)
    at Object.<anonymous> (/Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/input.js:147)
    at Module._compile (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/native-compile-cache.js:121)
    at Object.value [as .js] (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/compile-cache.js:255)
    at Module.load (internal/modules/cjs/loader.js:645)
    at Function.Module._load (internal/modules/cjs/loader.js:560)
    at Module.require (internal/modules/cjs/loader.js:685)
    at require (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/native-compile-cache.js:68)
    at Object.<anonymous> (/Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/parse.js:4)
    at Object.<anonymous> (/Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/parse.js:43)
    at Module._compile (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/native-compile-cache.js:121)
    at Object.value [as .js] (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/compile-cache.js:255)
    at Module.load (internal/modules/cjs/loader.js:645)
    at Function.Module._load (internal/modules/cjs/loader.js:560)
    at Module.require (internal/modules/cjs/loader.js:685)
    at require (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/native-compile-cache.js:68)
    at Object.<anonymous> (/Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/lazy-result.js:8)
    at Object.<anonymous> (/Applications/Atom Nightly.app/Contents/Resources/app.asar/node_modules/postcss/lib/lazy-result.js:490)
    at Module._compile (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/native-compile-cache.js:121)
    at Object.value [as .js] (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/compile-cache.js:255)
    at Module.load (internal/modules/cjs/loader.js:645)
    at Function.Module._load (internal/modules/cjs/loader.js:560)
    at Module.require (internal/modules/cjs/loader.js:685)
    at require (/Applications/Atom Nightly.app/Contents/Resources/app.asar/src/native-compile-cache.js:68)

#21396 took care of this for me locally. Let's switch back our CI to the nightly channel after tomorrow and make sure that our builds are green again?

atom/github is successfully building against nightly again. Looks like this is fixed 🌠


Put lowercases in french documentation when needed

In French typography a ":" is followed by a lowercase.

Thank you!


Using local images

How can I use local images?
Code snippet:
new Photo
{
URL = "music.jpg",
Title = "Korben"
},

I'll add the feature in few days.

You can let us load an image from stream or byte array to be able to load saved on device storage / cache.

I updated the code as follows and it works.
Code snippet:
new Photo
{
URL = "file:///user/images/music.jpg",
Title = "Korben"
},
and that works.

How can I use local images?
Code snippet:
new Photo
{
URL = "music.jpg",
Title = "Korben"
},

thank you @dentonsavage


finish-mobile-version

This pull request, the finished mobile version is about building a responsive portfolio for all mobile screen devices usage. In this project, I use Html and CSS to make my portfolio responsive on all mobile screen devices.

@KanzaTahreem, Please, what exactly do you want to see?

@Shedrack-Sunday, @KanzaTahreem, @GedeonTS, The sections you are looking for have been removed and I don't know how to write them again because it was not Html and CSS that were used to write them.

@Shedrack-Sunday, @KanzaTahreem, @GedeonTS, The sections you are looking for have been removed and I don't know how to write them again because it was not Html and CSS that were used to write them.

Hello @david30-maker, please copy this README, then edit all the sections according to your project. Just commit it when you're done. It will show here

@vickymarz, @wayungi, please check my pull request submission and update me on what is next for me to do

@Shedrack-Sunday, @vickymarz, @GedeonTS, @wayungi, @KanzaTahreem, Please check my pull request for review, Thanks

Hey @david30-maker
I can see that you are struggling for creating a professional README. I would like to guide you for that. Here is my slack handle 👉 Kanza Tahreem Let's fix this 💪

@david30-maker, please  reach out to me on zoom/sack

@topeogunleye, please can you check I want to submit my pull request for the changes, the provision for submission is still showing the request pending. Thanks

@wayungi, @KanzaTahreem, @topeogunleye, @Shedrack-Sunday, Please I haven't received a message for approval but my project is showing submit, let me know if I should go ahead to merge and submit. Thanks

@david30-maker Yes, I approved it,  see the comment I made above ☝🏽. You may merge and submit. 😉

Alright, Thanks
On Thu, Jan 19, 2023 at 3:53 PM Didier Peran Ganthier <
@.***> wrote:

@david30-maker https://github.com/david30-maker Yes, I approved it, see
the comment I made above ☝🏽. You may merge and submit. 😉
—
Reply to this email directly, view it on GitHub
https://github.com/david30-maker/finish-mobile-version/pull/1#issuecomment-1397099642,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AWZ5RQTJEYDR5P26ZKLPJ3DWTFIN5ANCNFSM6AAAAAAT6L5M7M
.
You are receiving this because you were mentioned.Message ID:
@.***>

@topeogunleye, please can you check I want to submit my pull request for the changes, the provision for submission is still showing the request pending. Thanks

@david30-maker, apologies for the late reply. I'm glad this has been resolved


Funky timer formatting

Whenever the time in seconds is a one digit number, it's formatted like x:y instead of x:0y.

Fixed


Cbc UI d2 652 python sharing sdk

I can't find this test in the py sdk, is it worthwhile to add it?
https://github.com/IABTechLab/uid2-client-java/blob/master/src/test/java/com/uid2/client/KeyParserTests.java#L14

I can't find this test in the py sdk, is it worthwhile to add it? https://github.com/IABTechLab/uid2-client-java/blob/master/src/test/java/com/uid2/client/KeyParserTests.java#L14

The Python sdk doesn't have this class

I think the equivalent Python code is in _parse_keys_json. Is there a test that covers this code, if not can we add one?


In Description please which WinVersion

In Description please which WinVersion ..
but i don't know if it's a thing of WinVersion, i have running Win7

Theoretically it should run on Win 7, I remember the exact same error message somebody reported on another app I used to code (staxrip) but I don't remember the reason. The GUI targets .NET 4.7.2 so you can try to install that and maybe it runs then.
If you really want to run it or see this error often also for other apps you can google it and try some of the suggestions.
The GUI is a normal .NET app that don't do anything special so totally common on Windows so there might be a issue with your system.
Unfortunately I can't test it on Win 7, sorry if I can't help much.

Hi @stax76
maybe is too new also not really well.. i guess, i have all updated up to the newest
net version, i fellow the whole Security-updates and security-preview's for Win7, load it down and save it for building a batch-patch.cmd for fresh installed Win7 with 15 Gig .. so should be all updated..
well, maybe have an other a solution, let wait us for an other one, who view this and have a solution for your Proggy  :) :+1:
best regards
blacky

resolved with Rlease 1.2 :) :+1:  super :smile:

Good to hear, thanks for reporting.

you service my as user, i service u as The Programmer,
Both worlds needs together.. and today don't need's razio anymore  ;)
let it clean up together.. :smile:
it works in //linuxserver/ and in local fs..
something could be also something for Reactos..
... if.. the latest netfx works in ros... finally ...
(if the WinXP step would not be)


check for searchguard.ssl.http.enabled in es_facts

Pull request acceptance prerequisites:

[X] Signed the CLA (if not already signed)
[X] Rebased/up-to-date with base branch
[ ] Tests pass
[ ] Updated CHANGELOG.md with patch notes (if necessary)
[X] Updated CONTRIBUTORS (if attribution is requested)

Thanks @hex2a, the contribution is appreciated 👍


Idea: Single User usage

Actually Winds allows every user who knows the Url where the app is being hosted, to be used by everyone.
Would be cool to have the possibility to configure only one user, in order to make it more personal ( say you want to host this into your own website hosting space ).
Thank you in advance :)

it's a solid idea. how would you like this to work in terms of configuration?

I admit I have no real vision at the moment, but I'm pretty sure that it may escalate quickyl from 1 user to many, but still a closed circle.
I think we need to make sure that a first admin user with a random password at first setup is created.
Then afterwards is up to the user to add more or keep just one.
Of course an admin page where to manage them is required. Additional users may be added by inviting via email ( see Ghost ).


"Check for Updates" button in Manimsideview won't work with latest mobject gallery version

I just updated all plywood galleries to the latest version (https://github.com/kolibril13/plywood-gallery/issues/24), including the mobject gallery, which has now the latest plywood infrastructure:
https://github.com/kolibril13/mobject-gallery/blob/04a50f43acccdf548c5bbf18304d417bfa0e56a9/gallery_assets/gallery_parameters.json#L1-L12
That means, that the "Check for Updates" button in Manimsideview won't work any more.
The plywood gallery extension and the manim sideview extension are now 100% equivalent, so maybe it would be even possible to deprecate the manim sideview gallery feature and link people to the plywood gallery extension. But I guess it also won't hurt to simply keep it in the sideview repo.

Another comment:
Since this latest mobject gallery version, I remove indentation manually on the front end like this:
https://github.com/kolibril13/mobject-gallery/blob/f41ac0a876f2b87ba081a23283bd1185371c1daf/index.html#L436-L438
It was the easiest solution and it works fine. But on the long run, I think the indentation correction should happen in the plywood backend, so that the json file can be used as it is.

The plywood gallery extension and the manim sideview extension are now 100% equivalent, so maybe it would be even possible to deprecate the manim sideview gallery feature and link people to the plywood gallery extension.

As far as i remember, this was already planned for when plywood gets stable enough.


Refactor

See https://docs.github-did.com/v2/

Codecov Report

Merging #21 into master will decrease coverage by 61.33%.
The diff coverage is 25%.

@@            Coverage Diff             @@
##           master     #21       +/-   ##
==========================================
- Coverage   69.66%   8.32%   -61.34%
==========================================
  Files           1      97       +96
  Lines          89     937      +848
  Branches        8      37       +29
==========================================
+ Hits           62      78       +16
- Misses         27     825      +798
- Partials        0      34       +34

Impacted Files
Coverage Δ

...ckages/app/src/components/DIDVerifer/DIDVerifer.js
0% <ø> (ø)

...omponents/ExpansionPanelList/ExpansionPanelList.js
75% <ø> (ø)

...ages/app/src/components/DIDResolver/DIDResolver.js
0% <ø> (ø)

packages/api/src/express/routes/version/index.js
63.63% <ø> (ø)

packages/api/src/lib/winston.js
100% <ø> (ø)

...nents/DisplayPayloadDialog/DisplayPayloadDialog.js
0% <ø> (ø)

packages/api/src/express/onErrorResponse.js
33.33% <ø> (ø)

...ackages/app/src/components/AppBar/DrawerContent.js
0% <ø> (ø)

packages/app/src/components/DIDWallet/DIDWallet.js
0% <0%> (ø)

packages/app/src/redux/wallet/handlers.js
0% <0%> (ø)

... and 111 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fe4a095...005d4e6. Read the comment docs.


feat: add timestamp, read and action to notifications

What it solves
Adds extra 'state' flags for notifications
How this PR fixes it
timestamp (for showing the date in the notification centre), read (whether the notification is (un-)read) and action (likely a ReactElement link) have been added to the notification system.
How to test it
Dispatch a notification and observe the timestamp/read flags in the store by default. read will be set to true unless the notification is not manually dismissed.
action is not used yet and will not be present in the store.

what are we using the dismissed state for?

My question is intended to sound more like: is it the only way to close the notification?

Just one question: what are we using the dismissed state for?

dismissed is how we signify that the user manually closed a notification.


UX: APP: iPod Touch: Decider "Next" button displaying outside speech bubble

iPod Touch:

Saw this happening even with the correct dialog text css, so must be something else.

This was working locally yesterday but appears to be acting up today on the app - moving to backlog

occuring again on iphone device.

lets fix #278 first to see if it fixes this problem


Add a hot key to make it full screen and vice-versa

A key combination, most probably, option/alt+ shift + f or any better which doesn't coincide with system and browser keys. Pressing this combination will make the screen show in full screen and again will make it resize to original size and position.

@poush I'd like to take a swing at this.
Some clarification questions:

In testing the apps I happened to have on this machine, the only I saw using CTRL+SHIFT+F were other electron apps like Discord and Slack, it mapped over to maximize. In that case, should I use control/command + shift + f to match that convention or use alt + shift + f to avoid conflict if player is not focused?

If Alt, how should Mac be addressed? Electron doesn't expose a OptionOrAlt like it does CommandOrControl, so should I utilize Alt in all cases or do alt for windows/linux and option for Mac?

Are we talking about true full screen or just maximize?

Thanks for your interest.

The issue is that we assume that our app is always on focus (always on top). To perform any actions we don't want the user to first focus it. Similarly, different keys for focused and unfocused situations could be confusing. So from these two combinations, I would suggest for Alt+shift+f to avoid conflicts and please let me know if you have a better key combination in your mind.

I think using Alt would be sufficient to work on all platforms.

Here, I would suggest full screen because if you see the use case of the picture to picture mode (which has a resizable screen), it would make sense if we have the full-screen option. Let me know if you have other views on it.


Reintroduce the daily intake visualization already in the code as a Knowledge panel

What

[ ] Reintroduce the daily intake visualization already in the code as a Knowledge panel on the product page

Code pointers

https://github.com/openfoodfacts/smooth-app/issues/304
https://github.com/openfoodfacts/smooth-app/commit/194da755efa94f7d43b1b48e7d7dbdeb0df1ea1f

Screenshot

The feature as it appeared in Quick view (note: we would move it to the full Product page)

Part of

#499

@teolemon

File not found
Either this file doesn’t exist or you don’t have permission to view it. Ask the file owner to verify the link and/or update permissions.

pierre@MacBook-Pro-de-Pierre smooth-app % git rev-list -n 1 HEAD -- packages/smooth_app/lib/cards/data_cards/smooth_intake_recommendation_card.dart
194da755efa94f7d43b1b48e7d7dbdeb0df1ea1f

@M123-dev I've added the screenshot, and I've found the removal commit

We want this on the Scan page ? I don't think we have space there :(

No, I was thinking of moving it on the Product Page, at the bottom

[ ] Getting rid of the dark mode hardcode
[ ] Putting it as disabled by default (behind a flag), I think we can enhance usability

This seems interesting. @teolemon I want to work on this issue. Please assign this to me.

The first step would be to re-add it, but behind a switch in the dev mode. To first test

@M123-dev At 194da5 deleted files like smooth_quantity_selector_card.dart import missing files which is not present at the time of removal. That implies it was removed before it.

Which imports are missing, maybe I can help with this


Save user preferences to account on server

We said to do this but I forgot. PWAs don't retain cookies, so that's an issue.

4f40ba37ad88f6b3d701cd337745dddd6dc32749
26775e68dbcddbdb775fccd384a5553864e3db59


Fix performance regression for (group/hash) aggregates without doc-values

Fixes https://github.com/crate/crate/issues/15224.
The reasoning behind this change has been described in https://github.com/crate/crate/issues/15224#issuecomment-1873465335.
Summary of the changes / Why this improves CrateDB
Checklist

[x] Added an entry in the latest docs/appendices/release-notes/<x.y.0>.rst for user facing changes
[x] Updated documentation & sql_features table for user facing changes
[x] Touched code is covered by tests
[x] CLA is signed
[x] This does not contain breaking changes, or if it does:

It is released within a major release
It is recorded in the latest docs/appendices/release-notes/<x.y.0>.rst
It was marked as deprecated in an earlier release if possible
You've thought about the consequences and other components are adapted
(E.g. AdminUI)

Can one of the admins verify this patch?

ok to test

I forgot to add a note to the changelog. I've addressed both of your remarks.


Magnolia macro doesn't "use" instances with path-dependent types

I've found an interesting case where magnolia needs an implicit to be in scope for a member variable in a case class, but the compiler complains that the implicit is not used. Commenting the implicit out causes magnolia to fail (as expected).
The slightly strange thing about this construction is that the implicit for the member variable lives in another object which uses type members - I wonder if this is some issue in magnolia (my macro-fu is very basic at best) or possibly even in the compiler.
import magnolia1.{CaseClass, Magnolia}
import scala.language.experimental.macros

//
// Ordinary Show example, nothing to see here :)
//
trait Show[A] {
  def apply(a: A): String
}

object Show {
  def join[A](cc: CaseClass[Show, A]): Show[A] =
    (a: A) =>
      cc.typeName.short +
      cc.parameters
        .map { p => p.typeclass(p.dereference(a)) }
        .mkString("(", ", ", ")")

  type Typeclass[A] = Show[A]
  implicit def gen[A]: Show[A] = macro Magnolia.gen[A]
}

//
// Because Reasons my code holds my typeclass
// instances in a trait which uses type members
//
trait ShowHolder {
  type A
  val show: Show[A]
}

object ShowHolder {
  def apply[A0](showA0: Show[A0]): ShowHolder =
    new ShowHolder {
      override type A = A0
      override val show: Show[A] = showA0
    }
}

//
// Now try to use ShowHolder
//
// - if you leave `bringIntoScope` where it is, the
//   compiler complains that it's unused
//
// - if you comment it out, obviously `Show.gen`
//   fails because there's no instance for `holder.A`
//
object ShowMain extends App {
  def showForTuple2(holder: ShowHolder) = {
    implicit val bringIntoScope: Show[holder.A] = holder.show
    Show.gen[(holder.A, holder.A)]
  }
}

Since I have -Xfatal-warnings enabled, the only way I can get this to work is to mark my implicits @unused (even though they're not).
Scala version: 2.13.8 and 2.13.10 (both tested)
Magnolia version: magnolia1_2 % 1.1.2

The reason is that macros are a bit special when it comes to warnings because they generate code and they might use code from their scope as you already noticed. That means that there are potentially two places where a warning can come from - your own code (before the macro is expanded) or the generated code (after the macro is expanded). Neither is ideal. If you warn before expanding the macro you might get an unused warning for code the macro needs (like your case). If you warn after expanding the macro, you might get a warning from the generated code and as a user there is not much you can do apart from filing an issue to the macro authors.
Because neither choice works for everyone the Scala compiler has an option to control it:

-Wmacros:before - to warn before expanding a macro
-Wmacros:after - to warn after expanding a macro

this would fix your example but could potentially cause other warnings from another macro (try it)

-Wmacros:both - to warn both before and after expanding the macro

Another way to resolve your issue is to annotate that implicit in question with @unused

Ahh I hadn't seen that compiler option before #til
Can confirm that switching to -Wmacros:after removes the problem (although as you say, comes with its own risks) so I'll have to think about what approach to take in our build.
I had thought I'd seen a case where the equivalent but with a type parameter instead of a type member (i.e. ShowHolder[A] in the example above) didn't display this issue - so thought / assumed it might be a possible issue in magnolia (or even the compiler) - but I now can't reproduce that behaviour, so I'll go ahead and close this.
Thanks for your help :)


Sql alchemy binding separation

I separated type.dto.common into type.dto.common and type.dto.sqlalchemy.common to remove the dependency on sqlalchemy.
Then I separated the sqlalchemy in type.dto.match in type.dto.match and type.dto.sqlalchemy.match.
Now I'm able to use MatchDetail without the need to have sqlalchemy installed in my system.
When someone wants to use the MatchDetail with the previous slqalchemy functionalities s/he should import type.dto.sqlalchemy.match instead.
Referring to issue #4

With the last changes there is no actual change relative to dto.match. I just moved some code in a separate file, and imported it all inside inside dto.match.


Ignore this

Objective-C binary sizes
*****************STATIC******************
  New size                      Old size
   662,645       Core (<)        669,448

 1,948,199      Total (<)      1,955,026

***************FRAMEWORKS****************
  New size                      Old size
10,843,348      Total (<)     10,843,751

 No significant differences in binary sizes


() takes exactly 1 argument (0 given)

The line
return os.path.join(git_directory(), "bigstore", "objects", hash_function_name)
calls git_directory with no arguments... git_directory is defined as
git_directory = lambda git_instance: git_instance.rev_parse(git_dir=True)
and takes one argument.
Am I missing something? Or is the current version broken?
Command:
git bigstore init .
Stack Trace:
Reading credentials from .bigstore configuration file.
Bigstore has already been initialized.
Traceback (most recent call last):
  File "/usr/local/bin/git-bigstore", line 73, in <module>
    args = parser.parse_args()
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1701, in parse_args
    args, argv = self.parse_known_args(args, namespace)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1733, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1942, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1898, in consume_positionals
    take_action(action, args)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1807, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1096, in __call__
    subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1733, in parse_known_args
    namespace, args = self._parse_known_args(args, namespace)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1942, in _parse_known_args
    stop_index = consume_positionals(start_index)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1898, in consume_positionals
    take_action(action, args)
  File "/usr/local/Cellar/python/2.7.9/Frameworks/Python.framework/Versions/2.7/lib/python2.7/argparse.py", line 1807, in take_action
    action(self, namespace, argument_values, option_string)
  File "/usr/local/bin/git-bigstore", line 15, in __call__
    init()
  File "/usr/local/lib/python2.7/site-packages/bigstore/bigstore.py", line 449, in init
    mkdir_p(object_directory(default_hash_function_name))
  File "/usr/local/lib/python2.7/site-packages/bigstore/bigstore.py", line 82, in object_directory
    return os.path.join(git_directory(), "bigstore", "objects", hash_function_name)
TypeError: <lambda>() takes exactly 1 argument (0 given)

Using sudo pip install git+git://github.com/lionheart/git-bigstore/@222f0abae37963e7290e636acb749939ea83ab08 for now which seems to work fine

@richardklafter Give it another shot from pip--I think this was just an instance of sloppy programming.


Left clicking an extension in the extension viewlet a section time changes the focus color

Left click once:

Left click 2+ times:

@isidorn @sandy081 this isn't a feature, it's a bug. It seems like the keyboard action of selecting is getting mixed up with what the mouse is meant to do as clicking it a second time focuses the extension editor. The explorer, search and scm all have lists like this and do not do this.

This will gets fixed along with #91603


CSX profile upload

@tacaswell, @danielballan
Before I make a mess, could you please instruct me on how to do this canonically?
Thank you...

what is the bl phone number?

631 344 1423
Thanks
On Mar 5, 2016 11:44 AM, "Thomas A Caswell"<EMAIL_ADDRESS>wrote:

what is the bl phone number?
—
Reply to this email directly or view it on GitHub
https://github.com/NSLS-II-CSX/xf23id1_profiles/issues/1#issuecomment-192684990
.


Fix concurrent MIDI issue

This issues fixes issue #1390.

I see this patch is for macOS.
Have you checked the issue doesn't happen on Windows and Linux?

I have not been able to reproduce this on Windows or Linux yet, however: when I look at the implementation of win32midiinputdevice.cpp, I don't see code that could lead to the same issue. I will try and set up the same situation (a virtual MIDI device is harder to set up on Windows than Mac).
I have meanwhile renamed the PR so that it more accurately reflects the issue.

For what it's worth, I just tried testing with LoopBe on Win11.  I used MultiPlay 3.x to send 2 MIDI messages in one click, and both of them came through successfully (I added some temporary qDebug lines to prove they both came into win32midiinputdevice.cpp::MidiInProc(), and they came through as two separate calls to this function. As such, I suspect the loop fix proposed for macx doesn't apply in the windows world.

Awesome, thanks for checking @kpr0th!

Coverage: 27.969%. Remained the same when pulling fbe57ff7c92bec062d1ecfc67289c3fb8d3c3ff4 on Timebutt:fix-concurrent-MIDI-issue into af27264da26663febc0fbe559e9789ee020b9f3f on mcallegari:master.


Add numba version

This PR is for adding numba version and comparison with pure python and cython version. It actually shows that numba version is faster than cython, but that doesn't happen when using parallel=True option and using numba.prange. Cython times improve when using openMP, so I wonder why numba version couldn't improve with this flag on.

@gbeckenkamp I couldn't set you as a reviwer I don't know why :p


ProjectReferences may break recorded tests when clients upgrade to new ServiceVersions

Based on guidance from the arch board, clients should default to the latest service version - even for service previews in preview package versions. However, when we convert PackageReferences to ProjectReferences, this can cause test recordings to fail if clients and/or tests using other clients aren't passing an explicit service version to the appropriate client options.

While this might show its head with ProjectReferences and recordings right now as we chatted this is a larger potential issue were we need to be able to configure our client libraries and/or test environments align on the service version they are testing. We also end to determine if we want to or should be testing multiple service versions.

We may need to parameterize the api-version for recordings somehow - a mapping of sorts that works with the record look-up.
/cc @pakrym

FYI @JimSuplizio a scenario that you may need to support with the recording framework.


v2.4.1 wrath - Problem with loot display

not all loot is displayed, so some of items are cant be needied/greedied/passed. Most of the time this happens when doing hc+ which drops some more items now
Interface/AddOns/NDui_Plus/Modules/Misc/LootRoll.lua:270: attempt to index global 'M' (a nil value)
Count: 1

Call Stack:
[string "=[C]"]: ?
[string "@Interface/AddOns/NDui_Plus/Modules/Misc/LootRoll.lua"]:270: in function `func'
[string "@Interface/AddOns/NDui/Init.lua"]:23: in function <Interface/AddOns/NDui/Init.lua:18>
[string "=[C]"]: ?
[string "@Interface/AddOns/NDui_Plus/Modules/Misc/LootRoll.lua"]:42: in function <Interface/AddOns/NDui_Plus/Modules/Misc/LootRoll.lua:41>

Thanks for the feedback, should be fixed in 2.4.2


vendor: support alt registries

Adds support for alt registries to cargo vendor. It mostly worked before, but panicked when trying to display the .cargo/config instructions.
This isn't entirely elegant, as the source replacement looks like this:
[source.crates-io]
replace-with = "vendored-sources"

[source."file:///Users/eric/Proj/rust/cargo/target/cit/t0/alternative-registry"]
registry = "file:///Users/eric/Proj/rust/cargo/target/cit/t0/alternative-registry"
replace-with = "vendored-sources"

[source."file:///Users/eric/Proj/rust/cargo/target/cit/t0/gitdep"]
git = "file:///Users/eric/Proj/rust/cargo/target/cit/t0/gitdep"
branch = "master"
replace-with = "vendored-sources"

[source.vendored-sources]
directory = "vendor"

The duplication of the URLs is a little unfortunate.  It could use the name of the registry, but that is not readily available and is tricky to obtain.  I feel like that is a challenge for another day.
Closes #7674.

r? @alexcrichton
(rust_highfive has picked a reviewer for you, use r? to override)

@bors: r+

:pushpin: Commit 723748fabfe30e79bc7012d58dfd9fdc3349e7b1 has been approved by alexcrichton

:hourglass: Testing commit 723748fabfe30e79bc7012d58dfd9fdc3349e7b1 with merge 720613d48334e6f24fab286da90f07ce0d4bd150...

:broken_heart: Test failed - checks-azure

I think we have to wait for the next nightly to be published (6 or 7 hours from now?).

:hourglass: Testing commit 723748fabfe30e79bc7012d58dfd9fdc3349e7b1 with merge 4647b1d142a81de3bbd6be36105fa959a9c01f8f...

:sunny: Test successful - checks-azure
Approved by: alexcrichton
Pushing 4647b1d142a81de3bbd6be36105fa959a9c01f8f to master...


use writecore release for RecoveryServices

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

USE WRITE CORE PR: https://github.com/Azure/azure-sdk-for-net/pull/46242

API change check
API changes are not detected in this pull request.


finish searcher part, next step is to finish server test

see title. before add http server, need to test searcher first.

see title


Add details about Data Submission workflow.

Discussion

https://docs.nmrxiv.org/docs/submission-guides/submission/upload


Update to .NET 5.0

Closes  #402

Does the Boogie upgrade to 5.0 mean that Dafny will have to be upgraded to 5.0?

Does the Boogie upgrade to 5.0 mean that Dafny will have to be upgraded to 5.0?

Dafny is already on .NET 5.0.

@akashlal :  Any concerns?

Sounds good to me. No concerns.
On Mon, Jun 28, 2021, 21:22 Shaz Qadeer @.***> wrote:

@akashlal https://github.com/akashlal : Any concerns?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/boogie-org/boogie/pull/403#issuecomment-869800607,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAOMQY5XJ5CHM5IOFQ2S233TVCLDHANCNFSM47NUWEBA
.

Is Corral going to switch too? Or maybe it already did? @akashlal? Thx!

@zvonimir Corral hasn't switched yet, but we should do it after boogie moves.

@bkragl :  Merge?


Unable to get ghcide working Error ghcide: : can't load .so/.DLL

`ghcide version: 0.0.6 (GHC: 8.6.5) (PATH: /home/mayanks/.local/bin/ghcide) (GIT hash: 2f5d0d3e69abb5f321bf28b1049ae7a7fc9f619e)
Ghcide setup tester in /home/mayanks/work/transporter.
Report bugs at https://github.com/digital-asset/ghcide/issues
Step 1/6: Finding files to test in /home/mayanks/work/transporter
Found 7 files
Step 2/6: Looking for hie.yaml files that control setup
Found 1 cradle
Step 3/6, Cradle 1/1: Loading ./hie.yaml
Step 4/6, Cradle 1/1: Loading GHC Session
ghcide: : can't load .so/.DLL for: /nix/store/qn76sklvyalzw9ilnxz6sh0020gl2qn6-glibc-2.27/lib/libpthread.so (/nix/store/qn76sklvyalzw9ilnxz6sh0020gl2qn6-glibc-2.27/lib/libpthread.so.0: undefined symbol: __libc_vfork, version GLIBC_PRIVATE)
`

What OS are you using?

Manjaro i3

Any updates on this? Having the same issue.

@epeery what OS?

Arch Linux

One interesting thIng I might note is that when I add a cabal.project file to the root of the project the error changes to:
Step 4/6, Cradle 1/1: Loading GHC Session
ghcide: CradleError (ExitFailure 1) ["Failed to parse result of calling cabal","","cabal: Unrecognised target syntax for ''.\n\n",""]

Seems to be also present on https://github.com/mpickering/haskell-ide-engine/issues/63

Could you check if LD_PRELOAD contains something and if it does run ghcide with LD_PRELOAD= ghcide ...
If that's not the case, it just system glibc interfering with Nix glibc, something that https://github.com/NixOS/nixpkgs/pull/31263 wants to solve.

Running LD_PRELOAD=ghcide ghcide gives me the error:
ERROR: ld.so: object 'ghcide' from LD_PRELOAD cannot be preloaded (cannot open shared object file): ignored.
Seems like that thread you linked is likely the solution. Hopefully it gets merged soon :)

LD_PRELOAD= ghcide is the correct command - there's a space between equal sign and ghcide.

Ah got it. My bad. Thanks.
The LD_PRELOAD error goes away but the can't load .so/.DLL error persists.

I'm having the same issue but in my case the message is:
can't load .so/.DLL for: libgmp.so (libgmp.so: cannot open shared object file: No such file or directory)
I'm using Ubuntu 18.04.2 LTS via Windows Subsystem Linux
The package in mentioned is located at /usr/lib/x86_64-linux-gnu/ However, it seems GHC cannot find it.
Inside /etc/ld.so.conf.d I have two files: libc.conf  x86_64-linux-gnu.conf. The latter contains the path to /usr/lib/x86_64-linux-gnu/. I also tried adding path to the former, no change. The GHC still cannot find libgmp.so.
Any ideas?

Could I ask if you're using vscode and if so, if it's installed with Nix? This applies to everyone.

@fabioluz could you run nix-store -qR /nix/store/xhg5lzac77x3y6ww54qjrvi3ygyjxzq9-ghcide-0.1.0-exe-ghcide

@domenkozar Yes, I'm using VSCode, it was not installed with Nix. Only ghcide was installed with Nix. Anyway, I get the same error if I run ghcide inside the project folder, so it should not be related to vscode, right?
The project was generated by Stack, which is was not installed with Nix.
The command you mentioned above showed a bunch of packages, more than 100.
/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27
/nix/store/jxny7lfjvxwxli237wp3fl09wll0jlfh-gcc-7.4.0-lib
/nix/store/f0wi5c6j4zl8idrd6qw3bsis7cmdy7j7-gmp-6.1.2
...

Thanks! Can you also run echo $LD_LIBRARY_PATH?

Since this happens when loading GHC session, most likely glibc mismatch comes between ghc and ghcide. Maybe ghcide should just pin ghc since versions need to be matched anyway.

Could I ask if you're using vscode and if so, if it's installed with Nix? This applies to everyone.

No I’m using Neovim which I installed from the Arch package repos.

Thanks! Can you also run echo $LD_LIBRARY_PATH?

It does not show anything. Empty line.

I'm happy to help debug this if someone allows me to https://tmate.io/ and use audio :)
Sure! I can do it in around 4 hours from now. Let me know when we can do it.

I'm in CET, so anytime withing working hours will do :)

I'm also available if that helps. Today is pretty open for me

I just started seeing this on NixOS, too. @domenkozar's suggestion about a glibc mismatch seems likely as my cachix-installed ghcide is now saying,
Step 4/6, Cradle 1/1: Loading GHC Session
ghcide: <command line>: can't load .so/.DLL for: /nix/store/5ar4gjxiykc3vpn40whgsd0cgqxgfng7-ghc-8.6.5/lib/ghc-8.6.5/ghc-prim-0.5.3/libHSghc-prim-0.5.3-ghc8.6.5.so (/nix/store/6yaj6n8l925xxfbcd65gzqx3dz7idrnn-glibc-2.27/lib/libm.so.6: version `GLIBC_2.29' not found (required by /nix/store/5ar4gjxiykc3vpn40whgsd0cgqxgfng7-ghc-8.6.5/lib/ghc-8.6.5/ghc-prim-0.5.3/libHSghc-prim-0.5.3-ghc8.6.5.so))

Let me know how I can give you the session id for https://tmate.io/

@acowley can you investigate if $LD_LIBRARY_PATH is set somewhere?

I've updated https://github.com/cachix/ghcide-nix to be built with glibc-2.30, could everyone using it try again?

@domenkozar That seems to have fixed it for me. Thanks!

@domenkozar It is working for me as well. Thanks!

I still encountered some issues on Ubuntu 18.04 using nix.
Most of the files are fine except one of those with stuffs using QuasiQuotes and TemplateHaskell.
The log is as following:
[nix-shell:~/workdir/argus]$ ghcide
ghcide version: 0.1.0 (GHC: 8.6.5) (PATH: /nix/store/02wcifqr53wjs1m5vn670b1avfy071az-ghcide-0.1.0-exe-ghcide/bin/ghcide)
Ghcide setup tester in /home/shen/workdir/argus.
Report bugs at https://github.com/digital-asset/ghcide/issues

Step 1/6: Finding files to test in /home/shen/workdir/argus
Found 9 files

Step 2/6: Looking for hie.yaml files that control setup
Found 1 cradle

Step 3/6, Cradle 1/1: Implicit cradle for /home/shen/workdir/argus
Cradle {cradleRootDir = "/home/shen/workdir/argus", cradleOptsProg = CradleAction: Default}

Step 4/6, Cradle 1/1: Loading GHC Session

Step 5/6: Initializing the IDE

Step 6/6: Type checking the files
File:     /home/shen/workdir/argus/src/Model.hs
Hidden:   no
Range:    1:0-100001:0
Source:   typecheck
Severity: DsError
Message:
  Unexpected usage error
  can't load .so/.DLL for: /nix/store/siks2gcfwx6qwh27m7c5r5lixcr621bd-glibc-2.27/lib/libpthread.so
  (/nix/store/siks2gcfwx6qwh27m7c5r5lixcr621bd-glibc-2.27/lib/libpthread.so.0: undefined symbol:
  __libc_vfork, version GLIBC_PRIVATE)
Files that failed:
 * /home/shen/workdir/argus/src/Model.hs

Completed (8 files worked, 1 file failed)

[nix-shell:~/workdir/argus]$ nix-store -qR /nix/store/02wcifqr53wjs1m5vn670b1avfy071az-ghcide-0.1.0-exe-ghcide/bin/ghcide | grep glibc
/nix/store/sc7lh1dhvd7fg28d7dxwsqzfcinnr6j5-glibc-2.30
/nix/store/ad115wnlpndz5xkkvz19k2ihsb5cdqpz-glibc-2.30-bin
/nix/store/bq2ycrgyzi3zkq2mwb3j46a684smqxcr-glibc-2.30-dev

[nix-shell:~/workdir/argus]$ ghc --version
The Glorious Glasgow Haskell Compilation System, version 8.6.5

Could you run nix-store -qR $(which ghc) | grep glibc

I had this same issue as well, and I've been following this thread. After updating ghcide this issue is gone, but another issue is back:
Step 4/6, Cradle 1/1: Loading GHC Session
> Resolving dependencies...
> Build profile: -w ghc-8.6.5 -O1
> In order, the following will be built (use -v for more details):
>  - shopping-cart-<IP_ADDRESS> (lib) (configuration changed)
> Configuring library for shopping-cart-<IP_ADDRESS>..
> Warning: The 'license-file' field refers to the file 'LICENSE' which does not
> exist.
> Preprocessing library for shopping-cart-<IP_ADDRESS>..
ghcide: <command line>: cannot satisfy -package-id aeson-<IP_ADDRESS>-Em7rwiOS5bO6ogTsCliW9g
    (use -v for more information)

I can't seem to get around this cannot satisfy -package-id error. Any hints?
I'm running ghcide within a nix-shell because I'm on Ubuntu, and it doesn't work outside of a Nix shell.

Nevermind, I managed to fix it with these exports:
export NIX_GHC="$(which ghc)"
export NIX_GHCPKG="$(which ghc-pkg)"
export NIX_GHC_DOCDIR="$NIX_GHC/../../share/doc/ghc/html"
export NIX_GHC_LIBDIR="$(ghc --print-libdir)"

Got them from this issue: https://github.com/mpickering/hie-bios/issues/25

@domenkozar
Here is output of the commands. I guess it means my ghc is still using the old glibc.
[nix-shell:~/workdir/argus]$ nix-store -qR $(which ghc) | grep glibc
/nix/store/siks2gcfwx6qwh27m7c5r5lixcr621bd-glibc-2.27
/nix/store/ngf4xva3wvw1m0pb84v9h4jgl784lxkh-glibc-2.27-bin
/nix/store/yjdnj9pcjgb09xif4ngmf13vb8npklys-glibc-2.27-dev

[nix-shell:~/workdir/argus]$ nix-store -qR /nix/store/02wcifqr53wjs1m5vn670b1avfy071az-ghcide-0.1.0-exe-ghcide/bin/ghcide | grep glibc
/nix/store/sc7lh1dhvd7fg28d7dxwsqzfcinnr6j5-glibc-2.30
/nix/store/ad115wnlpndz5xkkvz19k2ihsb5cdqpz-glibc-2.30-bin
/nix/store/bq2ycrgyzi3zkq2mwb3j46a684smqxcr-glibc-2.30-dev

Maybe it's a question out of ghcide. How can I force my ghc to link with glibc-2.30 by default? I'm using the nixpkgs-unstable channel.

Seems like you're using outdated version of nixpkgs-unstable channel. nix-channel --update and reinstalling ghc should do it.

I realized that I have to actually run sudo /nix/var/nix/profiles/default/bin/nix-channel --update due to the use of nix-daemon/multi-user configuration of nix and problem solved. Thanks.

I managed to fix it with these exports:
export NIX_GHC
export NIX_GHCPK
export NIX_GHC_LIBDIR
export NIX_GLIBC_PRIVATE

None of the export stuff worked for me. What helped me with this issue was found here: https://github.com/digital-asset/ghcide/issues/538
ghcide is in nixpkgs now:
nix-repl> pkgs = import <nixpkgs> {}
nix-repl> pkgs.haskell.packages.ghc883.ghcide
«derivation /nix/store/5kx45z9b8xvvbacq724ivpsq882m0g91-ghcide-0.1.0.drv»

Since the issue is due to mismatched glibc versions one can avoid the issue by including ghcide in the shell.nix on a per project basis assuming you're using GHC provided by the shell.nix from nixpkgs and not haskell.nix.

@willbush how did you include ghcide in your shell.nix ? I have the following issue: https://github.com/haskell/haskell-ide-engine/issues/1781

@razvan-flavius-panda I did it in this repo. Specifically here.
That nix expression was based on a template I found here: https://github.com/jwiegley/hello There's a lot of ways to write shell.nix for Haskell projects and I'm definitely not sure what the best way to do it. However, I do prefer to pin nixpkgs to a specific version using niv.
Here are some links I've been collecting on the topic:

https://github.com/Gabriel439/haskell-nix

https://www.fosskers.ca/blog/nix-en.html

https://www.haskell.org/cabal/users-guide/nix-integration.html

https://cah6.github.io/technology/nix-haskell-1/

https://cah6.github.io/technology/nix-haskell-2/

https://discourse.nixos.org/t/nix-haskell-development-2020/6170
also see:
https://www.reddit.com/r/haskell/comments/fp0g4n/nix_haskell_development_2020_howto/

The last one might be the most up-to-date. I've only skimmed it so far.

@domenkozar  not sure if lastest version of ghcide could have fix this, or if we could do something in ghcide to fix it, so i will close this if nobody disagree


clsparseSCsrMatrixfromFile() returns clsparseInvalidSize.

I am trying to run default benchmark function SpMdV on test file: Bell & Garland cant.mtx file.
I am getting error clsparseInvalidSize. on further debugging the actual error occurs at validateMemObject() due to current_size != required_size. This error was not present before, I don't know why this is occurring? I tried other bell & garland matrices the output is same.
Environment: W9100, Ubuntu 14.04 LTS.
The same behaviour is also observed on windows.
However the older version codes are working fine.

To make it work locally please comment lines from 660 from mm-reader.cpp:
Fix should looks like this:
         clsparseStatus validationStatus = clsparseSuccess;
        // = validateMemObject(pCsrMatx->values,
        //                              mm_reader.GetNumNonZeroes() * sizeof(cl_double));

        // I dont want to reallocate buffer because I suppress the users buffer memory flags;
        // It is users responsibility to provide good buffer;
        // if (validationStatus != clsparseSuccess)
        //    return validationStatus;

        validationStatus = validateMemObject(pCsrMatx->colIndices,
                                             mm_reader.GetNumNonZeroes() * sizeof(cl_int));

The issue is related to mm_reader. This object for reading mtx file is reading the file twice. The first pass is just to read the header of the file. From header it is taking the necessary number of rows, cols and nnz to be allocated. The second pass is reading the values and values locations. To optimize the code (reduce the acutal number of nnz) Kent was additionally testing the read values aganist zero. This lead to inconsistency between declared parameters from header and the actual number of nnz values.
@kknox I see several options

Do not check for val == 0 during the reading function
Check for val == 0 but not validate the matrix values buffer size

I personally vote for option nr 1. Because it is not our responsibility to properly creating the sparse matrix even if there are explicit zeros. We may also combine the 1 & 2 with some extra flag like bool check_for_extra_zeros.
In addition checking for explicit zeros may false some benchmarks when comparing to results from publications. I didn't see that anybody was checking for explicit zeros. In case of cant.mtx the difference between header and read number of zeros is 62451. I nuderstand that it may be waste of memory, but again it was the user who created that file.

Hi @jpola,
I wrote a fix for this problem while trying to debug Issue #141. Correct me if I'm wrong, but it seems that the validateMemObject function as written is too strict. On line 67 it is testing "if(current_size != required_size)" and returning clsparseInvalidSize (which is then silently causing failures in a lot of our benchmarks).
I haven't submitted a PR for this yet, but I fixed the problem seen by @kvaragan by changing it to current_size < required_size. See https://github.com/jlgreathouse/clSPARSE/commit/39c40cf2a871d90215f9d12f11d4028cbff17735. I believe this should be OK, because the validate function should ensure that whatever we want to put in the buffer should fit. I don't think strict size equality is required.

Hi @jpola, I agree nr 1 is not needed, I am also surprised why sparse storage format inputs are having some elements with val == 0. Maybe for very large MTX files we can use this technique for fear of not enough memory but otherwise I feel its not needed.

I also agree with at least including the option to save explicit zero values into the matrix when reading it in. All of the academic benchmarks we compare against save the explicit zero values, as does our internal code we used for the papers we wrote about CSR-Adaptive.
The UFL collection claims that keeping these around is useful because "they can represent important information, such as entries that will become nonzero later on in a subsequent interation of a nonlinear solver".

@jlgreathouse, unfortunatelly I can't agree on your suggestion regarding current_size < required_size. If the buffer will be smaller than required the algorithms will have undefined behaviour.

Err, you're right. Sorry, I ended up doing exactly the opposite of what I'd hoped to do there. The general point still stands, though: should we not compare "if(current_size >= required_size)" instead of != ?
That, of course, doesn't fix the problem being discussed here but may prevent issues in the future.

I was thinking about the same. We should allow users to allocate larger buffers than required. They i.e. may store several vectors in one buffer and play with the offsets (even if they are not fully supported right now).
@kknox
This function is your child :) How do you wish this issue to be resolved?

@jpola To give credit where it's properly due, the mm-reader code was originally written by one Chip Freitag; I've ported and refactored his code for clsparse use.
On to the question: I agree the 'explicit' zeroes setting should be configurable, and we should probably change the default to include explicit zeroes.  We could easily implement this with flags that are passed into the API to control mm-reader behavior.
As for the problems with validateMemObject, I think Jakub and Joseph are narrowing down what the appropriate behavior should be.  There should be no harm in allocating buffers bigger than they need to be, and we want to help catch user error only if we minimize false positives.
As another general comment, I also believe that sanity checks like these should only be done in DEBUG builds, and RELEASE builds should minimize all calls into the OpenCL runtime layer.

I will create two separate tickets. First for sanity checks, second for mm-reader.

Do these two separate tickets supersede this one?  Is this now closed?  Or is this ticket still tracking a unique problem?  I'm wondering about the code  in @jlgreathouse fork that will eventually be merged.

I sat down to hammer on this a bit, and here are some updates:
@jpola, I've convinced myself that the patch I linked (https://github.com/jlgreathouse/clSPARSE/commit/39c40cf2a871d90215f9d12f11d4028cbff17735) is correct. Note that "if(current_size < required_size)" is the failure case. If the current size of your buffer is not large enough to hold the required number of NNZs, return clSparseInvalidSize.
The current_size != required_size case happens for two reasons where we allocate more space than we actually need:

As mentioned above, we currently do not read in explicit zero values from matrix market files. As such, the NNZ listed in the MM header is larger than the NNZ we actually put into the data structure

@kknox, I don't think this one is Chip's fault -- someone added in 'const int exp_zeroes = 0;'. It used to come from a command line parameter. :)

When reading in a symmetric matrix, we take the NNZ in the matrix market header and multiply it by two in order to decide how much space to allocate. In actuality, a symmetric matrix will have less than 2x the number of non-zeros, because values on the diagonal are not stored twice.

As such, fixing only Issue #146 will not solve this current issue. I believe that we must have the patch https://github.com/jlgreathouse/clSPARSE/commit/39c40cf2a871d90215f9d12f11d4028cbff17735 and allow slight over-allocations of space in these cases.
Thoughts?

@jlgreathouse
The exp_zeroes = 0 was totes me when I ported mm_reader; couldn't port the parser to the library.


Update Rust edition to 2021 across all crates

Changes mostly done by cargo fix --edition.

Codecov Report

Merging #3392 (12c959d) into main (d464e34) will decrease coverage by 1.22%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main    #3392      +/-   ##
==========================================
- Coverage   32.21%   30.99%   -1.22%
==========================================
  Files         302      133     -169
  Lines       36931    14363   -22568
==========================================
- Hits        11897     4452    -7445
+ Misses      25034     9911   -15123

see 169 files with indirect coverage changes


[TextField] style of multiline textfield

Added ability to use inputStyle property for customizing inner textarea element style when multiLine={true}

Can you update the doc? (propType of the component and /doc)

inputStyle propType is already there and inputStyle property works for input element inside TextField, but didn't work for textarea element. I've updated the /doc.

Thanks

@krvital could you please rebase?

@krvital Can you add iconStyle to the propTypes list of this component?
Thanks!


Update windows-audio-architecture.md

Fix a typo: "any given device"

Thanks, Matt, appreciate the sharp eye.  The change should appear on the live site within 5-10 minutes


Type 'CAShapeLayerFillRule' (aka 'NSString') has no member 'evenOdd'

I'm getting this compiler error with 1.2.1 on the line:
fillLayer.fillRule = CAShapeLayerFillRule.evenOdd

Never mind, xcode just lost it's mind - I restarted it and the issue went away!

I got similar issue with you. I've restart xcode and my mac, but this issue cannot resolve. When I comment out that line, It's working well.

fillLayer.fillRule = kCAFillRuleEvenOdd


Add standard app layout

Type

Refactoring

Description
This PR extracts a common layout pattern into a separate layout component. The component has:

the main content area
a top bar
a side bar
a toolstrip alongside the sidebar
(optionally) a drawer

The sidebar opens and closes using CSS transition (which might perform better than react-spring on genome browser page). The drawer also opens and closes using CSS transition (which will almost certainly perform better, because it's based on animating the transform property). When opened, the drawer still leaves a narrow space through which the user can see the main content area.
The layout currently exists only as a Storybook component. If approved, it will be a separate task to integrate it into the app.
The StandardAppLayout component can be viewed in Storybook — either locally, or at https://ensembl.github.io/ensembl-client/storybook/index.html?path=/story/components-shared-components-layout-standardapplayout--with-drawer
Note: the layout component, as it's implemented in this PR, is not responsible for showing the modal inside the sidebar. It is up to the parent component to decide whether it wants to put regular sidebar content or the modal content inside layout's sidebar.

Should the folder src/ensembl/src/shared/components/layout be named as layouts  as we will be having more layouts like this?

Should the folder src/ensembl/src/shared/components/layout be named as layouts as we will be having more layouts like this?

I think I have a mild preference towards layout singular, much like the button singular that we already have (but which contains PrimaryButton and SecondaryButton). From the purely syntactic and conventional perspective, I would rather see import {StandardAppLayout} from '/path/layout' (similar to import {BrowserRouter, StaticRouter} from 'react-router-dom') than import {StandardAppLayout} from '/path/layouts'. But if you have a strong preference of the opposite, we can change layout to layouts.

I don't strongly prefer one way to the other but there are many folders that are named in plural like services, typings, stories, tests, favicons, fonts and also node_modules. I also noticed that Ant Design and Bootstrap calls this section as just Layout.
We can either keep it consistent within our app or follow what the other frameworks are doing. I guess the former will be slightly better?

there are many folders that are named in plural like services, typings, stories, tests, favicons, fonts and also node_modules

That's right. I believe that's determined by how people think of stuff that goes inside those folders — whether as of different things or as of essentially the same thing. Intuitively, tests, stories, fonts, etc. are things that are very different from one another, as are the contents of the components folder, where one "thing" is one component and another "thing" is a different component. But in the case of, say components/button(s) or components/layout(s) we can think of the stuff inside that folder as being either different things (different buttons, different layouts), or being variations of the same thing (variations of a button, variations of a layout).
I would say, that the way of thinking of similar components as of different varieties of the same component is fairly common among developers who build components (in React or other frameworks). That's why it's common, I believe, to see folder structures that say "components/singular-component-name/different-files-related-to-component". In other traditions it is different. For example, in Jekyll projects there is a dedicated _layouts folder — which tells us that Jekyll developers think of layouts as of different individual things, not as variants of the same one.
The naming reflects the way we think about things. I don't believe that the fact that the tests, or the stories, or the services folder are named in plural should have any bearing on how we name a particular component folder. The question to ask here is  — how do we prefer to think about what goes into a given folder: as of many different things or as of a single one.
As a side note, we have a similar situation with files named "helper(s)". Some files are named in singular, some in plural. This may be easier to standardise than folder names.
@imran-ebi do you have any opinion on this?

@azangru I don't have any strong opinions on whether these directory names need to be singular or plural. Both look fine to me and using them interchangeably doesn't seem to do any harm.


Fix our dockerfile (once again)

This should fix a few problems with the dockerfile. It also adds a CI build for it, so we won't have breaking changes being merged in the future.
I think the nix build would also benefit from having a CI check (not implemented here)

I think the nix build would also benefit from having a CI check (not implemented here)

For sure.
Theres a lot Devdebt that would make our life easier. The tests in master dont even run in my local machine.

Even the tests in master dont run in my local machine.

Why not?

failures:
    tests::test_get_block
    tests::test_get_block_hash
    tests::test_get_block_header
    tests::test_get_blockchaininfo
    tests::test_get_height
    tests::test_get_roots
    tests::test_send_raw_transaction
    tests::test_stop

i didnt debuged but, i did literally
git clone git@github.com:vinteumorg/floresta.git
cd floresta
cargo test

Even if its a missing thing that floresta depends, this cannot happen.

You have to cargo build first, it's in the readme

You have to cargo build first, it's in the readme.

As i said, something that i missed. But i still thinks thats something not ideal to the project.
The capacity of cloning a branch and quickly decides if the branch is broken by itself depends on the master at least being portable enough to avoid building.
I see two good ways to make this work:

Refactor start_florestad() so it creates the dir, or just overwrite it (its for tests anyway).

sort tests by its importance for the node functionality. we already have this but by its architecture unit, docs and int. Sorting by minimal or node(for tests that only regards the florestad's functionality as compared to the bitcoin node).

Anyway, this issue is not place for discuss this. Lets open another one to track this if you liked the idea.


Created Github Workflow that auto commit

I have created a Workflow that automatically commits when an issue is opened, a pull request is created, an issue is closed, or an issue is assigned or unassigned.
The issue number that resolves this particular issue is: https://github.com/Rakesh9100/ML-Project-Drug-Review-Dataset/issues/85

To run this Workflow It Needs Some Permission please give it permission to run the workflow

Pls specify what permission are you talking about to provide @Nishitbaria

Hey, @Rakesh9100 , please see my video I have explained the reasons why the workflow is not working and how you can give the necessary permissions.
screen-capture.webm

@Rakesh9100  I have mentioned the video please review it I have explained all things related to the permission that need to run the workflow


On self.m_k in MaskedHiddenLayer

In MADE used in AFLayer, it seems as though hidden_order='sequential' always, and so the m_k values of the hidden layers are arange(0, data_dim).repeat(...), so taking values in {0,...,data_dim-1} :
https://github.com/harvardnlp/TextFlow/blob/0eb8552ae5b370f69e440bca6c6c226b1734c1ba/MADE.py#L38-L39
Also input_order=arange(data_dim) + 1: https://github.com/harvardnlp/TextFlow/blob/0eb8552ae5b370f69e440bca6c6c226b1734c1ba/flows.py#L275
As it is now, the hidden units i at the first hidden layer with m_k[i] = 0 will be unconnected, since previous_m_k = input_order = arange(data_dim) + 1, taking values in {1,...,data_dim}. c.f. how masking is defined:
https://github.com/harvardnlp/TextFlow/blob/0eb8552ae5b370f69e440bca6c6c226b1734c1ba/MADE.py#L44
Should the m_k values of the hidden layers be arange(1, data_dim).repeat(), so that they take values in {1,...,data_dim-1} and you don't get unconnected hidden units? This also seems to be what the MADE paper suggests. I think this is a minor issue, as these unconnected units will be ignored anyways, but think it'd be good to remove redundancy and make the code clearer by aligning it with the method as presented in the original paper.

This is intended, as it provides a natural way to incorporate conditional information. This code comes from more general-purpose flow code, and it's best to keep it as general as possible.

Oh I see, the cond_inp are represented as zeros in previous_m_k, so the hidden units i with m_k[i]=0 will be connected to the conditional input. Thanks for the clarification.


not able to download 1080p video

it automatically fallback to 720p while 1080p is available

I am pleased to inform you that a new version has been released which includes a fix for the reported issue.
npx fem-dl@latest v0.1.7


Implement Spline evaluation

Fixes https://github.com/SouthEndMusic/SplineGrids.jl/issues/3.

Plot a simple random surface with:
using SplineGrids
using GLMakie

cp_grid_size = (5, 6)
degrees = (3, 2)
ns_sample_points = (30, 40)
dim_out = 3

spline_dimensions = SplineDimension.(cp_grid_size, degrees, ns_sample_points)
spline_grid = SplineGrid(spline_dimensions, dim_out)
spline_grid.control_points[:,:,1] .= ones(cp_grid_size[1]) * range(0, 1, length = cp_grid_size[2])'
spline_grid.control_points[:,:,2] .= range(0, 1, length = cp_grid_size[1]) * ones(cp_grid_size[2])'
spline_grid.control_points[:,:,3] .= rand(cp_grid_size...)

evaluate!(spline_grid)

surface(spline_grid.eval[:,:,1], spline_grid.eval[:,:,2], spline_grid.eval[:,:,3], axis=(type=Axis3,))

To do left: complete inline documentation.


[packagers] Flatpak OOB when using BINARY tar.gz archive

trace.log
The actual artifacts (If needed to reproduce): https://github.com/xpipe-io/xpipe_staging/releases/tag/0.5.36

The expectation for  Zip and Tar distributions is that the root entry matches the archive filename, such thatxpipe-portable-linux-x86_64.tar.gz should have a root entry named xpipe-portable-linux-x86_64. However, in this case the root entry is xpipe-0.5.36.
See https://jreleaser.org/guide/latest/concepts/distributions/binary.html#_creating_a_distribution

I see. How strict is this requirement in terms of the possibility to remove this? In my opinion this kinda negates the point of the transform attribute of distributions. Because you have to know the final name when you package it and are therefore not allowed to transform it.

Indeed. The archive assembler may be used to create the archive as its name accepts name templates. But if the archive is created by external means then the problem persists. It may be possible to inspect an archive whose root entry has a different value than the expected one but I'm sure other packagers will continue to fail. For example, Docker and Macports  still expect the root entry to match the archive name.
The code needed to solve this issue is not trivial because of the different places that need to be updated but it's possible.

Oh well that sounds complicated. I can just create another jreleaser distribution specifically for packagers as I don't want to change my naming scheme for my other releases.
I guess jreleaser should warn/error when such an invalid archive name is detected.

Perhaps. However, I think relaxing the expectations would also be a good thing to do. I'll run a code spike and see how far I can get.

One more thing, it's also expected that artifacts define a version in their names. xpipe-portable-linux-x86_64.tar.gz does not define a version. This could also cause trouble down the line. Would xpipe-portable-0.5.36-linux-x86_64.tar.gz be acceptable?

Found one more thing. The unixArchive inspector expects a single distribution with a bin directory. xpipe configures 2 applications as part of the distribution: app and cli. This results in binary files being copied without executable permissions:
 - install -Dm644 app/bin/xpiped /app/app/bin/xpiped
 - install -Dm644 cli/bin/xpipe /app/cli/bin/xpipe

I have a fix for arbitrary binaries in flatpak and spec. Still, compound distributions won't work with other packagers. Flatpak descriptor for xpipe should find the following binaries
 - install -Dm755 app/bin/xpiped /app/app/bin/xpiped
 - install -Dm755 app/lib/runtime/bin/java /app/app/lib/runtime/bin/java
 - install -Dm755 app/lib/runtime/bin/keytool /app/app/lib/runtime/bin/keytool
 - install -Dm755 app/lib/runtime/bin/rmiregistry /app/app/lib/runtime/bin/rmiregistry
 - install -Dm755 app/lib/runtime/bin/xpiped /app/app/lib/runtime/bin/xpiped
 - install -Dm755 cli/bin/xpipe /app/cli/bin/xpipe

One more thing, it's also expected that artifacts define a version in their names. xpipe-portable-linux-x86_64.tar.gz does not define a version. This could also cause trouble down the line. Would xpipe-portable-0.5.36-linux-x86_64.tar.gz be acceptable?

Where exactly is that requirement coming from? The reason I don't include the version is that it makes it easier to handle the artifacts in other scenarios like in a workflow file where I can just refer to them via the fixed name.

Found one more thing. The unixArchive inspector expects a single distribution with a bin directory. xpipe configures 2 applications as part of the same distribution: app and cli. This results in binary files being copied without executable permissions:
 - install -Dm644 app/bin/xpiped /app/app/bin/xpiped
 - install -Dm644 cli/bin/xpipe /app/cli/bin/xpipe

Technically these 2 applications should be distributed with separate archives. There's no current workaround for packagers to support this layout. Have to think for a moment how to expose such configuration as some packagers may be OK with this kind of "compound" distribution (flatpak, spec) but others may not (sdkman).

The main reason for this separation was that some users confused the executables when using the portable distribution. I guess for packagers, I can merge these two directories into one such that there is only one bin directory.

Where exactly is that requirement coming from? The reason I don't include the version is that it makes it easier to handle the artifacts in other scenarios like in a workflow file where I can just refer to them via the fixed name.

The expectation from JReleaser based on common practices for naming artifacts is <artifactId>-<version>[-classifier]. In this case artifactId is just a name (thus 'xpipe-portable' is fine) while classifier matches the target platform.

The main reason for this separation was that some users confused the executables when using the portable distribution. I guess for packagers, I can merge these two directories into one such that there is only one bin directory.

Based on what I can gather by looking at the files, app is a desktop application that bundles a Java runtime (built with jpackage) while cli is a command line application compiled to native code using GraalVM native (?). If app and cli have no dependency to one another that is, you may install x-pipe/app separate from x-pipe/cli, then they technically should have separate distributions (archives) for NATIVE_INSTALLER (app) and BINARY (cli). In the case of flatpak, the app manifest is different for desktop apps than for cli apps (see stereotype property).
Now, I understand is convenient for users to download a single archive that delivers both apps, both default templates for package managers as currently set by JReleaser expect a single distribution type per archive. It may be possible to support multiple distributions but you'll have to provide your own templates for the time being. If this were really to be needed then the template generate command will help you get started.

For more reference, https://github.com/jreleaser/jreleaser/tree/main/core/jreleaser-templates/src/main/resources/META-INF/jreleaser/templates/binary/flatpak defines the set of possible templates used by the flatpak packager for both cli and desktop stereotypes of a BINARY distribution.
The key difference is the .desktop file and an extra <launchable> element in the metainfo file.
Moreover, it's possible to bundle multiple binaries in the same distribution and let flatpak know about them (see https://www.freedesktop.org/software/appstream/docs/chap-Metadata.html#tag-provides) but the default JReleaser template does not handle that case (it expects a single binary at the moment). This could be added to JReleaser as well. Filed https://github.com/jreleaser/jreleaser/issues/1326

Where exactly is that requirement coming from? The reason I don't include the version is that it makes it easier to handle the artifacts in other scenarios like in a workflow file where I can just refer to them via the fixed name.

The expectation from JReleaser based on common practices for naming artifacts is <artifactId>-<version>[-classifier]. In this case artifactId is just a name (thus 'xpipe-portable' is fine) while classifier matches the target platform.

I can look into adapting that naming scheme for the artifacts, that shouldn't be an impossible task.

The main reason for this separation was that some users confused the executables when using the portable distribution. I guess for packagers, I can merge these two directories into one such that there is only one bin directory.

Based on what I can gather by looking at the files, app is a desktop application that bundles a Java runtime (built with jpackage) while cli is a command line application compiled to native code using GraalVM native (?). If app and cli have no dependency to one another that is, you may install x-pipe/app separate from x-pipe/cli, then they technically should have separate distributions (archives) for NATIVE_INSTALLER (app) and BINARY (cli). In the case of flatpak, the app manifest is different for desktop apps than for cli apps (see stereotype property).

The app and cli actually have a dependency on each other. The app can't be compiled using GraalVM native image as it dynamically loads modules at runtime. The cli is therefore separate to decrease start up time, and handle the instance management of the daemon. You can interact with the daemon through the cli, e.g. the command to start the app would be xpipe open, that is also the launch command that I use in the .desktop file.
I guess my project does not fit into the standard distribution definitions, so I will see how I can work around that.

The app and cli actually have a dependency on each other. The app can't be compiled using GraalVM native image as it dynamically loads modules at runtime. The cli is therefore separate to decrease start up time, and handle the instance management of the daemon. You can interact with the daemon through the cli, e.g. the command to start the app would be xpipe open, that is also the launch command that I use in the .desktop file.
I guess my project does not fit into the standard distribution definitions, so I will see how I can work around that.

Gotcha. In this case a single archive as you have it today is the way to go. Which means the default template provided by flatpak may fall short of what you need. I'd recommend generating a local copy of the templates and adapt them to your needs. Make sure to set desktop as stereotype for the distribution (you could also set it at the project level).
gradle jreleaserTemplateGenerate --distribution-name portable --packager-name flatpak.

So to avoid further problems, should I move the portable distributions to the jreleaser files type instead if they don't fit the usual criteria for any distribution type? And then create a separate packager distribution that somehow fits the criteria.

It depends. If you’d like to configure additional packagers such as homebrew, sdkman, scoop, and perhaps chocolatey then yes, a single distribution type per archive would be recommended if using the default templates provided by JReleaser.
But xpipe’s case is different as app depends on cli for better user experience. If separate distros and archives were to be published now users would have to install 2 distros instead of just one.
JReleaser will support multiple binaries per distribution in the future. That should help you avoiding custom templates for most cases.
Now for Homebrew, I think I’d be best if the app distro were to be packaged as a .dmg or .pkg which should make the formula contain a cask (that’s how UI apps are installed in osx) while the cli could remain as a regular formula. In which case 2 distinct formulas will be needed, with xpipe (app) depending in xpipe-cli (cli), such that installing the former also installs the latter while installing the latter does no install the former (assuming cli may be used in standalone mode).
Long story short, I think keeping the portable archive as you have it to be installed via flatpak is OK. But also using a custom template to expose 2 binaries instead of just the one. Do not modify the archive nor split into several distros unless you want to explore other packagers where the aforementioned restrictions apply. Hopefully JReleaser will provide what’s needed in a timely manner.

🎉 This issue has been resolved in v1.6.0 (Release Notes)


Update README.md

Users should build the site before attempting to run the server

Running the server is just a local web server for viewing changes as you make them (refresh -> change is visible). Building compiles the entire website from /source/ to /build/. You only build when outputting to production.


Redesign DeserializerAdapter trait to support deserializing using DeserializeSeed

Since the CDR format is not self-describing, the target type needs to be known for deserialization. The current RustDDS implementation ensures this by requiring a serde::Deserialize implementation for all target types. Unfortunately this makes it impossible to deserialize when the type information is only available at runtime. For example, it is not possible to base the deserialization on type information parsed from ROS2 msg files at runtime.
This pull request solves this limitation by adding basic support for deserialization based on serde's DeserializeSeed trait, which allows to use additional state for deserialization.
The main changes are:

Add a new Decode trait for deserializing a byte slice with access to the RepresentationIdentifier. The trait is implemented for the following new types:

PlCdrDeserializer is an empty newtype that implements Decode for all types that implement PlCdrDeserialize.
CdrDeserializeDecoder is an empty newtype that implements Decode for all types that implement serde::Deserialize.
CdrDeserializeSeedDecoder wraps a DeserializeSeed implementation and implements Decode using the seed for deserialization.

Redesign the DeserializerAdapter trait

Add a from_bytes_with method that takes an additional decoder: impl Decode argument. The method has a default implementation that calls the decoder.
Add a new associated type named Deserialized, which describes the type that the decoder will deserialize to.

Most of the time, the type is equal to the generic type D.
Some adapters, mainly DAWrapper, need to do an additional transformation after deserializing. This transformation is possible by implementing a new transform_deserialized trait method. For example, DAWrapper wraps the deserialized value into a NoKeyWrapper.

Create a new DefaultDecoder sub-trait, which describes adapters that can be used without supplying a decoder (since they provide a default one).

Gate the from_bytes and from_vec_bytes methods on the DefaultDecoder trait and provide a default implementation using the *_with variants and the default decoder.
The DefaultDecoder trait is implemented:

for CDRDeserializerAdapter if D implements serde::Deserialize.
for PlCdrDeserializerAdapter if D implements PlCdrDeserialize.

Use the DefaultDecoder trait to support both stateful and stateless deserialization in most functions, without duplicating the code
Create new try_take_one_with and as_async_stream_with API methods to allow ros2-client and others to plug in custom stateful deserializers.

I tried my best to limit the scope of this PR as much as possible and to document all the changes. Unfortunately, I failed to make this a non-breaking change, mainly because of the new DeserializerAdapter design. I hope you still consider this PR, as this feature is critical for our use case in the dora project.
Please let me know if you have any questions or ideas for improvements!
(edit: The commit list got a bit messy, sorry. I can squash if you like.)

With these changes, we can update ros2-client to provide take_seed and async_stream_seed functions on Subscription. I opened a draft PR for this at https://github.com/jhelovuo/ros2-client/pull/20 to show the required ros2-client changes.

I'm not sure why the "Code Coverage" CI job fails, but it appears to be broken on the master branch as well: https://github.com/jhelovuo/RustDDS/actions/workflows/coverage.yml . So I don't think that the failure is related to this PR.

Hello,
Thank you for the interesting idea and PR. I was not aware of the DeserializeSeed trait before. I can see the use case here.
If the deserialization would be changed as you propose, how would the serialization go then?

Serialization doesn't need to be changed, since the serde::Serialize trait gives access to the runtime value that should be serialized. So we can use the type information parsed from ROS2 message files in a custom Serialize implementation.
For example:
pub struct TypedPythonValue<'a> {
    pub value: &'a pyo3::Any,              // given from Python
    pub type_info: &'a Ros2TypeInfo,       // parsed from ROS2 message files
}

impl serde::Serialize for TypedValue<'_> {
    fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
    where
        S: serde::Serializer,
    {
        match &self.type_info.data_type {
            DataType::UInt8 => {
                // the ROS2 topic expects an `u8` value, so try to convert the Python value
                let value: u8 = self.value.extract().map_err(serde::ser::Error::custom)?;
                serializer.serialize_u8(value)
            }
            .... // handle other ROS2 data types
            _ => todo!(),
        }
    }
}

This approach doesn't work with the standard serde::Deserialize trait because it is completely stateless, i.e. there is no self argument. So for deserialization based on runtime type information, we need to use DeserializeSeed.

Codecov Report
Attention: Patch coverage is 52.17391% with 66 lines in your changes are missing coverage. Please review.

Project coverage is 73.26%. Comparing base (28ed26f) to head (4e8e37d).
Report is 130 commits behind head on master.

Files
Patch %
Lines

src/dds/no_key/simpledatareader.rs
0.00%
25 Missing :warning:

src/dds/with_key/simpledatareader.rs
48.64%
19 Missing :warning:

src/dds/adapters.rs
52.94%
16 Missing :warning:

src/serialization/cdr_deserializer.rs
77.77%
6 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #313      +/-   ##
==========================================
- Coverage   73.53%   73.26%   -0.28%
==========================================
  Files          94       95       +1
  Lines       19845    20192     +347
==========================================
+ Hits        14594    14793     +199
- Misses       5251     5399     +148

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

Thanks for keeping your PR up to date. I assume this means you are using it for yourself.
We are considering accepting this PR, even though it adds extra complication. We may have another use case where it will be useful. This is just to let you know that the PR is not dead.

Thanks for the update! Yes, we're using this PR for the https://github.com/dora-rs/dora project. We need it because we precompile Python bindings on a different machine, which need to be able to deserialize ROS2 messages based on type information parsed at runtime.

Looking at the central part of the new design, namely, new code to adapters.rs , terms "decode" and "transform" are used. Are these interchangeable? If yes, perhaps only one term should be used for clarity?

Looking at the central part of the new design, namely, new code to adapters.rs , terms "decode" and "transform" are used. Are these interchangeable? If yes, perhaps only one term should be used for clarity?

This question makes no sense. Apparently, "decode" and "deserialize" are more like synonyms. So the pipeline is
byte slice → deserialize/decode → value of type Self::Deserialized → call
[Self::transform_deserialized] → value of type D
And it makes sense to use "decode", because the term "deserialize" is already taken.
Am I now on the correct track?

Another question: What is the use case for transform_deserialized() ?
In this PR and the corresponding ros2-client PR it always seems to be the identity function.

I just created a new branch deserialize-with-seed for developing this feature. Please redirect the PR there.

Is there a specific reason in
fn from_bytes_with<S>(
      input_bytes: &[u8],
      encoding: RepresentationIdentifier,
      decoder: S,
    ) -> Result<D, S::Error>
    where
      S: Decode<Self::Decoded>,
    {
...
    }

for the decoder to be passed by value rather than reference? Correspondingly, Decode::decode_bytes() takes self rather than &self.  (Or maybe &mut self?)
If using a reference is possible, then we could get rid of the Clone requirement in DefaultDecoder::Decoder.  Requiring Clone is a bit awkward, because it would really do the cloning at every sample processed.

So the pipeline is
byte slice → deserialize/decode → value of type Self::Deserialized → call [Self::transform_deserialized] → value of type D

Yes, exactly!

Another question: What is the use case for transform_deserialized() ?

There are some adapters that add a wrapper around the deserialized values. With the normal Deserialize trait, this could happen behind the scenes because the adapters could just call deserialize on the desired type. With DeserializeSeed, the adapters need to explicitly specify the type that the DeserializeSeed instance should create, which might be different than the output type D.

In this PR and the corresponding ros2-client PR it always seems to be the identity function.

Yes, it's almost always the identity function. The one exception is no_key::DAWrapper, which wraps a DeserializerAdapter<D>, but returns a DeserializerAdapter<NoKeyWrapper<D>>. So it's transform_deserialized call wraps the result in a new NoKeyWrapper instance:
NoKeyWrapper::<D> {
    d: DA::transform_deserialized(deserialized),
}

(This implementation is at src/dds/no_key/wrappers.rs#L94-L96.)

I just created a new branch deserialize-with-seed for developing this feature. Please redirect the PR there.

I just tried, but it looks like all the commits of this PR are already part of the branch.

Is there a specific reason in
fn from_bytes_with<S>(
      input_bytes: &[u8],
      encoding: RepresentationIdentifier,
      decoder: S,
    ) -> Result<D, S::Error>
    where
      S: Decode<Self::Decoded>,
    {
...
    }

for the decoder to be passed by value rather than reference? Correspondingly, Decode::decode_bytes() takes self rather than &self. (Or maybe &mut self?)
If using a reference is possible, then we could get rid of the Clone requirement in DefaultDecoder::Decoder. Requiring Clone is a bit awkward, because it would really do the cloning at every sample processed.

I just added that code last week to get things working again with the latest changes from master. I'm not satisfied with that part of the implementation either, so it would be great if we could find a better solution.
The reason for https://github.com/jhelovuo/RustDDS/pull/313/commits/4e8e37d6405a21f6d54c1d1255e7ae1d65f41720 was the loop introduced in https://github.com/jhelovuo/RustDDS/commit/6908f40f68ddebbcc3f059b02cb5f7e64cac4986, which calls deserialize_with multiple times.
I agree that taking a (mutable?) reference to the decoder would be the cleaner solution and would be fine for our use case. Unfortunately, the DeserializeSeed::deserialize method takes self by value, so I'm not sure how we could avoid a Clone bound...
(Side note: If your DeserializeSeed type T only needs &self, you could implement DeserializeSeed for &T as well. Then the .clone() call would be just a pointer copy.)

So the pipeline is
byte slice → deserialize/decode → value of type Self::Deserialized → call [Self::transform_deserialized] → value of type D

Yes, exactly!

Ok. Your technical implementation is brilliant. This is quite complicated abstract metaprogramming.
In order to make it more accessible to others, I decided to rename some things. The basic renaming idea is
deserialize = decode + transform.
In your original code, there was confusion between deserialize and decode, as these were used as synonyms. Now it should be more clear that the whole two-step process is "deserialize" and the first step is called "decode".
I am migrating this PR to PR #334 , so we can continue the work there.

Is there a specific reason in

...code...

for the decoder to be passed by value rather than reference? Correspondingly, Decode::decode_bytes() takes self rather than &self. (Or maybe &mut self?)
If using a reference is possible, then we could get rid of the Clone requirement in DefaultDecoder::Decoder. Requiring Clone is a bit awkward, because it would really do the cloning at every sample processed.

I just added that code last week to get things working again with the latest changes from master. I'm not satisfied with that part of the implementation either, so it would be great if we could find a better solution.
The reason for 4e8e37d was the loop introduced in 6908f40, which calls deserialize_with multiple times.

The new loop was logically necessary, because it can happen that in a with_key DDS topic, e.g. most of the Discovery topics, we could receive a Dispose instead of data update in a DATA submessage. The Dispose is indicated by sending a Key instead of Data, or alternatively, sending an MD5 hash of the Key.
In case we, as a DataReader, join an already long-running Topic, it may have some past data values published that we can no longer receive, because the History buffers are not deep enough. But we can at some point receive Dispose notifications of these samples we have never heard of.  In case the Dispose was via a hash only, there is no way we can tell the application about the Dispose, because we do not know the actual key, so it is best to just ignore it, and move on to the next incoming sample. This is why the looping is needed.
This is something we could perhaps work around to avoid the need for cloning.

I agree that taking a (mutable?) reference to the decoder would be the cleaner solution and would be fine for our use case. Unfortunately, the DeserializeSeed::deserialize method takes self by value, so I'm not sure how we could avoid a Clone bound...
(Side note: If your DeserializeSeed type T only needs &self, you could implement DeserializeSeed for &T as well. Then the .clone() call would be just a pointer copy.)

Ok, the pass-by-value of self in DeserializeSeed::deserialize is something we cannot work around.
Then we must keep the Clone bound, and document that there will be quite a lot of cloning, about one .clone() per deserialized sample, so the application designer knows to invest in making it cheap, like a pointer copy.

Thanks a lot!


Add missing textarea styles for the feedback component

Missed in https://github.com/alphagov/static/pull/2680

BeforeAfter

</table

Makes sense, but it seems like this only happens for pages rendered by some applications - where is this problem occurring?

When the application (e.g. frontend) imports the styles there's no problem, but some app (e.g. collections) don't have textareas – hence the problem.


Review and develop new design for VectorStore

Current vectorstore interface is lacking in two important areas

Does not support other types of search other than similarity search
Does not support full api coverate for admin cases, e.g. delete, update operations

In its current form it also assumes that the content is next to the embeddings. Does it make sense to separate these two out? One could keep the actual data on disk, while keeping the (embedding) vector in a different place (e.g. Postgres).

In its current form it also assumes that the content is next to the embeddings. Does it make sense to separate these two out? One could keep the actual data on disk, while keeping the (embedding) vector in a different place (e.g. Postgres).

Such a case could be handled by using delegation or better, a domain object that describes the file path (an identifier, actual path, or some other form of resource locator).

I started exploring a Domain-bound VectorStore abstraction mostly related to lifecycle usecases (insert, update, delete). The overlap between AI and data is where we want to associate data with some embedding and where we want to retrieve data (by similarity returning some way to access content).
Most databases provide some sort of batching for inserting data, a few other batching for updating/deleting. It makes sense to distinguish updating from inserting.
Going a step further, a lot of applications have some sort of data already that they want to associate with embeddings. Using a domain-object-oriented approach makes much more sense than trying to pour everything through Document, especially since the additional metadata is the actual data that is relevant for other cases.
A typical application would have batch orchestration for initial data embedding while transactional updates target individual objects. I assume that Spring AI isn't the best place to provide production-ready batching, instead some orchestrator like Spring Batch might provide something.
It would be good to have a component that can associate objects with embeddings (T Embedder.embed(T), List<T> Embedder.embedAll(List<T>)) potentially with different strategies (a single vector field, a vector field per model identifier, a data row per model identifier). So separating VectorStore from the actual EmbeddingModel would be a good thing.
Another aspect is that there should be a way to translate domain objects into the representation that is needed for RAG (maybe a DomainVectorStoreRetriever<T> accepting Function<T, Document> to glue components into their current form).
Keeping the lifecycle separate, supporting it with a tiny abstraction that isn't overly prominent for getting-started-quickly seems a neat balance.
I toyed around with an initial domain-based design at https://github.com/mp911de/spring-ai/commit/c58f6ee9aec598a7aa689e7134cc8c13de14b893 that explores the domain part. From the lifecycle side, it feels much more natural and avoids duplicating model-metadata of the "other component" that deals with the non-AI part of data.

In its current form it also assumes that the content is next to the embeddings. Does it make sense to separate these two out? One could keep the actual data on disk, while keeping the (embedding) vector in a different place (e.g. Postgres).

Such a case could be handled by using delegation or better, a domain object that describes the file path (an identifier, actual path, or some other form of resource locator).

After my comment I indeed explored that idea and implemented it. It makes a lot of sense to extend already existing domain entities with embeddings and just treat it as yet another way of retrieving data from the store.

Another aspect is that there should be a way to translate domain objects into the representation that is needed for RAG (maybe a DomainVectorStoreRetriever accepting Function<T, Document> to glue components into their current form).

I've had an experience that might make sense to share here. We are processing documents that surpass the limit of 8191 tokens of the current embedding models and for that reason the data cannot be embedded in its entire form. The only way I see is to chunk the documents into chunks of max 8000'ish tokens and embed them per chunk, which leads to a vector per chunk. However, a chunk (quite literally) tells only part of the story and you might miss context when supplying only matching chunks to the LLM via RAG. So I was playing with the idea of having multiple embedding vectors refer to the same object, such that when any of the vectors of the chunks match that you supply retrieve the entire document and use that for RAG. I think that kind of supports the idea of having a DomainVectorStoreRetriever<T>.

Regarding the design of VectorStore, I have the following opinions:

The current ETL pipeline is implemented synchronously, which feels unnatural and lacks the essence of a “pipeline.” If the design is revised, I suggest considering a “stream”-based implementation. This approach would allow data to flow through the pipeline while transforming, making it more intuitive. An example of such an interface is discussed in #1253.

Currently, VectorStore has an unnecessary dependency on the embedding model, resulting in boilerplate code in methods like doAdd. This creates challenges during refactoring processes, such as in getEmbedding, as highlighted in #1239. By accepting DocumentWithEmbedding—a document transformed via a stream—rather than calculating embeddings internally, VectorStore’s dependency on the embedding model can be eliminated.


feat: sub modules for pmndrs and native components

Hey, @Tinoooo I just pushed the docs with the native passes documentation. I really don't know how to document Output could you help me with that one and a review?


Proposed Eval command.

Compensation needed for the lack of "self" in bot code. I also forgot to define self._eval = {}

Rewrote the entire thing myself, but squash merge on CLI kind of messed up.
Press F.