ScrollPane layout problem with ontop scrollbars and content smaller than scrollpane area

When using on top scroll bar for a scroll pane with a content widget smaller than the display area, the layout seems incorrect to me since the content widget won't get extended to the complete scrollpane area (this is a followup to PR #2709).
If we take, the ScrollPaneScrollBarsTest test and just change the displayed text by a rect (to clearly see the bounds) with this simple modification ;
// vertOnlyLeftTable.add(new Label("VERT-ONLY-LEFT", skin));
// vertOnlyRightTable.add(new Label("VERT-ONLY-RIGHT", skin));
vertOnlyLeftTable.add(new Image(skin.getDrawable("default-rect"))).height(20).expandX().fill();
vertOnlyRightTable.add(new Image(skin.getDrawable("default-rect"))).height(20).expandX().fill();

You get the following result :

and when scrolling:

I would think that the correct behavior would be to have the content widget expands to the complete scrollpane area.
Looking at the code, this seems to be due to the fact that the size of the widget is set after adjusting the display area of the content pane:
		if (fade) {
			// Make sure widget is drawn under fading scrollbars.
			if (scrollX) areaHeight -= scrollbarHeight;
			if (scrollY) areaWidth -= scrollbarWidth;
		} else {
			if (scrollbarsOnTop) {
				// Make sure widget is drawn under non-fading scrollbars.
				if (scrollX) widgetAreaBounds.height += scrollbarHeight;
				if (scrollY) widgetAreaBounds.width += scrollbarWidth;
			} else {
				// Offset widget area y for horizontal scrollbar at bottom.
				if (scrollX && hScrollOnBottom) widgetAreaBounds.y += scrollbarHeight;
				// Offset widget area x for vertical scrollbar at left.
				if (scrollY && !vScrollOnRight) widgetAreaBounds.x += scrollbarWidth;
			}
		}

		// If the widget is smaller than the available space, make it take up the available space.
		widgetWidth = disableX ? areaWidth : Math.max(areaWidth, widgetWidth);
		widgetHeight = disableY ? areaHeight : Math.max(areaHeight, widgetHeight);

A way to make it better would be to extend the size of the content widget when scrollbars are on top before adjusting the size of the display area (if you agree, I will prepare a proper PR):
		if (scrollbarsOnTop) {
			// If the widget is smaller than the available space, make it take up the available space.
			widgetWidth = disableX ? areaWidth : Math.max(areaWidth, widgetWidth);
			widgetHeight = disableY ? areaHeight : Math.max(areaHeight, widgetHeight);

			if (fade) {
				// Make sure widget is drawn under fading scrollbars.
				if (scrollX) areaHeight -= scrollbarHeight;
				if (scrollY) areaWidth -= scrollbarWidth;
			} else {
				// Make sure widget is drawn under non-fading scrollbars.
				if (scrollX) widgetAreaBounds.height += scrollbarHeight;
				if (scrollY) widgetAreaBounds.width += scrollbarWidth;
			}
		} else {
			if (fade) {
				// Make sure widget is drawn under fading scrollbars.
				if (scrollX) areaHeight -= scrollbarHeight;
				if (scrollY) areaWidth -= scrollbarWidth;
			}
			// Offset widget area y for horizontal scrollbar at bottom.
			if (scrollX && hScrollOnBottom) widgetAreaBounds.y += scrollbarHeight;
			// Offset widget area x for vertical scrollbar at left.
			if (scrollY && !vScrollOnRight) widgetAreaBounds.x += scrollbarWidth;

			// If the widget is smaller than the available space, make it take up the available space.
			widgetWidth = disableX ? areaWidth : Math.max(areaWidth, widgetWidth);
			widgetHeight = disableY ? areaHeight : Math.max(areaHeight, widgetHeight);
		}

Using this little modification, I made the following test situations ;

content smaller/bigger than scrollpane, ontop = true, fade = true, => Ok
content smaller/bigger than scrollpane, ontop = true, fade = false, => Ok
content smaller/bigger than scrollpane, ontop = false, fade = false, => Ok
content smaller/bigger than scrollpane, ontop = false, fade = true, => somewhat wrong (view area clipped where the scrollbar would be) => but is there any use case for this ?

What do you think ?
NB : On a side note, I think there is a tiny little bug in ScrollPaneScrollBarsTest, the initial state of scrollpane flags are not set. You need to add this at the end of the constructor to get correct result (I will do a proper PR when discussion is ok):
		for (ScrollPane pane : scrollPanes) {
			pane.setFadeScrollBars(doFade);
			pane.setScrollbarsOnTop(doOnTop);
		}

Well, my commit fixes scrollbars on top with scrollbars at left or top edge, but doesn't address that the widget doesn't fill the size of the scrollpane. Hmm.

Yep, I was checking it at the moment. Needs to adjust when extending the content widget.

Think I got it!

Nice ! thanks. Just added a little PR for the remaining bug in the test (ontop is supposed to be to true but it is not set to the scrollpanes)


Describe full type-resolution algorithm in plain English

Datatype-resolution is most important part of schema-ddl. All logic is inside TypeSuggestions.scala module, but it should be described in plain English somewhere in README.md or discourse at least.

Great idea - I think we should add this to the Iglu wiki as it's core behavior.

Migrated to iglu.


[QoL]Add a !online or !timeonline command, allowing for your total playtime for your current session to be broadcasted to other runelite users

Hello!
I'm sure a lot of players who use runelite also have pures, or other builds that they are frequently creating and spending many hours training to perfection. I know I do!
I'm asking if it's possible to create a command that would show your total playtime for your current session to other runelite users that would help you combat situations where you are going to either reset your aggression timers, or bank, and coming back to someone being in your spot, who have recently or just logged into your world.
Jagex changed how drops function a while back, and if you log into a world, you will not be able to see any drops for x amount of minutes before having been logged into said world, so a lot of people who log into worlds are having a hard time believing people who claim to have been there for many hours already, and this causes a lot of conflicts and arguments between players that is unnecessary and shouldn't be taking place to begin with.
I feel like this is very much a quality of life update, and I am sure a lot of people would appreciate an update like this coming to the client. The client is already great, and has many awesome features readily available, and that's awesome!
Thank you for reading this.
P.S.
I realize that you can't really hold jeopardy on a spot, or a world at all, but we have played this game for many years together, and there has always been a mutual respect for one another, and I think that adding this feature would just add another layer to that.

I don't think this would be useful as players could spoof that information very easily.

I like this, its innocent enough and people can spoof our existing stuff already anyway.

I don't think this would be useful as players could spoof that information very easily.

This would easily be avoidable by simply adding a colour that doesn't exist in the game to any text broadcasted using this command. Honestly baffled you hadn't thought of this.

I like this, its innocent enough and people can spoof our existing stuff already anyway.

Sure, they could try to do this, but it would be, as the aforementioned states, be avoidable by simply adding a colour to the next shown when using this command.

I don't think this would be useful as players could spoof that information very easily.

This would easily be avoidable by simply adding a colour that doesn't exist in the game to any text broadcasted using this command. Honestly baffled you hadn't thought of this.

I like this, its innocent enough and people can spoof our existing stuff already anyway.

Sure, they could try to do this, but it would be, as the aforementioned states, be avoidable by simply adding a colour to the next shown when using this command.

"Spoofing" means sending forged http requests to the server which shares the data between clients, not people typing out the text themselves. So using "color" would not fix this at all.
That said, we have a number of spoofable things already like !kc. I think as long as there is no financial incentive for people to spoof things it is mostly acceptable to have, as it limits it to just a handful of isolated people doing it instead of having organized groups of people trying to do it.

I don't think this would be useful as players could spoof that information very easily.

This would easily be avoidable by simply adding a colour that doesn't exist in the game to any text broadcasted using this command. Honestly baffled you hadn't thought of this.

I like this, its innocent enough and people can spoof our existing stuff already anyway.

Sure, they could try to do this, but it would be, as the aforementioned states, be avoidable by simply adding a colour to the next shown when using this command.

"Spoofing" means sending forged http requests to the server which shares the data between clients, not people typing out the text themselves. So using "color" would not fix this at all.
That said, we have a number of spoofable things already like !kc. I think as long as there is no financial incentive for people to spoof things it is mostly acceptable to have, as it limits it to just a handful of isolated people doing it instead of having organized groups of people trying to do it.

Ah, yes. I've seen people fake kill counts before, and to my knowledge, this is going into the files, and editing the contents. I'm not really savvy when it comes to coding, so I wouldn't know where to look; but wouldn't this just result in a static time that doesn't change (unless they do it themselves, every time they wish to use the command) when you use the command? A person using this, would use !online, or !timeonline and it would say [03:43:57] for example, twice, when using the command, even if the command is executed with an interval of 20 seconds or more?
Just going by what I've been told by other players, and I'm just submitting this to help people who are having troubles finding worlds due to being low levels and targeted as a result thereof.
Thanks for your reply!

wouldn't this just result in a static time that doesn't change (unless they do it themselves, every time they wish to use the command) when you use the command? A person using this, would use !online, or !timeonline and it would say [03:43:57] for example, twice, when using the command, even if the command is executed with an interval of 20 seconds or more?

It could be that way if it was done poorly, but one could also just use the client code/developer tools and modify the plugin to make it send the real time + 2 hours or whatever they want instead of the intended value.
Regardless, I agree with deathbeam, this is harmless so sure why not. But I don't think players in the examples you describe would change their actions because of this command.

It could be that way if it was done poorly, but one could also just use the client code/developer tools and modify the plugin to make it send the real time + 2 hours or whatever they want instead of the intended value.
Regardless, I agree with deathbeam, this is harmless so sure why not. But I wouldn't expect players in the examples you describe would change their actions because of this command.

I see!
I was not aware that this was a thing... My bad. Like I said, I'm not very savvy when it comes to coding, and I had no idea that developer mode is something available to anyone and everyone.
To adress your concern though, this was mostly directed at situations where you accidentally end up crashing somebody, because there is a confusion between the two players.
You run to the bank, to restock on supplies, and come back to find that somebody has taken your spot, so you could nicely ask them to leave, and they would often times be hesitant, because you have no way of proving otherwise; with this command, I hope that more people are able to retain their training spots, or any other spot for activity by quoting their total playtime for their session, in hopes of the other player recognizing this, and leaving. The same would apply to running to reset the aggression timers that have expired.
I agree though that anyone could just spoof this, and use it to trick others, but would it be possible to not only have this command display your total playtime for this session, but also any experience and in what skill you've gained as well?
This game is very competitive, and making builds for activities such as PKing, or bossing/revenants has never been more relevant than now, so it is rough finding a world to train your account in. Though, most people aren't dicks to one another, and would simply hop if the opposing player has the grounds to back up their claims.
I am just trying to be nice here, but I see a lot of people fall victim to this and it sucks to see, because this is a game we all know and love. I wish to see it prosper for a lot longer, because it is a great game. I grew up playing this game, so it is only natural for me to feel this sentimental towards it, and its players.
If this suggestion lacks grounds, or enough support to warrant an implementation, there is always tomorrow. I do hope that this garners enough attention though, to make it into the live game.
Thank you for your reply, and your clarification.

you could nicely ask them to leave, and they would often times be hesitant, because you have no way of proving otherwise (...) most people aren't dicks to one another, and would simply hop if the opposing player has the grounds to back up their claims.

I agree most people aren't as toxic as a lot of people say. And I can relate to the situations you're describing, I've trained many pures at sand crabs. I just think that those who would be willing to leave with this timer, already would have been willing to leave previously. There already kind of is some level of 'proof' that you were already there in this situation: you arrive to that spot very shortly after the other player got there. If that wasn't enough for them to believe you and hop, I think it would be rare that this timer makes a difference. You could also just have been logged into that world doing something else. Again though: nothing against the suggestion, just wanted to clarify these concerns in relation to your proposed use cases.


Why isn't Bootstrap using placeholder selector to eliminate unneeded classes and naming collision?

Hi!
I haven't found other issues talking about this, so here goes. I've always wondered why Bootstrap still uses extra classes that don't mean something on their own like btn class. You wouldn't and shouldn't use that on its own as it is there to provide some sort of inheritance to share with other classes, namely btn-primary, btn-block, etc....
However, why do we have to do this while we have a placeholder selector feature in Sass? It would allow for the same effect except without having actual CSS for the btn class.
So looking at the source code what I mean is that instead of having code that looks like this :
.btn {
  display: inline-block;
  font-weight: $btn-font-weight;
  color: $body-color;
  text-align: center;
  vertical-align: middle;
  user-select: none;
  background-color: transparent;
  border: $btn-border-width solid transparent;
}

.btn-block {
  display: block;
  width: 100%;
}

We can have something that looks like this:
%btn {
  display: inline-block;
  font-weight: $btn-font-weight;
  color: $body-color;
  text-align: center;
  vertical-align: middle;
  user-select: none;
  background-color: transparent;
  border: $btn-border-width solid transparent;
}

.btn-block {
  @extend %btn;
  display: block;
  width: 100%;
}

That way developers won't need to remember to add a useless class like btn or border on every element. Furthermore, since having a class named btn for buttons is already a popular thing, developers can include Bootstrap later in a project without having to account for what a class like that may break.
This also doesn't result in duplicate CSS, since classes that inherit from the same placeholder selector are grouped together in one selector in the final CSS output.
One more advantage to this that it wouldn't break any existing code-base. Yes a class like btn would stop having its own CSS but that's okay because it was never used by itself in the first place.
Thank you for reading this. I'm interested in hearing what you guys think.

What would have to change is how pseudo selectors are being processed. Otherwise all the selectors for each btn-.. would have to be duplicated
Right now there are selectors named .btn:disabled and more
They need to be reworked into using the 'starts with' selector [class^="btn-"]:disabled. Could that be done for all selectors that Bootstrap uses, and browser it targets? If it does your suggestion works. Otherwise there is the issue of increased size for bootstrap css Files.

Even the base idea of using SASS placeholder, would increase the final CSS

@herbalite @wolfy1339 I have actually tried this locally and thought it might result in duplicate CSS, but it didn't. Selectors are actually grouped together. Here's how the base btn styles look like in the final CSS:
a.btn.disabled,
fieldset:disabled a.btn, .btn-primary, .btn-secondary, .btn-success, .btn-info, .btn-warning, .btn-danger, .btn-light, .btn-dark, .btn-outline-primary, .btn-outline-secondary, .btn-outline-success, .btn-outline-info, .btn-outline-warning, .btn-outline-danger, .btn-outline-light, .btn-outline-dark, .btn-link, .btn-lg, .btn-group-lg > .btn, .btn-sm, .btn-group-sm > .btn, .btn-block {
  display: inline-block;
  font-weight: 400;
  color: #212529;
  text-align: center;
  vertical-align: middle;
  -webkit-user-select: none;
  -moz-user-select: none;
  -ms-user-select: none;
  user-select: none;
  background-color: transparent;
  border: 1px solid transparent;
  padding: 0.375rem 0.75rem;
  font-size: 1rem;
  line-height: 1.5;
  border-radius: 0.25rem;
  transition: color 0.15s ease-in-out, background-color 0.15s ease-in-out, border-color 0.15s ease-in-out, box-shadow 0.15s ease-in-out;
}

That doesn't seem like a lot of duplicate CSS to me. What do you think?


[ bug Report ] Invisible Lara in the Hub system

TombEngine version
v1.4
Tomb Editor version
v1.7.1
Describe the bug
In 2 levels connected with the “hub system” enabled in some contexts Lara becomes invisible
To Reproduce
In a hub level, there is an "end of level" trigger in a water room. When returning from the level to the hub, Lara becomes transparent at certain points.
If Lara goes into the water, she becomes permanently transparent. If I try to get out of the water, the game freezes but does not crash.
If I go to the next level from the water room and return to the hub, Lara will be permanently transparent even outside the water. If I go into the water and try to get out of the water, the game freezes but does not crash. The log does not report any errors.
Expected Behaviour
Lara does not have to become transparent
Additional Content
Lwmte suspects it's a bug in the renderer.

https://github.com/user-attachments/assets/42ed0ebe-194c-4e1a-9f23-405d7086a615
https://github.com/user-attachments/assets/08d108cc-003a-457a-85ba-0b71696a9967
Minimal reproduction project
https://mega.nz/file/z5USUSDA#FFmnLezypOdaguCLY1qdAVcLL-M7ElVCNgrw2z9D0-U

l.m. had the same problem in his level
https://github.com/user-attachments/assets/e5717be9-5d0f-40aa-9ad6-06e80ad1b53d
Test level:
DL.zip

choose Tremendous Tower Level to begin, pickup key, go through door, finish level to Georgous Gardens, then go back to first level and then lara will become invisible.

Updated package: https://we.tl/t-VWqX6woG6P

Fixed


Script objects created do not seem to have a notion of tokenstream because of which we lack source level information in a number of places

Script objects created do not seem to have a notion of tokenstream because of which we lack source level information in a number of places

Stack traces
Script::Source
Support for regenerating script source code given just a kernel ir file

Necessary for source display in Observatory relies on this.
Host != Device file system -> may not have easy access to files on disk

Discussion notes:
The current design might change, but it's: statements and expressions in the binary have an offset relative to the start of their script. Scripts are represented as a (index into a) URI (table) in the binary. An expression or statement implicitly belongs to the script of the enclosing member. This allows members in the same class (for instance) to come from different URIs but not code in the same member (e.g., due to inlining) to come from a different URI.
Each script has a line starts table which allows mapping an offset to a line and column position.
We are currently only missing the width of the (original) token.

@kmillikin


Please show all tabs when add a existen module

Description of problem
When you have some tabs with child tabs, and root tab is disabled (main menu for example). when you want to add an existent module from any of those child tabs, in tabs tree is not showed. This because parent tab is disabled.

To replicate create a page(menu), and then create a child page(submenu), and place some content. Html module for example
Disable main page(menu)
Create an additional page(location is not important)
In this additional page, try to add an existent module from submenu. You will not be able to navigate to that submenu. Only option is to search submenu, but can be a problem if you have some pages with same name, in different parents.

Description of solution
In other modules, you always show all tabs, ignoring disable state, this should be the same for "add existent module" function.
Affected version

[x] 9.3.2
[x] 9.3.1
[x] 9.2.2
[x] 9.2.1
[x] 9.2
[x] 9.1.1
[x] 9.1
[x] 9.0

Affected browser
All browsers

This issue duplicates #2594 and should be fixed by #2811 , if you can reproduce after 9.4.0 is release, please open a new issue. 9.4.0 is currently available as a release candidate that is not meant for production, if you do test with that RC, please take precautions (backup and local testing only, etc.)


grid.request.onload results in empty responseText

I'm currently experencing a problem where MVC6.Grid has stopped working with the latest release of .Net Core 3.
Firstly I had encountered this error "Synchronous operations are disallowed. Call WriteAsync or set AllowSynchronousIO to true instead.", when trying to pass the partial view to the grid.
To get around this I had to include this line in startup.cs:
services.Configure<IISServerOptions>(options => {     options.AllowSynchronousIO = true; });
This then results in an error in this code:
grid.request.onload = function () { if (200 <= grid.request.status && grid.request.status < 400) { success(grid.request.responseText); } else if (error) { error(grid.request.responseText); } };
as it ends up passing an empty success responseText to grid.startLoading(function (result){ ... }
which then overwrites the outerHTML with an empty string, so you get an empty grid.
Do you know how to fix this issue?
I'm just using a standard ajax grid, where I'm populating it with data when a button is pressed.

Synchronous IO errors will be fixed with https://github.com/aspnet/AspNetCore/issues/15367. Empty result usually means that there was an exception while rendering the view. Is there any exceptions in the error log.

Not currently seeing any errors in the log or stack trace. Would using .Net Core 3.1 preview work in the meantime?

It works on core3 too, not sure why are you getting empty responses though. Maybe you have a small repro? Because documentation page and our projects are using core3 without problems.

Just tried 3.1 and am still getting the Synchronous errors.
It's really weird, my grid implementation was working fine last week.
Not really sure what has changed to cause these errors.
Still no idea why the responseText is empty.
It causes this error afterwards:
Uncaught TypeError: Cannot read property 'classList' of undefined
at window.location.href (mvc-grid.js:181)
at XMLHttpRequest.grid.request.onload (mvc-grid.js:283)
mvc-grid.js:181

Ah sorry I thought the Synchronous IO fix was in the current preview.
It's actually being published as preview 3, so I'll just wait for that, and hope it fixes my issue.

I'm 99 % sure it won't fix your issues. You can try running web project and see it working on core3.

Can you tell me what's supposed to be in the grid.request.responseText on a 200 code?

Grid content, you can see an example here.

Is there a way in the meantime that I can get around the SynchronousIO problems without specifiying:
services.Configure<IISServerOptions>(options =>             {                 options.AllowSynchronousIO = true;             }); in startup.cs
I'm almost 100% that is causing the responseText problems.

Async IO is caused by an exception in view rendering, which gets swallowed by it. If you can't share a repro or replicate the problem on the web project and don't see any exceptions in the logs then I don't know how more I can help :disappointed:

Just created a test repo with the same concept as my application.
Hopefully this helps in diagnosing the problems.
Test Repo
In the actual application I'm populating the grid with data from a SQL DB.

You are getting this exception
System.ArgumentException: Value cannot be null or empty. (Parameter 'viewPath')
   at Microsoft.AspNetCore.Mvc.ViewEngines.CompositeViewEngine.GetView(String executingFilePath, String viewPath, Boolean isMainPage)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.HtmlHelper.RenderPartialCoreAsync(String partialViewName, Object model, ViewDataDictionary viewData, TextWriter writer)
   at Microsoft.AspNetCore.Mvc.ViewFeatures.HtmlHelper.PartialAsync(String partialViewName, Object model, ViewDataDictionary viewData)
   at AspNetCore.Views_Shared_MvcGrid__Grid.ExecuteAsync() in \TestMVCGridProject\Views\Shared\MvcGrid\_Grid.cshtml:line 156

Which means you updated grid dll to 5.1.1, but you are still using 5.0.1 views and static resources which are rarely compatible.

Thank you so much.
For future reference, should the view templates and static files update with the nuget package update?
Or should I always replace those files from the new ones available from this repo?

They need to be replaced by hand, nuget used to update content in pre core days, but now it can't.

Ok, Noted for future updates.
Thank you for being such a big help with this issue!


Update embassy-sync version to 0.6.0

A new version of embassy-sync has been released that supports zero initialization. I think this is a pretty important change to update

A new version of embassy-sync has been released that supports zero initialization. I think this is a pretty important change to update

Upgrading to the latest version of embassy-sync always does make sense, though I wonder what you mean by "zero initialization".
Are you confusing embassy-sync with something else?
Looking at their changelog it seems the only reason they bumped to 0.6 is because of a backwards incompatible change in their broadcast queue.

OK. I went a little overboard with the wording. There was only one commit in  PR about this


Provide module id in npm package definition

This PR is small chore to add main field in package.json to point code, allows to resolution of module path via module import. Bundlers like webpack relies on module path when picking up npm modules.

Thanks!


[Bug] JsonLogginConfigurationTests not running correctly

Bug description
siehe https://github.com/it-at-m/refarch-templates/pull/279#issuecomment-2377020168
Expected behaviour
test sind grün
Steps for reproduction
siehe pipeline durchlauf
Affected version
akutelle
Affected template
refarch-backend, refarch-eai
Last working version (optional)
aktuelle
Operating system
No response
Browser
No response
Relevant log output (optional)
No response
Screenshots (optional)
No response
Additional context (optional)
No response
No duplicate

[X] I confirm that this issue is not a duplicate

Code of Conduct

[X] I agree to follow this project's Code of Conduct

I debugged the issue and found out that:

@DirtiesContext for the JsonLoggingConfiguration class starts a fresh Spring context with the correct "json-logging" profile
The logback configuration itself dos not change and which ever configuration is used first is used throughout the whole test suite (I debugged this by adding <statusListener class="ch.qos.logback.core.status.OnConsoleStatusListener" /> to the logback configuration file
I tried different versions of @DirtiesContext (e.g. BEFORE_CLASS), when using BEFORE_EACH_TEST_METHOD tests work but the wrong json-logging is also used for follow up test classes
I tried the solution mentioned in https://stackoverflow.com/a/39292587


Better specfile

This is pretty much the same as https://github.com/DBuildService/dock/commit/6c4d0421d7983aa677bcb0cd025a8781af8fd330 to keep things in good shape and consistent.

+1


Remote OS Command Injection Vulnerability in GPT-SoVITS UVR5 Module

Summary
The web backend server for GPT-SoVITS lacks proper user input sanitization in the UVR5 Module, which leads to remote OS command injection vulnerability. This flaw allows attackers to execute arbitrary commands, compromising the system and causing critical security risks.
Due to this vulnerability and the web server's default public exposure, the GPT-SoVITS server is unsuitable for deployment in public production environments until the vulnerability is patched.
Affected Versions

Relased version < 20240821v2

As for today (2024.12.10), all versions of repo code

Details
The vulnerability originates from the _path_audio_ functions implemented in three classes, AudioPre class, MDXNetDereverb class and BsRoformer_Loader class. The UVR5 module relies these classes to pre-process audio data.
Under the hood, these classes externally call ffmpeg program to conduct the data pre-process operation, in which the shell command can be polluted by the user input as one out of the three vulnerable snippets shows:
#https://github.com/RVC-Boss/GPT-SoVITS/blob/a70e1ad30c072cdbcfb716962abdc8008fa41cc2/tools/uvr5/vr.py#L151
#https://github.com/RVC-Boss/GPT-SoVITS/blob/a70e1ad30c072cdbcfb716962abdc8008fa41cc2/tools/uvr5/mdxnet.py#L223
#https://github.com/RVC-Boss/GPT-SoVITS/blob/a70e1ad30c072cdbcfb716962abdc8008fa41cc2/tools/uvr5/bsroformer.py#L180
def run_folder(self,input, vocal_root, others_root, format):
  	...
    path_vocal = "%s/%s_vocals.wav" % (vocal_root, os.path.basename(path)[:-4])
    ...
    opt_path_vocal = path_vocal[:-4] + ".%s" % format
    ...
    os.system(
        "ffmpeg -i '%s' -vn '%s' -q:a 2 -y" % (path_vocal, opt_path_vocal)
    )

The tool can be reached in the web UI.

An attacker can exploit this command injection vulnerability by crafting malicious inputs. These inputs can be provided via the HTML forms or modified in the HTTP request, as highlighted in the screenshot below:

PoC (Proof of Concept)
An attacker can easily achieve remote command execution (RCE) by first uploading an audio file and injecting malicous payloads into the vocal output folder path in a UVR5 conversion operation.

Install and Deploy the GPT-SoVITS following the official instructions (dependencies like: ffmpeg, UVR5 models) with WebUI enabled
Turn on the UVR5-WebUI

Download a legitmate audio file: https://www.signalogic.com/melp/EngSamples/Orig/ENG_M.wav
Run the exploitation script: https://gist.github.com/superboy-zjc/2ed271471849418580686ac3c5de30cd. This PoC script dememstrate how to exploit the vulnerable function _path_audio_ in the AudioPre class. It is almost the same in other two vulnerable functions.

Replacing cmd value with your desired to trigger an RCE attack:
python SoVITS-uvr5-exp.py -f test.wav -u http://proof-of-concept:9873 -cmd "ping XXX"

Patch
Replace this block:
os.system(
    "ffmpeg -i '%s' -vn '%s' -q:a 2 -y" % (path_vocal, opt_path_vocal)
)

With:
import subprocess

subprocess.run(
    ["ffmpeg", "-i", path_vocal, "-vn", opt_path_vocal, "-q:a", "2", "-y"],
    check=True
)

Any update? @RVC-Boss I can help with the patch after a confirmation of the vulnerability. Thanks!


Request to add an issue link for the site

It would be nice to include a direct link for site feedback to the page so it's easy for users to open issues against this repository.

@coreyvickery thinking we can add a "Submit a bug" button in the top right of the header. Would you supply a design for that?

@castastrophe @dcaryll
Here's an example of a Submit a bug button in the title area. I think the positioning can work with all of the component names. The location on mobile has to be a little different.
https://xd.adobe.com/view/2bc4c20e-e89e-4a19-8f6d-172a5a64db36-7e97/

@castastrophe @dcaryll
Here's an example of a Submit a bug button in the title area. I think the positioning can work with all of the component names. The location on mobile has to be a little different.
https://xd.adobe.com/view/2bc4c20e-e89e-4a19-8f6d-172a5a64db36-7e97/

@coreyvickery @dcaryll We currently have a submit a bug feature (or really "Submit feedback") on ux.redhat.com through Usersnap. Are we considering these implementations synonymous (and thus closing this request)? Or do we want to handle this differently?

@markcaron I'm more of a fan of the Submit feedback function because it's more robust than a simple button (thanks to Usersnap). However, you need to be a Red Hat employee to use Usersnap, so it depends on how open source we want the website to be right now.

if there's going to be a direct link, perhaps it should be to the discussion board rather than issues. so far we've been treating the issue board as directly actionable things only - bug reports or prioritized work.

@bennypowers @markcaron Here's what I've been experimenting with. The link can't stay in the header navigation at small breakpoints, so I'm advocating to move it into the menu.
https://xd.adobe.com/view/112e18ea-305f-4017-bc39-d5edbb9db801-447d/
cc @marionnegp

@bennypowers Thoughts about the above?

position and responsiveness LGTM
labeling and target: does this just dump on the repo readme? does it link to /issues/new?
What about when I'm on an element page, does it now link to the element's subdirectory in the repo file tree?

I would think a GitHub link on the top level would take me to the repo readme, maybe inside the component pages themselves we can link to the specific sub-directory?  We could also add a link in the readme to the discussion board though for new ideas / thoughts?

@bennypowers Thoughts on what Brian is proposing?

makes sense.

@markcaron @hellogreg Could we close this and open a new issue for adding GitHub to the website instead?

Closing in favor of #1443


Visibility/debugging: Reflect the selected items in the page itself

Easy Visibility/debugging for contributors:
Instead of having the selected items logged through the console, it will be good to reflect the selected items in the page itself. For each list, display the selected items. (make life easier for the contributors  🥱 )

@ouwais1922 thank you very much for the idea though this can be implemented easily since the components accepts a selected state which you can always display it if you would like to


msgpack.decode(new Uint8Array(data)) leads to "Uncaught TypeError: this.readUInt32BE is not a function"

I tracked that down to the following method in decode-buffer.js:
DecodeBuffer.prototype.append = function(chunk) {
  var prev = this.offset ? this.buffer.slice(this.offset) : this.buffer;
  this.buffer = prev ? Buffer.concat([prev, chunk]) : chunk;
  this.offset = 0;
};

I think the buffer assignment has to be replaced to:
  this.buffer = prev ? Buffer.concat([prev, chunk]) : Buffer(chunk);

The current version of msgpack.decode requests a Buffer instance instead of a native Uint8Array instance as a chunk. Buffer has some of utility methods compared to Uint8Array's.
I guess msgpack.decode(Buffer(new Uint8Array(data)) would work.
By the way, the module was started when node.js was v0.12. It seems that Buffer implementation is changed since io.js. The module may have better to support Uint8Array. It'd fast on browsers.
$ ~/.nvm/v0.10.41/bin/node -pe 'Buffer(1) instanceof Uint8Array'
false
$ ~/.nvm/versions/node/v0.12.9/bin/node -pe 'Buffer(1) instanceof Uint8Array'
false
$ ~/.nvm/versions/node/v4.2.3/bin/node -pe 'Buffer(1) instanceof Uint8Array'
true
$ ~/.nvm/versions/node/v5.3.0/bin/node -pe 'Buffer(1) instanceof Uint8Array'
true

I use msgpack in chrome. Buffer is not exported to global namespace.

I use msgpack in chrome. Buffer is not exported to global namespace.

I see. Now it accepts Uint8Array instance since version 0.1.17!
var data = [0x81, 0xA3, 0x66, 0x6F, 0x6F, 0xA3, 0x62, 0x61, 0x72];
msgpack.decode(new Uint8Array(data)); // => {foo: "bar"}


Parameter style definitions inconsistent with the examples

The style values table has a type column which I assume indicates which data types can be serialized using that style (e.g. a deepObject style on a query parameter for any data type other than object would constitute an invalid spec), in the same way that I assume the in column determines which locations each style is valid in.
But this is not consistent with the examples, and further brings up some curiosities:

The simple parameter style only supports arrays, yet it is the default style for path parameters and the only style for header parameters.

Is this supposed to mean header parameters are only ever allowed to be arrays? No primitives can ever be passed to a header parameter?
There is an example path parameter object that uses the default style—simple—that contradicts this by defining a schema with data type string.

In the style examples table pipeDelimited and spaceDelimited both offer examples of serialized objects, not just arrays, and the simple style has examples of an object as well as a string primitive.

These contradictions imply that my initial assumption about the type column in the style values table is incorrect. But there is no explanation about what it is supposed to indicate, then.

There are other ambiguities too, like whether explode: true is allowed for pipeDelimited and spaceDelimited parameters (and what that would look like), and the same regarding explode: false for deepObject.
My question is essentially: are the examples in the parameters section of the specification normative?
If the examples section is non-normative (which you would usually expect from a standard), then it is at least inconsistent, but would also imply that the type column of the style values table is the source of truth, and I just have a hard time believing that simple-style parameters were intended to only ever be arrays.

From a quick review, I think Simple should support primitive and object, as shown in the examples.  I don't know why it was limited to arrays.  I'll get that fixed.
I'm not going to try and defend deepObject.  There was a strong desire from community members to support this, but there is no standard, so we allowed specifying it but the exact syntax would have to be communicated between API consumer and provider in some other way.
pipeDelimited and spaceDelimited were carry overs from OpenAPI v2 and don't fit naturally in the RFC6570 model.  I'm on the fence as to whether we should just drop the object examples or expand the potential usage.
The examples were created primarily around what would a URI Template library do with the values.  The examples should be consistent with the style constraints and I'm a little surprised that it has gone this long without someone point out how badly I screwed up there.
Thank you for taking the time to point out these inconsistencies.  I will fix the obvious ones.  I'm curious to your opinion on whether we should support space and pipe delimited objects.  I'm tending towards no.

My opinion about pipe- and spaceDelimited is the same as yours; I don't think we should support them. The final parsed result is the same regardless of which serialization style you use, so I prefer less options because it simplifies clients and also simplifies the standard.
Encoding is also a problem. For pipeDelimited it's probably safe to assume that you would just percent-encode any pipe characters within the arguments, but what about spaces within arguments for spaceDelimited parameters? The standard doesn't mention what to do in that case, so a client won't know how to serialize it unless the parameter description happens to explain how to escape spaces.

I also find this confusing. @darrelmiller is it going to be fixed?

Hi! I think it would at least make sense to say that deepObject is not supported when explode=false.
This should also require updating the way the default value of explode is defined:

When style is either form or deepObject, the default value is true. For all other styles, the default value is false.

This was actually fixed some time ago for 3.0.4 and 3.1.1 - I'm not sure what PR it was but simple shows primitive, array, object and both pipeDelimited and spaceDelimited show array, object.  Since the example table always showed these, the lack of the correct types in the style table was an accidental omission, and can be fixed without breaking compatibility.  The patch releases should be out within the next couple of months.


It is hoped that it can provide more powerful screenshot recognition functions

Description of the new feature / enhancement
It is hoped that it can provide more powerful screenshot recognition functions, such as recognizing and extracting the text and QR code in the screenshot, or identifying and extracting people, animals and other pictures in the screenshot, so as to facilitate users to use for other purposes. this can also help you train artificial intelligence algorithms, which is beneficial to both users and you, and even provides screenshots for screenshots in the screenshot area.
And the function of long screenshots under any software interface.
Scenario when this would be used?
When the user needs to take a screenshot on the computer and needs to identify and decompose the contents of the picture.
Supporting information
No response

/dup #4859


[WIP]ROS-21 v0 api desinging

This PR handles the v0 api design for ros. It is currently in WIP state and requires discussion and review comments for merge ready. The flow of creating this PR is threefold as of now. Two ways that I could think of after reading various references are:

either, using blueprints with view functions and registering the same to generate URLs for ros app and defining the api and api calls alongside of it: read here.
or, using just the project layout as defined by flask-restful: read here

So, what are your opinions here, if we try to combine them together like this in order to, create our URLs to communicate to the host inventory similar to drift-backend: read here to do versionining on the API?

[x] Remove existing dummy data
[ ] Fetch hosts lists from Inventory
[ ] Fetch insights-client id to retrieve these hosts
[ ] Define api calls/endpoints
[ ] Structure the api layout
[ ] Version this api v0

Please feel free to add comments if I'd missed any points.

As next step, removed blueprints and added v0 api over resources layout.

As next step, removed blueprints and added v0 api over resources layout.

As next step, defined the api with realtime data.

As next step, defined the api with realtime data.

@apuntamb @patilsuraj767 merging this PR.
Please make sure to address any unresolved comment(s) in separate PRs.

@apuntamb @patilsuraj767 merging this PR.
Please make sure to address any unresolved comment(s) in separate PRs.

Thanks @apuntamb!

Thanks @apuntamb!


Expiring node.js version

Summary
Adventist is currently using Node.js 16, but it is expiring on June 12, 2024. AWS is recommending upgrading it to Node.js 20. Below is the emailed alert on this. This will likely affect our other clients on r2-besties, so it might be worthwhile to do the same for them.
AWS Alert
Hello,
We are contacting you as we have identified that your AWS Account currently has one or more AWS Lambda functions using the Node.js 16 runtime.
We are ending support for Node.js 16 in Lambda on June 12, 2024. This follows Node.js 16 End-Of-Life (EOL) reached on September 11, 2023 [1].
As described in the Lambda runtime support policy [2], end of support for language runtimes in Lambda happens in several stages. Starting on June 12, 2024, Lambda will no longer apply security patches and other updates to the Node.js 16 runtime used by Lambda functions, and functions using Node.js 16 will no longer be eligible for technical support. Also, Node.js 16 will no longer be available in the Console, although you can still create and update functions using Node.js 16 via AWS CloudFormation, the AWS CLI, AWS SAM, or other tools. Starting July 15, 2024, you will no longer be able to create new Lambda functions using the Node.js 16 runtime. Starting August 15, 2024, you will no longer be able to update existing functions using the Node.js 16 runtime.
We recommend that you upgrade your existing Node.js 16 functions to Node.js 20 before June 12, 2024. A list of your Node.js 16 functions can be found in the 'Affected resources' tab of the AWS Health Dashboard.
End of support does not impact function execution. Your functions will continue to run. However, they will be running on an unsupported runtime which is no longer maintained or patched by the AWS Lambda team.
This notification is generated for functions using the Node.js 16 runtime for the $LATEST function version. The following command shows how to use the AWS CLI [3] to list all functions in a specific region using Node.js 16, including published function versions. To find all such functions in your account, repeat this command for each Region:
aws lambda list-functions --region us-west-2 --output text --query "Functions[?Runtime=='nodejs16.x'].FunctionArn"
From 180 days before deprecation, you can also use Trusted Advisor to identify all functions using the Node.js 16 runtime, including published function versions [4].
If you have any concerns or require further assistance, please contact AWS Support [5].
[1] https://github.com/nodejs/Release
[2] https://docs.aws.amazon.com/lambda/latest/dg/runtime-support-policy.html
[3] https://aws.amazon.com/cli/
[4] https://docs.aws.amazon.com/awssupport/latest/user/security-checks.html#aws-lambda-functions-deprecated-runtimes
[5] https://aws.amazon.com/support
Sincerely,
Amazon Web Services
Amazon Web Services, Inc. is a subsidiary of Amazon.com, Inc. Amazon.com is a registered trademark of Amazon.com, Inc. This message was produced and distributed by Amazon Web Services Inc., 410 Terry Ave. North, Seattle, WA 98109-5210

Reference: https://health.aws.amazon.com/health/home?region=us-east-1#/event-log?eventID=arn:aws:health:us-east-1::event/LAMBDA/AWS_LAMBDA_PLANNED_LIFECYCLE_EVENT/AWS_LAMBDA_PLANNED_LIFECYCLE_EVENT_156364ed4344b5dbc2779fffee16a28077c65bd589e2f1d5c12c814f6124e64c&eventTab=details

Acceptance Criteria

[ ] Node.js 16 is upgraded to Node.js 20

We transitioned the stack over to use the nodejs18 version we have community support for and have that live now behind the CloudFront CDN that is configured to both instances b2 and s2.


DNSPolicy improve default behaviour

/closes #359
Removes the default DNSPolicy creation from the gateway controller.
Update labelling of resources created by the DNSPolicy. Brings it in-line with how the other kuadrant polices work, and makes it easier to find resources created by a policy.
kubectl get dnsrecords -A --show-labels
NAMESPACE                NAME                   READY   LABELS
multi-cluster-gateways   myapp.mn.hcpapps.net   True    gateway-namespace=multi-cluster-gateways,gateway=prod-web,kuadrant.io/dnspolicy-namespace=multi-cluster-gateways,kuadrant.io/dnspolicy=prod-web

tested locally. Works as expected.

/lgtm


kcsp updates 11.1.21

Pre-submission checklist:
Please check each of these after submitting your pull request:

[x] Are you only including a repo_url if your project is 100% open source? If so, you need to pick the single best GitHub repository for your project, not a GitHub organization.
[x] Is your project closed source or, if it is open source, does your project have at least 300 GitHub stars?
[x] Have you picked the single best (existing) category for your project?
[x] Does it follow the other guidelines from the new entries section?
[x] Have you added your SVG to hosted_logos and referenced it there?
[x] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[x] Does your project/product name match the text on the logo?
[x] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[ ] ~15 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

Build failed because of:
Empty twitter for StorageOS (member): https://twitter.com/Storage_OS
Empty twitter for StorageOS: https://twitter.com/Storage_OS
Empty twitter for Karmada: https://twitter.com/karmada_io
Empty twitter for Spotinst Functions: https://twitter.com/spot_hq
Project Cilium has been added to incubating category but does not belong to CNCF
Page: /, section Special
Kubernetes Certified Service Provider
Kubernetes Training Partner is out of bound

@AndreyKozlov1984 can you please take a look at the KCSP section? I think we've outgrown the box on the landscape view. Thanks!


Vsf bridge and vsf external checkout compatibility

As https://github.com/Vendic/vsf-external-checkout produced different links, that had no route/controller in Magento we changed the listening frontController from vscheckout to vue/cart/sync.
We also needed to make it compatible with https://github.com/DivanteLtd/magento1-vsbridge as that one provides the basic magento API endpoints and coud be reused for this.

Everything works fine, except path to the config value, where the success page url is stored. We will solve this in another PR. Testen on VSF 1.9 and external-checkout extension 2.0. @pkarw you can accept this PR :+1: thanks @nhp !

Thanks, merged in :)


Fix SKPaint.Clone

Clone should operate on SkCompatPaint
Description of Change
Fix crash on dispose of a cloned SKPaint
Bugs Fixed

Fixes #2899

API Changes
None.
Behavioral Changes
None.

Required skia PR
None.

PR Checklist

[x] Has tests (if omitted, state reason in description)
[ ] Rebased on top of main at time of PR
[ ] Merged related skia PRs
[ ] Changes adhere to coding standard
[ ] Updated documentation

Targetted wrong branch


Section Numbering Problem in additional-sfrs section of Modules

Appears to be caused by manually inserted section for an audit table. So it looks like the code assumes that there will be no manually inserted sections.

Fixed by adding <additional-sfrs> and <modified-sfrs> elements to the level calculation.

And so closed.


Create instruction on how to replace default logger to custom one

Root case #418
Provide an example as well (replace log4j with console.syste,.out)

closed due to #418 was fixed


Apply DDAU pattern everywhere

Some (older) components are not designed with DDAU in mind. Have to fix that.
Will probably be a breaking change!
Let's collect all issues here:

[ ] ButtonGroup: value should be oneway, call change action
[ ] Alert: dismiss

Closed in favor of #158


equals on HyperLogLogPlusAggregator is wrong

The equals method on HyperLogLogPlusAggregator uses equals on the current state of the aggregator which is a HyperLogLogPlus. This class does not have an equals method. Instead, the equals method should serialise the HyperLogLogPlus and compare those.

Merged into develop.


Animator tool to send latest animation to farm with latest lookDev

Goal
To create an easy way for the animator to send their animation to the farm to be able to test motion blur or to see intersections easier.
Motivation
It would be great to be able to see your latest anim wip on the latest lookdev, many thing aren't always as clear in maya viewport as it is when seeing it rendered. Things like intersections, eye direction, motionblur ect.
Implementation
A simple button that that creates an alembic, apply the latest lookDev, sends it to the farm and deliver a quicktime to a folder on the server (daily?) for the animator to review.
The settings and resolution should be half the quality of the final rendered image and speed should be more of a priority then quality for this type of renders.

that creates an alembic, apply the latest lookDev, sends it to the farm and deliver a quicktime to a folder on the server (daily?) for the animator to review.

What I had in mind is similar, but a little different. You'd actually publishing things as usual, and then send a "recipe" of which assets at which versions you would like to render. To the artist, this would just be the shot, the shot is the recipe.
$ cd my/shot
$ render

This is something we could do via the Launcher, by just browsing to a shot, and saying "Render this with everything latest, please". You'll be able to do it via ftrack as well, which means Rickard could do it without artist intervention.

Sounds great! as long as we can render a "simpler" render that is faster since its for animator reviewing.
Having a preset to just have SSS shaders would be cool as well. a special lookdev for checking SSS intersections and motionblur or something similar.


Yandex.Cloud Driver

Description
New Docker Machine driver for Yandex.Cloud

Please sign your commits following these rules:
https://github.com/moby/moby/blob/master/CONTRIBUTING.md#sign-your-work
The easiest way to do this is to amend the last commit:
$ git clone -b "yandex-cloud-driver" git@github.com:yandex-cloud/docker-machine.git somewhere
$ cd somewhere
$ git rebase -i HEAD~842357575048
editor opens
change each 'pick' to 'edit'
save the file and quit
$ git commit --amend -s --no-edit
$ git rebase --continue # and repeat the amend for each commit
$ git push -f

Amending updates the existing PR. You DO NOT need to open a new one.

PR with documentation.

Hi @guillaumerose !
Could you review and merge PR?

Hi @GennadySpb, thanks for the pull request.
We are not accepting additional drivers into the core because they are supported by 3rd party plugin binaries. Feel free to distribute your own and add it to the AVAILABLE_DRIVER_PLUGINS.md though.

Hi @guillaumerose !
I have been create PR to AVAILABLE_DRIVER_PLUGINS.md.
Another question: where is the AVAILABLE_DRIVER_PLUGINS.md page located at https://docs.docker.com/ ?


Copy address clicking QR

74 PSF tokens rewarded. TXID: 9077f8bdb7049c5c1cabd04fbdfbd48fdae047d8d89c156e1164f9ff49a195e0

:tada: This PR is included in version 1.9.16 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Allow Collection.init() type-safe population

When using the Collection.init() method on a relation, Mikro allows us to populate a OneToMany collection, however it is returning Promise<this> instead of a `Promise<Loaded<..>>. Because of this, we are forced to manually cast the collection to a populated type for compilation.
Additional context
    "@mikro-orm/cli": "^4.5.9",
    "@mikro-orm/core": "^4.5.9",
    "@mikro-orm/migrations": "^4.5.9",
    "@mikro-orm/nestjs": "^4.3.1",
    "@mikro-orm/postgresql": "^4.5.9",
    "@mikro-orm/reflection": "^4.5.9",

That method returns the collection instance (so a fluent interface), not entities. What you are asking for is a breaking change.
What you want can be achieved via em.populate().

how would it be a breaking change? unless there are some classes which extend Collection or otherwise alter the return type throughout the fluent call chain, the signature can simply return Collection<T & Loaded> or LoadedCollection, etc?
and while yes, it can be achieved with em.populate, that is assuming you want to pass it down multiple layers. there can be instances where you can rely on the entitles them selves to lazy load if you weren't falling a strict DTO-like pattern

I got confused by the Loaded type suggestion, makes sense to make it a LoadedCollection I guess.


✨  Add in key event for pointer and annotation tool

Description

Added key events for using the pointer and annotation tool. "a" key for annotation tool, "z" key for pointer tool, and "x" key for cancel all state and revert back default cursor.
Please delete options that are not relevant.

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

Tested on?

[x] Windows
[ ] Linux Ubuntu
[ ] Centos
[ ] Mac
[ ] Others  (State here -> xxx )

Checklist:

[x] My code follows the style guidelines of this project
[ ] I have performed a self-review of my own code
[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] Any dependent changes have been merged

This is good. But add the info to here

So user can get the information.

This is good. But add the info to here

So user can get the information.

@devenyantis Thanks, I will add on the key info

@devenyantis  Key info added


Arduino library: memset not defined

I found that the library (used with the examples) wouldn't compile for me when targeting my Adafruit Huzzah32 ESP32-based board.
The specific issue appears to be that it can't find memset. I'm still not very familiar with Arduino and its idiosyncrasies, but some Googlging reveals that this can be fixed by importing Arduino.h.
I've added the following to the top of exixe.cpp and everything works fine:
#include "Arduino.h"

This could be an issue with my setup but it is, as far as I can tell, the default configuration for the ESP32 boards and I'm not aware I've done anything strange.

Thanks for the discovery, I don't have a ESP32 myself and apparently it's different.
Isn't memset in string.h?  Try#include <string.h> instead of Arduino.h and see if it works. I'll test and update the library tomorrow.

A collection of other symbols seem to be missing with just #include <string.h>.
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp: In constructor 'exixe::exixe(int)':
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:18:19: error: 'OUTPUT' was not declared in this scope
   pinMode(cs_pin, OUTPUT);
                   ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:18:25: error: 'pinMode' was not declared in this scope
   pinMode(cs_pin, OUTPUT);
                         ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:19:24: error: 'HIGH' was not declared in this scope
   digitalWrite(cs_pin, HIGH);
                        ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:19:28: error: 'digitalWrite' was not declared in this scope
   digitalWrite(cs_pin, HIGH);
                            ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp: In member function 'void exixe::spi_write()':
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:25:24: error: 'LOW' was not declared in this scope
   digitalWrite(cs_pin, LOW);
                        ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:25:27: error: 'digitalWrite' was not declared in this scope
   digitalWrite(cs_pin, LOW);
                           ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:26:11: error: 'delay' was not declared in this scope
   delay(10);
           ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:29:24: error: 'HIGH' was not declared in this scope
   digitalWrite(cs_pin, HIGH);
                        ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp: In member function 'void exixe::crossfade_init(unsigned char, unsigned int, unsigned char, unsigned char)':
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:107:34: error: 'millis' was not declared in this scope
   animation_start_frame = millis() / EXIXE_ANIMATION_FRAME_DURATION_MS;
                                  ^
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp: In member function 'unsigned char exixe::crossfade_run()':
/Users/jbmorley/Documents/Arduino/libraries/exixe/exixe.cpp:116:41: error: 'millis' was not declared in this scope
   unsigned long current_frame = (millis() / EXIXE_ANIMATION_FRAME_DURATION_MS) - animation_start_frame;
                                         ^
Multiple libraries were found for "WiFi.h"
 Used: /Users/jbmorley/Documents/Arduino/hardware/espressif/esp32/libraries/WiFi
 Not used: /Applications/Arduino.app/Contents/Java/libraries/WiFi
exit status 1
Error compiling for board Adafruit ESP32 Feather.

Thank you so much!


add impl of oauth and device login

Codecov Report

Merging #700 into develop will decrease coverage by 0.83%.
The diff coverage is 16.92%.

@@             Coverage Diff             @@
##           develop     #700      +/-   ##
===========================================
- Coverage     33.5%   32.66%   -0.84%
===========================================
  Files           83       86       +3
  Lines         3188     3306     +118
  Branches       479      496      +17
===========================================
+ Hits          1068     1080      +12
- Misses        2040     2143     +103
- Partials        80       83       +3

Impacted Files
Coverage Δ

...in/java/com/microsoft/azure/auth/WebAppServer.java
0% <0%> (ø)

.../microsoft/azure/auth/AzureCredentialCallable.java
58.82% <58.82%> (ø)

...java/com/microsoft/azure/auth/QueryStringUtil.java
77.77% <77.77%> (ø)

...java/com/microsoft/azure/auth/AzureAuthHelper.java
21.79% <8.06%> (-56.78%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 39f49f2...fd53fb6. Read the comment docs.

Closed as the new PR is reviewed and merged


freetds bump

Status
Ready/In Progress/In Hold (Reason for hold)
Related Content Pull Request
Related PR: link to the PR at demisto/content
Related Issues
Related: link to the issue
Description
A few sentences describing the overall goals of the pull request's commits.

Problem was not with FreeTDS driver


Support GGUF

GGUF is the new file format specification that we've been designing that's designed to solve the problem of not being able to identify a model. The specification is here: https://github.com/ggerganov/ggml/pull/302
llm should be able to do the following:

continue supporting existing models (i.e. this change should be non-destructive)
load GGUF models and automatically dispatch to the correct model.

load_dynamic already has an interface that should support this, but loading currently only begins after the model arch is known
use the new information stored within the metadata to improve the UX, including automatically using the HF tokenizer if available

save GGUF models, especially in quantization

llm could do the following:

convert old models to GGUF models with prompting for missing data
implement the migration tool mentioned in the spec, which does autonomous conversion for users based on hashes

To give an update on the state of GGUF: Halfway August GGUF was merged into llama.cpp (https://github.com/ggerganov/llama.cpp/pull/2398#issuecomment-1686986770). It’s full specification can be found here.
Recap of what GGUF is:

binary file format for storing models for inference
designed for fast loading and saving of models
easy to use (with a few lines of code)
mmap (memory mapping) compatibility: models can be loaded using mmap for fast loading and saving.

Hi all, any updates on this?

Hi - sorry about the lack of updates, I've been extremely busy for the last ~two months and haven't had much free time to work on llm. I'm hoping this will ease up soon and we can start catching up proper.

Any updates on gguf?


#84696 Modify git add/revert to use splitInChunks

This PR fixes #84696
Hi, this is my first time contributing to open source code.

Modified the add and revert functions to also split the commands across chunks, mainly copying it from the checkout function.
Modified paths.lengths to paths.length > 0 to be consistent with the previous PR adding chunk splitting. (TS 3.7 would allow for this to be paths?.length)

Is using a limiter necessary for either of the 2 functions?
If the revert command throws an error, is it necessary to deal with any chunks that would have already finished running?

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: jzyrobert sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

@joaomoreno Do you mind rerunning checks on this? I wanted to give a hand on this issue and fix tests, but it seems that all of the failing ones are flaky.

Should I rebase this onto master again?

@jzyrobert No, it's OK.

Can we get this PR resolved? I am using insiders and keep butting up against this issue as I do a lot of architectural refactoring on a big project in vscode.

@joaomoreno I've rebased it on top of recent updates, do you have any opinions on this?

Thanks! 🍻


Error output written in mysql_log from mysql_tquery/mysql_pquery is corrupted

Version R41.
I was testing if some certain errors are given in OnQueryError and just noticed that:
[ERROR] €¬q
[ERROR] ÈÏ/
[ERROR] Ä
[ERROR] ¸¶/

I executed the same faulty query and each time random characters are given.

Another one I just found: using cache_num_rows() to a place without active cache reports line 182 from a_mysql.inc file which actually makes sense since the function is in the include. Instead of calling cache_get_row_count or moving the function to the script, is there any other way to report the previous place was called from for that specific function?

Which query were you executing?
And I'm already aware of that problem with calling natives in non-public functions. I'm working on a way to incorporate a small AMX call trace into log calls.

So far, any query with error in the syntax or table/column that does not exist gives that output.

I run samp-server.exe many times because of testing and I mostly click on X button instead of typing exit. I get the message that "SA-MP dedicated server has stopped working". I downgrade to R40 and this doesn't happen. Updated to R41 again to make sure and it happens.

It's getting even more weird. Since today morning, when a fault query (table does not exist) is executed, samp-server.exe stops working immediately. I don't know if the debug output from Visual Studio is at all useful: http://pastebin.com/raw/Vb8vbQyb

I can't seem to reproduce the error message bug with a query like SEuLECT 1234 or SELECT * FROM invalidtable.
I also can't reproduce the "server stopped working" issue. The log core reacts when you press the X button and shuts down itself correctly, but this is already implemented in R40. There are no notable changes between the log-core used in R40 and the one in R41. The worst thing you should experience is a "freeze" when pressing the X button. That means the log core is writing all remaining logs into their log files in background.
And that error code you receive on crash stands for INVALID_CRUNTIME_PARAMETER, and I'm not really sure if this is caused by the MySQL plugin nor the log core.
If you're running this through Visual Studio, you could compile everything in Debug configuration and launch your SA:MP server through the Visual Studio debugger. This way you should get more descriptive error messages.

Can you also tell me more about your setup (system, used plugins, used includes, ...)?

I'm not sure if I can manage to link them all together and compile the plugin myself.
In the AppVeyor, the "Configuration: Debug" option does anything useful? Because if I run it, I get: http://i.imgur.com/S7UJrTY.png

Yes, the AppVeyor debug build should work too. You can press "Retry" in that window and the debugger should open with a call trace somewhere. Take a look at that trace and you should see if that crash was really caused within the SA:MP server or an other plugin.

The second box is given all the time when clicking "Retry" and no debugger opens. I tried the version for the commit above but the problem still persists.

I finally found the reason causing all this. I was testing the specifiers in logtimeformat (server.cfg) and I had apparently an invalid specifier.
I also re-downloaded the last compiled version and it reports the errors again.

Ah, I see. I should probably check if the datetime format string is valid first and then set it.


Doesn't relaunch after game is closed - Windows 8.1

As the title states: EmulationStation doesn't relaunch once an emulator has closed on a fully updated Windows 8.1 system. There are threads in the forum stating the same issue is present under Windows 10.
Tested across multiple emulators.

Log file:
lvl2:   EmulationStation - v2.0.1a, built Mar  8 2015 - 16:10:39
lvl2:   Creating surface...
lvl2:   Created window successfully.
lvl2:   Checking available OpenGL extensions...
lvl2:    ARB_texture_non_power_of_two: ok
lvl2:   Loading system config file C:/Users/arcade-machine/.emulationstation/es_systems.cfg...
lvl2:   Parsing XML file "C:/Users/arcade-machine/.emulationstation/gamelists/snes/gamelist.xml"...
lvl2:    req sound [basic.launch]
lvl2:      (missing)
lvl2:   Attempting to launch game...
lvl2:    getting volume as 100 ( from float 1)
lvl2:           C:\ProgramData\chocolatey\bin\zsnesw.exe -m "F:\Emulation\Nintendo, Super\Roms\90 Minutes - European Prime Goal (Europe).smc"
==============================================
==============================================◙lvl2: ○Creating surface...◙

Hang appears to happen in createSurface() function, between lines 33 and 71.

I'm almost a hundred percent sure this has to do with an AMD driver.  See issue 474

Got the same issue on Win 8.1 and AMD A10 APU / R9 280X GPU.
An older version of Emulationstation worked like charme, but after updating to the latest version this error occurs. The task is still opened in background and eats my CPU (Taskmanager) but no window opens after exiting Retroarch...

amd 15.8 Beta drivers fixes this

Confirmed - and if you couldn't update to the 15.8 AMD driver, try the 15.9 that is out now.  Worked great on my end.


[Epic] Clean URLs

We need to change our URLs so that they are more human readable.
Existing URLs: https://docs.google.com/spreadsheets/d/1AL4YDc892AguREWtgQa6vAHtogb5kW9SIF9YUpiqOMk/edit#gid=0
We need to map our existing URLs to new formats. Here is some example that can be followed:
When users clicks on an application from home page or clicks on a page from the application, they are taken to this page:
base_url/applications/618511f258966f7066a109bd/page/618511b058966f7066a109b9
The link to application from home page can be like this:
base_url/apps/my-application-618511f258966f7066a109bd
Link to a page from the application can be like this in view mode:
base_url/my-application/page/my-page-618511b058966f7066a109b9
And in edit mode:
base_url/my-application/page/my-page-618511b058966f7066a109b9/edit
RACI matrix
| ------------- | ------------- |
| Responsible | @arunvjn  , @nayan-rafiq |
| Accountable | @hiteshjoshi|
| Consulted | @Nikhil-Nandagopal, @mohanarpit, @areyabhishek|
| Informed | @kocharrahul7  |

Things to work next on the scope definition document.

URL definitions.
Unicode support.
Edge cases for homepage deletion and new url for homepage.

@hiteshjoshi @nayan-rafiq the URL format seems a little verbose. Why not simply:
base_url/app-name/page-name-id

Update : @arunvjn is working on the POC and have some success.

Not sure if related - I would also like custom slugs if possible.
For e.g. I could have an app for applying for leave as https://myappsmith.com/leaves or a general one like /portal etc. This is more relevant for public URLs that do not have appsmith login flow / homepage to chose that APP from.

@bizcover-bot you can make the application public and share it with your users

I know this is closed, but the other two topics that were considered duplicate of this actually wanted to have cleaner URLs, current URLs, as of Aug 2022, are not user friendly. It could easily be something really usable, like
base_url/[workspace]/[appName]/[pageName]/[mode]

or a more restlike approach, which is not that great for human accesible pages, but still.
base_url/workspace/[workspace]/app/[appName]/page/[pageName]

Honestly, I cannot understand why the workspace is at the end, is at least, counter intuitive.


chore(deps): updates ependencies

I update and test local.
Everything work except the @playwright/test that require download new browser.

I test local with npm run test not with npm run build.
So I check it later.


Bump fulcio chart for ctlog

Bumping fulcio chart for ctlog version update

@cpanato bumped

github is dead lol
lets wait


AppVeyor build issues with 1.4.0

Our build started to fail with the 1.4.0 release. The build hangs after downloading the python installer. When I switch back to the 1.3.2 version, then python-2.7.13 is installed. That works.
With the python-2.7.14 installer, it fails.
It could be due to one of the recent AppVeyor default image changes or specific to our configuration. Could you please advise? By the way, if you need help, assistance investigating the problem, let me know.

I've opened PR #81 for the python installer issue.

Thanks to @pmacn and @jacobq - they're the true heroes and fixed this. Use<EMAIL_ADDRESS>and you should be good. Sorry for the trouble!

Thank you for the fix, I can confirm; AppVeyor works perfectly with 1.4.2.
Does anyone know why the build with node v4.0.0 fails? I have tried to reproduce it on my Windows image using node v4.0.0 but, I was unable to reproduce the failure. Thanks!
$ node --version && npm --version && npm install --ignore-scripts && npm run test
v4.0.0
2.14.2

><EMAIL_ADDRESS>test C:\Users\kittaakos\dev\windows-build-tools
> standard "src/*.js" && npm run build && mocha

><EMAIL_ADDRESS>build C:\Users\kittaakos\dev\windows-build-tools
> babel src -d lib

src\compatible.js -> lib\compatible.js
src\constants.js -> lib\constants.js
src\download.js -> lib\download.js
src\environment.js -> lib\environment.js
src\index.js -> lib\index.js
src\install\index.js -> lib\install\index.js
src\install\launch.js -> lib\install\launch.js
src\install\tailer.js -> lib\install\tailer.js
src\utils.js -> lib\utils.js

  DownloadTools
    downloadTools (installer)
Downloaded BuildTools_Full.exe. Saved to C:\Users\kittaakos\.windows-build-tools\BuildTools_Full.exe.
      √ should attempt to download the installer (765ms)

  Environment
    √ should attempt to lauch the environment script
    √ should attempt to lauch the environment script with the add to path variable

  Install - Launch
    √ should attempt to lauch the installer script

  Install - Tailer
    √ should register the location of the logfile
    √ waitForLogFile() should resolve when the logfile exists

  6 passing (796ms)

BTW, please note the typo in in the spec descriptions: lauch -> launch.

@kittaakos I believe that error is indicating that node 4.x does not understand/support destructuring.
See http://node.green/#ES2015-syntax-destructuring--declarations
Basically, instead of const { x } = { x: 10 } you used to have to do var temp = { x: 10 }; var x = temp.x
const { getWorkDirectory } = require('./get-work-dir')
      ^
SyntaxError: Unexpected token {

Thanks, @jacobq! Do you happen to know why didn't it fail on my Windows image although I set my node version to 4.0.0? I would like to understand what was the difference between the AppVeyor node and my environment but apparently, I am missing something. Cheers!

@kittaakos I think I was editing my comment as you were writing your reply. The difference is most likely due to which revision of the code you were actually testing. AppVeyor was testing a PR for the 2.0 branch, which hit a problem with src\utils\get-build-tools-installer-path.js meanwhile your results show testing of src\utils.js -> lib\utils.js, which suggests to me that you are most likely on a different git branch, e.g. master.

Makes sense, that's why I was not finding that module. Thank you for the explanation!

FYI: VC build tools are pre-installed on Appveyor including Python 2.7 if you use the MSVC 2015 Image or newer. There is no need to use windows-build-tools per se.


Using SharePoint Online templates - export

Is there anyway to download a Lookbook template as a zip file? The article seems to be missing information.
[Enter feedback here]
Is there anyway to download a Lookbook template as a zip file? The article seems to be missing information.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: b9f52c85-8e87-991f-5002-149f011fe8d0
Version Independent ID: 352955bd-4e1a-126a-a026-440d2b62de38
Content: SharePoint site template and site script overview
Content Source: docs/declarative-customization/site-design-overview.md
Service: sharepoint-online
GitHub Login: @VesaJuvonen
Microsoft Alias: vesaj

I don't think so, please see the documentation at the following URL: https://docs.microsoft.com/en-us/sharepoint/add-sample-site. The only way to provision the templates is to click at the button "Add to your tenant".


Karaoke app

Find which Hanzi are known by the user, and display (tone marked) pinyin above unknown characters.
Would make karaoke much more accessible, and a good learning experience.
Of course, extendable to other languages.

What would the output format be? Maybe a chunk of text with both Hanzi and pinyin, or are you thinking of sometime producing real-time output? :o

Maybe it could go into some software that'll highlight lines from left to right, and sync to some data about when each syllable starts. Needs research!

I like the idea of "graded readers" for songs, but this distracts from core domain work so closing the ticket


Remove public constructor for Any<T1, ..> classes

Related forum thread:
http://forums.bridge.net/forum/bridge-net-pro/bugs/2278-open-1327-any-t1-classes-shouldn-t-have-public-constructors
Expected
x3 should not be possible
Actual
x1 is an int, which is either-an-int-or-a-string, which is good.
x2 is a string, which is either-an-int-or-a-string, which is good.
x3 is not an int or a string, which is bad.
Steps To Reproduce
Any<int, string> x1 = 123;
Any<int, string> x2 = "test";
Any<int, string> x3 = new Any<int, string>();

Made Any class constructor protected to allow inheritance (used in Bridge.Angular, for example).


Statically import libsql targets

I am not sure if there are more targets, but developing with bun using the single-file executable mode the libsql driver fails to be imported from the embedded file system as Bun doesn't detect what files are to be imported if done dynamically.
As the code dynamically imports the proper target module we can get a false positive if the built file is run in a project that has the node_modules/@libsql/* target folders so it's recommended to test this by copying the binary file to another directory or simply running it from somewhere else:
e.g:

Image Error Transcript:
error: Cannot find module "@libsql/linux-x64-gnu" from "/$bunfs/root/test-build"

@penberg I wasn't sure about neon, have to check how it works, but I am trying another approach now as this way it actually seems to be importing all into the binary

Hey, there are any updates regarding this? It would be nice to drop the @neon-rs/load dependency!


Simple support for reveal.js and other autolinks

Would be super cool to be able to, on a third-party site or local reveal.js presentation, include a bit of CE javascript that turns appropriately-tagged HTML <code> blocks into links. Or something like that. I mean, the code blocks need to remain, but a little pop-up "edit on CE" link would be cool. Similar to embed.

Well... unofficial hackery can be found in some of my presentations: https://github.com/mattgodbolt/ce-behind-the-scenes/blob/master/compiler-explorer.js
That's good enough for me...for now!


Changing default stream alignment to 64B.

This is not a real fix for the issue but will let us test how critical this value is.

Might also want to revert https://github.com/google/iree/commit/74cab16fa7fafa0c92d41c3d86105ac73d5c4af3 to show the effects.

Good idea! Will push that after this benchmark finishes because I've been waiting for 45mins already :P

Benchmarks didn't update, but full run rebased onto the latest mahesh changes looks good: https://gist.github.com/c29d328dd4369e8504f62c085498a209

I think this breaks MiniLM Bert. It crashes during the runtime.
I'm able to compile and run the model at commit 7e1322e4477412b7a26c6bb9b6747bd06e7e2f1b. I'm not able to run the model at this commit.
To repro:
$ iree-translate --iree-mlir-to-vm-bytecode-module --iree-hal-target-backends=dylib-llvm-aot --iree-input-type=mhlo ~/input.mlir --iree-llvm-target-triple=x86_64-pc-linux-gnu --iree-llvm-target-cpu-features=host -o /tmp/a.vmfb --iree-llvm-loop-unrolling=true

$ iree-benchmark-module --module_file=/tmp/a.vmfb --driver=dylib --task_topology_group_count=0 --function_input=1x128xi32=0 --function_input=1x128xi32=0 --function_input=1x128xi32=0 --entry_function=predict

The input file can be found at https://storage.googleapis.com/iree-shared-files/nod-perf/hanchung/ir.zip

Define crash - segfault in codegen?
If bumping the alignment up to the next power of two makes things crash that's pretty scary. I'll try a diff and see if anything stands out.

I'm not pretty sure, I didn't see the stack trace. The last command I found in Tracy is iree_hal_device_submit_and_wait.

The only difference in the output between before/after on the host is an attribute (I'll verify it's being respected) and a single constant integer inside of the generated LLVM IR (indicating alignment):

it does result in some very different x86 though once LLVM gets ahold of it:

I'll verify the buffer base address really is at 64 bytes in the binary and in memory.

Looks like constants may not be aligned right! It does run just fine for me, though.
[module.predict+000007EC]    vm.call.varadic @hal.command_buffer.push_descriptor_set(%r17(!hal.command_buffer/0x0000022D0AAB15D0), %r4(!hal.executable_layout/0x0000022D0B5A3010), %i15(0), %i15(0), %r3(!hal.buffer/0x0000022D7FE61AC0), %i15(0), %i14(133405184), %i2(1), %r0(!hal.buffer/0x0000022D0B5BB720), %i15(0), %i3(512), %i13(2), %r2(!hal.buffer/0x0000022D0B5BBC40), %i15(0), %i3(512), %i12(3), %r16(!hal.buffer/0x0000022D064A1040), %i15(0), %i9(2557440))
[module.predict+00000826]    vm.call @hal.command_buffer.dispatch(%r17(!hal.command_buffer/0x0000022D0AAB15D0), %r5(!hal.executable/0x0000022D0B43A860), %i2(1), %i16(6), %i13(2), %i2(1))
dispatch
  binding[0]: 0000022D0BB6AEA0 / 32
  binding[1]: 0000022D0B5BB7A0 / 32
  binding[2]: 0000022D0B5BBCC0 / 0
  binding[3]: 0000022D064A10C0 / 0

I'll figure out why alignment isn't propagating right.

🤦  everything is right in the file, but the ancient debug c++ that loads files into memory doesn't align things

(and can't 'cause it's following the silly abseil pattern of using std::string to allocate memory for binary buffers - time to nuke this all)

I've fixed file alignment for constants but it turns out the allocated heap buffers have only ever been aligned to 16B. This means that the 32B alignment value was also wrong before but just happened to line up most of the time. For me the only time I get issues is using -iree-llvm-target-cpu-features=host; I think because AVX is only kicking in with that flag and 64B alignment:

I'll get the file changes in that fix alignment for constants but it was chance that it ever worked with transient buffers and I'll need to find a good solution for that.

(file fix https://github.com/google/iree/pull/8505, issue for tracking alignment of heap buffers https://github.com/google/iree/issues/8506)

Thanks for looking into this tricky issue! :D


Use unoptimized RMSNorm kernel if pointers are not aligned

Description
The optimimzed RMSNorm kernels access data with vectorized loads/stores, and can thus experience alignment issues if the data pointers are misaligned (e.g. they are views within a larger buffer). This PR falls back to unoptimized kernels (with entry-wise memory accesses) if data pointers are not aligned.
This is the same fix as https://github.com/NVIDIA/TransformerEngine/pull/490.
Type of change

[ ] Documentation change (change only to the documentation, either a fix or a new content)
[x] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)

Changes

Use unoptimized RMSNorm kernel if pointers are not aligned

Checklist:

[x] I have read and followed the contributing guidelines
[x] The functionality is complete
[x] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] My changes generate no new warnings
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes

/te-ci

LGTM!
Would you like to add a unit test for this as well?


Rename cirq.google.ExpWGate's attributes to more closely match cirq.PhasedXGate

Part of https://github.com/quantumlib/Cirq/issues/952

Automerge: started

Automerge done.


Set an empty string for entrypoint if false-y

Another take on #125.

Any chance we can make the workflow public here, would love to patch it, but cant see the failures in CI.

Is this in a merge-able state?/Is there anything blocking this fix? entrypoint: false is still not working as of v3.8.0 (from: https://github.com/buildkite-plugins/docker-buildkite-plugin/issues/138)

Hey @lox! any idea on the state of this PR? Do you think is worth closing it and working on a new one? thanks!

Hey folks! we are going to close this one because it's not ready to merge and it's very outdated. We have in our backlog to fix entrypoint, but feel free to submit a new PR 🙏🏻. Thanks!

We'll follow up on the open issue #138 😊


[build_dashboard] No legend for the grid

The old build dashboard has a ? button that shows a legend for what the different grid items mean.  I don't see any equivalent on the new build dashboard.

/cc @CaseyHillers

The legend statuses were moved to help clear some of the clutter. The status for a color is still accessible by clicking on a task and hovering over it's icon (see below)

James, do you think this is okay? We could also surface this information via the overlay and just have some text at the top that says:
Status: In Progress or whatever the status of the task is.


Implement 'url' API endpoint

This implements the '/files/{id}/url' endpoint, which gets a URL to download a file

Thanks @stridervc!


K8SSAND-1325 ⁃ Java garbage collection properties cannot be configured correctly

What happened?
I created a Cassandra 4.x cluster with some GC properties set in my manifest, but they were not configured correctly in the Cassandra pod. This is closely related to https://github.com/k8ssandra/k8ssandra/issues/1353 in that it has to do with how [cass]-config-definitions] handles JVM GC properties for Cassandra 4.x.
First I tried to configure some G1 settings:
apiVersion: k8ssandra.io/v1alpha1
kind: K8ssandraCluster
metadata:
  name: test
spec:
  auth: false
  cassandra:
    serverVersion: "4.0.3"
    storageConfig:
      cassandraDataVolumeClaimSpec:
        storageClassName: standard
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 5Gi
    config:
      jvmOptions:
        heapSize: 512Mi
        additionalOptions:
          - "-XX:MaxGCPauseMillis=600"
          - "-XX:ParallelGCThreads=4"
    networking:
      hostNetwork: true
    datacenters:
      - metadata:
           name: dc1
         size: 1

The additionalOptions property basically appends properties to the list of properties passed to Cassandra's JVM. -XX:MaxGCPauseMillis=500 is already passed, so -XX:MaxGCPauseMillis=600 is appended to the list. It isn't clear which one will take precedence, but there should only be one instance of -XX:MaxGCPauseMillis in the properties passed to the JVM.
I didn't see -XX:ParallelGCThreads previously set so that was ok.
I tried configuring CMS with the following settings:
    config:
      jvmOptions:
        heapSize: 512Mi
        additionalOptions:
          - "-XX:+UseConcMarkSweepGC"
          - "-XX:SurvivorRatio=4"

Again those properties are simply appended to the list of properties passed to the JVM. -XX:+UseG1GC appears earlier in the list, so presumably it is used.
I don't think we can rely on using additionalOptions for configuring GC properties.
Did you expect to see something different?
I expect to be able to configure GC properties.
How to reproduce it (as minimally and precisely as possible):
Deploy the above manifest. When the Cassandra pod is ready grab Cassandra's logs with:
kubectl logs test-dc1-default-sts-0 -c server-system-logger > cassandra.log
Look for the log statement that starts with INFO  [main] 2022-03-13 22:35:40,855 CassandraDaemon.java:642 - JVM Arguments.
Environment

K8ssandra Operator version:
1.0.1

┆Issue is synchronized with this Jira Task by Unito
┆friendlyId: K8SSAND-1325
┆priority: Medium

A few data points:

In (almost?) all JMVs, in case of duplicate JVM options, the last occurrence wins (see here). It's not a documented feature, but it just happens to be that way for every JVM out there. So I would expect our options to be taken into account rather than the default ones, since cass-config-builder usually appends additional stuff at the end of each file. If that's not happening, maybe it's because files such as jvm.options, jvm-server.options. etc. are treated differently, it would be good to investigate that:

In C* 4.0, the various jvm*.options files are read into $JVM_OPTS in cassandra.in.sh, see here.
In C* 3.11, these files are read inside cassandra-env.sh, see here.

Currently config.jvmOptions.additionalOptions is being inserted in cassandra-env.sh, at the end of the script, see here. I did this recently so that our JMX-related JVM options are not accidentally overwritten by previous lines in this script.

As I said, I would expect anything specified in config.jvmOptions.additionalOptions to prevail, because they are applied last. Maybe what's happening is that contradictory GC settings end up being applied by the JVM?

In general, Cassandra startup scripts generated by cass-config builder are tricky to customize in a fine-grained fashion because they take decisions for you that might contradict your expectations, e.g.: JMX which is a mess since it is configured in many files in ways that are not always customizable. I fear that garbage collection is another example.
All config-builder templates for jvm*.options files accept a parameter called garbage_collector, see here. The default is always G1GC. We might want to expose this setting in k8ssandra-operator, Having it default to G1GC is maybe interfering when you specify a different GC, e.g. with -XX:+UseConcMarkSweepGC.

I tried the above definition using G1. Here is what I observed:
Contents of /etc/cassandra:
cassandra-env.sh
cassandra-jaas.config
cassandra-rackdc.properties
cassandra-topology.properties
cassandra.yaml
commitlog_archiving.properties
cqlshrc.sample
hotspot_compiler
jvm11-clients.options
jvm11-server.options
jvm8-clients.options
jvm8-server.options
jvm-clients.options
jvm.options
jvm-server.options
logback-tools.xml
logback.xml
metrics-reporter-config-sample.yaml
README.txt
triggers

Contents of jvm.options (trimmed to relevant bits):
-XX:G1RSetUpdatingPauseTimePercent=5
-XX:MaxGCPauseMillis=500
-XX:+UseG1GC

Contents of jvm-server.options: nothing relevant to GC.
Contents of jvm11-server.options (trimmed to relevant bits):
-XX:G1RSetUpdatingPauseTimePercent=5
-XX:MaxGCPauseMillis=500
-XX:+UseG1GC
-XX:+ParallelRefProcEnabled

Contents of cassandra-env.sh (only relevant bits):
echo $JVM_OPTS | grep -q UseConcMarkSweepGC
USING_CMS=$?
echo $JVM_OPTS | grep -q +UseG1GC
USING_G1=$?

# We only set -Xmn flag if it was not defined in jvm-server.options file
# and if the CMS GC is being used
# If defined, both Xmn and Xmx should be defined together.
if [ $DEFINED_XMN -eq 0 ] && [ $DEFINED_XMX -ne 0 ]; then
    echo "Please set or unset -Xmx and -Xmn flags in pairs on jvm-server.options file."
    exit 1
elif [ $DEFINED_XMN -ne 0 ] && [ $USING_CMS -eq 0 ]; then
    JVM_OPTS="$JVM_OPTS -Xmn${HEAP_NEWSIZE}"
fi

# We fail to start if -Xmn is used with G1 GC is being used
# See comments for -Xmn in jvm-server.options
if [ $DEFINED_XMN -eq 0 ] && [ $USING_G1 -eq 0 ]; then
    echo "It is not recommended to set -Xmn with the G1 garbage collector. See comments for -Xmn in jvm-server.options for details."
    exit 1
fi

if [ "$JVM_ARCH" = "64-Bit" ] && [ $USING_CMS -eq 0 ]; then
    JVM_OPTS="$JVM_OPTS -XX:+UseCondCardMark"
fi

JVM_OPTS="$JVM_OPTS $JVM_EXTRA_OPTS"

JVM_OPTS="$JVM_OPTS -Dcassandra.system_distributed_replication_dc_names=dc1"
JVM_OPTS="$JVM_OPTS -Dcassandra.system_distributed_replication_per_dc=1"
JVM_OPTS="$JVM_OPTS -Dcom.sun.management.jmxremote.authenticate=false"
JVM_OPTS="$JVM_OPTS -XX:MaxGCPauseMillis=600"
JVM_OPTS="$JVM_OPTS -XX:ParallelGCThreads=4"
JVM_OPTS="$JVM_OPTS -javaagent:/opt/metrics-collector/lib/datastax-mcac-agent.jar"
JVM_OPTS="$JVM_OPTS -javaagent:/opt/management-api/datastax-mgmtapi-agent-0.1.0-SNAPSHOT.jar"

Note that the last lines above are the ones added by K8ssandra.
Now contents of JVM flags as reported by the logs, trimmed to relevant ones only:

-XX:G1RSetUpdatingPauseTimePercent=5 (from jvm11-server.options)
-XX:MaxGCPauseMillis=500 (from jvm11-server.options)
-XX:+UseG1GC (from jvm11-server.options)
-XX:+ParallelRefProcEnabled (from jvm11-server.options)
-XX:MaxGCPauseMillis=600 (from cassandra-env.sh)
-XX:ParallelGCThreads=4 (from cassandra-env.sh)

Indeed -XX:MaxGCPauseMillis appears twice, but I'm confident that the last appearance wins. IMO for G1 at least, everything is working as expected. I will try CMS next.

So with CMS, things get indeed into the weeds. I tried the definition above with CMS, and cassandra didn't even started.
Contents of files didn't change except, of course, cassandra-env.sh that had at the end:
JVM_OPTS="$JVM_OPTS -XX:+UseConcMarkSweepGC"
JVM_OPTS="$JVM_OPTS -XX:SurvivorRatio=4"

The reason was:
cassandra@k8ssandra-0-worker4:/$ cat /var/log/cassandra/stdout.log
Error occurred during initialization of VM
Multiple garbage collectors selected

Hopefully, there is a workaround: add -XX:-UseG1GC to additionalOptions:
    config:
      jvmOptions:
        heapSize: 512Mi
        additionalOptions:
          - "-XX:-UseG1GC" # add this
          - "-XX:+UseConcMarkSweepGC"
          - "-XX:SurvivorRatio=4"

With this change, Cassandra started, and indeed CMS is used, as we see from the logs:
INFO  [main] 2022-03-15 11:54:44,199 CassandraDaemon.java:638 - Par Eden Space Heap memory: init = 119275520(116480K) used = 119275520(116480K) committed = 119275520(116480K) max = 119275520(116480K)
INFO  [main] 2022-03-15 11:54:44,199 CassandraDaemon.java:638 - Par Survivor Space Heap memory: init = 29818880(29120K) used = 14816800(14469K) committed = 29818880(29120K) max = 29818880(29120K)
...
INFO  [main] 2022-03-15 11:54:44,199 CassandraDaemon.java:638 - CMS Old Gen Heap memory: init = 357957632(349568K) used = 0(0K) committed = 357957632(349568K) max = 357957632(349568K)

We can even see that the survivor ratio was correctly applied: 119275520/29818880 = 4.
Conclusions:

It is risky to attempt to change GC without changing the garbage_collector config-builder variable – I suggest that we found a way to expose it in K8ssandra.
There is hopefully a workaround, but one needs to "cancel" all GC settings before applying new ones, e.g. add -XX:-UseG1GC before -XX:+UseConcMarkSweepGC.
This workaround has one downside: in cassandra-env.sh, 2 variables are used for sanity checks: USING_CMS and USING_G1. Given that K8ssandra JVM options arrive after their declarations, these variables will always be set to false and true respectively, effectively making the sanity checks inoperable.

We currently don't expose the jvm11-server-options key in K8ssandra operator. The JvmOptions type has:
type JvmOptions struct {
	HeapSize          *resource.Quantity `json:"heapSize,omitempty"`
	HeapNewGenSize    *resource.Quantity `json:"heapNewGenSize,omitempty"`
	AdditionalOptions []string           `json:"additionalOptions,omitempty"`
}

HeapSize and HeapNewGenSize go into initial_heap_size, max_heap_size and heap_size_young_generation, under the jvm-server-options key. And as I said, AdditionalOptions end up under the cassandra-env-sh key as additional-jvm-opts.
We will need to modify JvmOptions and add more fields there, I think this is a backwards compatible change as long as the new fields are optional. I suggest that we don't change AdditionalOptions since this field is key for JMX to work properly.

I am also in favor of leaving AdditionalOptions as is. If possible I prefer to avoid exposing jvm-server-options. I think it should be an implementation detail. We should just add additional properties in the JvmOptions struct and the operator should configure them as necessary in the CassandraDatacenter.

Also see my comment here: https://github.com/k8ssandra/k8ssandra/issues/1353#issuecomment-1068157469.
Your proposed config:
    jvm-server-options:
      initial_heap_size: "512m"
      max_heap_size: "512m"
    jvm11-server-options:
      garbage_collector: G1GC
      max_gc_pause_millis: 550

is indeed correct. I wonder however if we shouldn't explicitly disable GC setting in jvm-options:
    jvm-options:
      garbage_collector: NONE
    jvm-server-options:
      initial_heap_size: "512m"
      max_heap_size: "512m"
    jvm11-server-options:
      garbage_collector: G1GC
      max_gc_pause_millis: 550

I wonder however if we shouldn't explicitly disable GC setting in jvm-options

That might be the sane thing to do so configs aren't spread across too many files. My primary concern though is making sure we provide a simpler, uniform interface to the user which I think should be JvmOptions.
On a related note @Miles-Garnsey created https://github.com/k8ssandra/cass-operator/issues/296 the other day.
If there are scenarios where we can determine one garbage collector would be better over another and the user has not explicitly configured GC, then I think the operator should apply the settings.