Version banner template example doesn't work because latest_version is hardcoded to 'master'

(First off, thanks for your work! My project was stuck on an old version of Sphinx due to using sphinxcontrib-versioning, and I'm almost done switching us over to sphinx-multiversion 🥳 )
Background
The template example for "version banners" is exactly what I'd like to use: when the user is viewing an unreleased version, warn them that it is in development; and if they are viewing and older released version, warn them that there's a newer version available.
Problem
However, the example doesn't work on my repo (using Sphinx 2.4.4 and sphinx-multiversion 0.2.4). The latest_version HTML context variable was None.
It seems like this context variable is based on the smv_latest_version config value, set to "master" by default. My repo doesn't have a "master" branch, which I guess is why the context variable is None.
Ideally, I'd like to configure sphinx-multiversion to dynamically use the greatest released tag version as the latest version, instead of a static config value.
Mapping to sphinxcontrib-versioning's options:

smv_latest_version seems to be the same as scv_banner_main_ref
the feature I would like to have is scv_banner_greatest_tag
another related feature is scv_banner_recent_tag (for treating the most recently created tag as the latest version)
scv_sort defines the sort ordering

Potential improvements

Add documentation for the smv_latest_version config value.
Add a boolean config like smv_latest_version_is_greatest_tag (or extend the semantics of smv_latest_version to read a certain sentinel value?

If you're supportive of this idea, but feel that it's low priority, I may be able to find some time at work to contribute a patch.
Thanks!

FWIW the example didn't work for me either, and would also be interested in having a version-based page banner :)

FWIW the example didn't work for me either, and would also be interested in having a version-based page banner :)

Actually it's not hard coded to master, just undocumented. If you look at Mixxx (for which @Holzhaus originally wrote this extension), you can actually set smv_latest_version in conf.py.
https://github.com/mixxxdj/manual/blob/5165e797563e46accfd2dbb10fe80fc5109c414a/source/conf.py#L106
I agree however, that it would be nice if it just took the latest version as defined by smv_released_pattern.

For our own project, I did some digging into the issue.
First, if you look at the documentation of @Holzhaus's own project Mixxx (for which he built sphinx-multiversion in the first place), you can see that you can actually set smv_latest_version in conf.py.
https://github.com/mixxxdj/manual/blob/5165e797563e46accfd2dbb10fe80fc5109c414a/source/conf.py#L106
Instead of hard coding this into the conf file, I thought it would be much more convenient to override this value using the -D option of sphinx-build/sphinx-multiversion. However, then i got the following warning:
unknown config value 'smv_outputdir_format' in override, ignoring

So I did some further digging. The smv config is added to Sphinx here (if 'add_default = True'):
https://github.com/Holzhaus/sphinx-multiversion/blob/cdc9cb591a738e67a5aacc458f9c95e2b88a94c2/sphinx_multiversion/main.py#L41
However, in main, when we loop over all the git refs, the function is called without the add_default argument (which itself is False by default. So by calling with
current_config = load_sphinx_config(confpath, confoverrides, add_default=True)

I could enable setting the latest version from the cli.
Maybe this also helps someone else.
Oh, I also had to use the alternative regex pattern for 'smv_released_pattern' as described in #66 .


Windows release script fails to create link

Fixes the examples and external directory creation and linking for windows release script.

Test PASSed.

Test PASSed.


Consider using AppImage and AppImageUpdate instead of shell script for universal platform support

This proposal is around replacing shell script with AppImage format. I'm not recommending  any replacement of distro specific packages (deb/rpm)
https://appimage.org  is a packaging tool that makes executable installers that run on every major linux platform.
They also have built-in update tooling https://docs.appimage.org/packaging-guide/updates.html
AppImages are also signed for security - https://docs.appimage.org/packaging-guide/signatures.html
They can also be mounted and inspected - https://docs.appimage.org/user-guide/run-appimages.html
There's already a large list of appimages that are popularly used - https://appimage.github.io/apps/
AppImage is an infintely better way to distribute k3s versus shell scripts.

Sorry, I am trying to understand the advantages of using AppImage versus our current system, can you please elaborate on how it is infinitely better? It appears that we have solved most of the issues AppImage is trying to solve by using static + muslc compilations of k3s (also a self extracting binary). The installation script provides a single line command for installing that k3s binary to systemd or openrc, does AppImage help with that use-case?

@erikwilson You are right, appimage has no relation with the shell scripts that you provide.
But what would be the best is to have linux repos with the binaries inside so we can just update k3s with a yum update or apt update.

We are certainly happy to help package maintainers create OS specific releases, it looks like Alpine and Arch have some packages currently available:
https://pkgs.alpinelinux.org/packages?name=k3s&branch=edge
https://aur.archlinux.org/packages/k3s-bin/
For the most part we have upstreamed our containerd changes for better arm support. There is some challenge because we tend to test and ship with specific versions of supporting packages that are newer than what may be used in the OS distributions. For instance alpine ships containerd 1.2.9 and strongswan 5.7.2 where we are now shipping containerd 1.3.0 and strongswan 5.8.1 with k3s v0.10.0-alpha1.
We have considered providing an auto-update feature, perhaps a command like k3s update/upgrade could help minimize that effort.

Hi
Thank you guys for your comments. Sorry if the intent was lost in the
poorly worded title.
My original purpose for this issue was to address the primary problem of
predictable updates and signing...in a context where the k3s devs won't be
able to support all the myriad packaging formats there are.
Appimage was one. https://cnab.io/ could be other. Not sure if k3s is
viable to be delivered as a docker image itself.
However I wanted to highlight that as k3s goes closer towards prediction
usecases, some sort of robust signing/update/package management features
are needed.
On Tue, 8 Oct, 2019, 22:58 Erik Wilson<EMAIL_ADDRESS>wrote:

We are certainly happy to help package maintainers create OS specific
releases, it looks like Alpine and Arch have some packages currently
available:
https://pkgs.alpinelinux.org/packages?name=k3s&branch=edge
https://aur.archlinux.org/packages/k3s-bin/
For the most part we have upstreamed our containerd changes for better arm
support. There is some challenge because we tend to test and ship with
specific versions of supporting packages that are newer than what may be
used in the OS distributions. For instance alpine ships containerd 1.2.9
and strongswan 5.7.2 where we are now shipping containerd 1.3.0 and
strongswan 5.8.1 with k3s v0.10.0-alpha1.
We have considered providing an auto-update feature, perhaps a command
like k3s update/upgrade could help minimize that effort.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rancher/k3s/issues/871?email_source=notifications&email_token=AAASYU27KQFL26KWEQ5XFNDQNS7KZA5CNFSM4I6G4XB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAU63UI#issuecomment-539618769,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAASYU2RZ76FOFWBXANGKU3QNS7KZANCNFSM4I6G4XBQ
.

Who uses CNAB? (serious question) Is it an Azure only thing?
What is needed for it? Just a bundle.json file? How do I use it?
PRs are welcome! (please see /contrib)

Updates should be as simple as replacing the binary and restarting. Some update logic could probably be added to roll updates out to nodes.
Validation of the binary is accomplished at the moment by checking the hash of the executable, if you trust github certs and build process this should be sufficient.
Yes there is a k3s docker image, there is also k3os and k3d, please look into those.

I think we should be moving as much functionality from the install script into the k3s binary as possible. I would like to see some commands like the following:
# validate the binary
k3s verify binary

# validate system requirements
k3s verify system

# install a systemd or openrc service
k3s install (... args ...)

# upgrade k3s
k3s upgrade (version)

# auto-update k3s (download & log but no restart)
k3s auto-update (on|off) (schedule)

# auto-upgrade k3s (download & restart)
k3s auto-upgrade (on|off) (schedule)

@erikwilson Are you sure putting more « complex » features does not go against k3s purpose which is to be as light as possible ?
I know that on Windows or Mac OS there is no other choice but on Linux, we have package managers so I think it would be best to use it instead and call for volunteers to maintain packages/repo for each distribution like the guys from alpine and arch Linux does.
What do you think ?

The commands are light in executable size, memory, and cpu footprint. They may not be ideal from a package maintainer's point of view so could maybe be disabled for distributions. But, they may not be ideal from a cognitive overhead view also either even tho the purpose is to make using k3s easier, so maybe better to stick to the install script. Personally I don't think many people are using the distro packages at the moment, I would prefer to have access to the latest k3s/k8s release rather than wait for it to trickle down through a distro anyways.

Yes there is a k3s docker image, there is also k3os and k3d, please look
into those.

Question in general - why is that not the primary release mechanism for k3s
? does it have a limitation that needs k3s to run natively and through the
script ? If there was a way that docker-based k3s works well, then that
should be the primary release mechanism. docker will give you signing,
updates, versioning...pretty much all the benefits of a package manager.

I think we should be moving as much functionality from the install script
into the k3s binary as possible. I would like to see some commands like the
following:

totally totally  agree here.

@erikwilson https://github.com/erikwilson Are you sure putting more
« complex » features does not go against k3s purpose which is to be as
light as possible ?

K3s is as simple as possible when it comes to kubernetes. But that does
mean batteries included. It can be argued that k3s should not have network
plugins or ingresses bundled in. But I'm not very sure that a few mb in
size decrease is what will make k3s compelling versus k8s. The problem with
k8s is complexity of getting started and bundling+batteries is
simplification.
I would go as far as to say that even if k3s is double the size of k8s, but
is super simple... that would solve what most people are looking for.
On Wed, Oct 9, 2019 at 4:43 AM Erik Wilson<EMAIL_ADDRESS>wrote:

The commands are light in executable size, memory, and cpu footprint. They
may not be ideal from a package maintainer's point of view so could maybe
be disabled for distributions. But, they may not be ideal from a cognitive
overhead view also either even tho the purpose is to make using k3s easier,
so maybe better to stick to the install script. Personally I don't think
many people are using the distro packages at the moment, I would prefer to
have access to the latest k3s/k8s release rather than wait for it to
trickle down through a distro anyways.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rancher/k3s/issues/871?email_source=notifications&email_token=AAASYU7INTM74QFFWGYRKBLQNUHZPA5CNFSM4I6G4XB2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEAV5FBA#issuecomment-539742852,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAASYU7AKYVTEGTGSFVJZLTQNUHZPANCNFSM4I6G4XBQ
.

Closing due to age


NullReferenceException when using Information alias

This issue is continuing the issue at Nunit/Nunit#930.
Some information about this issue (by @CharliePoole):
I'm running it on Windows 7.
I'm using the 0.6.0 zip from github cake-build/cake
My build file is the sample file shown here: https://github.com/cake-build/cake#2-create-a-cake-script

So here is what I just did, starting from scratch...
1. Created a temp directory
2. Made sure that the security setting on the zip was cleared
3. Unzipped, creating a temp\Cake directory
4. Began typing in the script a bit at a time, running it after each change.

All the results were as expected until I filled in the last bit, which I already suspected.
That was the line
Information("Hello World!");
It gave me a null reference error.

I tried setting verbosity. It ran without error but still didn't produce the "Hello" output. If I used different settings I got more or less info from Cake but still no "Hello World!"

Finally, after running with verbosity set a few times, I removed that setting from the command line. It still didn't display anything but also didn't give an error.

So it seems to be something that only occurs right after install up to the point where you use a verbosity argument. Is it remembering that argument? Is it initializing something?

And of course, it's still not saying hello to me :-(

Next I'll try the bootstrapper.

Here you go...
D:\temp>Cake\Cake.exe build.cake -verbosity=Diagnostic
Analyzing build script...
Analyzing D:/temp/build.cake...
Processing build script...
Creating script session...
Adding reference to mscorlib.dll...
Adding reference to System.dll...
Adding reference to System.Core.dll...
Adding reference to System.Data.dll...
Adding reference to System.Xml.dll...
Adding reference to System.Xml.Linq.dll...
Adding reference to Cake.Core.dll...
Adding reference to Cake.Common.dll...
Adding reference to Cake.exe...
Importing namespace Cake.Common...
Importing namespace Cake.Common.Build...
Importing namespace Cake.Common.Diagnostics...
Importing namespace Cake.Common.IO...
Importing namespace Cake.Common.IO.Paths...
Importing namespace Cake.Common.Net...
Importing namespace Cake.Common.Security...
Importing namespace Cake.Common.Solution...
Importing namespace Cake.Common.Solution.Project...
Importing namespace Cake.Common.Solution.Project.Properties...
Importing namespace Cake.Common.Solution.Project.XmlDoc...
Importing namespace Cake.Common.Text...
Importing namespace Cake.Common.Tools...
Importing namespace Cake.Common.Tools.Cake...
Importing namespace Cake.Common.Tools.Chocolatey...
Importing namespace Cake.Common.Tools.Chocolatey.ApiKey...
Importing namespace Cake.Common.Tools.Chocolatey.Config...
Importing namespace Cake.Common.Tools.Chocolatey.Features...
Importing namespace Cake.Common.Tools.Chocolatey.Install...
Importing namespace Cake.Common.Tools.Chocolatey.Pack...
Importing namespace Cake.Common.Tools.Chocolatey.Pin...
Importing namespace Cake.Common.Tools.Chocolatey.Push...
Importing namespace Cake.Common.Tools.Chocolatey.Sources...
Importing namespace Cake.Common.Tools.Chocolatey.Upgrade...
Importing namespace Cake.Common.Tools.DupFinder...
Importing namespace Cake.Common.Tools.Fixie...
Importing namespace Cake.Common.Tools.GitReleaseManager...
Importing namespace Cake.Common.Tools.GitReleaseManager.AddAssets...
Importing namespace Cake.Common.Tools.GitReleaseManager.Close...
Importing namespace Cake.Common.Tools.GitReleaseManager.Create...
Importing namespace Cake.Common.Tools.GitReleaseManager.Export...
Importing namespace Cake.Common.Tools.GitReleaseManager.Publish...
Importing namespace Cake.Common.Tools.GitReleaseNotes...
Importing namespace Cake.Common.Tools.GitVersion...
Importing namespace Cake.Common.Tools.ILMerge...
Importing namespace Cake.Common.Tools.ILRepack...
Importing namespace Cake.Common.Tools.InspectCode...
Importing namespace Cake.Common.Tools.MSBuild...
Importing namespace Cake.Common.Tools.MSTest...
Importing namespace Cake.Common.Tools.NSIS...
Importing namespace Cake.Common.Tools.NuGet...
Importing namespace Cake.Common.Tools.NuGet.Install...
Importing namespace Cake.Common.Tools.NuGet.Pack...
Importing namespace Cake.Common.Tools.NuGet.Push...
Importing namespace Cake.Common.Tools.NuGet.Restore...
Importing namespace Cake.Common.Tools.NuGet.SetApiKey...
Importing namespace Cake.Common.Tools.NuGet.SetProxy...
Importing namespace Cake.Common.Tools.NuGet.Sources...
Importing namespace Cake.Common.Tools.NuGet.Update...
Importing namespace Cake.Common.Tools.NUnit...
Importing namespace Cake.Common.Tools.OctopusDeploy...
Importing namespace Cake.Common.Tools.OpenCover...
Importing namespace Cake.Common.Tools.ReportGenerator...
Importing namespace Cake.Common.Tools.ReportUnit...
Importing namespace Cake.Common.Tools.Roundhouse...
Importing namespace Cake.Common.Tools.SignTool...
Importing namespace Cake.Common.Tools.WiX...
Importing namespace Cake.Common.Tools.XBuild...
Importing namespace Cake.Common.Tools.XUnit...
Importing namespace Cake.Common.Xml...
Importing namespace Cake.Core...
Importing namespace Cake.Core.Diagnostics...
Importing namespace Cake.Core.IO...
Importing namespace Cake.Core.Scripting...
Importing namespace System...
Importing namespace System.Collections.Generic...
Importing namespace System.IO...
Importing namespace System.Linq...
Importing namespace System.Text...
Importing namespace System.Threading.Tasks...
Compiling build script...
Error: System.NullReferenceException: Object reference not set to an instance of
an object.
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.InitializerDimensions(BoundA
rrayInitialization init)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitAllElementInitializers(A
rrayTypeSymbol arrayType, BoundArrayInitialization inits, Boolean skipConstants)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitVectorInitializers(Array
TypeSymbol arrayType, BoundArrayInitialization inits)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitArrayCreationExpression(
BoundArrayCreation expression, Boolean used)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitExpression(BoundExpressi
on expression, Boolean used)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitArgument(BoundExpression
argument, RefKind refKind)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitArguments(ReadOnlyArray 1 arguments, ReadOnlyArray1 parameters)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitCallExpression(BoundCall
call, Boolean used)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitExpression(BoundExpressi
on expression, Boolean used)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitStatement(BoundStatement
statement)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitBlockStatement(BoundBloc
k block)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.EmitStatement(BoundStatement
statement)
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.Generate()
at Roslyn.Compilers.CSharp.CodeGen.CodeGenerator.Run(MethodSymbol meth, Bound
Statement block, ILBuilder builder, Module module, DiagnosticBag diagnostics, Bo
olean optimize, Boolean emitSequencePoints)
at Roslyn.Compilers.CSharp.Compiler.GenerateMethodBody(TypeCompilationState c
ompilationState, MethodSymbol method, BoundStatement block, DiagnosticBag diagno
stics, Boolean optimize, Boolean emitSequencePoints, Func2 mapFileNameToDebugDo c, ReadOnlyArray1 namespaceScopes)
at Roslyn.Compilers.CSharp.MethodBodyCompiler.ProcessGeneratedMethods(TypeCom
pilationState compilationState)
at Roslyn.Compilers.CSharp.MethodBodyCompiler.CompileNamedType(NamedTypeSymbo
l symbol)
at Roslyn.Compilers.CSharp.MethodBodyCompiler.<>c__DisplayClass6.b__4()
at System.Threading.Tasks.Task.InnerInvoke()
at System.Threading.Tasks.Task.Execute()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNot
ification(Task task)
at Roslyn.Compilers.CSharp.MethodBodyCompiler.WaitForWorkers()
at Roslyn.Compilers.CSharp.MethodBodyCompiler.CompileAllMethodBodies(Compilat
ion compilation, Module moduleBeingBuilt, Boolean generateDebugInfo, DiagnosticB
ag diagnostics, CancellationToken cancellationToken)
at Roslyn.Compilers.CSharp.Compiler.CompileAllMethodBodies(Compilation compil
ation, Module moduleBeingBuilt, Boolean generateDebugInfo, DiagnosticBag diagnos
tics, CancellationToken cancellationToken)
at Roslyn.Compilers.CSharp.Compilation.Emit(ModuleBuilder moduleBuilder, IAss
emblyLoader assemblyLoader, Func2 assemblySymbolMapper, CancellationToken cance llationToken, Boolean recoverOnError, ReadOnlyArray1& compiledAssemblyImage)
at Roslyn.Compilers.CSharp.Compilation.CommonEmit(ModuleBuilder moduleBuilder
, IAssemblyLoader assemblyLoader, Func2 assemblySymbolMapper, CancellationToken  cancellationToken, Boolean recoverOnError, ReadOnlyArray1& compiledAssemblyIma
ge)
at Roslyn.Scripting.CommonScriptEngine.TryEmitSubmission(CommonCompilation co
mpilation, DiagnosticBag diagnostics, Type delegateType, Boolean collectible, Ca
ncellationToken cancellationToken, Delegate& factory)
at Roslyn.Scripting.CommonScriptEngine.Compile(String code, String path, Diag
nosticBag diagnostics, Session session, Type delegateType, Type returnType, Canc
ellationToken cancellationToken, Boolean isInteractive, Boolean isExecute, Commo
nCompilation& compilation, Delegate& factory)
at Roslyn.Scripting.CommonScriptEngine.Execute[T](String code, String path, D
iagnosticBag diagnostics, Session session, Boolean isInteractive)
at Roslyn.Scripting.Session.Execute(String code)
at Cake.Scripting.Roslyn.Stable.RoslynScriptSession.Execute(Script script)
at Cake.Core.Scripting.ScriptRunner.Run(IScriptHost host, FilePath scriptPath
, IDictionary2 arguments) in c:\Source\Git\cake-build\cake\src\Cake.Core\Script ing\ScriptRunner.cs:line 175    at Cake.Commands.BuildCommand.Execute(CakeOptions options)    at Cake.CakeApplication.Run(IEnumerable1 args)
D:\temp>

@CharliePoole Are you running this on a 32 or 64-bit OS?

@patricksvensson 32
On Sat, Nov 14, 2015 at 5:58 AM, Patrik Svensson<EMAIL_ADDRESS>wrote:

@CharliePoole https://github.com/CharliePoole Are you running this on a
32 or 64-bit OS?
—
Reply to this email directly or view it on GitHub
https://github.com/cake-build/cake/issues/521#issuecomment-156698114.

@CharliePoole
Ok, this is actually a known Roslyn issue (see #241) when running on 32-bit platforms with .NET 4.6 installed. This was a problem for 64-bit OS as well since we compiled with Prefer 32-bit enabled and we solved this in #247 by compiling for MSIL (Any CPU).
Good news is that there is are two workarounds for this. One that should work without problem, and another one that might work (depending on your preferences). Let us know if any of this worked for you.
Workaround 1
The first workaround, is to use the Mono script engine. You do this by appending the -mono flag. This should work without problem.
D:\temp>Cake\Cake.exe build.cake -verbosity=Diagnostic -mono

Workaround 2
The second (potential) workaround (if you really want to use the Roslyn script engine) is to use the experimental Roslyn bits. I'm not 100% sure this will work, but it's worth a try. You do this by appending the -experimental flag.
D:\temp>Cake\Cake.exe build.cake -verbosity=Diagnostic -experimental

Both workarounds work for me, so I think you can close this. Is there any kind of environment variable where I can set up -mono as standard for all commands?

@CharliePoole glad to hear that you got this working!  I will close this issue.  We will update our documentation to mention this issue with 32bit Operating Systems.
I am not aware of any global flags or environment variables that you can use, but perhaps @devlead and @patriksvensson might be able to suggest something.


Consistent source code styling

The purpose of this PR is to format the codebase with a single style, so that it appears written with "one voice". In addition, it expands the linting rules to include more best practices.
There are several "relaxers" in the linting rules that can be removed once we move to a modular system that includes calling out explicit dependencies.
Changes:

Replaced hard tabs 2 spaces
Added appropriate whitespace around functions, blocks, object members, array indexes, etc
Replaced jshint with eslint and expanded rules
Fixed existing code to follow new linting rules

@brianbaker @markhealey can you take a look at this when you get a chance? It took about 12 hours to complete, so it might take a little bit to review.
Are there any rules to want to remove from linting? Any you want to add?
There's a refactor of the CDS example app included in this PR. I probably should have left it out of this branch, but I think they need a touch up and it seemed to fit with the formatting theme.

Minor thing, but might be a good time to alphabetize the tasks in the Gruntfile.js.  It is almost there but I think over time some of the tasks got out of order.


HA stops responding after a little while, started with upgrade to 0.99.0

Home Assistant release with the issue:
0.99.0 and 0.99.2
Last working Home Assistant release (if known):
0.98.5
Operating environment (Hass.io/Docker/Windows/etc.):
Docker installation
Description of problem:
HA stops responding to web UI or iOS client.  If I restart, things work fine for a little while.  After another hour or so it stops working again.
Additional information:
I've enabled debug logging, but I don't see anything in home-assistant.log that's relevant.

Any suggestions on how to debug this?

After upgrading to 0.99 (and possibly also after upgrading to 0.98), I observe home assistant spawning a large number of zombi processes of command "ssl_client". This eventually exhausts resources on my server and I need to hard reboot as I can no longer login (because process limited is reached so no shell can be spawned).

(note 3852 is PID of home assistant within a docker container)
Running "docker stats" confirm that one of my two home assistant instances is indeed using a lot of processes:

This container has been running for less than an hour so about 2 ssl_client processes are spawned per minute.
Please check by running similar commands over a period of time if you have a similar issue

Thanks for the info.  Unfortunately my docker HA isn't showing anything ssl_client processes.

I've tracked this down to a custom component.  Either HACS or something else.  I'm going to close this out.


Sidebery navbar panel icons are displayed as flipped unless "Show in one line" is turned on

Description
Panel icons on Sidebery navbar are displayed as flipped if the setting is "horizontal + 'Show navigation bar in one line' turned off" or "vertical". It doesn't happen with "horizontal + 'Show navigation bar in one line' turned on" settings.
What especially doesn't make sense to me is that the navbar layout shown in the preview screenshot on the README seems to be possible only when the "Show in one line" option is turned off, because one-lined navbar seems it doesn't really respect the elements list on the settings.
The screenshots below are taken with a slightly modified CSS config, but the issue occurs with the official config too.
Environment

Floorp 11.10.1 (Firefox 115.8.0)
Sidebery 5.1.1

Screenshots

(Obvious but) resolved by fixing this line:
https://github.com/KiKaraage/ArcWTF/blob/6eef0115ae9db1ec0965f6e899a3a41e73feff71/sidebery/sidebery-css#L12
to the following:
.NavigationBar .nav-item:not([data-class="panel"]) {

Thank you for the feedback! I hope the current CSS fixes it well


[Documentation] func CertFromPEM has a pemData as input . What is the pemData byte array ? Is it the certificate in the byte array ?

Documentation related to component

Please check all that apply

[ ] typo
[x] documentation doesn't exist
[ ] documentation needs clarification
[ ] error(s) in the example
[x] needs an example

Description of the issue

Hi Team,
I am trying to generate a token .
I have a private key( privatekey.pem), thumbprint and certificate file (cert.pem). I dont have a secret.
I was trying to understand and generate token for myself.
From what I understand I should use CertFromPEM then the returned values in the NewCredFromCert.
CertFromPEM has 2 inputs pemData []byte, password string. So I have converted my cert.pem from string to byte array and using privatekey.pem (copied the string code) as my password .
It gives me the error "no private key found" .
Please suggest what needs to be changed and provide an example.

Thanks. I was confused and later realized I have to combine the public and private keys which will act as the pemData. This can be closed.


Unable to have soft cover like newspaper

Hi all,
I would like to have soft covers like newspaper. Even if I add data-density="soft" to my divs, The covers are always hard.

BOOK TITLE

Page Header 1

   ...

THE END

Could you please tell me how to have soft covers?
Thank you.

It looks like setting the showCover property to true makes the lines 61-64 run in Collection/PageCollection.ts so it will always make the cover hard. You might fork/download the code and modify it to suit your needs.

I achieved this by having blank pngs as first and last pages, but it would be nice to have the option for data-density="soft" support on cover pages.

Same requirement in my project. It would be really nice to have a flexibility of setting covers' density when showCover is true.

Same issue here. Soft covers would be really nice.

I achieved this by having blank pngs as first and last pages, but it would be nice to have the option for data-density="soft" support on cover pages.

Could you explain exactly how did you fix this?

I achieved this by having blank pngs as first and last pages, but it would be nice to have the option for data-density="soft" support on cover pages.

Could you explain exactly how did you fix this?

Set showCover: false, use data-density="soft" for every page and set up dummy page containers for the first and last page containing a placeholder image.

I just omitted showCover: true and for the first page image used a transparent png, so the second image acts as a soft cover. Eventually I changed the first image to a png with instructions on how to use the magazine feature, so I could give hints on keyboard support I had added. You can see it in action here:
https://bmxmuseum.com/reference/6490
Here is the code I am using to init (with some preload logic in case it's useful for someone else):
var pageFlip = new St.PageFlip(
	flipbook,
		{
			width: 400,
			height: 550,
			drawShadow: true,
			flippingTime: 300,
			size: "stretch"
			// showCover: true,
		}
);
pageFlip.loadFromHTML(flipbook.querySelectorAll('.page'));
pageFlip.on('flip', (e) => {
	// Preload next few pages
	flipbook.querySelectorAll('.page:nth-child('+(e.data + 3)+') img, .page:nth-child('+(e.data + 4)+') img').forEach(function(el) {
		el.setAttribute('loading', 'eager');
	});
});

        // This is the dummy page

    [...]

I achieved this by having blank pngs as first and last pages, but it would be nice to have the option for data-density="soft" support on cover pages.

Could you explain exactly how did you fix this?

Set showCover: false, use data-density="soft" for every page and set up dummy page containers for the first and last page containing a placeholder image.

dummy pages?

'Dummy page' meaning placeholder page. In order to make a "soft" cover on the right hand side work, it actually needs to be the second page in the source code. The first page serves as a simple placeholder in that case...

        // This is the dummy page

    [...]

How did you work this out @foxacid ?

Can you create a demo or post your code?
I suspect you just added an empty container. In order for this to work you need to add a transparent image, too.

I made a fork adding the ability to have soft pages for front/back. So, for anyone happy with the state of this project, but really needs that functionality, you're welcome to use my fork at https://github.com/maxfahl/StPageFlip. Since this repo seems to be dead, I just forked it and modified things to my hearts content. Be aware that I will not do maintain it actively. If anyone wants to contribute, just let me know.
Changes made in the fork:

Allow soft pages for cover and back, which was not possible in original version.
Removed everything related to image pages, only keeping the HTML part.


Use Events

Hi,
How can i use your package with events, i would like use in a module Event and Listener.
Thanks you
Regards
Damien

Hello,
You can use events like any laravel application.
Create your event & listener, and register them either in your module service provider, or in a dedicated events service provider.

Ok, thanks you for your answer.

I've created a new documentation page explaining exactly how to proceed:
https://nwidart.com/laravel-modules/v2/advanced-tools/registering-module-events
👍


Added hello world app using cloudevents in node js

hi @rsriniva ,
I have not added many changes from the base app. Please review when free.
Thanks.

@somishraredhat is this ok to merge mate?

Sry Aykut, I forgot about this. Just merged now.


setup docs

Setup documentation template using sphinx.
For issue #133

Codecov Report

Merging #134 into 2.0 will not change coverage.
The diff coverage is n/a.

@@         Coverage Diff         @@
##            2.0   #134   +/-   ##
===================================
  Coverage   100%   100%
===================================
  Files         1      1
  Lines        66     66
===================================
  Hits         66     66

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 221333a...e11c4b7. Read the comment docs.


Graceful Edge upgrade over the air (OTA)

[ ] Remote update when new version is available
[ ] Three update profiles - recommended (user doesn't need to see and can ignore), urgent (user must be notified), critical (user must take action: agree or reject).
[ ] Manual vs Automated update option. User can choose to enable automated updates.

What are the stable options for Raspberry Pi & docker OTA updates?
Resources:
https://hub.mender.io/t/how-to-update-a-docker-container-with-a-raspberry-pi-3/1106
https://www.balena.io/docs/reference/OS/updates/update-process/

We introduced docker-compose.yaml with watchtower in our quick start guide about a month ago. It has worked well for users and has been able to seamlessly update ambianic-edge when new versions are released.
Closing this issue as the watchtower solution has worked well and it is no longer a pain point reported by end users.


Config save to S3

Hi,
Is it possible or is there a plan for saving the config&dashboards (current state) of superset on S3, like Zeppelin can: https://zeppelin.apache.org/docs/0.8.0/setup/storage/storage.html#notebook-storage-in-s3?
My motivation is keeping the image on kubernetes as basic as possible and keep state elsewhere on cloud.
Superset version
0.27.0
Expected results
Configs are saved on s3.
Actual results
Configs can be exported as YAML.
Steps to reproduce
N/A

Backup your metadata database and put the backup file on s3?

Is there a documentation/recommendation about the best practice of exporting and importing config/dashboards/etc (current state actually)?
I can see some scripts in the images such as: dashboard_import_export_util.py, import_util.py, superset_config.py but it would be great if someone can guide me through.
Thanks a lot...


Way to compress message selectively

When use permessage deflate, compresion works better for large message than small message. Could Beast support a message size threshold parameter to enable compression for large messages only?

I agree, there should be a way to turn compression on or off for each message.

I think the "on off for each message" that @vinniefalco describes covers that, but here is my use case:
Most the peers are in the same subnet over a gigabit connection, and compression slows down the transfer noticeably for them. There are a few remote hosts where I would like to enable compression. I would like to be able to turn compression on/off depending in the peer address.
A message size threshold as initially suggested is not flexible enough.

I would like to be able to turn compression on/off depending in the peer address.
You can do that now, fill out a permessage_deflate and call stream::set_option with it.

Thanks. That looks straightforward. I could have figured that out by thinking before writing...

Is this still unresolved?

I believe we did add the per-message compression flag, can we please check?

The current per-message compression flag is a stream-level setting.
For better performance, the feature requested is to ask to compress message selectively, e.g. to compress message with size > 500 bytes only.


Adding Scrubs to the MediDrobe

About the PR

This PR adds the blue, green, and purple scrubs to the MediDrobe.
Media

[X] I have added screenshots/videos to this PR showcasing its changes ingame, or this PR does not require an ingame showcase

Changelog

:cl:

Management has realized they forgot to supply station medical staff with scrubs. Scrubs can now be obtained from Medidrobe vendors

Finally. Scrubs enjoyers stay winnin'.


Fix syntax on SELinux installation for clearlinux

Hello,
I think the commit message is clear but with Ansible version 2.9.0 (I don't know if it fails with previous versions) it fails :smile:
Do you need more details ?

thanks! :+1:


org.jetbrains.skiko.RenderException: Cannot init graphic context / Insufficient Memory

Describe the bug
On running the gradle task run, an empty window is displayed, and some error messages are logged.
Affected platforms

Desktop (macOS)

Versions

Kotlin version*: 1.8.0
Compose Multiplatform version*: 1.6.2
OS version(s)* (required for Desktop and iOS issues): 14.4.1
OS architecture (x86 or arm64): arm64
Device (model or simulator for iOS issues):
JDK (for desktop issues): 17

To Reproduce
Steps and/or the code snippet to reproduce the behavior:

In IntelliJ IDEA, run the gradle task run. Optionally, add the java VM option -Xmx4096m to the run configuration.
Wait and observe the output.

Expected behavior
A window with application UI should be displayed.
Screenshots

Additional context
Log:
[SKIKO] warn: Failed to create Skia Metal context!
java.lang.RuntimeException: Can't wrap nullptr
[SKIKO] warn: Exception in draw scope
org.jetbrains.skiko.RenderException: Cannot init graphic context
2024-05-06 12:47:52.292 java[40676:28196535] WARNING: Secure coding is not enabled for restorable state! Enable secure coding by implementing NSApplicationDelegate.applicationSupportsSecureRestorableState: and returning YES.
2024-05-06 12:47:52.558 java[40676:28196546] GLDRendererMetal command buffer completion error: Error Domain=MTLCommandBufferErrorDomain Code=8 "Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)" UserInfo={NSLocalizedDescription=Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory), NSUnderlyingError=0x600003410960 {Error Domain=IOGPUCommandQueueErrorDomain Code=8 "(null)"}}
2024-05-06 12:47:52.591 java[40676:28196615] GLDRendererMetal command buffer completion error: Error Domain=MTLCommandBufferErrorDomain Code=8 "Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)" UserInfo={NSLocalizedDescription=Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory), NSUnderlyingError=0x60000340f360 {Error Domain=IOGPUCommandQueueErrorDomain Code=8 "(null)"}}
2024-05-06 12:47:52.660 java[40676:28196546] GLDRendererMetal command buffer completion error: Error Domain=MTLCommandBufferErrorDomain Code=8 "Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)" UserInfo={NSLocalizedDescription=Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory), NSUnderlyingError=0x60000340fc90 {Error Domain=IOGPUCommandQueueErrorDomain Code=8 "(null)"}}
2024-05-06 12:47:52.661 java[40676:28196546] GLDRendererMetal command buffer completion error: Error Domain=MTLCommandBufferErrorDomain Code=8 "Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)" UserInfo={NSLocalizedDescription=Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory), NSUnderlyingError=0x60000340f480 {Error Domain=IOGPUCommandQueueErrorDomain Code=8 "(null)"}}
2024-05-06 12:47:52.662 java[40676:28196546] GLDRendererMetal command buffer completion error: Error Domain=MTLCommandBufferErrorDomain Code=8 "Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)" UserInfo={NSLocalizedDescription=Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory), NSUnderlyingError=0x600003421650 {Error Domain=IOGPUCommandQueueErrorDomain Code=8 "(null)"}}
2024-05-06 12:47:52.663 java[40676:28196546] GLDRendererMetal command buffer completion error: Error Domain=MTLCommandBufferErrorDomain Code=8 "Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)" UserInfo={NSLocalizedDescription=Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory), NSUnderlyingError=0x60000342e0a0 {Error Domain=IOGPUCommandQueueErrorDomain Code=8 "(null)"}}
2024-05-06 12:47:57.515 java[40676:28196546] GLDRendererMetal command buffer completion error: Error Domain=MTLCommandBufferErrorDomain Code=8 "Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)" UserInfo={NSLocalizedDescription=Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory), NSUnderlyingError=0x600003418690 {Error Domain=IOGPUCommandQueueErrorDomain Code=8 "(null)"}}
2024-05-06 12:47:57.547 java[40676:28196615] GLDRendererMetal command buffer completion error: Error Domain=MTLCommandBufferErrorDomain Code=8 "Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory)" UserInfo={NSLocalizedDescription=Insufficient Memory (00000008:kIOGPUCommandBufferCallbackErrorOutOfMemory), NSUnderlyingError=0x600003418600 {Error Domain=IOGPUCommandQueueErrorDomain Code=8 "(null)"}}

When "Insufficient Memory" appeared, I checked "Activity Monitor" in macOS and observed both IntelliJ IDEA and MainKt (the application) did not exceed the customized JVM memory limit. Their sum did not exceed the customized JVM memory limit set on IntelliJ IDEA as well.
"Insufficient Memory" does not always appear (e.g. the first screenshot).
This issue does not consistently appear all the time. I just want to know how could I workaround this when it appears, except restarting my computer.

Hey, what is the device you are running this on?

Hi @elijah-semyonov
I am running on a M2 MacBook Pro.
I can only provide the project I am working on to you, as I am not sure what is going on. The issue is not frequently reproducible. But once I hit it, I would keep hitting for a while. The file Main.kt may have what you need. Please let me know if I use Compose Desktop in a wrong way.
https://github.com/sunny-chung/hello-http/blob/f2dac5216bf8b96c28b57e43a1d6ac03870b7470/src/jvmMain/kotlin/com/sunnychung/application/multiplatform/hellohttp/Main.kt

Are there some specific circumstances, that you could note, that lead to this?
Do you have some heavy graphics application running along? I've played with an app for several minutes but didn't manage to reproduce the behavior.

It apparently only reproducible with IntelliJ IDEA and after I upgraded Compose Multiplatform from 1.6.0-rc02 to 1.6.2. It only happens while launching the application.
Near the end of Main.kt, there is a dummy Window. It will be dismissed after some background loading and after 500ms. Perhaps that relates to the issue you mentioned.

I have just reproduced with:

Run the gradle task run with IntelliJ IDEA.
Without closing the application, run a test case in the "ux-and-transport-test" module of the project using the green triangle button in IntelliJ IDEA. The window created in the test case is an empty window.
Stop all the tasks.
Rerun the test case. The window created is normal now.
Repeat step 1 - 4 for several times.
Without running the application, run the gradle task :ux-and-transport-test:check, the window created is an empty window.
Stop it and then run the gradle task run. Now the application also shows an empty window, with the reported error messages output in the console.
Rerun the gradle task run, the issue still exists.
Stop it, run the gradle task clean and then run, the issue still exists.

During the time, I have no other heavy graphic-demanding applications running.

Please check the following ticket on YouTrack for follow-ups to this issue. GitHub issues will be closed in the coming weeks.


Issue with libcublas-<IP_ADDRESS>-1-r0

Just pulled new updates and started trying to build with dunfell-l4t-32.4.3
Also building with GCC-8 and contrib/config/gstreamer-1.14.conf
Seeing the following crop up:
NOTE: Executing Tasks
ERROR: libcublas-<IP_ADDRESS>-1-r0 do_compile: Execution of '/work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/temp/run.do_compile.1292767' failed with exit code 1:
mv: cannot stat '/work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/aarch64-linux-gnu/lib*': No such file or directory
WARNING: /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/temp/run.do_compile.1292767:1 exit 1 from 'mv /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/aarch64-linux-gnu/lib* /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/'

ERROR: Logfile of failure stored in: /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/temp/log.do_compile.1292767
Log data follows:
| DEBUG: Executing shell function do_compile
| mv: cannot stat '/work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/aarch64-linux-gnu/lib*': No such file or directory
| WARNING: /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/temp/run.do_compile.1292767:1 exit 1 from 'mv /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/aarch64-linux-gnu/lib* /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/'
| ERROR: Execution of '/work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/temp/run.do_compile.1292767' failed with exit code 1:
| mv: cannot stat '/work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/aarch64-linux-gnu/lib*': No such file or directory
| WARNING: /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/temp/run.do_compile.1292767:1 exit 1 from 'mv /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/aarch64-linux-gnu/lib* /work/smarter-yocto/poky/build/tmp/work/armv8a_tegra-poky-linux/libcublas/<IP_ADDRESS>-1-r0/libcublas-<IP_ADDRESS>-1/usr/lib/'
|
ERROR: Task (/work/smarter-yocto/poky/build/conf/../../../meta-tegra/recipes-devtools/cuda/libcublas_<IP_ADDRESS>-1.bb:do_compile) failed with exit code '1'
NOTE: Tasks Summary: Attempted 3205 tasks of which 2542 didn't need to be rerun and 1 failed.

Summary: 1 task failed:
  /work/smarter-yocto/poky/build/conf/../../../meta-tegra/recipes-devtools/cuda/libcublas_<IP_ADDRESS>-1.bb:do_compile

disregard, it appears to have been something corrupted into my build state.  Blowing everything away and starting from scratch solved the problem.


Account ID Hashing

This PR adds one-way hashing for account IDs as to prevent these from being stored in the analytics DB.

Thanks @idea404 I've ported your work into this PR


fix multiple worker problem

Addresses issue #203. Note that the creator of uvloop advises against conforming to wsgi for asyncio web servers. See https://github.com/python/asyncio/issues/481#issuecomment-271098847

This change also removes some unnecessary code and incorrect logging messages.

@seemethere I'm still working on the tests -- when you use serve_multiple directly it's causing an issue when rebinding to the port. Give me a couple hours before merging please, as I'll be adding a test case for this and fixing the issue with using serve_multiple directly.

@r0fls Will do, thanks for all the good work!

Well, that didn't go as planned... the utils for testing don't appear to be working with this, which will require investigation. So, I've disabled the multiprocessing tests for now. Since MP wasn't really working before, this is still a significant improvement. The below example shows this is working:
import asyncio
import uvloop
from sanic.response import text

import os
from sanic import Sanic

app = Sanic(__name__)

@app.route('/')
async def hello(request):
    print(os.getpid())
    return text("OK")

if __name__ == '__main__':
    app.run(workers=4)

Then you can see multiple processes responding:
wrk -t50 -c100 http://localhost:8000
3704
3703
3704
3701
3703
3701
3704
3702

You should also see improvements from wrk. I clocked in at this with 4 workers:
$ wrk -t50 -c100 http://localhost:8000
Running 10s test @ http://localhost:8000
  50 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     2.70ms    1.01ms  10.75ms   72.88%
    Req/Sec   746.49    181.38     6.51k    82.84%
  372222 requests in 10.10s, 45.79MB read
Requests/sec:  36856.63
Transfer/sec:      4.53MB

vs this with 1
$ wrk -t50 -c100 http://localhost:8000
Running 10s test @ http://localhost:8000
  50 threads and 100 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     5.18ms    1.81ms  15.60ms   60.86%
    Req/Sec   388.54     75.63   808.00     67.54%
  193713 requests in 10.03s, 23.83MB read
Requests/sec:  19317.26
Transfer/sec:      2.38MB


add an optimization for immutable structs without type arguments

Codecov Report

Merging #1 into master will decrease coverage by 4.52%.
The diff coverage is 78.78%.

@@            Coverage Diff             @@
##           master       #1      +/-   ##
==========================================
- Coverage   94.52%   90.00%   -4.53%
==========================================
  Files           1        1
  Lines          73      100      +27
==========================================
+ Hits           69       90      +21
- Misses          4       10       +6

Impacted Files
Coverage Δ

src/LazilyInitializedFields.jl
90.00% <78.78%> (-4.53%)
:arrow_down:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d74d92...30cdb66. Read the comment docs.

So this seems to improve the performance when using non-lazy field and slightly decrease performance when using the lazy field. Interesting.

Note that the proper way of doing this is to have support in julia for declaring individual fields as immutable, which is https://github.com/JuliaLang/julia/issues/9448#issuecomment-68077080.
I heard that https://github.com/JuliaLang/julia/pull/37847 gets us a bit on the way to that.

Another question, is it worth adding @ifdebug or is @static if isdebug() good enough?

Note that the proper way of doing this is to have support in julia for declaring individual fields as immutable, which is JuliaLang/julia#9448 (comment).
I heard that JuliaLang/julia#37847 gets us a bit on the way to that.

@KristofferC Now that Julia master allows annotating mutable struct fields as const, will that be useful for the implementation of this PR?


fix: build ffi script

[!WARNING]
This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

#320  👈
#319
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @AngelOnFira and the rest of your teammates on  Graphite


[RFC] split rustpython_stdlib from rustpython_vm

Summary

Split rustpython_stdlib from rustpython_vm which contains non-necessary rust modules for vm itself.
Detailed Explanation

Advantages:

We will have pressure to define public APIs by dogfooding, because rustpython_stdlib cannot access rustpython_vm private.
Compiling will be faster by deviding crates. At least fot rustpython_stdlib.
stdlib can be a feature. Without it, the binary size will be smaller than now.

Drawbacks, Rationale, and Alternatives

Unresolved Questions

Question on this since #3132 got me thinking.
Doesn't vm require minimal support from the stdlib? sys seems like it defines things that are also used in vm/, importlib should be around for importing things and io and encodings also are required if I recall correctly.
I can see how a subset of the stdlib could be separated but not really seeing how all could be yanked and placed in a new crate with could be a nostdlib feature, a limited-stdlib maybe. This is just a quick though on it, though, what's your idea on this?

I think modules defined in sys.builtin_module_names must be included in vm. Other modules will be splited.
I also think the name vm::stdlib doesn't fit anymore.
Maybe vm::module include builtin modules (since it is defined in sys.builtin_module_names),
and they will be re-exported through rustpython_stdlib again

Something I've been thinking about - oh which it seems also @DimitrisJim - some things should be core to the interpreter, right? Like _signal, or _weakref, or _codecs - imo, anything that's required to start up rustpython (and to be honest, I thought it was a given that sys/builtins would stay in the vm). CPython makes this distinction via "built-in" modules vs dlls. _io is <module 'io' (built-in)>, but zlib is <module 'zlib' from '/usr/lib/python3.9/lib-dynload/zlib.cpython-39-x86_64-linux-gnu.so'>. Obviously we don't have extension modules, but I think it's more productive/useful to put those ancillary modules in a separate crate while keeping the actually core ones in rustpython-vm.

Oh, right - like sys.builtin_module_names. These all seem preeetty core to the interpreter, except for things like functools/itertools but those are still very heavily used in all kinds of python code, they aren't like zlib which is domain-specific.
('_abc', '_ast', '_codecs', '_collections', '_functools', '_imp', '_io',
 '_locale', '_operator', '_peg_parser', '_signal', '_sre', '_stat', '_string',
 '_symtable', '_thread', '_tracemalloc', '_warnings', '_weakref', 'atexit',
 'builtins', 'errno', 'faulthandler', 'gc', 'itertools', 'marshal', 'posix',
 'pwd', 'sys', 'time', 'xxsubtype')

Maybe the distinction we could have, if we kept these in rustpython-vm, is corelib instead of stdlib? Obviously a different set of priorities than Rust's core vs std, but a similar concept imo.

#3129 is now working version

I have another public stdlib api suggestion in #3165

This is finished by #3620


Crash in Scatter, ScatterLayout and ScatterPlane when clear() on empty canvas

Versions

Python: 2.7.12 at least
OS: Ubuntu 16.04
Kivy: 1.10 at least
Kivy installation method: From pip

Description
In Scatter, ScatterLayout and ScatterPlane get kivy crash wen executing a
canvas.clear()  or  canvas.before.clear() on empty canvas.
If however canvas has some lines or other shapes, then no-crash.
Crash error message is: IndexError: list index out of range
Code and Logs
Here is small script that demonstrates the error,
click the clear before canvas button, and the program should crash with index error:
from kivy.config import Config
Config.set('graphics','fullscreen','auto')

from kivy.uix.scatter import Scatter
from kivy.lang import Builder
from kivy.base import runTouchApp

from kivy.graphics import Rectangle

Builder.load_string('''
<RootLayout>:
    Button:
        text: 'clear before canvas'
        on_press: root.canvas.before.clear()
        pos: (200,0)
''')

class RootLayout(Scatter):
    def __init__(self, **kwargs):
        super(RootLayout, self).__init__(**kwargs)

runTouchApp(RootLayout())

https://github.com/kivy/kivy/blob/d9f551a2199576eee2f0a0f7806f7d32abafc773/kivy/data/style.kv#L101-L107
Scatter has some canvas instructions by default. So canvas.before.clear() is bad idea I think.

I mean the canvas actually is not empty. You removed canvas instructions which are used by Scatter.
That's why the error occured
from kivy.uix.scatter import Scatter
from kivy.lang import Builder
from kivy.base import runTouchApp

Builder.load_string('''
<RootLayout>:
    Button:
        text: 'clear before canvas'
        pos: (200,0)
        on_press:
            cbefore = root.canvas.before
            print('\\n'.join(str(inst) for inst in cbefore.children))
            cbefore.clear()
            print('------------------------')
            print('\\n'.join(str(inst) for inst in cbefore.children))
            print('------------------------')
''')

class RootLayout(Scatter):
    def __init__(self, **kwargs):
        super(RootLayout, self).__init__(**kwargs)

runTouchApp(RootLayout())

output
<kivy.graphics.context_instructions.PushMatrix object at 0x7f1444ee94f8>
<kivy.graphics.context_instructions.MatrixInstruction object at 0x7f1452f09378>
------------------------

------------------------
[INFO   ] [Base        ] Leaving application in progress...
 Traceback (most recent call last):
   ...

The clearing itself actually doesn't matter that much. The error is quite simple - you're indirectly using PopMatrix by not clearing all its calls in Scatter in canvas.before/canvas/canvas.after and there is nothing to pop:
from kivy.lang import Builder
from kivy.base import runTouchApp

runTouchApp(Builder.load_string('''
Widget:
    # just to show that there is no previous canvas
    canvas.before:
        PopMatrix:
'''))

Which falls down to a simple stack[-1] error while the stack list is empty. IMO that shouldn't be fixed, instead it should have the same behavior as e.g. glPopMatrix() and we might just raise an Exception:

It is an error to push a full matrix stack or to pop a matrix stack that contains only a single matrix. In either case, the error flag is set and no other change is made to GL state.

ref message https://github.com/kivy/kivy/commit/088eea6f6fbe7d8086109ae4b2aa716ccabbfb24, yet I haven't found any glPushMatrix()/glPopMatrix() calls, so I did a lil' investivation and apparently those functions aren't used because:

If glPushMatrix and glPopMatrix serve you well, you can stick with it. But if the GL matrix stack is not large enough for you (depend on implementation but it is 64 depth in general) then you can implement your own matrix stack with a stl data structure. For matrix operation, you can use glm. Then pass the top of the stack to the function glUniformMatrix4fv for your shader.

which falls down basically to the glPopMatrix() warning from the docs.

Some widgets have instructions in canvas.before and canvas.after. Using clear will remove the instructions defined by default. You should NOT do that in any widget except if you control all the instructions in the widget.
Other best approach is:

create your own canvas and put your own instructions in it. You can use clear as much as you want.
except complex widgets (like filechoose, etc), the content (canvas) it free from instructions. Put yours in it.
create a child and put your instructions in it.

Canvas before/after is mostly used for stencil and matrix instructions. RelativeLayout, Scatter, ScatterPlane, etc.. Therefore, if you clear one of them, you'll broke the widget. Embed your own instructions in a neutral sub widget.
This is not something we would solve, that's the cost of the Kivy flexibility :)


docs: rfc of inverted index for SST file

I hereby agree to the terms of the GreptimeDB CLA
What's changed and what's your intention?
rendered
Checklist

[ ]  I have written the necessary rustdoc comments.
[ ]  I have added the necessary unit tests and integration tests.

Refer to a related PR or issue link (optional)

Question: if we want to support other index types such as geospatial indexing, is the API design extensible?

We have designed the Index data to be housed within Puffin. As a container, Puffin opens up opportunities for us to iterate formats and develop other types of indexes. When there is a need for a geospatial index, we can design a new format in the same file and assign a new type such as geospatial-index-v1.

Cool, let's make it work.


Batch/transaction support for Subcollections

Hey! First of all, thanks for the great library.
I'm using fireorm in one of my projects now and I have an issue with batch writes and transactions in subcollections. I couldn't find any examples in your tests and I couldn't find any docs about it. Is batch writes and transactions in subcollections currently supported?
Here's my use case: I have a base object with a subcollection, and that subcollection has its own nested subcollection, so roughly A -> B -> C. I would like to create an A object, insert around 10k B objects in it, and insert around 500-1k C objects in each of those B objects. Obviously, doing the inserts one by one takes a long time and I would like to use the batch insert mechanism.

I haven't tested transactions within subcollections of subcollections 🤔. Theoretically it should work but is untested as of now.
Are you seeing any errors? Do you get the typing information for the inner subcollection? I know @joemckie had a workaround to make this work, not really sure what it was.

I don't have any errors because I'm stuck at the very first step. I just don't know how to get the batch repository for a subcollection 🙂
const batch = createBatch();
batch.getRepository(DUNNO_WHAT_TO_PUT);

I haven't implemented batches or transactions yet, but it looks like this functionality might not be completed yet.
https://github.com/wovalle/fireorm/blob/master/src/Batch/FirestoreBatch.ts#L24 getRepository should accept a collectionPath argument which is used to instantiate the BaseFirestoreBatchRepository.

@joemckie sooo... should I wait for the bugfix? if I understand correctly, we just need to add an optional parameter to that function and provide it to the BaseFirestoreBatchRepository constructor, i.e.
getRepository<T extends IEntity>(entity: Instantiable<T>, collectionPath?: string) {
  return new BaseFirestoreBatchRepository(this.batch, entity, collectionPath);
}

I guess that would be fine 😅

Hi,
I've just started using subcollections and a little bit confused about using batch write in subcollections.
First of all, I read this article - https://medium.com/@wovalle/introducing-fireorm-ba7734644684
Example of using collection described there:
class Album {
  id: string;
  name: string;
  year: number;
}

@Collection()
export class Band {
  id: string;
  name: string;
  formationYear: number;
  genres: Array<string>;
  @SubCollection(Album)
  albums?: ISubCollection<Album>;
}

Please, note, that current ISubCollection interface (that is alias of IRepository<T>) does not expose createBatch() method - https://github.com/wovalle/fireorm/blob/f66fe7e8c85a093d335f72a67044a73b6fa385ab/src/types.ts#L80
So, how to use batch write to subcollection?
I tried the next scenario:
import { ISubCollection, SubCollection, Collection, getRepository } from 'fireorm';

class Child {
  id: string;
  name: string;
}

@Collection()
class Parent {
  id: string;
  name: string;
  @SubCollection(Child)
  children?: ISubCollection<Child>;
}

(async _ => {
  try {
    const repo = getRepository(Parent);

    const created = await repo.create({
      name: 'Father'
    });

    const batch = created.children?.createBatch();

    [
      {name: 'First'},
      {name: 'Last'},
    ].forEach(child => {
      batch.create(child);
    });

    await batch.commit();

    console.log(await created.children.find()); // empty array
  } catch (e) {
    console.error(e);
  }
})();

Expected behaviour
1 document in "Parents" collection and 2 documents in "children" subcollection (of this "perent" doc) were created.
Actual behaviour
1 document in "Parents" collection and 2 documents in "children" collection (as root collection) were created.
I'm observing this in (at least) 0.17.0 and 0.14.0 (that is referenced in this thread as on that resolves the issue).
0.12.1 (that I have installed) works as expected


Browser Automation Framework is undetected

Hey, it seems like BAS ( https://bablosoft.com/shop/BrowserAutomationStudio ) is currently undetected,
from the looks of it they use a custom compiled chrome version.

It appears that the fingerprint feature is a paid service. There is a lot of cool stealthy software out there.
We mostly give attention to FOSS-privacy tools. I will take a look, but will probably not make modifications to show it can be detected.

Nice. I will definitely check it out.
I might introduce some concepts in the future in the analysis section.
Everything that does not blend-in with normal traffic on the site can be detected. As an example, generate 200 – 1000s requests in 24 hrs, and usually, it can be traced. If you would like to test this for fun, I can try to tag and name your browser in the analysis section.

No need to sign, and randomizing fingerprints is optional, but not needed.

Okay, try now. In the analysis section, you see “bad” for the hidden fingerprint, but let me know if I missed the target. If you open a different or clean browser, it should go away.

I'm just looking at last 2 hrs today, but can pick a different date.

Could probably hook console.log and check the console outputs. Depending on how BAS has tampered with the function you might be able to detect it with the given types.
(function () {
    const _console = console.log.bind(console);
    let log_history = [];
    console.log = function () {
        log_history.push(arguments);
        _console.apply(this, arguments);
    }
    console.log("Hello World");
    for (let i = 0; i < log_history.length; i++) {
        if (log_history[i][0] !== "Hello World") {
            return false;
        }
    }
    //restore the hook
    console.log = _console;
    return true;
})();

Do you have a copy of the BAS chromium binary? Want to try something.

Do you have a copy of the BAS chromium binary? Want to try something.

Yes i do. want me to upload it or anything?

Yeah that would be great

Doesn't appear to run, can you check if there are any run parameters specified on the binary? Tried to get it running through selenium and puppeteer but neither worked. I assume its the worker.exe file?
https://superuser.com/questions/415360/how-do-i-find-out-command-line-arguments-of-a-running-program

Doesn't appear to run, can you check if there are any run parameters specified on the binary? Tried to get it running through selenium and puppeteer but neither worked. I assume its the worker.exe file?
https://superuser.com/questions/415360/how-do-i-find-out-command-line-arguments-of-a-running-program

yes its worker.exe
full cli
"C:\Users\LTGT\AppData\Roaming\BrowserAutomationStudio\apps\25.4.1\Worker\chrome\worker.exe" --remote-debugging-port=16668 --unique-process-id=0pMhvMU2 --parent-process-id=25524 --no-proxy-server --no-sandbox --force-device-scale-factor=1 --noerrdialogs --window-position=0,0 --disable-smooth-scrolling --disable-auto-reload --disable-component-update --disable-features=GpuProcessHighPriorityWin,GpuUseDisplayThreadPriority --disable-gpu-program-cache --disable-gpu-shader-disk-cache --disk-cache-size=1 --enable-features=ViewportHeightClientHintHeader --lang=en --reduce-user-agent-minor-version --window-size=1040,688 --user-data-dir="C:\\Users\\LTGT\\AppData\\Roaming\\BrowserAutomationStudio\\apps\\25.4.1\\prof/kUjSYtdK" about:blank

It using JavaScript to fake fingerprint and it can detected and heavy


First draft of the docs for TF + Datasets

I might cc a few of the other TF people to take a look when this is closer to being finished, but it's still a draft for now.

Some links are still missing I think :)

This is probably quite close to being ready, so cc some TF people @gante @amyeroberts @merveenoyan just so they see it! No need for a full review, but if you have any comments or suggestions feel free.


Fix formatting issues in controller

This PR fixes issues raised in https://github.com/wso2/k8s-wso2am-operator/pull/26#discussion_r552390957 and https://github.com/wso2/k8s-wso2am-operator/pull/22#discussion_r552392672

@pubudu538 Shall we review and merge this PR?

@pubudu538 Shall we review and merge this PR?


Attribute cannot have the same name as its type

I am trying to make the class attribute having the same name as its type. The following code does not work and show the ConfigError
class A(BaseModel):
    pass

class B(BaseModel):
    A: A = None

ConfigError: unable to infer type for attribute "A"

However, if I make the attribute no default value then it works
class A(BaseModel):
    pass

class B(BaseModel):
    A: A

Is this behavior intended?

This is related to https://github.com/python/mypy/issues/1775
I ended up using an alias
so using your example
before
class A(BaseModel):
    pass

class B(BaseModel):
    A: A = None`

after
class A(BaseModel):
    pass

_A = A

class B(BaseModel):
    A: _A = None


Upgrade from 0.66.3 to .068.2 Ios App multiple target

Description
We have a target for the app and a target for an appClip
it was ok on 0.66.3 but after upgrading to 0.66.8 it opens the two bundlers at the same time when building the app target that makes conflict loading bundles on the app
How can we disable parallel building in such a case?
it was checked with manual and automatic order on the build scheme and results are the same
Version
0.68.2
Output of npx react-native info
System:
OS: macOS 12.0.1
CPU: (8) x64 Apple M1
Memory: 62.33 MB / 8.00 GB
Shell: 5.8 - /bin/zsh
Binaries:
Node: 14.17.0 - ~/.nvm/versions/node/v14.17.0/bin/node
Yarn: 1.22.10 - ~/.npm-global/bin/yarn
npm: 6.14.13 - ~/.nvm/versions/node/v14.17.0/bin/npm
Watchman: 2<IP_ADDRESS> - /opt/homebrew/bin/watchman
Managers:
CocoaPods: 1.11.3 - /opt/homebrew/bin/pod
SDKs:
iOS SDK:
Platforms: DriverKit 21.4, iOS 15.4, macOS 12.3, tvOS 15.4, watchOS 8.5
Android SDK: Not Found
IDEs:
Android Studio: 2021.2 AI-212.5712.43.2112.8609683
Xcode: 13.3.1/13E500a - /usr/bin/xcodebuild
Languages:
Java: 11.0.12 - /usr/bin/javac
npmPackages:
@react-native-community/cli: ^5.0.1 => 5.0.1
react: 17.0.2 => 17.0.2
react-native: 0.68.2 => 0.68.2
react-native-macos: Not Found
npmGlobalPackages:
react-native: Not Found
Steps to reproduce
add another target that runs react-native
upgrade to the latest version
Snack, code example, screenshot, or link to a repository

keep it up


List and display data source views in Poke

Description

This PR brings some fanciness to Poke, including:

A variety of generic, reusable components such as links and copy buttons.
A PokeLayout component that consistently wraps the navigation bar and applies necessary padding.
A conditional data table that loads data on demand, only when needed.

The primary change is the addition of conditional listing for the workspace's data source views. This prevents unnecessary slowdowns during page loading, as these views aren't always required. It also includes a display view for a data source, allowing users to view information and the tree displayed in the Assistant Builder.

Risk

Deploy Plan

PMRR.


[EngSys] shorten -no-test-proxy=true test option

The "=true" part is no longer needed after PR #28978

API change check
API changes are not detected in this pull request.

Looks that the failures are due to existing issues: known failure, flaky test, and formatting
Will override and open a follow-up PR for formatting.

Will override and open a follow-up PR for formatting.

Oops accidentally pushed formatting changes here...

/check-enforcer override


Check problem with signed transactions on Zeitgeist

https://github.com/polkadot-js/api/issues/5710

Error can be replicated by trying to transfer token from this dapp
https://test.staging.zeitgeist.pm/portfolio/5CFh4qpiB5PxsQvPEs6dWAhzgAVLHZa8tZKxeE9XsHBg4n9t?mainTab=Balances

No update from partner so we're closing this issue


Recreate directory tree

Hi,
This following command htmlclean src -o public only considers src/*.html files:
Would it possible to take care of src/**/*.html and recreate directory tree?

Hi mathpere.
Thank you for your post.
I agree to your proposal. It's useful.
But htmlclean must know where the root of directory tree is. New option might have to be added.
I think more about it.

Thanks for your feedback.
I don't understand why you would need new option.
For this given structure:
src
|--- index.html
|--- subfolder1
         |--- view1.html
         |--- view2.html

If I modify line 268 of htmlclean binary like this:
pair(paths[0] + (paths[0].substr(-1) !== '/' ? '/' : '') + '/**/*.html', output);

I create dir destination: mkdir www
And I execute this command: htmlclean src -o public
I get this structure:
www
|--- index.html
|--- view1.html
|--- view2.html

Could we use the exact same option by modifying the behavior when HTML files like view1.html, view2.html are not located at the root of src or am I wrong?

If you want only that behavior, htmlclean don't have to be modified. Just do htmlclean "src/**/*.html" -o www.
A subfolder1 directory is missing in www.
"recreate directory tree" you said. That is, subfolder1 must be made, right?

If the structure is:
src
|--- index.html
|--- subfolder1
|        |--- view1.html
|        |--- view2.html
|--- subfolder2
         |--- view1.html
         |--- view2.html

view1.html in www are mixed from subfolder1/view1.html and subfolder2/view1.html. view2.html too.

Oh I'm sorry, my comment was not very clear.
By executing htmlclean src -o www I get this structure
www
|--- index.html
|--- view1.html
|--- view2.html

But I would like
www
|--- index.html
|--- subfolder1
         |--- view1.html
         |--- view2.html

So I confirm: htmlclean should first automatically create subfolder1 folder (e.g. [mkdirp] (https://github.com/substack/node-mkdirp))

Yes, therefore, I think new option have to be added.
htmlclean must know where the root of directory tree is. htmlclean must be given the root via new option, when it can't presume the root.
In your sample, a subfolder1 directory missing in www. Then mkdir subfolder1 is needed. It's very simple. because it is clear that src is root.
But, actually, the paths that is given might be not root, or the plurality of those are mixed, maybe.
For example:

src/content1/*.html (I want to make content1 directory in www.)
src/@(content1|content2)/*.html
src{1,2}
project1/src (Don't make src directory in www.)

I rewrote. Please check new version 2.6.0.
The --root option was added.

If you specify one root directory (even if it includes pattern) to input (like your sample), you have to do nothing. Because the specific directory is considered as the root.
Note that if a directory is specified to the input, it is considered as directory/**/*.html. Not directory/*.html.

And, I close this issue. If there is problem, please reopen it.

Not yet sure if this is an issue but when I execute this command:  htmlclean src  -o www
I get this following error Error: ENOENT, no such file or directory ''

Thank you for test.
Please check again in verbose mode.
htmlclean src -o www -v

And, permission of www is ok?

Sorry, I mistook...
I fixed it.
Please try v2.6.1.

I tried version 2.6.2, it works like a charm, thank you!

Thank you for report :grinning:


Remove gitleaks-action

Resolves #30

Looks like the trufflehog failure is just because I deleted the gitleaks config file, which contains a string that trufflehog considers a secret.

I've reviewed the failing secrets analysis check and concluded it's a false positive, trufflehog is flagging text in the deleted gitleaks.toml file. Safe to ignore.


Offer audio ABR

hls.js does not feature adaptive audio track selection: https://github.com/video-dev/hls.js/issues/1152
On LEVEL_LOADED, if level's audio GROUP-ID differs from current GROUP-ID, set audioTrack to a track of the same group, if possible with the same NAME (NAME should always be present).

Add hlsjs config option audioABR, defaulting to true.

ABR in this case will also mean: the audio track will change upon manual video quality selection if appropriate.


Remove Python (CPython) compilation

Now that the cpython buildpack uses the new dependency workflow to compile cpython, we can remove cpython compilation from dep-server.
I chose to leave the cpython code references alone, for several reasons:

Historical: "Just in case" we need to reference them again, it's easiest just to come and look without worrying about commit history.
Convenience: I wasn't sure what I could change without side effects in other compilation processes. Besides, it looked hard to do.

Also see #232

BLOCKED on #235

@joshuatcasey Could you rebase the PR with main?

Unblocked now that #235  is merged

@brayanhenao done


Create a filter/sample that shows how to configure form size limits

We want an example of a filter that shows how to configure the limits used to read the form (by replacing FormFeature). This should be an IAuthorizationFilter configured to run before antiforgery.
See aspnet/Mvc#4533 for more information

86147eda46a476b8acc409df0eaa9b0eca730e94

Has this sample been published anywhere??

@M-Curtis it's located here - https://github.com/aspnet/Entropy/blob/dev/samples/Mvc.FileUpload/Controllers/RequestFormLimitsController.cs


fix: flickering caused by IK when moving vertically with platforms

What does this PR change?
Fixed https://github.com/decentraland/unity-explorer/issues/1029
How to test the changes?

/world goerli
Go to the scene with the moving platforms and stand on the vertically moving one

Our Code Review Standards
https://github.com/decentraland/unity-renderer/blob/master/docs/code-review-standards.md

I still see the fee going into the platform in the video


Python Desktop app calling Linux files

The desktop app (https://github.com/clariusdev/cast/blob/master/desktop/src/python/pysidecaster.py)  contains code that tries to load linux libraries into windows python app causing errors.
libcast = ctypes.CDLL('./libcast.so', ctypes.RTLD_GLOBAL) # load the libcast.so shared library
pyclariuscast = ctypes.cdll.LoadLibrary('./pyclariuscast.so') # load the pyclariuscast.so shared library
Removing this code and using the most up to date window's binaries (cast.dll and cast.lib) does not successfully make a connection. This code used to work before the latest update.

as we distribute pyclariuscast with specific python versions, ensure you remove the version from the .so name (ie. remove 310 from pyclariuscast310.so)


Utilize pod process namespace sharing instead of docker executor

Is this a BUG REPORT or FEATURE REQUEST?: FEATURE REQUEST
What happened:
Currently, artifact saving is performed through the docker executor, which implements artifact copying via a docker cp command to copy out artifacts from the main container. The problem with this approach is it requires mounting of docker.sock of the host, which is insecure, and unacceptable in some secure environments.
In K8s, there is an (alpha) feature to share the process namespace and the filesystem between containers in a pod:
https://kubernetes.io/docs/tasks/configure-pod-container/share-process-namespace/
Instead of utilizing the docker executor, we could simply create the pod spec with shareProcessNamespace: true and access the filesystem of the main container to copy files directly. Similarly, the actual waiting and killing of the process, would only need to be done via normal kill command from the argoexec sidecar, as opposed to a docker kill.
Using process namespace sharing provides us an ideal solution which addresses a lot of the security, and scalability concerns with our current docker, kubelet, or k8s API server approaches.
NOTE that this is an alpha feature and needs an feature gate (--feature-gates=PodShareProcessNamespace=true) configured for it to be enabled.

/cc @edlee2121 @JulienBalestra @gaganapplatix
I don't think we need to spend any effort around a pure K8s API server executor implementation, since I think this should be the future of how we do artifact management, process waiting, process killing. Longer term, I think the docker/kubelet executors will eventually go away if the argoexec sidecar can already access the main container's filesystem easily through process namespace sharing.

For copying artifacts, we would share process namespaces and access files via /proc/pid/root?

That’s right

I have quickly tested process namespace sharing (now beta feature enabled by default) on k8s v1.12 and is working as expected to access other container's filesystem.

Could you let us know, any further development on finalizing the approach from the above list?

Coming to this from kubeflow pipeline community.
https://github.com/kubeflow/pipelines/issues/678
We have phased out Docker support in IBM Kubernetes Service, and are now relying on containerd, This is one blocking issue currently impacting the adotption for us. Any further updates on thsi?

Seems like this might be the problem:
https://github.com/argoproj/argo/blob/23d733bae386db44ec80639daf91b29dbf86b335/workflow/controller/workflowpod.go#L71-L79
Users shouldn't be required to mount hostpath.

Perhaps you can pass in an option into your Custom Resource to disable this as some users might just be using argo just for running workflows and might not require copying files.

Perhaps you can pass in an option into your Custom Resource to disable this as some users might just be using argo just for running workflows and might not require copying files. One option could be to have the code within the container running the job connect to s3 or some cloud storage to store these files. or even minio could be a drop in option.

Do you know how this option can be passed or is it existing at all ?

Fixed.
K8s, Kubelete, and PNS executors will no longer mount docker.sock. The docker executor still needs docker.sock access even if not copying out artifacts, in order to perform a docker wait for waiting for the container to complete.

Thanks @jessesuen - can we now use either K8s and PNS executors, and get same functional behaviour from Argo, if not scalability?

can we now use either K8s and PNS executors, and get same functional behaviour from Argo, if not scalability?

The main difference is the K8s executor puts some additional polling load on k8s API server for waiting for container completion, whereas PNS is polling the local linux kernel (procfs) for container completion. Also PNS requires K8s 1.12 for PNS support (without the feature gate).
I'm copying my pro/con list from the other issue.

Docker:

+ supports all workflow examples
+ most reliable and well tested
+ very scalable. communicates to docker daemon for heavy lifting
- least secure. requires docker.sock of host to be mounted (often rejected by OPA)

kubelet

+ secure. cannot escape privileges of pod's service account
+ medium scalability - log retrieval and container polling is done against kubelet
- additional kubelet configuration may be required
- can only save params/artifacts in volumes (e.g. emptyDir), and not the base image layer (e.g. /tmp)

K8s API

+ secure. cannot escape privileges of pod's service account
+ no extra configuration
- least scalable - log retrieval and container polling is done against k8s API server
- can only save params/artifacts in volumes (e.g. emptyDir), and not the base image layer (e.g. /tmp)

PNS

+ secure. cannot escape privileges of service account
+ artifact collection can be collected from base image layer
+ scalable - process polling is done over procfs and not kubelet/k8s API
- processes will no longer run with pid 1
- artifact collection from base image may fail for containers which complete too fast
- cannot capture artifact directories from base image layer which has a volume mounted under it
- immature


WTD Updated verbiage to be gender inclusive

@pkbilkhu : Thanks for your contribution! The author, @davidmu1, has been notified to review your proposed change.


[SPARK-36784][SHUFFLE][WIP] Handle DNS issues on executor to prevent shuffle nodes from getting added to exclude list

What changes were proposed in this pull request?
When a DNS issue happens on the executor node, shuffle nodes would get added to the exclude list due to FetchFailed exception. The change here is to have a configuration host value to test DNS resolution against before marking it as a FetchFailed Exception.
Why are the changes needed?
This would prevent shuffle nodes from getting added to the exclude list due to DNS issues
Does this PR introduce any user-facing change?
No
How was this patch tested?
Added Unit Tests

Can one of the admins verify this patch?

I am not sure if we can generalize this expectation - UnknownHostException at a node does not imply a local DNS resolution issue alone.
For example, if a node is decommissioned/removed - fetch failures could result due to UnknownHostException as the host is no longer resolvable. If we do not trigger a re-execution of the parent stage, tasks in the current stage will keep failing and result in application failure.

@mridulm Thanks for reviewing it, Mridul. As you mentioned, this may not be the most reliable solution. I will keep this as a WIP and look into ways to resolve DNS without using the JVM or OS cache.

if a node is decommissioned/removed - fetch failures could result due to UnknownHostException as the remote host is no longer resolvable

Agreed, but the proposed changes will accommodate for this situation. If an UnknownHostException is seen, we don't immediately assume that there are DNS issues. We first check if some other "known good" hostname is resolvable, as defined by spark.network.dnsHealthCheck.host, and only if that host isn't resolvable, we assume there is a local DNS issue. If no such host is configured, we fall back to the current behavior, which is to assume that UnknownHostException indicates an issue with the remote host. See Executor#isDNSResolvableIfConfigured for more details.
This isn't perfect, but I think it acts as a very practical heuristic that should cover the majority of cases.
cc @venkata91 as well


v2 MRT exports all @tabler/react-icons during dev mode

mantine-react-table version
2.0.0-beta.6
react & react-dom versions
18.3.0
Describe the bug and the steps to reproduce it
When working with v2 MRT, the Network tab in devtools is having performance issues because for each tabler icons we get a chunk request thus producing over 5700 requests.

FYI I'm not using any tabler icons in that repo.
Minimal, Reproducible Example - (Optional, but Recommended)

Copy minimal example from https://github.com/sbsrnt/wire.
Open network tab and pay attention to the amount of requests that are happening (5.5k+, for each icon chunk).
Go to https://github.com/sbsrnt/wire/blob/master/src/layout/Content/Content.tsx and comment out <LibrariesList /> (or in LibrariesList component, comment out any code related to MRT)
Notice that there are no chunk requests for tabler icons

Screenshots or Videos (Optional)
No response
Do you intend to try to help solve this bug with your own PR?
No, because I do not have time to dig into it
Terms

[X] I understand that if my bug cannot be reliably reproduced in a debuggable environment, it will probably not be fixed and this issue may even be closed.

This is just during dev mode without any tree shaking though, right?

This is just during dev mode without any tree shaking though, right?

Yeah seems to be only in dev mode - https://sbsrnt.github.io/wire/
I will update the title

Anyway to solve this issue? It takes 10-15 seconds to load simple table when dev tools is open. I always use dev tools open so it's really making it harder to use it. It affects development experience a lot. I've tried to merge chunks in vite but couldn't find a way to do that in development mode. I really like this library but because of this issue I cannot use it.

After hours of debugging, I found that downgrade @tabler/icons-react ^3.19.0 => 3.11.0 fix this issue for me.

Yeah it's an issue with the icon library - https://github.com/tabler/tabler-icons/issues/1233

Yeah, on further looking into that issue, it's not great. I guess a brute force solution would be to just copy the raw icon svgs into the MRT source code and remove tabler icons as a peer dep. Hopefully they just ship a fix though.

I will close this as it is an issue with Tabler and it's only in dev mode. Feel free to reopen if this changes.


Use of environment-scoped variables

Do you have guidance on how we should use variables and environments together? For example, let’s say one of my deployment steps is to run some database scripts, and in order to do that I need to pass a connection string into the YAML pipeline. Different environments will have different connection strings. What’s the recommended way to do this? As far as I can see, we can’t scope variables to environments currently.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 77d95db6-9983-7346-d0eb-4b7443e4e252
Version Independent ID: 0a22cccc-318d-592f-d1ab-09ec01d88087
Content: Environment - Azure Pipelines
Content Source: docs/pipelines/process/environments.md
Product: devops
Technology: devops-cicd
GitHub Login: @shashankbarsin
Microsoft Alias: shasb

Thanks for your question. It looks like you're working through an issue with your scenario or implementation, rather than an issue with the documentation. Here are a couple of options where you might consider asking your question:

Azure DevOps on Stack Overflow
Azure DevOps Support Bot

Hi @WilliamAntonRohm, I would argue that this is an issue with the documentation. Having environment-specific variables in releases is an extremely common thing to do, and there's nothing in the documentation to advise how to do this. It's also possible that this feature is coming soon and isn't there yet, but I think it's a reasonable question to ask about the documentation.

@johndowns -- thank you for replying; you make a good point.  I've re-opened this issue and notified the article's author.

Agreed. It would be quite helpful to have documentation on how to achieve this. Or guidance as to what the recommended approach should be.

variables are set at pipeline scope or at stage, job scope. In the case of environment, they are scoped to a specific deployment job. Hence variables at job scope will be equivalent to setting  variables at environment scope.
We do have plans to support linking variables/key vaults to an environment (no time lines). This would be an alternate option to handle environment scoped variables.

@clancy (and anyone else who is interested) - I did a little more investigation into this and have found an approach that hopefully is workable. It doesn't use environments at all.

Non-secret variables - you can define non-secret variables within the YAML files themselves. This can be done both at the level of the whole pipeline (which is equivalent to setting a release-wide variable in the classic model), and/or on individual stages (which is equivalent to setting a variable at the scope of a single release stage in the classic model).
Secret variables - the best option right now seems to be to use variable groups. You create and configure these in the Azure Pipelines UI. Secrets can be added into the group directly, or variable groups can also link to a Key Vault for storing secrets. Once the variable group is created, you can refer to it inside the YAML template. (Note that once you refer to a variable group you need to switch the syntax of all of your variable definitions to the key/value model of defining variables.)
Task groups - in the classic release model you may have used a task group to define the sequence of steps to follow to run a release, and then added that task group into each of the release stages. In YAML there is a nicer option, which is to use a YAML template, which essentially embeds that file inline. This means you could have a file that defines all of your release steps, and then instantiate it for each of your environments by including it as a stage.

Here's an example that shows all of the above:
Parent template:
trigger:
- master
pr: none

pool:
  vmImage: 'ubuntu-latest'

variables:
  MySharedVariable: xyz

stages:

- stage: build
  jobs:
  - job: run_build
    pool:
      vmImage: 'Ubuntu 16.04'
    steps:
    - checkout: self
    - script: echo Build

- stage: deploy_dev
  dependsOn: build
  jobs:
  - job: deploy_dev
    variables:
      - group: MySecrets-Dev
      - name: EnvironmentName
        value: Dev
    steps:
      - template: pipeline-deploy.yml

- stage: deploy_test
  dependsOn: deploy_dev
  jobs:
  - job: deploy_test
    variables:
      - group: MySecrets-Test
      - name: EnvironmentName
        value: Test
    steps:
      - template: pipeline-deploy.yml

Child template (pipeline-deploy.yml):
steps:
- script: echo Deploying to $(EnvironmentName) with secret $(ExternalApiKey)

And I have two variable groups set up:

MySecrets-Dev

ExternalApiKey = xxx (and configured as a secret)

MySecrets-Test

ExternalApiKey = yyy (and configured as a secret)