panicked at 'expected a package' in prost_build::Config::compile_protos

why require a package? Am I missing something?
The hello world example in the docs doesn't have one:
syntax = "proto3";

message SearchRequest {
  string query = 1;
  int32 page_number = 2;
  int32 result_per_page = 3;
}

https://github.com/danburkert/prost/blob/1944c27c3029d01ff216e7b126d846b6cf8c7d77/prost-build/src/message_graph.rs#L25

Yes, prost requires a package declaration.

Oh. I see that in the README now. In case it would help, I submitted a PR to document it in the API.

Turns out compiling a package-less https://github.com/hyperium/tonic app fails in a very unhelpful error (from an unwrapped tonic_build::compile_protos call):
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: NotPresent', src/libcore/result.rs:1165:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace.

How can we make this output a more helpful error message?
cc @hyperium @LucioFranco

I'm going to close this out, there have been two commits landed to master which improve the situation:

8df3a14e8c58d2b5dbf7ec7eb2ee8b22b412afe1 by @joelgallant improves the error message associated with a missing package specifier.
8c0c18031fadbb51be887514ff8b0d208517d613 by @danburkert changes the panic to an error, and adds the error condition to the compile_protos function doc.

Please feel free to re-open or file a new issue if there's something more that can be improved.


feat(podinfo): add secure port for end-to-end tls

add secure-port argument to podinfo
add cert-path argument to podinfo
add http server for secure port
normalise http/https server start
add manifests for example secure-port implementation (relies on cert-manager)
add script to create a cluster via kind to test podinfo locally (with support for self-signed certs)

Closes: #105

I believe the build is failing due to this:
https://github.blog/changelog/2020-10-01-github-actions-deprecating-set-env-and-add-path-commands/
I'll take a look at solving this tomorrow.

You need to bump<EMAIL_ADDRESS>

You need to change kind to<EMAIL_ADDRESS>
Fixed. I also fixed the add-paths in the release-notes action.


Enter key handler on Select elements

Vue.js version
2.1.8
Reproduction Link
https://jsfiddle.net/nickfederighi/o8fwf0gh/5/
Steps to reproduce
Assign an event handler to a select element to override the default behavior of opening the element. See below:
<select @keyup.enter.prevent="keyHandler">
What is Expected?
I would expect the keyHandler method to fire when the select element has focus and the 'enter' key is pressed
What is actually happening?
Enter key presses on select elements are not handled by the listener. All other keys appear to work fine. See my jsfiddle for an easy reproduction. Set focus on the select element and press any keyboard key and the handler will be called. Pressing the enter key has no impact.
The default behavior on browsers is for the enter key to expand the select list, but I would like it to submit the form data to be consistent with other inputs on the form that have the same behavior. Adding the 'prevent' modifier to the  inline handler doesn't appear to override this behavior.

What browser did you test this with? On OSX Sierra, Chrome 55 it seems to work:

Fascinating. I'm on Windows 10, Chrome 55

I can confirm that on Firefox 50.1.0 on Windows that the behavior is as expected. This appears to be a Windows Chrome issue.

Does anyone have an idea about how to rework around this Chrome bug? If not, I will close this issue because it's not fixable in Vue.

@LinusBorg I agree with you.
@nickfederighi You should look up https://bugs.chromium.org/p/chromium/issues/list to see if the bug report already exists and report it if not

Thanks for the help on this, guys, but I believe I've solved the problem on Chrome for Windows. There seems to be a difference between browsers as well as  Windows vs OSX on chrome as to whether the default select behavior is triggered by keyup or keydown. Changing the handler to @keydown allowed me to intercept the default behavior.
Fixed jsFiddle:
https://jsfiddle.net/nickfederighi/o8fwf0gh/7/

@nickfederighi thanks for the suggestion. Experiencing a similar issue on a form with multiple buttons on OSX chrome66. Catching the keydown event instead is a good workaround.


Documentation

Use GitHub wiki to give a more in depth analysis of the project, featuring design decisions, ci/cd, tech stack, functionality and more.

[x] Motivation

TechStack

[X] Deno
[x] Svelte
[x] MongoDB
[x] Heroku

CI/CD

[ ] CodeQL
[ ] Deno
[ ] Build and Deploy

We done


Got TBeam v1.1 915Mhz works

I finally got 915Mhz to work on my TBeam v1.1
I had to modify/change the platformio.ini to use the SX1276
Can you make this change permanent in future builds?

So I should add this mod as a board for using with 915 MHz ??

Sure. It gives others in the US options for their TBeams. There are two versions. SMA on the boards use the SX1276 and U.FL on the board use the SX1262.

I would also like to see support for the SX1276.
I tried running your application on a T-Beam v1.2 with SX1276, NEO-6M, and AXP2101.
It mostly worked.  I could configure it for my wifi LAN, and send beacons, to APRS-IS, which appeared on aprs.fi.
When I tried setting setting the LoRa frequency and enabling RX/TX, all of that stopped working.
After that, I just get the initial "LoRa APRS (IGate&Digi)" screen which does not change.
It does not attach to my wifi LAN.  It does not present its own wifi access point.
Thanks!!!
P.S.  North America, South America, Australia, and Japan allow 902-928 MHz, or a subset of that, for LoRa.

Ok
Will add all versions then very soon for 915 MHz

all boards with 915 version updated


[Service Connector] az connection create: Add new param --customized-keys

Related command

Description

Testing Guide

History Notes

[Component Name 1] BREAKING CHANGE: az command a: Make some customer-facing breaking change
[Component Name 2] az command b: Add some customer-facing feature

This checklist is used to make sure that common guidelines for a pull request are followed.

[ ] The PR title and description has followed the guideline in Submitting Pull Requests.

[ ] I adhere to the Command Guidelines.

[ ] I adhere to the Error Handling Guidelines.

@kairu-ms Can you help merge the PR?


Floating point issues on Linux

Platforms:
g++10, clang++10 on Fedora Linux 32
MSVC v15.9 on Windows 10
There's some kind of very strange issue I've encountered with using doubles as properties that are passed between C++ and Lua. Essentially, when retrieving an internal 2D vector type Vec2, the x and y properties get corrupted. They always end up as 3 distinct values:

0
-1.08...e-19
-3.68...e+19

This happens when the internal Vec2 is converted into LuaVec2, and it only happens on Linux. If LuaVec2 is created via a constructor inside the Lua VM, it functions as expected. Additionally, when the LuaVec2 is converted back into Vec2 on the C++ side of things, it functions completely normally. Additionally, this only happens on Linux. When I tested it out on Windows, I received results that I originally expected!
IE:
--Returns internal mech class
player = Mech.new("PLAYER")
--sol::property that uses a getter/setter to convert between Vec2 and LuaVec2
ppos = mech.pos
print("x: "..ppos.x)
--On Linux, will output 'x: -1.084202e-19'
--On Windows will, output 'x:<PHONE_NUMBER>2354', which is the correct number according to C++ logs
otherMech = Mech.new("othermech")
--Correctly moves otherMech on top of player
otherMech.pos = ppos

--LuaVec2 is aliased to Vec2 inside the Lua VM for user clarity.
vec = Vec2.new(2.0, 3.0)
--This will output '2' on all systems
print("x: "..vec.x)

Here is the wrapper struct:
struct LuaVec2
{
    LuaVec2() : x(0.0f) y(0.0f) {}
    LuaVec2(double _x, double _y) : x(_x) y(_y) {}
    LuaVec2(Vec2 v) : x(v.x) y(v.y) {}

    Vec2 AsVec2()
    {
        return Vec2(x, y);
    }

    double x;
    double y;

};

And it's usertype declaration:
sol::state lua;
sol::usertype<LuaVec2> vec2_type = lua.new_usertype<LuaVec2>("Vec2",
        sol::constructors<LuaVec2(), LuaVec2(double, double)>());
vec2_type["x"] = &LuaVec2::x;
vec2_type["y"] = &LuaVec2::y;

I've used this exact same struct with a different Lua wrapper before and it worked fine on Linux. I should note that Vec2 uses float, while LuaVec2 uses double. I played around and made LuaVec2 use float, but it didn't resolve the issue.
This essentially makes the LuaVec2 object completely opaque for my Linux users. They can use the provided API and get it from the C++ side of things, and they can even pass it back to the C++ side of things and it will work perfectly fine. However they cannot read or write into it making it of very limited use. I tried on two computers running Linux and they both got the same erroneous result.

Guess I need to download me a Fedora Image so I can install it on this VM so I can test this.
This might take some time. :D

Hmm, I set up a test outside of the codebase, isolating the offending data structures, and I couldn't get it to reproduce. I'm going to work on doing a deeper dive to see if this might actually be a me problem!

Alright so I've definitely determined that something wierd happens outside of my own code.
This code:
player = Mech.new("PLAYER")
logVec("Player", player.pos)
LOG("---- TEST TEST TEST ----")
TESTLOG(player.pos.x)

Will output the following:
[LOG] New LuaVec2: (108.541664, 114.583336)
[LOG] From Vec2: (108.541664, 114.583336)
[LUA] Player pos
(-1.084202e-19, 3.689349e+19)
[LUA] ---- TEST TEST TEST ----
[LOG] New LuaVec2: (108.541664, 114.583336)
[LOG] From Vec2: (108.541664, 114.583336)
[LUA] 0.000000

[LOG] indicates a log from C++. Specifically, I have the C++ side log every time a new LuaVec2 is created like so:

I have discovered the issue. For some reason, this define was in one of the make scripts:
LUA_32BITS
And for some reason ld doesn't throw a tantrum when you statically link a 64-bit executable to a 32-bit library.
Love the wrapper btw. This is easily the best one out there, you're doing great work.

Glad you figured it out!


testing java ffi: combase.dll not found correctly on windows

steps
with java-22 or java-23, on windows, check out dirs-dev repo, switch to branch for this pull request,
https://github.com/dirs-dev/directories-jvm/pull/61
and do sbt test
problem
combase.dll not found when using java ffi. the path displayed is in sbt temp directory.
expectation
testing java ffi works, just like it does on linux.

I do not have access to a windows machine currently but is it possible to workaround the issue via using System.load and specify an absolute path instead of using System.loadLibrary?

Btw this might be the referring to the same issue? The Stackoverflow answer suggests the actual cause of the issue is trying to load a 32 bit DLL from C:\Windows\System32

https://stackoverflow.com/a/9835908

Shall close this issue for now. Feel free to reopen it if new evidence emerges that suggest this is indeed caused by sbt.

I do not have access to a windows machine for the next few days but is it possible to workaround the issue via using System.load and specify an absolute path instead of using System.loadLibrary?
I had tested that the following snippet workarounds around the issue and makes the CI test pass.
if (Constants.operatingSystem == 'w') {
      String sysdir = System.getenv("WINDIR") + "\\system32";
      System.load(sysdir + "\\combase.dll");
      System.load(sysdir + "\\ole32.dll");
      System.load(sysdir + "\\shell32.dll");
}

what windows version was this on? for me, windows-11, this gives:
[error] Caused by: java.lang.ExceptionInInitializerError: Exception java.lang.UnsatisfiedLinkError: Directory separator should not appear in library name: C:\WINDOWS\system32\combase.dll [in thread "pool-1-thread-1"]

what windows version was this on? for me, windows-11, this gives:

I ran https://github.com/Friendseeker/directories-jvm/actions/runs/11535962968/job/32111615159, which runs the same CI job as https://github.com/dirs-dev/directories-jvm/pull/61

what windows version was this on? for me, windows-11, this gives:

This error is very confusing... I currently don't have a windows machine with me so cannot perform live investigation. But I had checked JVM source code and System.load should not throw "Directory separator should not appear in library name" error. Only System.loadLibrary does.
    @CallerSensitive
    @Restricted
    public void loadLibrary(String libname) {
        Class<?> caller = Reflection.getCallerClass();
        Reflection.ensureNativeAccess(caller, Runtime.class, "loadLibrary", false);
        loadLibrary0(caller, libname);
    }

    void loadLibrary0(Class<?> fromClass, String libname) {
        @SuppressWarnings("removal")
        SecurityManager security = System.getSecurityManager();
        if (security != null) {
            security.checkLink(libname);
        }
        if (libname.indexOf((int)File.separatorChar) != -1) {
            throw new UnsatisfiedLinkError(
                "Directory separator should not appear in library name: " + libname);
        }
        ClassLoader.loadLibrary(fromClass, libname);
    }

    @CallerSensitive
    @Restricted
    public void load(String filename) {
        Class<?> caller = Reflection.getCallerClass();
        Reflection.ensureNativeAccess(caller, Runtime.class, "load", false);
        load0(caller, filename);
    }

    void load0(Class<?> fromClass, String filename) {
        @SuppressWarnings("removal")
        SecurityManager security = System.getSecurityManager();
        if (security != null) {
            security.checkLink(filename);
        }
        File file = new File(filename);
        if (!file.isAbsolute()) {
            throw new UnsatisfiedLinkError(
                "Expecting an absolute path of the library: " + filename);
        }
        ClassLoader.loadLibrary(fromClass, file);
    }

oh, my bad, i overlooked that you changed from "loadLibary" to "load". i can reproduce now, thank you so much!

I did some more digging and... it probably is an sbt issue.
I tried an Intellij Java project
public class Main {
    public static void main(String[] args) {
        System.loadLibrary("combase");
    }
}

A Scala Project with Intellij build system
@main
def main(): Unit = {
  System.loadLibrary("combase")
}

A sbt project (run via sbt run Main)
@main
def main(): Unit = {
  System.loadLibrary("combase")
}

Only the sbt project failed to load combase...

Full Stacktrace for reference
Exception in thread "sbt-bg-threads-8" java.lang.UnsatisfiedLinkError: D:\Repos\untitled2\target\task-temp-directory\sbt_2b92f817\combase.dll: A dynamic link library (DLL) initialization routine failed
        at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
        at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:331)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:197)
        at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:139)
        at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2444)
        at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:916)
        at java.base/java.lang.System.loadLibrary(System.java:2063)
        at main$package$.main(main.scala:5)
        at main.main(main.scala:2)
        at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
        at java.base/java.lang.reflect.Method.invoke(Method.java:580)
        at sbt.Run.invokeMain(Run.scala:144)
        at sbt.Run.execute$1(Run.scala:94)
        at sbt.Run.$anonfun$runWithLoader$5(Run.scala:121)
        at sbt.Run$.executeSuccess(Run.scala:187)
        at sbt.Run.runWithLoader(Run.scala:121)
        at sbt.Defaults$.$anonfun$bgRunTask$6(Defaults.scala:2030)
        at sbt.Defaults$.$anonfun$termWrapper$2(Defaults.scala:1969)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at scala.util.Try$.apply(Try.scala:213)
        at sbt.internal.BackgroundThreadPool$BackgroundRunnable.run(DefaultBackgroundJobService.scala:367)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
        at java.base/java.lang.Thread.run(Thread.java:1583)

https://github.com/sbt/zinc/blob/0de8e03e57e48195d22aca7ff6f7bbb2a4020c88/internal/zinc-classpath/src/main/scala/sbt/internal/inc/classpath/ClassLoaders.scala#L166-L209
/**
 * Loads native libraries from a temporary location in order to work around the jvm native library uniqueness restriction.
 * See [[NativeCopyConfig]] for configuration details.
 */

The issue probably stems around here.

Cause: combase.dll cannot be moved.
Try running the following java snippet
public class Main {
    public static void main(String[] args) {

        System.loadLibrary("karma"); // rename combase.dll as karma.dll and place it elsewhere in java.library.path
    }
}

Now we reproduce the exact same stacktrace.
Exception in thread "main" java.lang.UnsatisfiedLinkError: C:\ProgramData\chocolatey\bin\karma.dll: A dynamic link library (DLL) initialization routine failed
	at java.base/jdk.internal.loader.NativeLibraries.load(Native Method)
	at java.base/jdk.internal.loader.NativeLibraries$NativeLibraryImpl.open(NativeLibraries.java:331)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:197)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:139)
	at java.base/jdk.internal.loader.NativeLibraries.findFromPaths(NativeLibraries.java:259)
	at java.base/jdk.internal.loader.NativeLibraries.loadLibrary(NativeLibraries.java:251)
	at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2451)
	at java.base/java.lang.Runtime.loadLibrary0(Runtime.java:916)
	at java.base/java.lang.System.loadLibrary(System.java:2063)
	at Main.main(Main.java:4)

The plot thickens...
When we run the tests when settings Test / fork = true, sbt does not do the same native lib classpath manipulation, and in fact, the loadLibrary all succeeded


Auto-instrumentation Agent only in Lambda

We have a collector running as a separate app collecting x-ray data from ECS instances. We are now planning to run some Java apps (docker images) on Lambdas as helpers to our main apps. How do we enable Auto-instrumentation Agent in our lambdas (sending data to our collector) and how to do it with docker images?
I see in the doc that it can be achieved through the layer but it's not supported with docker image? And also it looks like it comes with a bundle with the collector and we only need an agent.

We currently do not support running the lambda layers as container images. We will keep this issue open to track interest.

So there's no way to include the otel collector lambda layer in lambda functions that are packaged as docker images? This article https://aws.amazon.com/blogs/compute/working-with-lambda-layers-and-extensions-in-container-images/ makes it seem like we just copy the contents of the otel lambda layer into a /opts directory in the docker file for the function?

Is there any update on the above? I tried it with the above approach, but I am getting the below error, which I am not sure if it is a network issue or an actual issue with the code.
"unable to start, otelcol state is 3; request failed with status 403 Forbidden"
Would it work without auto-isntrumentation?
{
  "level": "info",
  "ts":<PHONE_NUMBER>.490942,
  "msg": "Launching OpenTelemetry Lambda extension",
  "version": "v0.35.0"
}
{
  "level": "info",
  "ts":<PHONE_NUMBER>.4959302,
  "logger": "telemetryAPI.Listener",
  "msg": "Listening for requests",
  "address": "sandbox:53612"
}
{
  "level": "info",
  "ts":<PHONE_NUMBER>.496056,
  "logger": "telemetryAPI.Client",
  "msg": "Subscribing",
  "baseURL": "http://<IP_ADDRESS>:9001/2022-07-01/telemetry"
}
TELEMETRY	Name: collector	State: Subscribed	Types: [Platform]
2024/01/05 03:33:39 attn: users of the prometheusremotewrite exporter please refer to https://github.com/aws-observability/aws-otel-collector/issues/2043 in regards to an ADOT Collector v0.31.0 breaking change
{
  "level": "info",
  "ts":<PHONE_NUMBER>.4981377,
  "logger": "telemetryAPI.Client",
  "msg": "Subscription success",
  "response": "\"OK\""
}
{
  "level": "info",
  "ts":<PHONE_NUMBER>.4982986,
  "logger": "NewCollector",
  "msg": "Using config URI from environment",
  "uri": "/var/task/collector.yaml"
}
{
  "level": "warn",
  "ts":<PHONE_NUMBER>.50137,
  "logger": "lifecycle.manager",
  "msg": "Failed to start the extension",
  "error": "unable to start, otelcol state is 3"
}
{
  "level": "info",
  "ts":<PHONE_NUMBER>.5031855,
  "msg": "done",
  "error": "unable to start, otelcol state is 3; request failed with status 403 Forbidden",
  "errorCauses": [
    {
      "error": "unable to start, otelcol state is 3"
    },
    {
      "error": "request failed with status 403 Forbidden"
    }
  ]
}
EXTENSION	Name: collector	State: Ready	Events: [INVOKE, SHUTDOWN]
INIT_REPORT Init Duration: 236.06 ms	Phase: init	Status: error	Error Type: Extension.Crash
{
  "level": "info",
  "ts":<PHONE_NUMBER>.59592,
  "msg": "Launching OpenTelemetry Lambda extension",
  "version": "v0.35.0"
}
{
  "level": "info",
  "ts":<PHONE_NUMBER>.5973806,
  "logger": "telemetryAPI.Listener",
  "msg": "Listening for requests",
  "address": "sandbox:53612"
}
{
  "level": "info",
  "ts":<PHONE_NUMBER>.5975645,
  "logger": "telemetryAPI.Client",
  "msg": "Subscribing",
  "baseURL": "http://<IP_ADDRESS>:9001/2022-07-01/telemetry"
}
TELEMETRY	Name: collector	State: Already subscribed	Types: [Platform]
{
  "level": "info",
  "ts":<PHONE_NUMBER>.5993073,
  "logger": "telemetryAPI.Client",
  "msg": "Subscription success",
  "response": "\"AlreadySubscribed\""
}
{
  "level": "info",
  "ts":<PHONE_NUMBER>.6002276,
  "logger": "NewCollector",
  "msg": "Using config URI from environment",
  "uri": "/var/task/collector.yaml"
}
{
  "level": "warn",
  "ts":<PHONE_NUMBER>.6008797,
  "logger": "lifecycle.manager",
  "msg": "Failed to start the extension",
  "error": "failed to resolve config: cannot resolve the configuration: cannot retrieve the configuration: unable to read the file file:/var/task/collector.yaml: open /var/task/collector.yaml: no such file or directory"
}
2024/01/05 03:33:39 attn: users of the prometheusremotewrite exporter please refer to https://github.com/aws-observability/aws-otel-collector/issues/2043 in regards to an ADOT Collector v0.31.0 breaking change
{
  "level": "info",
  "ts":<PHONE_NUMBER>.6143064,
  "msg": "done",
  "error": "failed to resolve config: cannot resolve the configuration: cannot retrieve the configuration: unable to read the file file:/var/task/collector.yaml: open /var/task/collector.yaml: no such file or directory; request failed with status 403 Forbidden",
  "errorCauses": [
    {
      "error": "failed to resolve config: cannot resolve the configuration: cannot retrieve the configuration: unable to read the file file:/var/task/collector.yaml: open /var/task/collector.yaml: no such file or directory"
    },
    {
      "error": "request failed with status 403 Forbidden"
    }
  ]
}
EXTENSION	Name: collector	State: Ready	Events: [INVOKE, SHUTDOWN]
INIT_REPORT Init Duration: 45.73 ms	Phase: invoke	Status: error	Error Type: Extension.Crash
START RequestId: 8b8d0eda-ad43-4027-9614-04096febd01e Version: $LATEST
RequestId: 8b8d0eda-ad43-4027-9614-04096febd01e Error: exit code 0
Extension.Crash
END RequestId: 8b8d0eda-ad43-4027-9614-04096febd01e
REPORT RequestId: 8b8d0eda-ad43-4027-9614-04096febd01e	Duration: 47.24 ms	Billed Duration: 48 ms	Memory Size: 1024 MB	Max Memory Used: 11 MB
XRAY TraceId: 1-65977892-16a54b5854e365727c8968c3	SegmentId: 7f011c767d09b210	Sampled: true

not stale, this feature is important


Add warnings (and annotations) to PromQL query results

Addresses some of https://github.com/prometheus/prometheus/issues/10839 and https://github.com/prometheus/prometheus/issues/11216 as a hackathon project.
Most of the code changes are just changing from storage.Warnings to notes.Warnings to move it to a more neutral location as the new warnings we are adding are not related to storage.
Annotations are created and handled separately from warnings here, but they are merged together with the warnings in the PromQL engine because the HTTP API and the UI don't support it yet and we want to avoid making external changes for now. By creating annotations separately though, it would make it easier in future when we want to display them differently.
Due to the way warnings are created, we end up with many duplicate warnings, or perhaps we find the warning is not needed later on, e.g. for the RangeTooSmallWarning, we may add this warning at each section of the range, or we may only add it at the beginning or end but still return a non-empty result for the query. For now we do the deduplication and the removal of the unnecessary warning at the final response writing step in the API.
New warnings (util/notes/warnings.go):

RangeTooSmallWarning         = errors.New("Need at least 2 points to compute, perhaps time range is too small")
MixedFloatsHistogramsWarning = errors.New("Range contains a mix of histograms and floats")
MixedOldNewHistogramsWarning = errors.New("Range contains a mix of conventional and native histograms")
InvalidQuantileWarning    = errors.New("Quantile value should be between 0 and 1")
BadBucketLabelWarning     = errors.New("No bucket label or malformed label value")
PossibleNonCounterWarning = errors.New("Metric might not be a counter (name does not end in _total/_sum/_count)")

Possible future improvements:

We might want to change notes.Warnings ([]error) in the PromQL engine to structs that contain both warnings and annotations (the new notes.Notes) to delay the merging of warnings and annotations (which would be helpful if we eventually separate them in the final output), and to have it as an independent struct to make any future changes easier (not done yet as it is not strictly necessary at this stage and I'm not sure if we want to do this)
With the above change, we can do the deduplication at an earlier stage each time we merge Notes to avoid collecting a long list of duplicate warnings until the final step. With a struct, we can easily add internal maps to support the deduplication in the merge function.

So, in summary, I think we have to crack to nuts here:

How to deal with the "not enough samples in the range" warning/annotation.
How to implement the plumbing for conveying warnings/annotations at all.

To reduce the problem surface, I think we should focus on (2) for now and only start thinking about (1) once the plumbing is in place.
WRT (2): To give some preference among the many thoughts I have expressed in the comments, my gut feeling is currently to…

…not think about deduplication but go for more specific warnings/annotations so that deduplication might not be needed (but could be revisited if needed).
…treat everything as Go errors utilizing the errors package tooling (which you have already done partially anyway).

In that way, we won't need any structures (like notes.Notes) but just specific error values and types. Maybe have an error value called PromQLAnnotation and then an (unexported) type promQLAnnotation with an Is method returning true for target == PromQLAnnotation.

Here some rough ideas how to approach the problem. Happy to discuss here or in a call to refine.

Let's make the annotation.Annotations type the only thing we hand around and propagate (essentially replacing storage.Warnings in the current upstream code). And let's get rid of annotations.Info and annotations.Errors.
annotations.Annotations could have map[string]error as the underlying type, with the stringified version of each error as the key, thereby providing de-duplication.
The errors we put into that map could wrap marker errors in a suitable way (see Go documentation and details below).

To expand on the last point:
Here is a slight variation of what you had already done:
var (
	PromQLInfo    = errors.New("PromQL info")
	PromQLWarning = errors.New("PromQL warning")

	BadBucketLabelWarning  = fmt.Errorf("%w: no bucket label or malformed label value", PromQLWarning)
	PossibleNonCounterInfo = fmt.Errorf("%w: metric might not be a counter (name does not end in _total/_sum/_count)", PromQLInfo)
)

func NewBadBucketLabelWarning(metricName, label string) error {
	return fmt.Errorf("%w: %s %s", BadBucketLabelWarning, metricName, label)
}

func NewPossibleNonCounterInfo(metricName string) error {
	return fmt.Errorf("%w: %s", PossibleNonCounterInfo, metricName)
}

For this example, I have only implemented two annotations here: BadBucketLabelWarning precisely as before, and NewPossibleNonCounterInfo as an info version of the previous PossibleNonCounterWarning.
Now a new version of the API could return the info annotations in a separate section by iterating through all annotations and testing them with errors.Is(annotation, annotations.PromQLInfo) and putting them into the new section if true or in the old warnings section if false, more or less like this:
for _, annotation := range annotations {
    if errors.Is(annotation, annotations.PromQLInfo) {
        // Add to info section.
    } else {
        // Add to warning section as before.
    }
}

@beorn7 I will fix the test errors soon (but the build works, you can test the warnings with quantile(3, up)), but meanwhile please have a look to see if this is what you meant.

I'm back from vacation, and I'll review this ASAP.
Have you found out anything about the Prometheus stability guarantees for the API? Can we just add another key to the JSON? Otherwise, we should inquire via the developer community channels.

Here my thoughts about adding position info:
Here is the the definition of a ParseErr. It contains a field of type parser.PositionRange and the whole Query as a string. It also has logic in its Error() string method to use this data to format a string with the correct position information (line number and position in line). I don't think we can just embed ParseErr into the errors in the annotations package, but maybe extract the logic somewhere where it can be used by both.
The more problematic part is how to even get the PositionRange and the full query as a string. I think the former mostly works. For example, if a PromQL function acts on a parser.VectorSelector, it can simply access the PosRange field in the VectorSelector. However, I haven't seen any obvious way to access the full query as a string. Maybe the engine could enrich the annotations with that string as the last step before handing back the query result including annotations to the caller.

@beorn7 Updated according to your comments, but still not sure how to add position info. For example, look at how we add the warning for invalid quantile in https://github.com/prometheus/prometheus/blob/f22466d167f28e3fac5ab9bc95a9c42bc7ed8fd2/promql/functions.go#L642 , I'm not sure how to get anything relevant from vals, args or enh there. We could manually pass in the string "quantile_over_time" if that helps though, and pass along the original function string for helper functions like histogramRate.

In this case, you could use the args arg.
Let's assume annotations.NewInvalidQuantileWarning would have a parser.PositionRange argument, then you could write:
annos.Add(annotations.NewInvalidQuantileWarning(q, args[0].PositionRange()))

That's the easy part, as said. The hard part is how to give the annotation a pointer to the full query string (needed to render the error string). We could try to set that later (once the evaluation is done), but we have to be careful to not interfere with the deduplication. Or we give the EvalNodeHelper a pointer to the query string…
If this turns out to be too involved, we could also implement it in a later PR. I already like things how they look now:

Wild thought: Maybe we could store the query string once per Annotations (make it a struct with the current map and the new string as field). Annotations should always be all for the same query.

Now it looks like PromQL warning: quantile value should be between 0 and 1 not 50.00 (at 19:21). Only implemented for the quantile warning as the rest aren't directly related to user input, only the returned metrics, so we display the relevant metric name instead. Do we need to pass in the full query string? It seems like that would require a lot of code changes, and the result is something we can already show (the 50). I'm using interface{} for now as I'm not sure if we want to stick with this approach, and there's an import cycle if we use parser.PositionRange. If we stick with this, I could try moving parser.PositionRange into its own package or something.

Now it looks like PromQL warning: quantile value should be between 0 and 1, not 50.00 (at 19:21).

It would be cool to keep the syntax consistent with the full errors we already have. But that's just a minor cosmetic thing. See here:

The real problem is that the position is just "characters into the query", ignoring line breaks. The full errors tell you the line number and the character position within that line. There is logic for that in the Error() method of ParseErr, as I said earlier. See the code here (interestingly, it only points to the start of the character range in question, so we should do the same, which also means we only need the start position as a parameter for the annotations).
For this logic, we need the full query string, which answers your other question.
My wild idea also mentioned above was to let the engine add the full query string to the Annotations (making it a struct with the current map as a field and then a separate field for the query string). This could be done after the fact (i.e. just before the Annotations are returned to the caller). Not sure if that works, but it would not require us to propagate the query string everywhere.
So ideally, the last warning in the screenshot above should read something like: PromQL warning: 1:20: quantile value should be between 0 and 1 but is 1.1 (BTW: Where is that weird trailing 0 coming from?).

Only implemented for the quantile warning as the rest aren't directly related to user input, only the returned metrics, so we display the relevant metric name instead.

I think it's still very helpful to display the position. See the first warning in my screenshot above. Is it referring to the first or the second occurrence of up? It would be cool if it read: PromQL warning: 1.25: no bucket label or malformed bucket label value on metric up.

I'm using interface{} for now as I'm not sure if we want to stick with this approach, and there's an import cycle if we use parser.PositionRange. If we stick with this, I could try moving parser.PositionRange into its own package or something.

Yeah, I was afraid that would happen. Since the quite elaborate logic to convert the start position into an actual "line:pos" pair has to go somewhere where it is accessible to the PromQL engine, too, we need to move stuff around anyway. Maybe it could live in the promql package itself?
Having said all that: I do think we should provide the line:pos information, but we don't have to do it in the first iteration. It might be better to finish this PR up without it if it gets too involved. And the add this information later.

I guess the formatting with the position info at the end is generally OK. (I don't think that the formatting of the parser errors is optimal, anyway – so maybe that would be a separate task to rework all those error messages into something more readable and nicer to the eye.)
About the lint error: It's for file promql/parser/lex_test.go, which isn't one of the autogenerated ones (is it?). It should be easy to fix. I left a comment there.
I'll do a detailed code review ASAP.

In the meantime, paging @roidelapluie again: It would be very good to get a general call about the changes in the API as discussed above and previously on Slack. Currently, we just add all the warnings and the one info annotation we have so far to the warnings. But there will be more "info" style annotations in the future (i.e. "warnings" that might be false positives and therefore should maybe go to their own info category). So:

Should we do it as it is done here now (everything as a warning), at least for now? (Assuming that changing warnings isn't considered a breaking change.)
Should we do it as it is done here now but hide the new annotations behind a feature flag so that we don't have to worry about a breaking change if we change our mind?
Should we introduce a new "info" section in the API now (and adjust the UI accordingly to display them differently, in a green box or something)?

Maybe @juliusv could also chime in, especially if we do the latter and add something new to the UI.

(Disclaimer: I still have to do the detailed review.)
I hijacked an unrelated meeting with @roidelapluie to find out what he thinks about this.
He brought up the concern that users that regularly create false positives for INFO level annotations might get rather annoyed by them (imagine someone using misnamed counters for historical reasons), so it would be good if INFO level annotations could be disabled (e.g. in the UI or even via config). On the other hand, we shouldn't jump to conclusions here. We can also try as is (i.e. have all the annotations as warnings for now) and only change if users are unhappy indeed. @roidelapluie agreed that moving an annotations from WARN to INFO isn't really a breaking change, so we could do it even without hiding everything behind a feature flag.
But it's quite important to never spam the user with many annotations. Currently, this can easily happen if an annotation is spawned for every element in a huge metric vector. So we definitely need to put a limit in there. I guess the UI should never display more than around 10 annotations, followed by a message like "10,453 more PromQL warnings".
While this could be implemented just in the UI, I think we should even go further and limit it in the API itself. We are just talking about helpful annotations here, and that shouldn't OOM the receiver just because 10,000 annotations are included. So maybe let's implement that in the API itself to only show the first 10 (or so) warnings and infos, and then add one more new warning/info that just tells you how many more have been skipped.

@beorn7 Added the limit for the API

Thank you for your thorough reviews and all your suggestions!

We already have https://github.com/prometheus/prometheus/issues/12945 and https://github.com/prometheus/prometheus/pull/12922 . Please discuss there. (I'm pretty sure we should not warn on nameless metrics. Once we have stronger typing in PromQL, we can do it properly.)


Too long HP bar

Steps to reproduce:

Just start the game

What I saw;
The HP bar of hero is too long to the right side.
What I expected:
HP bar should be matched to the hero.

In new version is still a problem - HP bar is too stretch vertical.


MSBuild 12.0 Microsoft.CodeContracts.targets imports wrong ContractAnalysis.targets

The MSBuild v12.0 version of Microsoft.CodeContracts.targets imports $(CodeContractsInstallDir)MsBuild\v4.0\Microsoft.CodeContractAnalysis.targets. This is a relatively minor issue, but causes some problems if you're trying to add the minimum requirements for Code Contracts into your source enlistment to support builds without the extension.
The problem is on line 638 with the definition of the CodeContractAnalysisTargets property. This line is fine in the MSBuild 3.5 and 4.0 versions of the file.
The best way to resolve this would be to set the property to $(MSBuildThisFileDirectory)Microsoft.CodeContractAnalysis.targets, which would prevent the need for the version number in any of the files (4.0 and up - 3.5 doesn't support MSBuildThisFile).
Most of the content could be consolidated to avoid the need for duplicating the file with each version, but this minor change would avoid at least one problem.

For now I would prefer a minor change. But, could you explain in what scenario current target path is causing problems? Is this related to build server workspace? I just want to know the nature of the problem.

It's not an issue if the entire MSBuild folder is there, but when adding to source control, it's preferable to reduce things to what's needed. Since the v12.0 file references the v4.0 folder, both folders have to be there. This would also be an issue if the 4.0 folder were removed at some point.
The simple change is to do the following, which avoids the need to worry about the version at all. This can be applied to the 4.0, 12.0, and 14.0 .targets files.
$(MSBuildThisFileDirectory)Microsoft.CodeContractAnalysis.targets

Fixed by #57.


Run without Dock icon

Thank you for the app… pretty great. Any chance to make it run as a background process without a Dock icon?

Absolutely, you could. Right now, the app was built as quickly as possible. Removing the Dock icon would also remove the menu bar, which I think will break the 1-4 action shortcut keys. That action window would need to be rebuilt (correctly, this time) to handle this.


[SUPPORT] Some classes  lacked and couldn't be imported in the  module of  hudi-spark3.2.x_2.12 and  hudi-spark3.3.x_2.12

Some problems i met when compiled the hudi source code on the branch-master
The compile operation failed in the module of  hudi-spark3.2.x_2.12, the error as below
_[ERROR] Failed to execute goal net.alchim31.maven:scala-maven-plugin:3.3.1:compile (default-cli) on project hudi-spark3.2.x_2.12: wrap: org.apache.commons.exec.Execut
eException: Process exited with an error: 1 (Exit value: 1) -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging._

At the package org.apache.spark.sql.parser ， there are two import errors ,the HoodieSqlBaseBaseVisitor, HoodieSqlBaseParser  lacked：
import org.apache.hudi.spark.sql.parser.HoodieSqlBaseParser._
import org.apache.hudi.spark.sql.parser.{HoodieSqlBaseBaseVisitor, HoodieSqlBaseParser}

The same problem also happended in the module of hudi-spark3.3.x_2.12
I didn't find the two classes above by searching in the whole project, how to solve this problem and if it's the lack of the two classess leand to the compile process failed or not ?

@GoodJeek how did you build the project? have you been following commands from the README.md ? our CI keeps building against different spark profiles so there is no issue

Hi @GoodJeek, Thanks for raising this issue.  I cannot reproduce your compilation failure after pulling and compiling Hudi code from scratch in the console.  These are the build commands I verified that are successful:
mvn clean package -DskipTests -Dspark3.2 -Dscala-2.12
mvn clean package -DskipTests -Dspark3.3 -Dscala-2.12

If you are using IntelliJ IDE, you should follow the developer setup on the Hudi website to properly compile and index the code.  These two sections are relevant: IDE Setup and Running unit tests and local debugger via Intellij IDE.
Closing this issue now as the problem cannot be reproduced.  Feel free to reopen this issue if you still have a problem after following the above guide.


[E2E][Triton] TIMM model pnasnet5large accuracy regression.

🐛 Describe the bug
Skip the model in pre-ci. Please retrieve the case, if triton gets fixing.
https://github.com/intel/intel-xpu-backend-for-triton/issues/1353
Versions
Latest main.

This issue have been fixed in latest triton main branch.  we'll update triton to resolve this issue.

Please keep this open issue, we've not updated triton.

@etaf please help to double check whether we can close it

@etaf  could we close this issue now.


Use"image" instead of "digest" to track scan overview.

Add column "image" and drop "digest" from scan_overview table.
The value of image will be "$repository:$tag"
Update API/DAO code.

Won't do this change.


image generation time  increased a lot on Mac M1

Describe your environment

GPU: mps
VRAM: 128G
CPU arch: arm
OS:  macOS
Python:  miniconda
Branch:  development
Commit: 720e5cd6513cd27e6d53feb6475dde20bd39841a

Describe the bug
after refactoring dream takes ~70% longer to generate images w.r.t. 1.13 release
development : 1:26 sec
main : 0:50 sec
To Reproduce
git checkout development (720e5cd6513cd27e6d53feb6475dde20bd39841a)
dream> "test" -s50 -W576 -H896 -C7.5 -Ak_lms

100%|  50/50 [01:25<00:00,  1.71s/it]
Generating: 100%|  1/1 [01:26<00:00, 86.13s/it]

>> Usage stats:
>>   1 image(s) generated in 54.13s
>>   Max VRAM used for this generation: 0.00G

git checkout main (751283a2de81bee4bb571fbabe4adb19f1d85b97)
dream> "test" -s50 -W576 -H896 -C7.5 -Ak_lms

100%| 50/50 [00:50<00:00,  1.00s/it]
Generating: 100%| 1/1 [00:50<00:00, 50.87s/it]
>> Usage stats:
>>   1 image(s) generated in 50.95s
>>   Max VRAM used for this generation: 0.00G

Environment
pytorch                   1.12.1                 py3.10_0    pytorch
pytorch-lightning         1.6.5              pyhd8ed1ab_0    conda-forge
torch-fidelity            0.3.0                    pypi_0    pypi
torchdiffeq               0.2.3                    pypi_0    pypi
torchmetrics              0.9.3              pyhd8ed1ab_0    conda-forge
torchvision               0.13.1                py310_cpu    pytorch

The usage stats may also throw an odd value.
1 image(s) generated in 50.95s but right above
Generating: 100%|  1/1 [01:26<00:00, 86.13s/it]

@eduardoarinopelegrin
thank you for your remark, Indeed I forget to mention the discrepancy with the time reported in the stats.
but the image realy takes 01:26 seconds to generate, I double checked with a stopwatch at the same time.

@magnusviri , are you seeing slowdowns on your M1?

With 64 GB RAM, I can reproduce the issue. As a small note, the usage stats seem correct now.
git checkout development
dream> "test" -s50 -W576 -H896 -C7.5 -Ak_lms
Generating: 100%|| 1/1 [02:23<00:00, 143.55s/it]
>> Usage stats:
>>   1 image(s) generated in 143.94s
>>   Max VRAM used for this generation: 0.00G

git checkout main
dream> "test" -s50 -W576 -H896 -C7.5 -Ak_lms
Generating: 100%|| 1/1 [01:35<00:00, 95.73s/it]
>> Usage stats:
>>   1 image(s) generated in 95.84s
>>   Max VRAM used for this generation: 0.00G

Environment
pytorch                   1.12.1          cpu_py39h9b0ea23_0    conda-forge
pytorch-lightning         1.6.5              pyhd8ed1ab_0    conda-forge
torch-fidelity            0.3.0                    pypi_0    pypi
torchdiffeq               0.2.3                    pypi_0    pypi
torchmetrics              0.9.3              pyhd8ed1ab_0    conda-forge
torchvision               0.13.0          cpu_py39h982e834_0    conda-forge

I'm pretty sure the slowdowns on M1 hardware are due to the memory optimizations in attention.py that we instituted over the weekend. I don't have an M1 myself so can't check personally, but here's what you can do to help isolate the issue:

Check out the development branch git checkout development; git pull This will bring you up to date with all the recent changes.
Run your timing tests to confirm that problem still occurs.
Check out the version of attention.py just prior to the optimizations git checkout bdb0651e ldm/modules/attention.py
Run your timing tests again. Note whether the checkout has fixed them.
Return attention.py to its current state git checkout HEAD ldm/modules/attention.py

Much obliged for any help you can give in tracking this down.

@lstein
I guess you right, I executed the test procedure you ask for
1/ git checkout development; git pull (eef788981cbed7c68ffd58b4eb22a2df2e59ae0b)
dream> "test" -s50 -W576 -H896 -C7.5 -Ak_lms

100%|| 50/50 [01:20<00:00,  1.61s/it]
Generating: 100%|| 1/1 [01:21<00:00, 81.05s/it]
>> Usage stats:
>>   1 image(s) generated in 81.25s
>>   Max VRAM used for this generation: 0.00G

2/ git checkout bdb0651e ldm/modules/attention.py
dream> "test" -s50 -W576 -H896 -C7.5 -Ak_lms

100%| 50/50 [00:49<00:00,  1.01it/s]
Generating: 100%|| 1/1 [00:50<00:00, 50.01s/it]
>> Usage stats:
>>   1 image(s) generated in 50.16s
>>   Max VRAM used for this generation: 0.00G

so this is the same time I get on the 1.13 Relase

I noticed similar behavior on m1 after changing size of image. When the first generation is like 384x384, second is 512x512, then speed drops (about 9 times slower). Terminal restart fixes it.

The bigger the image, the longer it takes to generate it (as there are more pixels). Although 9 times slower seems like a lot. Maybe using SWAP for 512x512 in your machine, but not for 384x384.
For me, 1 image(s) generated in 20.33s for 512x512 and 1 image(s) generated in 29.81s for 384x384. 50% increase in time for 77% increase in number of pixels.

I noticed similar behavior on m1 after changing size of image. When the first generation is like 384x384, second is 512x512, then speed drops (about 9 times slower). Terminal restart fixes it.

The bigger the image, the longer it takes to generate it (as there are more pixels). Although 9 times slower seems like a lot. Maybe using Swap for 512x512 in your machine, but not for 384x384.
For me, 1 image(s) generated in 20.33s for 512x512 and 1 image(s) generated in 29.81s for 384x384. 50% increase in time for 77% increase in number of pixels. Much more reasonable.

Of course, but in my case it’s 9 times slower than 512 without changing size :-) I mean, if I just generate 512 it’s about 1:20 for 50 samples klms, but if I do that after generating another size, for example 384, it takes 10-15 minutes. SOmetimes. Sometimes not.

@lstein
ldm/modules/attention.py (Development branch)
        # attention, what we cannot get enough of, by halves
        sim[4:] = sim[4:].softmax(dim=-1)
        sim[:4] = sim[:4].softmax(dim=-1)

        sim = einsum('b i j, b j d -> b i d', sim, v)
        sim = rearrange(sim, '(b h) n d -> b n (h d)', h=h)
        return self.to_out(sim)

Usage stats:
1 image(s) generated in 44.24s
Max VRAM used for this generation: 0.00G

ldm/modules/attention.py (With this code instead)
        # attention, what we cannot get enough of
        attn = sim.softmax(dim=-1)
        out = einsum('b i j, b j d -> b i d', attn, v)
        out = rearrange(out, '(b h) n d -> b n (h d)', h=h)
        return self.to_out(out)

Usage stats:
1 image(s) generated in 30.74s
Max VRAM used for this generation: 0.00G

Two options here. The easier one is to detect if we are running on M1 by examing device.type=='mps', and choosing the old or new code depending on the results. The other is what you suggested, to interrogate VRAM and the image size, and choose whether to use the optimization if VRAM will be inadequate.
For the latter, we need to measure how much VRAM is used. Also, we don't actually know whether the memory optimization on CUDA results in any improvement in VRAM usage on M1, do we? I don't completely trust reports that "I couldn't run a large image before, and now I can" because VRAM is shared with other processes.

@lstein
Yes, if device.type=='mps' is the easier solution. As for measuring VRAM, not sure how to do that in Windows/Linux, but in case some users with M1/M2 with low RAM (16? 8?) prefer the new code (that is, to trade speed for lower memory usage), since M1/M2 chips use unified memory, maybe we could do something like:
import psutil
use_halves = True if psutil.virtual_memory().available < 8 * 10**9 else False

then
use_mps = True if x.device.type == 'mps' else False # detect mps right before deleting x
del context, x

and finally
        # attention, what we cannot get enough of
        if use_mps and not use_halves:
            sim = sim.softmax(dim=-1)
        else:
            # by halves
            sim[4:] = sim[4:].softmax(dim=-1)
            sim[:4] = sim[:4].softmax(dim=-1)

        sim = einsum('b i j, b j d -> b i d', sim, v)
        sim = rearrange(sim, '(b h) n d -> b n (h d)', h=h)
        return self.to_out(sim)

To offer a bit more information about psutil, this is what psutil.virtual_memory() gives me (64 GB RAM). Something to note is, even though is says 10.7GB available, I can use 20GB extra (e.g. from python3.9 using 10GB to using 30GB in Activity Monitor) and still be fine (not use Swap). So maybe we should use psutil.virtual_memory().total instead of psutil.virtual_memory().available and assume users won't have many other things open at the time. This way, if a user has an M1/M2 with 8GB RAM, it would automatically use the 'by halves' version of the code.
% print(psutil.virtual_memory())
svmem(total=68719476736, available=10701914112, percent=84.4, used=4901261312, free=6921248768, active=4137091072, inactive=3623813120, wired=764170240)

From the psutil docs:  https://psutil.readthedocs.io/en/latest/

total: total physical memory (exclusive swap).
available: the memory that can be given instantly to processes without the system going into swap. This is calculated by summing different memory values depending on the platform and it is supposed to be used to monitor actual memory usage in a cross platform fashion.
used: memory used, calculated differently depending on the platform and designed for informational purposes only. total - free does not necessarily match used.
free: memory not being used at all (zeroed) that is readily available; note that this doesn’t reflect the actual memory available (use available instead). total - used does not necessarily match free.
active (UNIX): memory currently in use or very recently used, and so it is in RAM.
inactive (UNIX): memory that is marked as not used.
buffers (Linux, BSD): cache for things like file system metadata.
cached (Linux, BSD): cache for various things.
shared (Linux, BSD): memory that may be simultaneously accessed by multiple processes.
slab (Linux): in-kernel data structures cache.
wired (BSD, macOS): memory that is marked to always stay in RAM. It is never moved to disk.

Just to confirm, the code works on my Mac.
attention.py.zip

Also, we don't actually know whether the memory optimization on CUDA results in any improvement in VRAM usage on M1, do we?

Actually, I see worse performance memory-wise on my Mac with the new code.
"ocean" -s50 -W512 -H512 -C7.5 -Ak_lms -F -S2381025714
Old code

New code

Not sure if @i3oc9i  can reproduce this too?

@eduardoarinopelegrin
what do you want I test exactely, i'm not sure to understand what do you mean with old and new code ?
im on development branch eef788981cbed7c68ffd58b4eb22a2df2e59ae0b
with this enviroment
Python 3.10.5

pytorch                   1.12.1                 py3.10_0    pytorch
pytorch-lightning         1.6.5              pyhd8ed1ab_0    conda-forge
torch-fidelity            0.3.0                    pypi_0    pypi
torchdiffeq               0.2.3                    pypi_0    pypi
torchmetrics              0.9.3              pyhd8ed1ab_0    conda-forge
torchvision               0.13.1                py310_cpu    pytorch

So, basically, this seems to be the problematic part of ldm/modules/attention.py in the Development branch.
        # attention, what we cannot get enough of, by halves
        sim[4:] = sim[4:].softmax(dim=-1)
        sim[:4] = sim[:4].softmax(dim=-1)

        sim = einsum('b i j, b j d -> b i d', sim, v)
        sim = rearrange(sim, '(b h) n d -> b n (h d)', h=h)
        return self.to_out(sim)

and this is the old version, which was faster.
# attention, what we cannot get enough of
        attn = sim.softmax(dim=-1)
        out = einsum('b i j, b j d -> b i d', attn, v)
        out = rearrange(out, '(b h) n d -> b n (h d)', h=h)
        return self.to_out(out)

lstein was asking it the first version, even though it was slower, at least used less RAM on Macs. I just tried and it actually was slower + used more RAM (instead of less). So no benefit at all.
I was asking if in your machine it also used more RAM, apart from being slower.

@eduardoarinopelegrin what do you want I test exactely, i'm not sure to understand what do you mean with old and new code ?
in development branch eef7889 "ocean" -s50 -W512 -H512 -C7.5 -Ak_lms -F -S2381025714  50/50 [00:29<00:00, 1.67it/s]
in main branch 751283a "ocean" -s50 -W512 -H512 -C7.5 -Ak_lms -F -S2381025714 50/50 [00:20<00:00, 2.46it/swith this enviroment
Python 3.10.5

pytorch                   1.12.1                 py3.10_0    pytorch
pytorch-lightning         1.6.5              pyhd8ed1ab_0    conda-forge
torch-fidelity            0.3.0                    pypi_0    pypi
torchdiffeq               0.2.3                    pypi_0    pypi
torchmetrics              0.9.3              pyhd8ed1ab_0    conda-forge
torchvision               0.13.1                py310_cpu    pytorch

Yep, so simply check the memory usage as well on Activity Monitor. Do you see any differences between both branches?

@eduardoarinopelegrin
Development 15.4G   29sec
Main               11.65G 20Sec
So development slower and more RAM

Just to share that last nightly pythorch is on python 3.10 now
    ---------------------------|-----------------
    pytorch-1.13.0.dev20220905 |         py3.10_0        44.4 MB  pytorch-nightly
    torchaudio-0.13.0.dev20220905|        py310_cpu         5.6 MB  pytorch-nightly
    torchvision-0.14.0.dev20220906|        py310_cpu         6.4 MB  pytorch-nightly
    ------------------------------------------------------------

I will check if there is some improvment into a separate conda enviroment in order
to avoid to break my official dream-dev test environment

@i3oc9i; Thanks for the leg work
I immediately note that 1.13 is "nightly" (and tagged with "dev"); I'd expect VRAM usage to drop darmatically once they cut a release. Whether it's below 1.11/1.12 remains to be seen :)

Actually, I see worse performance memory-wise on my Mac with the new code.
I came here to share the same results. I switched to the development branch and I can relate the generation performance is painful to watch.

Let me rephrase: It's highly unlikely that the Torch team is going to release a huge memory usage regression.

@glonlas
I suggest you stay on main brach, developpment branch is a working in progress branch so things can be worse/broken, and there are actually other branches where performances  solutions are evaluated.

@i3oc9i The problem is some features such as inpainting are not available in the main branch.
Plus, if we want to add code (such as what I started and @glonlas took over, which is upscaling and face restoration for M1), I think it's better to develop it for the development branch, because they're refactoring code, so the main branch has a bit of an expiration date (and won't get all features implemented).
Oh, I just checked, and performance is back on the development branch! They brought this back
if device_type == 'mps':      #special case for M1 - disable neonsecret optimization
            sim = sim.softmax(dim=-1)
        else:
            sim[4:] = sim[4:].softmax(dim=-1)
            sim[:4] = sim[:4].softmax(dim=-1)

TL;DR - If you want something that works stably without issue, grab the release. If you want instability, but want to try out the latest features under development, git checkout development
HAVING SAID THAT

there are some issues with release 1.13
'main' has an Mx fix on it (which it shouldn't)

so the repo isn't really in a "normal" state right now - hopefully things will be more "usual" after the next release

If you want to develop, add features, or propose bug fixes, use the development branch. The main branch will only change for urgent bug fixes.

fixed in 1.14.1 release 9b28c65e4b0526956fd90ad86e2118536e4eaa9b


Windows cli updates

Summary

Updated files for windows file paths.
Added helper function to sanitize filepaths for windows.
some minor code cleanup.
Requires testing on non-windows OS.
solves issues with windows reading slashes as escape characters and the cli failing most commands.

bug-fix

NA
Test Plan

manual QA:
commands:

version [x]

build [x]

deploy [x]

diff [x]

bounce [x]

destroy

init [x]

bundle [x]

link [x]

unlink [x]

help [x]

repos [x]

api [x]

upgrade [/]

watch

info [x]

proxy [x]

logs [x]

ops [x]

utils

apply

test

push

template

from-grafana

login [x]

import [x]

crypto [x]

config [x]

profile [x]

create [x]

validate [x]

topsort [x]

serve [x]

shell

workspace

output

build-context

changed [x]

help

Checklist

[ ] My change requires a change to the documentation and I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.

I'll take a look when I get home.
I believe that function ended up not being used, so should have removed it.
If memory serves it was to handle the Abs functions that required a result and an error to be returned.
I'll see if the two mentioned functioned ( or others in that library) are a better solution for the minor Windows issues too.
Get Outlook for Androidhttps://aka.ms/AAb9ysg

From: David van der Spek @.>
Sent: Monday, April 25, 2022 3:58:16 AM
To: pluralsh/plural-cli @.>
Cc: dustinfaulkner91 @.>; Mention @.>
Subject: Re: [pluralsh/plural-cli] Windows cli updates (PR #47)
@DavidSpek commented on this pull request.

In pkg/utils/pathing/helpers.gohttps://github.com/pluralsh/plural-cli/pull/47#discussion_r857504204:

"strings"

+)
+
+// returns a string (filepath) with appropriate slashes, dependent on OS. Optional parameters and method overloading unavailable in Go.
+func SanitizeFilepath(filepath string) string {

  os := runtime.GOOS

  switch os {

  case "windows":

          return strings.Replace(filepath, "\\", "/", -1)

  default:

          return filepath

  }

+}
+
+// returns a string (filepath) with appropriate slashes, dependent on OS if error is empty. Optional parameters and method overloading unavailable in Go.
+func SanitizeFilePathWithError(filepath string, err error) (string, error) {
There were some places where a function is called that returns a path and an error. So for those cases this could be used.
—
Reply to this email directly, view it on GitHubhttps://github.com/pluralsh/plural-cli/pull/47#discussion_r857504204, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADSTJBMRVOBIS5FC2B736TDVGZ3ERANCNFSM5TMTCZVQ.
You are receiving this because you were mentioned.Message ID: @.***>

Superseded by https://github.com/pluralsh/plural-cli/pull/54.


Perf test for color filter with saveLayer

This should reveal the problem in:
https://github.com/flutter/flutter/issues/51975
Fixing that issue should significantly lower the frame rasterization
time.

@karhu: I've simplified your reproduction app, and put it inside Flutter's devicelab perf test suite. Do you think that you can directly create a PR like this next time?
As for the two problem you mentioned,

"saveLayer with ColorFilter disables the raster cache" will be tracked in https://github.com/flutter/flutter/issues/51975 and a fix would show a significant improvement in this test.
"picture is always recreated under fade transition" is actually something that you can fix in your code. You simply need to add a RepaintBoundary here to not let the fade animation rebuild everything below.

@liyuqian Thanks a lot for your work on this! I went over your changes and it  is indeed a great reference for future repro apps. Will definitely give it a try next time :+1:

The use of the color param as a flag to trigger the use of shadow.color looks a little wonky, but that's not a blocking concern.
LGTM.

Oh, I see it. Now I've removed color and changed everything to useColorFilter now. Thanks for the suggestion!


including missing files that need to be updated on version change

LGTM

Current coverage is 76.01%

Merging #299 into master will not affect coverage as of e1f0583

@@            master    #299   diff @@
======================================
  Files           41      41
  Stmts         4019    4019
  Branches       350     350
  Methods          0
======================================
  Hit           3055    3055
  Partial        145     145
  Missed         819     819

Review entire Coverage Diff as of e1f0583

Powered by Codecov. Updated on successful CI builds.


Clean Up Dead Code Button

Issue Type: Feature Request
Visual Studio (VS) Code already has the ablity to detect dead code, but to clean up the dead code the user must manually search for the dead code and remove it. However, it would be more efficient if there was a button in the debug side tab that would remove all dead code. There should be options for the user to remove dead variables, functions, methods, and or classes. There should be an option to remove dead code from all files open, just selected files, or just the current open file. If the user has two, or more, code files open, in the split editor, and the user selects the current file open then which ever file is active (A.K.A the file that the user is currently editing). Of course there should be a pop up warrning explaing that removing dead code will remove any variables, functions, methods, and or classes (depending on which the user selected to have removed) that are not in use, regardless of their implementation.
VS Code version: Code 1.60.0 (e7d7e9a9348e6a8cc8c03f877d39cb72e5dfb1ff, 2021-09-01T10:41:52.311Z)
OS version: Windows_NT x64 10.0.19043
Restricted Mode: No

This is a language function and unrelated to debug.


Move Bourbon

Move Bourbon under the Libraries category, because it is a library and not a framework.
My thinking is that Bourbon does not provide any style or pre-built components out of the box, which frameworks do. Bourbon is simply a suite of Sass tools to help you write CSS.

Oops. Thank you for catching this one and for the pull request.
You rock ✊

@tandavas Thank you!


.Net: Remove InvokeAsync extension on Kernel that accepts a string input instead of KernelArguments

Just to confirm, the plan here is to remove this and then make it easy to construct a kernel arguments from a single input string?
cc: @SergeyMenshykh


Job data should be cleaned more robustly on worker machine

Cleans it under normal execution, but not under disconnect.

Greatly improved by latest commit


Where is WDK for Windows 10, version 1909 ?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 3f5d6d55-cb35-6b9e-f8ca-e75f70826dc3
Version Independent ID: 1d913516-b1e0-e872-c337-e30ed5dffce8
Content: Previous WDK versions and other downloads - Windows drivers
Content Source: windows-driver-docs-pr/other-wdk-downloads.md
Product: windows-hardware
Technology: windows-devices
GitHub Login: @EliotSeattle
Microsoft Alias: eliotgra

Hi @sonic74.
There was  no Windows 10, version 1909 of the WDK that was released. The last publicly released version was Windows 10, version 2004. Before that was Windows 10, version 1903.
You can find other test builds of the WDK by joining the Windows Insider Program, including WDK Insider Preview builds.
Hope this helps.
Eliot


python312Packages.ffmpy: 0.4.0 -> 0.5.0

Diff: https://github.com/Ch00k/ffmpy/compare/refs/tags/0.4.0...0.5.0

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

nixpkgs-reviewd workflow started
Status and logs: https://github.com/nixosbrasil/nixpkgs-reviewd/actions/runs/12475963236

nixpkgs-reviewd workflow finished
Status and logs: https://github.com/nixosbrasil/nixpkgs-reviewd/actions/runs/12475963236
nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 367775

x86_64-darwin

:fast_forward: 4 packages marked as broken and skipped:

private-gpt
private-gpt.dist
python312Packages.private-gpt
python312Packages.private-gpt.dist

:x: 6 packages failed to build:

python312Packages.gradio
python312Packages.gradio-client
python312Packages.gradio-client.dist
python312Packages.gradio-pdf
python312Packages.gradio-pdf.dist
python312Packages.gradio.dist

:white_check_mark: 6 packages built:

python312Packages.ffmpy
python312Packages.ffmpy.dist
python313Packages.ffmpy
python313Packages.ffmpy.dist
zotify
zotify.dist

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 367775

aarch64-darwin

:fast_forward: 4 packages marked as broken and skipped:

private-gpt
private-gpt.dist
python312Packages.private-gpt
python312Packages.private-gpt.dist

:x: 2 packages failed to build:

python312Packages.gradio
python312Packages.gradio.dist

:white_check_mark: 10 packages built:

python312Packages.ffmpy
python312Packages.ffmpy.dist
python312Packages.gradio-client
python312Packages.gradio-client.dist
python312Packages.gradio-pdf
python312Packages.gradio-pdf.dist
python313Packages.ffmpy
python313Packages.ffmpy.dist
zotify
zotify.dist

nixpkgs-review result
Generated using nixpkgs-review.
Command: nixpkgs-review pr 367775

x86_64-linux

:fast_forward: 4 packages marked as broken and skipped:

private-gpt
private-gpt.dist
python312Packages.private-gpt
python312Packages.private-gpt.dist

:x: 6 packages failed to build:

python312Packages.gradio
python312Packages.gradio-client
python312Packages.gradio-client.dist
python312Packages.gradio-pdf
python312Packages.gradio-pdf.dist
python312Packages.gradio.dist

:white_check_mark: 6 packages built:

python312Packages.ffmpy
python312Packages.ffmpy.dist
python313Packages.ffmpy
python313Packages.ffmpy.dist
zotify
zotify.dist

gradio is broken on master https://hydra.nixos.org/build/282676642


Is there any way to apply a LORA to the model?

cogvideox fun has a method for generating loras.  Is there a possibility that you could add an apply lora function?

Sure, once I get access to a LoRA...haven't seen one yet.

https://github.com/kijai/ComfyUI-CogVideoXWrapper/commit/3efe90ba35467fb0ad92d97676d3ee140d872c2f
Woah! Are there actually LORAs available to try somewhere, or is this aspirational support?

https://github.com/kijai/ComfyUI-CogVideoXWrapper/commit/3efe90ba35467fb0ad92d97676d3ee140d872c2f
Woah! Are there actually LORAs available to try somewhere, or is this aspirational support?

Not publicly yet, as far as I'm aware, I got access to one to test the support (fun version lora) and it does work.


Update README.md: new header, text clean up, fix broken links

What's changed?

Updated header section. Added quick nav with discord link, badges:

Slightly updated Table of contents to be more readable
Fixed some links that lead to non-existing files (e.g., ./docs/llm-connections.md, whereas ./docs/how-to/llm-connections.md is correct). Added links to the Documentation website where possible.
Fixed formatting for the document so it is more user-friendly and easier to navigate

Possible small improvements for the future:

Discord badge can also be added. But widget support needs to be turned on in Discord channel settings for it to work.
Content in the main page for the website documentation (./docs/index.md) can also be updated the same way.

Love it! So good!


How to allow ExpectSet multitimes/Anytimes?

Sometimes, it's not care about the times of call ExpectActions, how to allow it anytimes?

Curious about this as well. It seems like this functionality isn't implemented, looking at the struct here? Or did I just miss something?

Looking forward to this feature. No requirement to anticipate the expiration time duration since my tested function computes the expired key's duration. Any updates available?


Initial PR

Pull Request
What does this PR do?

Add initial support for flutter through some helper methods and widgets
Add all the pagination techniques mentioned in this guide
More details to be added...

PR checklist

[x] Text highlighting helpers
[ ] General-purpose Query Builders

[x] Offset-based (offset + limit)
[ ] Page-based (page + hitsPerPage)

[ ] Infinite Scrolling ListView (using offset-based query builder)
[ ] Previous-Next ListView (using offset-based query builder)
[ ] PaginatedDataTable & DataTableSource helpers (using page-based query builder)
[ ] MeilisearchBar a search bar that displays results that can be selected

Example App Gifs

Infinite scroll list that gets data from 2 indexes, one for arabic, one for english book titles. (using multi-index query)

mostly the tests and whether there are missing scenarios /documentation


Failure when using with axum-login

My project uses axum-login for authentication, but to do this I need to pass axum_login::extractors::AuthContext around all server functions for authentication. To do this, I need to make custom handlers:
    async fn server_fn_handler(auth_context: AuthContext, path: Path<String>, headers: HeaderMap, request: Request<AxumBody>){
        handle_server_fns_with_context(path, headers, move |cx| {
            provide_context(cx, auth_context.clone());
        }, request).await;
    }

    async fn leptos_routes_handler(auth_context: AuthContext, Extension(options): Extension<Arc<LeptosOptions>>, req: Request<AxumBody>) -> Response{
            let handler = leptos_axum::render_app_to_stream_with_context((*options).clone(),
            move |cx| {
                provide_context(cx, auth_context.clone())
            },
            |cx| view! { cx, <TodoApp/> }
        );
        handler(req).await.into_response()
    }

and handle all routes with them:
        .route("/api/*fn_name", post(server_fn_handler))
        .leptos_routes_with_handler(routes, get(leptos_routes_handler) )

However, whenever I serve and load a page, my browser will fail to load the page, with this error in the cli:
thread 'tokio-runtime-worker' panicked at 'Session handle still has owners.: RwLock { mr: 536870911, s: Semaphore { permits: 536870911 }, c: UnsafeCell { .. } }', /home/jo/.cargo/registry/src/github.com-1ecc6299db9ec823/axum-sessions-0.4.1/src/session.rs:338:49

Unsure what's happening here, is leptos improperly holding onto something when it shouldn't?

So the panic is coming from axum-sessions here: https://github.com/maxcountryman/axum-sessions/blob/4c7480a205953abbc89bbef56391af2ac94533af/src/session.rs#L346-L348
It looks like they are implementing Clone using an Arc, but assuming that there is actually only one clone of something in existence. I'm pretty surprised to see something like this in a Rust library, honestly.
This is pretty incompatible with the way context works in Leptos: when you provide_context the framework holds onto a copy of something, and when you use_context it clones it.
I'm going to close this because I don't think it really has anything to do with Leptos, but with the way they've chosen to implement this library.

Closing, issue with axum-sessions

I'm pretty surprised to see something like this in a Rust library, honestly.

I'm not sure what's so surprising about interior mutability, however I do think there are ergonomic problems with this API. There is a reason for this however, so for future readers who stumble upon this: the underlying crate at issue here is async-session which has made the unfortunate choice to implement Clone such that it destroys the cookie value when clone is called. There aren't many great ways to work around this since the library authors haven't provided an escape hatch. Thus we're leveraging interior mutability here to avoid calling clone directly (because again that destroys the cookie value). It's actually not such an uncommon pattern in Rust, but the issue here is that it might be surprising if you aren't aware you need to manage the lock (which seems to be the issue in this case).

I'm not sure what's so surprising about interior mutability
What I specifically mean is surprising here is providing a type that implements Clone that panics on use if it has in fact been cloned, without documenting that it panics, as far as I can tell.

The pattern I would've expected here, given what you're describing, is something like Arc<RwLock<Option<_>>> using .write() to get a write lock and .take() to take the value out of the Option, rather than asserting that the Arc only has one living reference (in which case, what's the point of the Arc...?)

the issue here is that it might be surprising if you aren't aware you need to manage the lock (which seems to be the issue in this case).

It might be useful to document how this is supposed to be done... I can't find any discussion of managing a lock anywhere in the axum-login or axum-session docs.

Please feel free to open an issue or better yet a PR.


Expand the data model

As GBIF explores capabilities with a new data model we wish to produce exemplar datasets that demonstrate the output using the IPT.
This is an evolutionary change from the current model and removes the constraints of the star-schema inherent in the DwC-A. It is desirable to minimise the impact of these changes to the user community, and so we are looking to adapt the IPT in a manner that will remain familiar.
It is envisaged that:

The table schemas for the new model will be available, in a similar manner to those on rs.gbig.org today. The format of these may change (e.g. JSON, RDF or Avro Schema), but the initial implementation could use a similar XML to minimise changes if preferrable.

A user of the IPT will be able to upload spreadsheets, or connect to a database as they do currently

During data mapping, the user can select the target table to map data to in a similar manner to the current core and extension. The difference however, is that the table arrangement may not be in a star-format

On data publishing the IPT will prepare a Zip file (initially) containing the converted CSV files with header rows, an EML file as it does today, and a meta file that describes the relationships between the tabular data. In the first implementation, we should prepare this meta file in the Frictionless data package format. This may be revised to e.g. the W3C CSV on the web format or even Avro formats as explorations develop.

During the archive generation, the IPT will continue to perform key validation checks, including the existing and uniqueness of the necessary IDs, and checking the referential integrity of the relationships.

An installation of this branch (v3) of the IPT be available for those working on the data model to test.

Nice! Happy to see this on the roadmap and happy to see so many improvements in the IPT by @mike-podolskiy90.

It would be great if the IPT would then also allow to generate ColDP which is for most parts very close to frictionless data. There is even a frictionless tabular-data-package generated by the API that contains all possible fields for all possible entities.
Contrary to DwC-A ColDP does not use a semantic mapping of the data files but instead uses column headers and filename conventions to identify the terms/entities.

5. During the archive generation, the IPT will continue to perform key validation checks, including the existing and uniqueness of the necessary IDs, and checking the referential integrity of the relationships.

This is relevant for a question we received through the portal feedback, so it is great to see it will be incorporated in the new data model.


Sentry plugin does not work. The annotation is not picked up

I followed these steps exactly, even adding the "demo mode". Then I added the sentry.io/project-slug annotation to my component config. I also made another unrelated change to the config to make sure I knew the changes are being picked up. But somehow I am still seeing The sentry.io/project-slug annotation is missing. You need to add the annotation to your component if you want to enabled for this tool. wherever the Sentry data should be.
Expected Behavior
I expected to see the Sentry demo data
Current Behavior
I see The sentry.io/project-slug annotation is missing. You need to add the annotation to your component if you want to enabled for this tool.
Possible Solution
No idea
Steps to Reproduce
Follow these steps from the sentry-plugin NPM package, enabling "demo mode"

I've downgraded to 0.3.13 and it's working now

Hmm - that's very strange as it's only dependencies that got bumped.
Did you use yarn backstage-cli versions:bump to bump the sentry package or did you just do it manually?

I did it manually. Deleted the lockfile and changed the package.json version to be exactly 0.3.13.

Yeah I think the issue is that you would need to bump other dependencies too. Editing the lockfile could cause duplicate packages to be brought in, which could cause unexpected things to happen. If you use the yarn backstage-cli version:bump command it will do this de-dupe for you.

Try with that command and see if it helps

You might have missed this comment I left below the main message: https://github.com/backstage/backstage/issues/6429#issuecomment-878502128
Everything worked after I downgraded one patch version from the latest by editing the package.json file and regenerating a new lockfile (I didn't hand-edit the lockfile).

I'm seeing something similar with 0.3.18
is there some way to debug this further?
Interestingly, i'm also getting a similar error with the Sonar plugin too - perhaps theres something more fundemental going on?

I stand corrected - I'm not sure why, how or otherwise, but the plugin is now operational. I wish i could tell you what changed, but alas no such luck.
Sorry for the false negative.

@bob-bins Are you still having this issue even with recent versions of Backstage?

It has been working for me after I downgraded plugin-sentry from 0.3.14 to 0.3.13 and redeployed. Not sure what the real root cause was however

I had a similar issue with another plugin (sonarqube), and running yarn install to make sure all dependencies are set up properly did the trick.

I have a feeling this is due to multiple versions of @backstage/plugin-catalog-react. We'll likely bring in cross-version compatibility like we've done for @backstage/core-plugin-api, but for now the fix is to deduplicate the installations of that package, with yarn resolutions if needed.


data_recorder

Depends/supersedes #111
Working data recorder using record_data:=true.

undraft & merge when #111 is merged

@linhdoan8

why are there so many new files included in this PR when its only about the data recorder? @voshch
@tuananhroman check pls

why are there so many new files included in this PR when its only about the data recorder? @voshch @tuananhroman check pls

changed the base branch, only contains data recorder now


Move crd examples from kubeedge repo

Signed-off-by: fisherxu<EMAIL_ADDRESS>The crd will be removed in kubeedge repo.

/lgtm
/approve


Suggestions for changes

I just created a PR to trigger the CI.
You decide what to change and what not.

gonna merge them first for the rest parts


Pin version of schematic in build workflow

Describe the bug
Schematic is changing very rapidly, and it's hard to ensure that it functioning consistently for users on production (i.e. production is currently broken due to the workflow pulling in a newer version of schematic that is not compatible). We should pin to a specific version or commit to avoid automatically pulling in the latest version when building, and only update the pin when it has been tested on develop. This will also make rollbacks easier as the commit/version will be stored in git history.

[ ] Determine best SHA to pin schematic for balance of least breakage and compatibility with model, DCA. This is somewhat difficult to figure out with schematic's lack of real versioning and a clear changelog, so spending too much time going through the issues and commit history. It seems that currently lists do not work at all. The NF model went back to list validation to accommodate a submission (https://github.com/nf-osi/nf-metadata-dictionary/pull/140), but that ended up with us between a rock and a hard place: Any single-value entries will fail validation because https://github.com/Sage-Bionetworks/schematic/issues/653 is still unresolved. But actual comma-delim list values also don't work, as documented both by you and Lingling: https://github.com/Sage-Bionetworks/schematic/issues/699 https://github.com/Sage-Bionetworks/schematic/issues/703. So manifests with lists can neither be newly submitted or updated.
[ ] Update workflow at https://github.com/nf-osi/NF_data_curator/blob/develop/.github/workflows/shinyapps_deploy.yml#L55

Could we consider this closed since we are now on the pypi releases?


added authorization for inteli api

Summary

added authorization for inteli api

Did you check if this uses the correct token so that we dont exhaust tokens?


scroller: invert list based on reading direction

Fixes LAND-1353.
This changes our logic about whether or not to invert the list. Rather than relying on load direction, we make the invert/uninvert decision base on reading direction (whether the user perceives to be scrolling up or down). If scrolling up, we invert. If scrolling down, we "uninvert". This avoids jumpiness associated with scrolling "up" in an uninverted state, or "down" in an inverted state (see https://github.com/TanStack/virtual/issues/659). We also assume that if a user has scrolled directly to a message they are going to begin reading "down" from that message.
This also moves a few variables around for easier readability, and moves a few variables into useMemo where it seems appropriate.
Tested on local dev moons on livenet.
PR Checklist

[ ] Includes changes to desk files
[x] Describes how you tested the PR locally (test ship vs livenet)
[ ] If a new feature, includes automated tests
[x] Comments added anywhere logic may be confusing without context

Moving to draft while I diagnose a bug when exiting/re-opening thread pane.

There is a jump I get consistently when a thread view is closed. The scroll position in the main conversation jumps to an older point, presumably the oldest that was fetched so far.
(We reviewed this on a call, just adding here to ensure we test that before merging)

I detected something that may be related to the thread open/close issue, or at least has a similar result.
On desktop,
If I open a chat with unreads,
click on the "n new messages since xyz. - Click to view"
takes me to the first new message (which - separate issue -  doesn't display in a consistent position, but in one case mid screen and in another near the bottom)
Then as soon as I start scrolling to continue reading below the oldest new one, the conversation jumps way back.

Ok, I think we're back in a state where someone else should try banging on this to find if there are any remaining issues.

@patosullivan here's a clip of the issue I was trying to describe in standup. Steps to reproduce:

Be scrolled back in the history and start scrolling down (towards newer messages)
Stop and initiate a scroll back up (towards older messages)

You can sometimes see oscillating behavior when the scroller is fighting your movement, other times it stays in place and doesn't move despite the trackpad input (see first & last pause in the video).
I think the real world case where this might occur is if you're scanning chat history quickly and you pass a message you actually want to read/focus on. Seemed to present even with a longer, more natural interval between switching directions than what's demoed.
https://github.com/tloncorp/landscape-apps/assets/90741358/15e5a8f6-2353-49e5-9355-96ab033651cc

I'm having a hard time reproducing the frozen state that @latter-bolden found, might need to pair to see exactly how to trigger that one.
I've been looking at how to reduce the jitter on the flip between inverted/uninverted, not coming up with much as of yet. Will keep trying.

Testing the latest locally, on desktop, I've noticed a new issue.

Go to a chat channel
Using the scroll wheel or equivalent gesture (with a magic mouse, that is vertical swipe with one finger)
Perform a short movement (I swipe vertically the equivalent to 200-300 px or the hight of 4-5 lines of text) and let go, to allow momentum to carry the motion until it stops.
Once it stops, perform the oposite scroll movement, again swiping vertically for a short distance, and let it stop
One more time, now in the oposite direction, perform a short scroll movement

After two momentum / full-stop cycles in oposite directions what I observe is that the third one stops abruptly after a fraction of a second and makes two or three short jumps back and forth.
I think letting the momentum stop by itself is the trigger


ALM Toolkit: 5.0.28

ALM Toolkit freezes when we try to "Update". App freezes upon update action.
We are using an Azure VM. I can provide more details if needs be, but stability on our VM seems to be an issue.
Has anyone else reported the app not responding upon attemepts to update?

Hi Julian, the most common cause of this type of experience is the TOM transaction is taking a long time. There can be various reasons for this like ongoing queries or other reasons. Generally, if you wait, it will go through. If you want to confirm this is the cause and the issue is repeatable, you can - instead of committing the update by clicking the Update button - script it out to a TMSL script using the Script button - and run the script from SSMS. This is basically the same as what ALM Toolkit is doing, but it may provide a bit more diagnosis info by running from SSMS.


Feature: new environment variable resolvers

Checklist

[ ] Affected Issues have been mentioned in the Closing issues section
[ ] Documentation has been written/updated
[ ] PR title is ready for inclusion in changelog

This adds 3 new resolvers to the API for interacting with environment variables

addOrUpdateEnvVariableByName
getEnvVariablesByProjectEnvironmentName
deleteEnvVariableByName

These resolvers are mostly designed to be used by lagoon-cli to make less calls against the API to retrieve this information.
Examples for adding or updating a variable on an environment and a project:
# add `environment` to add to environment only
mutation addOrUpdateVariableInEnvironment {
  addOrUpdateEnvVariableByName(
    input: {
      project: "high-cotton"
      environment:"Develop"
      name: "MY_VARIABLE"
      value: "my variable value"
      scope: GLOBAL
    }
  ) {
    id
    scope
    name
    value
  }
}
# omit `environment` to add to project only
mutation addOrUpdateVariableInProject {
  addOrUpdateEnvVariableByName(
    input: {
      project: "high-cotton"
      name: "MY_VARIABLE"
      value: "my variable value"
      scope: GLOBAL
    }
  ) {
    id
    scope
    name
    value
  }
}

Closing issues
closes #2556
closes #1677

If these are "are mostly designed to be used by lagoon-cli" what if put them behind a cli field? The purpose would be to wall off a section were we can be looser with BC concerns since we control both sides. I suggested a similar tactic for the UI parts. I'm a bit hesitant to add these new resolvers to the main query when we already have issues in the queue that would likely require BC breaks for the proposed schema changes.

They might be mostly designed for the CLI, but they are easier to use than the existing ones are in terms of user convenience. I flagged the existing ones as deprecated, but not removed them so users that like to do things the hard way still can.
I'm not sure what putting them behind a cli field means here though? Can you reference the ui parts you mentioned?
I'm struggling to see why putting them under something specifically for the cli would make a difference otherwise? I haven't changed any of the existing functionality, these are just new resolvers that store data the same as the old resolvers do. They just make adding, updating, and retrieving this data easier.

I'm not sure what putting them behind a cli field means here though? Can you reference the ui parts you mentioned? I'm struggling to see why putting them under something specifically for the cli would make a difference otherwise?

https://github.com/uselagoon/lagoon/pull/3229#discussion_r930446151
So something like:
query {
  cli {
    getEnvVariablesByProjectEnvironmentName {
    }
  }
}

This schema change only addresses some of the env var issues we have. It's easier to use, but not more flexible. Moving to a sub field is one way to say "hey this section of the api is not public/guaranteed/stable." Or we mark these as deprecated/alpha right now. It's also possible we don't get to anything before v3 and worrying about deprecation doesn't matter. Just wanting to avoid a situation where we have addEnvVariable @deprecated, addOrUpdateEnvVariableByName @deprecated, addOrUpdateEnvVariableByEnvironmentType @deprecated, addOrUpdateEnvVariableNew (that's exaggerating to be fair).

I'm not sure what putting them behind a cli field means here though? Can you reference the ui parts you mentioned? I'm struggling to see why putting them under something specifically for the cli would make a difference otherwise?

#3229 (comment)
So something like:
query {
  cli {
    getEnvVariablesByProjectEnvironmentName {
    }
  }
}

This schema change only addresses some of the env var issues we have. It's easier to use, but not more flexible. Moving to a sub field is one way to say "hey this section of the api is not public/guaranteed/stable." Or we mark these as deprecated/alpha right now. It's also possible we don't get to anything before v3 and worrying about deprecation doesn't matter. Just wanting to avoid a situation where we have addEnvVariable @deprecated, addOrUpdateEnvVariableByName @deprecated, addOrUpdateEnvVariableByEnvironmentType @deprecated, addOrUpdateEnvVariableNew (that's exaggerating to be fair).

Are there more than these issues that I can find?
https://github.com/uselagoon/lagoon/issues/1538 - IMHO this is a bad idea, would work better under my proposed organization structure.
https://github.com/uselagoon/lagoon/issues/2556 - This PR addresses this issue, and makes it easy
https://github.com/uselagoon/lagoon/issues/1677 - This PR addresses this issue, and makes it easy
https://github.com/uselagoon/lagoon/issues/3208 - Part of this was a bug, but unsetting variables could be useful, these resolvers or any others that are existing can't solve this directly either.
https://github.com/uselagoon/lagoon/issues/3286 - This could be solved by using new enums with the env type in the scope global_development or such, if not with new scopes, I'm sure these resolvers could be extended easily to support environment types..
If there are other issues, I'd love to know about them to see how these resolvers need to be more flexible to support potential changes?

I don't see these as being purely-CLI (or even purely-UI, as they would be used there too) - hence no need to put them in a sub-section.
I think it's ok to add them now (given we're not rewriting existing functions - they will act as they currently do, but these ones will bring Lagoon closer to the CRUD tools (in this space anyway) that our users expect it to be?

As a consumer it would be more convenient to be able to update an env var than the process to get / delete / add.
Our use case currently is secrets rotations.


feat: onramper support

Full lib-side OnRamper support. Because they support multiple payment gateways that use different id's we need to use a more complex mapping to make sure all gateways are included in our request

@gomesalexandre Thanks for the comments, all resolved


Update landscape.yml

pre-application stage
Pre-submission checklist:
Please check each of these after submitting your pull request:

[ ] Are you including a repo_url? You need to pick the single best GitHub repository for your project, not a GitHub organization.
[ ] Does the project meet the guidelines for new entries section?
[ ] Have you included a URL for your SVG or added it to hosted_logos and referenced it there?
[ ] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[ ] Does your project name match the text on the logo?
[ ] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[ ] ~5 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

Hey @slandath - I got the build errors fixed.
You can use the link below to confirm which offerings will show for Zowe V3.
https://deploy-preview-842--omp-landscape.netlify.app/card-mode?category=app-framework-zowe-v3,cli-zowe-v3,api-mediation-layer-zowe-v3,explorer-for-visual-studio-code-zowe-v3,comprehensive-zowe-v3,explorer-for-visual-studio-code-zowe-v3,explorer-for-intelli-j-zowe-v3,api-mediation-layer-zowe-v3,app-framework-zowe-v3,cli-zowe-v3,client-sdk-node-js-zowe-v3&grouping=category

@jmertic I updated one name at the request an applicant

Excellent - please then share direct links to the lists using the template @yarille has for them to review. You can let them know we will add these once the press announcement goes live.