Race condition on error strings cleanup?

This report is inspired by https://github.com/latchset/pkcs11-provider/issues/429#issuecomment-2276242417 and https://github.com/latchset/pkcs11-provider/issues/429#issuecomment-2276265853. This report is based on my understanding of the issue and may not be correct, we didn't manage writing a simple reproducer here and now.
Looks like, when we have a default library context and a non-default one with loaded provider, the lock object is freed on freeing non-default context and attempt to lock it causes crash on exit.
Looks like it has nothing to do with OpenSSL atexit() handler as the crash is reproduced when we build OpenSSL with no-atexit.
@simo5, @Jakuje and @AdamWill may give some more details.

What I see in the backtrace is that ldap is trying to call OpenSSL when OPENSSL_cleanup() was already called. Who called it or how was the call triggered is not clear. I do not really see a bug in OpenSSL here unless it was OpenSSL itself that called the cleanup. Are you sure about no-atexit being applied properly in the last tests?

We will double-check but I'm sure the build was done with no-atexit. I don't have the env in my disposal :(

We need to find out what calls OPENSSL_cleanup() then. If it is explicitly called by an application, then either this call must be removed or ldap must not do things in atexit or fini handler.

This is happening with OpenLDAP 2.6.8 which has the following commit:
commit 1d37f7b005163b4cf531da3cb98eeade2bb7b644
Author: Howard Chu<EMAIL_ADDRESS>Date:   Tue May 7 19:47:35 2024 +0100

    ITS#10209 libldap: only use OPENSSL_INIT_NO_ATEXIT if it's defined

    Fake OpenSSL clones like LibreSSL don't support it.

    In general we will make no effort to support fake OpenSSL clones.

diff --git a/libraries/libldap/tls_o.c b/libraries/libldap/tls_o.c
index f24158252b..d5861d9339 100644
--- a/libraries/libldap/tls_o.c
+++ b/libraries/libldap/tls_o.c
@@ -225,7 +225,12 @@ tlso_init( void )
        SSL_library_init();
        OpenSSL_add_all_digests();
 #else
-       OPENSSL_init_ssl(OPENSSL_INIT_NO_ATEXIT, NULL);
+#ifdef OPENSSL_INIT_NO_ATEXIT
+#define        OPENSSL_FLAGS   OPENSSL_INIT_NO_ATEXIT
+#else
+#define        OPENSSL_FLAGS   0
+#endif
+       OPENSSL_init_ssl(OPENSSL_FLAGS, NULL);
 #endif

        /* FIXME: mod_ssl does this */

Even before that, there was the following commit:
commit e08b80e8b7534d63080bdd4a2eb2a3d46071aef8
Author: Howard Chu<EMAIL_ADDRESS>Date:   Sun Feb 18 10:57:07 2024 +0000

    ITS#9952 TLS/OpenSSL: disable use of atexit()

    This will only have any effect if libldap is the first caller to
    initialize OpenSSL, but that should be all that matters when libldap
    is part of a dynmically loaded module. It prevents the crash in the
    example cases given.

diff --git a/libraries/libldap/tls_o.c b/libraries/libldap/tls_o.c
index 6847ef33b4..9bd830c196 100644
--- a/libraries/libldap/tls_o.c
+++ b/libraries/libldap/tls_o.c
@@ -225,7 +225,7 @@ tlso_init( void )
        SSL_library_init();
        OpenSSL_add_all_digests();
 #else
-       OPENSSL_init_ssl(0, NULL);
+       OPENSSL_init_ssl(OPENSSL_INIT_NO_ATEXIT, NULL);
 #endif

        /* FIXME: mod_ssl does this */

While openldap here may be doing something wrong (not certain but they may be trying to free the default openssl context which would be a bad idea in a library), this is a race condition in openssl that tries to access a lock after already having nulled it.
That simply should not happen, regardless of whether ldap is doing good or bad thing.
If openssl_cleanup() has already been called there should be a guard that makes it a noop if it is called again.
However this is a different issue than double calling cleanup, this is the code that frees the providers trying to cleanup the default context err_strings to my reading which sound suspicious.

The cleanup done in libldap in the fini handle is not about calling OPENSSL_cleanup() but it is calling into OpenSSL. The problem is something else before that does the OPENSSL_cleanup() call, that must be investigated and the call removed (or postponed after the fini call in libldap).

@t8m if openldap where to instantiate a new libctx, would that prevent an OPENSSL_cleanup() called on the default context from messing up until that other libctx is also freed ?
If not I do not think we can get out of this, because when you have multiple libraries loaded in a process you may not have any control on what is called in what order.

Just to be clear, note that openldap is simply calling SSL_CTX_free() which is the correct thing to do in a library. So if cleanup is proceeding while there are contexts still around I think it is something OpenSSL should deal with.
One way would be to mark the context as freed so that any call into it will not crash but will also be a no-op.
Leaking an allocation for the context structure itself in case the application fails to free the context is preferable to crashes.

OPENSSL_cleanup() is NOT libctx-specific. It should NOT be called from a library. After it is called, OpenSSL must never be used again (except of any other OPENSSL_cleanup() call).
There is no indication that libldap is calling OPENSSL_cleanup() by itself. The cleanup calls you see in the backtrace are not related to an OPENSSL_cleanup() call and they are legitimate.
However, something somewhere must have called OPENSSL_cleanup() before that and that must not be done if any library or the app is still having OpenSSL calls to do. This was always the case and we will not try to patch things around here as a hackish workaround/bug fix.

I have one more thing to add. For the reason that an application might want to call OPENSSL_cleanup() before it terminates to get a clean valgrind output, it is in general a bad idea to call OpenSSL from a library destructor. So actually it is partially a libldap fault, however the best way to resolve this is to simply not call OPENSSL_cleanup() at all. It is not really necessary.

In the end it comes down to ERR_unload_strings() being called after err_string_lock has been NULLed, which happens only in err_cleanup() if I read it right, and err_cleanup() is called only in OPENSSL_cleanup().
The thing here is that I assume python correctly called OPENSSL_cleanup() before exiting but the openldap (also correctly ?) is calling SSL_CTX_free() in its atexit handler, which necessarily comes after python is done.
So either it is illegal to do anything with openssl in atexit handlers, and openldap should just leak those contexts. Or OpenSSL handles gracefully freeing contexts after OPENSSL_cleanup() has been called.
This specific string freeing crash can be worked around by checking if err_string_lock is NULL in ERR_unload_strings(), but it is just trying to swipe the dust under the mat, pretty sure something else will end up blowing up.
This is the sequence as I understand it:
Process calls OPENSSL_cleanup() [which calls err_cleanup() which set err_string_lock = NULL] and then exit().
Atexit() handler kicks in at this point and openldap calls SSL_CTX_free() and this goes down as this sequence of calls:
 - SSL_CTX_free()
   - ssl_evp_cipher_free()
     - EVP_CIPHER_free()
       - evp_cipher_free_int()
         - ossl_provider_free()
           ...
           - ERR_unload_strings()
             -  CRYPTO_THREAD_write_lock(err_string_lock)  <- BOOM

However, something somewhere must have called OPENSSL_cleanup() before that and that must not be done if any library or the app is still having OpenSSL calls to do. This was always the case and we will not try to patch things around here as a hackish workaround/bug fix.

I guess the only fix here is to make openldap leak the SSL ctx and not try to free it in an atexit() handler

Yeah, or not call OPENSSL_cleanup() from python.

Yeah, or not call OPENSSL_cleanup() from python.

This would be ineffective, openldap would blow up in some other application calling OPENSSL_cleanup() ... which is a valid thing to do for an application.

So...is there an openldap bug report?

@simo5 what openssl bindings are being used in your python environment?  I'm looking at the python cryptography package, and while they don't appear to actually call OPENSSL_init_crypto/OPENSSL_cleanup at all (not sure how they avoid all the memory leaks), But it does appear to dynamically load openssl when cryptography is imported/used.  Is it possible that the library version which python loaded differs from the one that you built for openldap?  i.e. I'm wondering if, despite the fact that you built openssl with no-atexit, that python is loading a different version that does have atexit enabled, and the loading of libldap is just resolving the symbols to the former library rather than loading what you expect?
The test would be to run the server and then check /proc//maps to see the path that libssl and libcrypto were loaded at.  If its not what you expect, a solution might be to force python to look in a different path for libcrypto/ssl so that you properly get your no-atexit version.

@nhorman I do not think it matters how OPENSSL_cleanup() ends up being called, it is totally legal for an application to call it, and the destructor openldap calls at library unload will blow up regardless of atexit() or not.
This is the issue:
https://github.com/openldap/openldap/blob/c9ab732ec18e32c58643e44a45233a6ed87830b6/libraries/libldap/init.c#L500

It just so happens that if there is a non-default provider loaded this ends up crashing. If there isn't we just do not notice, but it is wrong either way.

@simo5 an application can absolutely call OPENSSL_cleanup, as long as it is aware of the libraries that its loading, and ensure that any dependent libs are no longer using openssl at the time cleanup is called, or after, but I grant you in this situation those conditions aren't met, and OPENSSL_cleanup shouldn't be called.
that said, I don't see anywhere in pyca/cryptography that happens.  What I'm postulating is that python is loading libcrypto at some point before libldap is loaded, and is doing so from a path that differs from the location that you have your version of openssl which was built with no-atexit (note I have no idea if this is true, its just a hyptohesis).  The result of that may be that:
a) python (ne pyca/cryptography) is loading a version of libcrypto that does have atexit handlers registered, and so is calling OPENSSL_cleanup implicitly when python exits
and
b) that libldap, when loaded is resolving its libcrypto symbols to the already loaded versions from (a) - i.e. you're not actually using your no-atexit build of openssl
I'm suggesting that, if you can force the library load path of python to use your no-atexit build, you might be able to avoid this problem.  Its still a crappy situation, but if the hypothesis is right, its a reasonable workaround.

@nhorman you are barking up the wrong tree sorry.
It would be nice to be able to know what libraries you are loading from an application but it is often not possible (people use to load in glibc things like nss_ldap which the application would not be aware of at all, and generally any library that is pluggable ... like openssl with providers makes it so that it is impossible for an application to generically and categorically know all the libraries it loaded).
So given that libldap can be pulled into random applications and applications are allowed to call OPENSSL_cleanup() before exiting, the only sane thing to do is to avoid trying to touch SSL context in ldap destructors, and just let them leak if the application does explicitly free the ldap context.

So given that libldap can be pulled into random applications and applications are allowed to call OPENSSL_cleanup() before exiting, the only sane thing to do is to avoid trying to touch SSL context in ldap destructors, and just let them leak if the application does not explicitly free the ldap context.

It means that for lack of memory leaks an application should be able to provide the context managed by it, not by the library

Apps can always provide their own context via ldap_set_option(.., LDAP_OPT_X_TLS_CTX, ...) but since most apps don't do this explicitly, libldap will still have to create one itself by default.
Leaking the context should typically be OK but if any app tends to load/unload modules repeatedly during a run it will become a problem. I would expect this is not a normal usage pattern though.

As I noted in https://bugs.openldap.org/show_bug.cgi?id=9952#c5 you guys could just fix your damn cleanup functions to check for the context already being NULL before executing.

Apps can always provide their own context via ldap_set_option(.., LDAP_OPT_X_TLS_CTX, ...) but since most apps don't do this explicitly, libldap will still have to create one itself by default.

Thanks, it's a great design!

Ok, so from my re-read here, the path forward is for openLDAP to not touch SSL contexts on their cleanup path.  Is there any reason for this to remain open?

Is there any reason for this to remain open?

IMO yes, a library that requires callers to leak memory is a broken library. It's frankly shocking that a debate was needed about whether to fix things properly. Even moreso that the outcome of that debate was "no". I've been developing with this codebase since SSLeay 0.4 but if this is how you guys want to do things I think it's time to abandon this project.

I'm sorry, but I don't see how you arrive at that conclusion.  To recap my read of this issue:

An issue was reported.  It is assumed that a crash occurs because a lock (specifically the err_string_lock) is freed on the freeing of one context and then its use is attempted by an other context.

@t8m pointed out that the only way that lock gets freed is via a call to OPENSSL_cleanup.  This is observable, as there are only two calls to CRYPTO_THREAD_lock_free(err_string_lock), one in do_err_strings_init() if the a failure occurs, which would present as an unsuable library (init failure), and one in err_cleanup(), which is only called from OPENSSL_cleanup.  As such, something must have called OPENSSL_cleanup.  If not via the atexit handler  ( disabled by configuration here ), then by some other caller, possibly in a different call stack.

@simo5 indicated that the solution going forward was to modify the openldap destructor to not touch SSL objects (allowing them to leak)

So I don't see how you arrive at the conclusions that openssl mandates memory leakage, or that  the outcome of the debate was "no" (which I think I correctly interpret as your view being that the openssl devs have agreed there is an openssl bug here, and that we're not going to fix it, please correct me if I'm wrong).  I don't see that conclusion at all, rather that the openssl team has presented a counter argument to the assertion of the root cause of the issue, and there doesn't seem to have been any follow up to identifying that alternate root cause.
That said, item (3) in my mind is not a foregone conclusion.  We could absolutely pursue continuing to look for the wayward copy of libcrypto (possibly linked to or loaded by another library component of openldap, or python itself, and find where OPENSSL_cleanup is getting called prior to openldaps destructors.  I'm happy to help out with that, But it sounds like thats not whats going to happen.  And if its not, then the question stands, why is this still open?

That said, item (3) in my mind is not a foregone conclusion. We could absolutely pursue continuing to look for the wayward copy of libcrypto (possibly linked to or loaded by another library component of openldap, or python itself, and find where OPENSSL_cleanup is getting called prior to openldaps destructors. I'm happy to help out with that, But it sounds like thats not whats going to happen. And if its not, then the question stands, why is this still open?

Finding the wayward copy is pointless. It should be obvious that you can't account for all of the possible ways libcrypto gets invoked in an arbitrary process. As such, the only viable course of action is to ensure that it's safe to call OpenSSL_cleanup redundantly. And doing that isn't hard.

Finding the wayward copy is pointless. It should be obvious that you can't account for all of the possible ways libcrypto gets invoked in an arbitrary process. As such, the only viable course of action is to ensure that it's safe to call OpenSSL_cleanup redundantly. And doing that isn't hard.

You can call OPENSSL_cleanup() redundantly. That isn't the problem here. What you cannot do is to do any other
OpenSSL calls (apart from those not touching error state or any other global state) after you called OPENSSL_cleanup(). We could add guards here and there but these wouldn't be thread safe and would not really solve the problem as the only thing that such call can do safely is to immediately return failure.

The best way to fix these problems is to not call OPENSSL_cleanup() at all. The global state does not need to be cleaned up. It should not cause any memory leakage reports as the allocated memory will be linked from the global state.

@hyc I understand that you think its pointless and it seems @simo5 agrees, let me try explain why I disagree
We agree that there is A problem in which openssl cleans up it's data structures prior to some components finishing their use, yes?
As I tried to elaborate above, I'm hypothesizing that the environment in question is using two versions of openssl, one that has atexit disabled, and another that does not.  If that is the case and symbol resolution occurs in a "wrong" or "bad" order symbols may get resolved such that the version of OpenSSL_crypti_iniy that gets bound is on that does atexit registration.  In that event you will get exactly the symptom you are seeing
While I understand you think that it's impossible to know every component that uses openssl, this is in fact the bug here (hypothetically).  That multiple components are using different versions of openssl, and the application is getting bound to the unexpected/wrong version with an incompatible config.  While we can't know all the components that use openssl we can perhaps influence the version that gets selected in such an environment, allowing for the use of an acceptable build time config.
If you don't want to explore that, then I'm back to my original question, why are we still working on this issue?  If you just want the openssl cleanup model changed to support this, that's fine, we have plans to remove at exit in 4.0 (it's going to be abi breaking so it has to wait for a major release).  I'm on my phone now so I can't look it up, but I can post the trackig issue here tomorrow.  At which point this seems like a duplicate to me.
Or you can just follow the proposed solution @simo5 proposed and let the SSL_CTX objects leak.  In my view it seems worthwhile to pursue my hypothesis to avoid that leakage, at least until the atexit code gets removed.
But Its your all's choice.  Let us know what you would like to do.

As I noted (here)[https://github.com/openssl/openssl/issues/25294#issuecomment-2392549785], I have a hypothesis for whats going on here, specifically that, while openLDAP builds to link against an openssl version that has atexit use disabled, another linked component (python IIRC), builds and links against an openssl version that still has atexit use enabled, leading to this issue.  A possible solution would be to simply change the python build configuration such that is disables atexit usage in openssl as well, which would solve the problem without having to diverge from upstream.  There doesn't seem to be interest in pursuing that approach however, so you are likely stuck with having to carry a custom patch, at least until we remove atexit usage entirely, which needs to happen on a major release boundary, and all components need to update to it.

@nhorman sorry but your hypothesis is provably wrong as we can reproduce on a system with a single dynamically linked OpenSSL. The analysis is clear that what is happening is that openldap in their destructor calls SSL_CTX_free() on a libctx after OPENSSL_cleanup() has already been called. So please let's not keep advancing theories. Let's decide what is the behavior OpenSSL expects so we can move on.
Note that removing atexit() is not sufficient to avoid all cases, because an application can be calling OPENSSL_cleanup() before exit() and OpenLdap libraries would then still call SSL_CTX_free() in their library destructor that necessarily executes after exit().
So either OpenSSL wants to make SSL_CTX_free() safe to call after OPENSSL_cleanup() or we must remove any call to OpenSSL functions in any destructor anywhere.

Do I understand right the libldap calls SSL_CTX_free() from library _fini() (a.k.a. library destructor)?
if it is the case, then this is not a race condition. It's rather matter of unfortunate ordering. If I remember correct the libc calls atexit handlers first (this is where the call to OPENSSL_cleanup() happens. Once all registered atexit handlers are done, the libc is going to call library destructors, this is where SSL_CTX_free() is coming from. call to SSL_CTX_free() comes after call to OPENSSL_cleanup() in this case. I've found it while I was trying to replace atexit() with library destructors (call OPENSSL_cleanup() from _fini()). But moving call to OPENSSL_cleanup() to library destructor (_fini()) invites also it's problems. The plan is to leave it as it is in 3.x versions.
In my opinion the only way how to sort it out is not to use atexit to call OPENSSL_cleanup() at all. And let application to call it explicitly if wants to do so.

@Sashan @nhorman The problem with OpenLDAP will happen not just if the OPENSSL_cleanup() is called within atexit() but also when OPENSSL_cleanup() is called from an application with OpenLDAP library being still loaded because if the OpenLDAP library unload happens after that, it will only at that point try to call SSL_CTX_free() which is already an invalid call.
So yeah @simo5, for safety reasons OpenLDAP cannot really call SSL_CTX_free() in the library destructor if it is to be used in such apps.
I believe this requires two things to clean this mess up:

OpenLDAP should use its own OSSL_LIB_CTX for that SSL_CTX
We need to change OPENSSL_cleanup() to be a no-op if there are existing non-default OSSL_LIB_CTX contexts that are not released yet.

What openldap can do is to opt-out from atexit. Calling OPENSSL_init_crypto(OPENSSL_INIT_NO_ATEXIT) at the beginning should prevent libcrypto from arming atexit handler. The openldap then can call OPENSSL_cleanup() from its library destructor. This is perhaps the solution which fits best to the current situation.

What openldap can do is to opt-out from atexit. Calling OPENSSL_init_crypto(OPENSSL_INIT_NO_ATEXIT) at the beginning should prevent libcrypto from arming atexit handler. The openldap then can call OPENSSL_cleanup() from its library destructor. This is perhaps the solution which fits best to the current situation.

No @Sashan, unfortunately that does not solve anything. As @simo5 and I said the same problem happens if an application calls OPENSSL_cleanup() explicitly before OpenLDAP is unloaded. Which can very well happen as OpenLDAP is used for example by the nss_ldap module.

@Sashan it's be nice if you red other messages fully before proposing "solutions".
That last thing libldap should do is call OPENSSL_cleanup() itself and break the other 50% of applications that do not blow up today.
@t8m has provided the only reasonable solutions if libldap for some reason needs to clean up the context.
But libldap should probbaly stop calling SSL_CTX_free() at all.
Libldap has no way to know if other libraries still are using the default context in their destructors as well, only a context allocated by the library itself for exclusive use is safe to free in a destructor as a general rule. Any context provided by the application should also not bee freed, because the application may still be using it after it calls dlclose() on the ldap library (unless the contract is that the application relinquish all control on the ctx after it gives it to openldap).

simo5@ sorry I was following my path on how to make sure OPENSSL_cleanup() is not invoked before library destructors. I agree libldap should not be calling OPENSSL_cleanup(). I think my suggestion to use OPENSSL_INIT_NO_ATEXIT is valid thing, which is worth to try.

We already disabled OpenSSL's use of atexit, back in February. Why are we still having this conversation?
https://git.openldap.org/openldap/openldap/-/commit/a5953812f0c03e802e61109ae18e8fed5f3f2df8
As for nss_ldap - that was deprecated over a decade ago. Distros should be using nss-pam-ldapd.

We already disabled OpenSSL's use of atexit, back in February. Why are we still having this conversation?

because this works if and only if there is a single call to OPENSSL_init_crypto()/OPENSSL_init_ssl() from the application. Consider a case where the application depends on more shared libraries (libldap + libwhatever). If those libraries further depend on libcrypto, then we are heading to a situation where libwhatever may override libldap's decision not to arm atexit, by simply calling OPENSSL_init_crypto() with its own set of flags.
we are just afraid to change the unfortunate behavior in 3.x release.


root: enable davix and ssl support

This patch enables CERN ROOT to open file urls with scheme davs:// and https://. The above two functionality of the updated root package are tested on CERN LXPLUS.
The inclusion of openssl in buildInputs will increase neither the closure size nor the rebuild frequency, since python3, which depends on openssl, is already a dependency.
This simple change significantly increases the usability of the root package toward remote file access.
Description of changes

Things done

Built on platform(s)

[X] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[X] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[X] Tested basic functionality of all binary files (usually in ./result/bin/) (Tested on CERN LXPLUS.)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Let's revert to the previous, simpler version.

Result of nixpkgs-review pr 200096 run on x86_64-darwin 1

6 packages built:

hepmc3
python310Packages.hepmc3
python39Packages.hepmc3
root
xfitter
yoda-with-root

Would be nice to get xrd also working in ROOT.

Need to fix the xrootd package first (#169677). There might be some hidden FHS assumptions, or something I don't understand.


How to implement??

The instructions you put in the readme aren't working for me.
Specifically, in the CircularSeekBar.java, the line, "package com.devadvance.circularseekbar;" gives me the error "package does not correspond to the file path (my project path)..."
and this " xmlns:app="http://schemas.android.com/apk/res/com.devadvance.circulartest"" in the xml isn't working either.
What am I doing wrong?
Also in CircularSeekBar.java, the word "boolean" is red, says "cannot resolve value"
/**
* Set whether the pointer locks at zero and max or not.
* @param boolean value. True if the pointer should lock at zero and max, false if it should not.
*/

having same issue

After deining your class with your project package name you can use this in your xml main Layout ``  xmlns:app="http://schemas.android.com/apk/res-auto"


ci: fix retest command

ref: https://github.com/envoyproxy/envoy/blob/main/.github/workflows/command.yml
fixes: https://github.com/envoyproxy/gateway/issues/2339

/retest

/retest

I'd like merge this first to verify if everything works as expected.


no-unused-css can't handle @media

I've encountered an error regarding no-unused-css rule.
It seems as if the rule wouldn't be able to handle @media calls in the scss files.

Versions:

angular: 6.0.3
codelyzer: 4.4.2

Steps to reproduce:

Create new angular project,
add codelyzer,
add no-unused-css rule,
change app.component.css to scss,
add @media rule,
make some change in app.component.ts to trigger tslint

Simple reproduction -> stackblitz (of course it wouldn't show ts-lint errors)

The rule is experimental and the chances are that we'll remove it at some point, or completely rewrite it.
I'll close this issue for now and consider it when we decide how we'll proceed.


Installation issue

Collecting nvidia-cublas-cu12==<IP_ADDRESS>
Downloading nvidia_cublas_cu12-<IP_ADDRESS>-py3-none-win_amd64.whl (439.9 MB)
---------------------------------------- 439.9/439.9 MB 1.6 MB/s eta 0:00:00
Collecting nvidia-cuda-cupti-cu12==12.1.105
Downloading nvidia_cuda_cupti_cu12-12.1.105-py3-none-win_amd64.whl (10.2 MB)
---------------------------------------- 10.2/10.2 MB 2.7 MB/s eta 0:00:00
Collecting nvidia-cuda-nvrtc-cu12==12.1.105
Downloading nvidia_cuda_nvrtc_cu12-12.1.105-py3-none-win_amd64.whl (19.3 MB)
---------------------------------------- 19.3/19.3 MB 2.5 MB/s eta 0:00:00
Collecting nvidia-cuda-runtime-cu12==12.1.105
Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-win_amd64.whl (821 kB)
---------------------------------------- 821.2/821.2 kB 3.5 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement nvidia-cudnn-cu12==<IP_ADDRESS> (from versions: 0.0.1.dev5, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>)
ERROR: No matching distribution found for nvidia-cudnn-cu12==<IP_ADDRESS>
[notice] A new release of pip is available: 23.0.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
(myenv) C:\Users\Administrator\Documents\GitHub\datadoc>streamlit run app.py
You can now view your Streamlit app in your browser.
Local URL: http://localhost:8501
Network URL: http://<IP_ADDRESS>:8501
2024-03-21 11:12:42.579 Uncaught app exception
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 534, in run_script
exec(code, module.dict)
File "C:\Users\Administrator\Documents\GitHub\datadoc\app.py", line 3, in
from main import get_llm_response
File "C:\Users\Administrator\Documents\GitHub\datadoc\main.py", line 2, in
from embed_data import chroma_db_store
File "C:\Users\Administrator\Documents\GitHub\datadoc\embed_data.py", line 4, in
from langchain_community.document_loaders import DirectoryLoader
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain_community\document_loaders_init.py", line 190, in
from langchain_community.document_loaders.sql_database import SQLDatabaseLoader
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain_community\document_loaders\sql_database.py", line 10, in
class SQLDatabaseLoader(BaseLoader):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain_community\document_loaders\sql_database.py", line 22, in SQLDatabaseLoader
query: Union[str, sa.Select],
AttributeError: module 'sqlalchemy' has no attribute 'Select'
Thanks for help in advance!

I have fixed some dependencies conflicts can you try again? Also mention which device you are using.
Don't use cached packages from your system.

Clear the cache: pip cache purge
Install packages: pip install -r requirements.txt
Or Directly use this: pip install -r requirements.txt -no-cache-dir

Thanks for you quick reply. However I got the same error message as above. I am using win10 PC without Nvidia GPU. It seem your app require this to run?
Collecting nvidia-cuda-runtime-cu12==12.1.105
Downloading nvidia_cuda_runtime_cu12-12.1.105-py3-none-win_amd64.whl (821 kB)
---------------------------------------- 821.2/821.2 kB 5.8 MB/s eta 0:00:00
ERROR: Could not find a version that satisfies the requirement nvidia-cudnn-cu12==<IP_ADDRESS> (from versions: 0.0.1.dev5, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>, <IP_ADDRESS>)
ERROR: No matching distribution found for nvidia-cudnn-cu12==<IP_ADDRESS>
[notice] A new release of pip is available: 23.0.1 -> 24.0
[notice] To update, run: python.exe -m pip install --upgrade pip
(myenv) C:\Users\Administrator\Documents\GitHub\datadoc>streamlit run app.py
You can now view your Streamlit app in your browser.
Local URL: http://localhost:8501
Network URL: http://<IP_ADDRESS>:8501
2024-03-21 15:15:15.500 Uncaught app exception
Traceback (most recent call last):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 534, in run_script
exec(code, module.dict)
File "C:\Users\Administrator\Documents\GitHub\datadoc\app.py", line 3, in
from main import get_llm_response
File "C:\Users\Administrator\Documents\GitHub\datadoc\main.py", line 2, in
from embed_data import chroma_db_store
File "C:\Users\Administrator\Documents\GitHub\datadoc\embed_data.py", line 4, in
from langchain_community.document_loaders import DirectoryLoader
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain_community\document_loaders_init.py", line 190, in
from langchain_community.document_loaders.sql_database import SQLDatabaseLoader
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain_community\document_loaders\sql_database.py", line 10, in
class SQLDatabaseLoader(BaseLoader):
File "C:\Users\Administrator\AppData\Local\Programs\Python\Python310\lib\site-packages\langchain_community\document_loaders\sql_database.py", line 22, in SQLDatabaseLoader
query: Union[str, sa.Select],
AttributeError: module 'sqlalchemy' has no attribute 'Select'

Here is the same issue I found:
https://github.com/langchain-ai/langchain/issues/18528#issuecomment-1978040612
Try using sqlalchemy==2.0.0 first then try the latest one i.e sqlalchemy==2.0.28.

I followed your instructions and then installed some other dependencies, and finally I can open your app.
But after I asked a question, it run and responded with a new error like this:
OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like sentence-transformers/all-MiniLM-L6-v2 is not the path to a directory containing a file named config.json. Checkout your internet connection or see how to run the library in offline mode at 'https://huggingface.co/docs/transformers/installation#offline-mode'.
Traceback:
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\streamlit\runtime\scriptrunner\script_runner.py", line 542, in _run_script
exec(code, module.dict)
File "C:\Users\Administrator\Documents\GitHub\datadoc\app.py", line 75, in
output = generate_response(user_input, model,explain_to_kid=explain_kid)
File "C:\Users\Administrator\Documents\GitHub\datadoc\app.py", line 41, in generate_response
response = get_llm_response(prompt, model, image_path, api_key, explain_to_kid)
File "C:\Users\Administrator\Documents\GitHub\datadoc\main.py", line 11, in get_llm_response
vectordb = chroma_db_store()
File "C:\Users\Administrator\Documents\GitHub\datadoc\embed_data.py", line 32, in chroma_db_store
embeddings = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2")
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\langchain_community\embeddings\huggingface.py", line 67, in init
self.client = sentence_transformers.SentenceTransformer(
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\sentence_transformers\SentenceTransformer.py", line 198, in init
modules = self._load_auto_model(
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\sentence_transformers\SentenceTransformer.py", line 1063, in _load_auto_model
transformer_model = Transformer(
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\sentence_transformers\models\Transformer.py", line 35, in init
config = AutoConfig.from_pretrained(model_name_or_path, **model_args, cache_dir=cache_dir)
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\transformers\models\auto\configuration_auto.py", line 1138, in from_pretrained
config_dict, unused_kwargs = PretrainedConfig.get_config_dict(pretrained_model_name_or_path, **kwargs)
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\transformers\configuration_utils.py", line 631, in get_config_dict
config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\transformers\configuration_utils.py", line 686, in _get_config_dict
resolved_config_file = cached_file(
File "C:\Users\Administrator\Documents\GitHub\datadoc\myenv\lib\site-packages\transformers\utils\hub.py", line 441, in cached_file
raise EnvironmentError
ChatGPT gave me the following instruction, but I don't know how to proceeds.
from transformers import AutoModel, AutoConfig
指定本地模型文件和配置文件的路径
model_path = '/path/to/your/local/model/file.bin'
config_path = '/path/to/your/local/config/config.json'
加载配置文件
config = AutoConfig.from_pretrained(config_path)
加载模型
model = AutoModel.from_pretrained(model_path, config=config)
Could you give me any advice? Thanks!

OSError: We couldn't connect to 'https://huggingface.co' to load this file, couldn't find it in the cached files and it looks like sentence-transformers/all-MiniLM-L6-v2 is not the path to a directory containing a file named config.json. Checkout your internet connection or see how to run the library in offline mode at

You can see it may be internet issue. Sometimes huggingface api takes time to download the models. We are using embeddings = SentenceTransformerEmbeddings(model_name="all-MiniLM-L6-v2") to create the embeddings.
Can you try it again? May be by creating whole project again. It doesn't indicate any error in the code itself.

After installed another dependency pip install chromadb and finally it works. Seem lots of dependencies needs to be included.

Chromadb wasn't in the requirements.txt file?

Yes, at least in my case. By the way I put one pdf file which A Barron magazine pdf and it seems it can't provide any answer.

Do these things:

Delete the existing chromadb folder that was present when you downloaded the repo.
Whenever you add the new files in the transcripts folder, make sure to delete chromadb folder before running the file.
Try to ask question that is present in the file. Instead of asking "what is present in the file", ask something specific related to any topic that is present in the file.

Thanks! By the way besides pdf file, does it support other format like txt or word?

Yes it supports docx,pdf,txt,md,pptx now. Pull the code to use it.


TrustRegion fix

Implementation of the suggestions made by Chris in this https://github.com/SciML/NonlinearSolve.jl/pull/132#issuecomment-1399911499

Thanks! This approach looks very nice


User avatar preview shows attachment icon instead of the image itself.

If you haven’t already, check out our contributing guidelines for onboarding and email<EMAIL_ADDRESS>to request to join our Slack channel!

Action Performed:

Login with any account
Open 2 tabs, from the first tab try to change the user avatar.
From second tab, go to any report, wait for the avatar changes, click on user to preview the avatar.
Notice that the preview modal shows the attachment icon, instead of the user avatar.

Expected Result:
It should show user avatar instead of attachment icon.
Actual Result:
The preview modal shows the attachment icon, instead of the user avatar.
Workaround:
unknown
Platforms:

Which of our officially supported platforms is this issue occurring on?

[ ] Android / native
[ ] Android / Chrome
[ ] iOS / native
[ ] iOS / Safari
[x] MacOS / Chrome / Safari
[ ] MacOS / Desktop

Version Number:
Reproducible in staging?: Needs reproduction
Reproducible in production?: Needs reproduction
If this was caught during regression testing, add the test name, ID and link from TestRail:
Email or phone of affected tester (no customers):
Logs: https://stackoverflow.com/c/expensify/questions/4856
Notes/Photos/Videos: Any additional supporting documentation
https://user-images.githubusercontent.com/43996225/229811342-4f0ed52e-ac92-473f-b3fc-77f52a4f4c35.mov
Expensify/Expensify Issue URL:
Issue reported by: @hungvu193
Slack conversation: https://expensify.slack.com/archives/C049HHMV9SM/p1680262415387269
View all open jobs on GitHub

Triggered auto assignment to @johncschuster (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

Bug0 Triage Checklist (Main S/O)

[ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
[ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)

If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead

[ ] This bug is reproducible using the reproduction steps in the OP. S/O

If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
If the reproduction steps aren't clear and you determine the correct steps, please update the OP.

[ ] This issue is filled out as thoroughly and clearly as possible

Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.

[ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

@johncschuster Huh... This is 4 days overdue. Who can take care of this?

@johncschuster Eep! 4 days overdue now. Issues have feelings too...

Triggered auto assignment to @slafortune (Bug), see https://stackoverflow.com/c/expensify/questions/14418 for more details.

Bug0 Triage Checklist (Main S/O)

[ ] This "bug" occurs on a supported platform (ensure Platforms in OP are ✅)
[ ] This bug is not a duplicate report (check E/App issues and #expensify-bugs)

If it is, comment with a link to the original report, close the issue and add any novel details to the original issue instead

[ ] This bug is reproducible using the reproduction steps in the OP. S/O

If the reproduction steps are clear and you're unable to reproduce the bug, check with the reporter and QA first, then close the issue.
If the reproduction steps aren't clear and you determine the correct steps, please update the OP.

[ ] This issue is filled out as thoroughly and clearly as possible

Pay special attention to the title, results, platforms where the bug occurs, and if the bug happens on staging/production.

[ ] I have reviewed and subscribed to the linked Slack conversation to ensure Slack/Github stay in sync

I was able to reproduce this once I removed by current profile pic and reloaded it.

Job added to Upwork: https://www.upwork.com/jobs/~0103169638f95d2d0e

Current assignee @slafortune is eligible for the External assigner, not assigning anyone new.

Triggered auto assignment to Contributor-plus team member for initial proposal review - @mananjadhav (External)

Proposal
Please re-state the problem that we are trying to solve in this issue.
Old picture appears in preview
What is the root cause of that problem?
The source state is only assigned with source props in constructor. So that, when source props is changed source state is still old value. That's why old picture appears in preview.
https://github.com/Expensify/App/blob/2d0ef06470d9f3a2c6b3692429fd7f3f78169336/src/components/AttachmentModal.js#L85
What changes do you think we should make in order to solve the problem?
We should add componentDidUpdate function to update source state when source props is changed.
What alternative solutions did you explore? (Optional)
NA
Result
Screencast from 12-04-2023 22:21:19.webm

Triggered auto assignment to @MonilBhavsar (External), see https://stackoverflow.com/c/expensify/questions/7972 for more details.

Proposal
Please re-state the problem that we are trying to solve in this issue.
Old picture appears in preview
What is the root cause of that problem?
The source state is only assigned with source props in constructor. So that, when source props is changed source state is still old value. That's why old picture appears in preview.
https://github.com/Expensify/App/blob/2d0ef06470d9f3a2c6b3692429fd7f3f78169336/src/components/AttachmentModal.js#L85
What changes do you think we should make in order to solve the problem?
We should add componentDidUpdate function to update source state when source props is changed.
What alternative solutions did you explore? (Optional)
NA
Result
Screencast from 12-04-2023 22:21:19.webm

This issue is being solved here I guess.
https://github.com/Expensify/App/issues/16528#issuecomment-1501676615

updating to weekly

The corresponding PR is deployed to production. Testing this now

Seems to have been fixed
@slafortune to be double sure, could you please test it and close this issue
https://user-images.githubusercontent.com/32012005/235474841-75161b4a-317a-4d34-b505-db7c81c78a4f.mov

Checked on staging and looks fixed to me as well.
https://user-images.githubusercontent.com/3069065/235682171-8ad61ff7-3637-4c67-b4f9-ee84ae530a44.mov

@MonilBhavsar @slafortune Still waiting on anything?

Issue not reproducible during KI retests. (First week)

Thanks!
We can close this


[d3d9] Kane & Lynch: Dead Men (8080) Unhandled render state D3DRS_ADAPTIVETESS_Y

The FOURCC value is KanL. I doubt it used somewhere in drivers, it looks like it was either for debugging purposes or to be treated as D3D9Format::Unknown (Edit: I mean "and applied with D3DRS_ADAPTIVETESS_Y". Name likely was "invented" to be unsupported format.).
Out of curiosity I created a hack to accept this format (where D3D9Format::ATOC check is in current code)
https://github.com/doitsujin/dxvk/blob/b4f2094c02bea9b241f558a1c1888ba28a88d3f0/src/d3d9/d3d9_device.cpp#L1695-L1697
and there is indeed noticeable difference (see screenshots). Actually I don't know what I'm doing, so this bug report is FYI.
Before

After

Software information

Native dlls: d3dx9_31, d3dcompiler_47, d3dx9_34.dll (present in game's installdir)
GFWL: xliveless (Bundled version failed to install)
Maxed settings (upped to AA:8/AF:16).

System information

Wine version: Proton-4.11-11 (local build)

Apitrace file(s)

'll try to do on request

Log files

d3d9.log: kaneandlynch_d3d9.log

What you linked to was just my code for ADAPTIVENESS_Y and ATOC?

IDK what KanL is meant to do, I imagine its meant to signify to the driver that this is Kane and Lynch for drive hacks.
Is there an actual problem with the game btw?

What you linked to was just my code for ADAPTIVENESS_Y and ATOC?

Yes, for context.

Is there an actual problem with the game btw?

No, the game works good.

This isn't a bug then.
If there's any difference on native drivers that's noticable, let me know.
(ie. does it look like ATOC or another feature is enabled?)


feat: Aggregator should check if gas is enough before respondToTask

Agg should be updated to check if the gas is enough to pay before sending tx

Done in #918


When generating department reports only generate for current year

I'm still not authorized to merge, so that'll have to be @tskeeley

I'm still not authorized to merge, so that'll have to be @tskeeley

You should be able to now. try again.

Can I drop this one into prod, or will it conflict with other commits?


Remove make_callback in kv.rs

It can be replaced by util::future::paired_future_callback now.

assign me.

@csmoe Seems that GitHub's assignees are limited to contributors. Don't worry, please go ahead :)

Thanks @csmoe
Wait for your PR soon.


Some problem about projection.csv amd obj_map.csv

When I run the file create_map.launch. 'projection.csv' amd 'obj_map.csv' will be deleted. why? And How should I solve this problem?

Hi. The mapping file first removes the previous teach files to create new one. You can remove the lines or I recommend change the file names after each teach run. Then you can use whatever teach path you want later.


Custom context menu disappear right after opening in preferencesEditor

Steps to Reproduce:

Set "window.titleBarStyle": "custom"
Open settings JSON editor
Left click on any ✏ next to a setting
No context menu shown

If I use right click, context menu does show.
Related to #71623

My bad. This is a duplicate of #75026

/duplicate


Jringle/cairo

This implements most of the script_ops in the scenic -> driver interface. I put #warning so that a warning pops up at compile time for the remaining unimplemented functions:
c_src/scenic/ops/cairo_script_ops.c:170:2: warning: #warning "cairo TODO: script_ops_draw_ellipse" [-Wcpp]
  170 | #warning "cairo TODO: script_ops_draw_ellipse"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:264:2: warning: #warning "cairo TODO: script_ops_draw_sprites" [-Wcpp]
  264 | #warning "cairo TODO: script_ops_draw_sprites"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:330:2: warning: #warning "cairo TODO: script_ops_arc_to" [-Wcpp]
  330 | #warning "cairo TODO: script_ops_arc_to"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:346:2: warning: #warning "cairo TODO: script_ops_quadratic_to" [-Wcpp]
  346 | #warning "cairo TODO: script_ops_quadratic_to"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:370:2: warning: #warning "cairo: TODO: script_ops_scissor" [-Wcpp]
  370 | #warning "cairo: TODO: script_ops_scissor"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:520:2: warning: #warning "cairo TODO: script_ops_fill_stream" [-Wcpp]
  520 | #warning "cairo TODO: script_ops_fill_stream"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:552:2: warning: #warning "cairo TODO: script_ops_stroke_linear" [-Wcpp]
  552 | #warning "cairo TODO: script_ops_stroke_linear"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:554:2: warning: #warning "cairo TODO: script_ops_stroke_radial" [-Wcpp]
  554 | #warning "cairo TODO: script_ops_stroke_radial"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:556:2: warning: #warning "cairo TODO: script_ops_stroke_image" [-Wcpp]
  556 | #warning "cairo TODO: script_ops_stroke_image"
      |  ^~~~~~~
c_src/scenic/ops/cairo_script_ops.c:558:2: warning: #warning "cairo TODO: script_ops_stroke_stream" [-Wcpp]
  558 | #warning "cairo TODO: script_ops_stroke_stream"
      |  ^~~~~~~

Everything else is there and ready for reviewing.

I added a few commits to fix issues in cairo I encountered while running cairo on our device with a modified version of @axelson hello_scenic_full

There are now only 6 unimplemented functions in cairo_sciprt_ops.c (PR comment updated)

I've now added some commits that allow building cairo to run on a rpi (tested on rpi3b)

https://github.com/ScenicFramework/scenic_driver_local/assets/100756/6a85af34-cb98-4585-b4f1-d1f0283de84a
https://github.com/ScenicFramework/scenic_driver_local/assets/100756/184abe9b-ec37-4aaa-b1d8-5c404871f718
https://github.com/ScenicFramework/scenic_driver_local/assets/100756/0d4dfffe-a17b-4880-8e72-538a0a8fb907
https://github.com/ScenicFramework/scenic_driver_local/assets/100756/5a9e6e14-5622-443f-bb18-8a422536e5cc
https://github.com/ScenicFramework/scenic_driver_local/assets/100756/103a997a-5e7b-4ff9-bef3-7b4cd10f1417


HWKAPM-691 Fix issue with service name not being displayed. It was du…

…e to it not being set on nodes that did not have a URI defined

@pavolloffay Could you review this one please?
Reviewer notes:

Service name was not set when the fragment completion time didn't have a URI.
Added guard on other places where the service name is set, to avoid situation where a 'null' service name overwrites a previously set value.

@pavolloffay Thanks!
Have updated the test but will keep the guard condition in case it may happen - it is not likely, but would prefer code to be resilient to the situation rather than overwrite.


[Image Preproc] Packing into a module

Main Contributor: @zhiao777774

TODO

[ ]  filter images: Some images under the same mod_organ may go thru preproc, some may not be preprocessed. Define this gate.
[ ]  Integrate @00757033  X-ray preproc
[ ]  Integrate ABD_CT preproc


Add clamav as secondary scanning mechanism

This builds on top of https://github.com/openshift/image-inspector/pull/53 for providing clam results and @pweil- https://github.com/openshift/image-inspector/pull/45 for the clam support.
I tried to merge these two together, make the clam actuall scan files in the Docker image (or container if we point it to the container root fs).
Some sample output:
# image-inspector -image docker.io/mfojtik/virus-test:latest -scan-type clamav -clam-socket /var/run/clamd.sock
2017/06/01 19:51:35 Pulling image docker.io/mfojtik/virus-test:latest
2017/06/01 19:51:39 Extracting image docker.io/mfojtik/virus-test:latest to /var/tmp/image-inspector-530387680
2017/06/01 19:51:43 ClamAv scan took 1s
2017/06/01 19:51:43 DEBUG: api.ScanResult{ImageName:"docker.io/mfojtik/virus-test:latest", ImageID:"", Results:[]api.Result{api.Result{Name:"clamav", ScannerVersion:"ClamAV 0.99.2/21723/Mon Jun 13 15:53:00 2016", Timestamp:time.Time{sec:63631936301, nsec:778170681, loc:(*time.Location)(0x8d9f40)}, Reference:"file:///virus", Description:"Eicar-Test-Signature", Summary:[]api.Summary(nil)}}}

(this actually found my hidden virus file I put inside the mfojtik/virus-test:latest image). Combining this with -post-results-url you can send the output of this test where you want to.
This also adds /results endpoint for the HTTP server where the compacted results are served as well (in case you don't want to use the POST-hook).

@simon3z @enoodle @pweil- added /results endpoint for the webdav server so you will be able to get the clam results and compacted openscap results in old fashion as well.

@pweil- i realized that we perhaps need something that indicates whether this is result of "container" scan or "image" scan. The difference between is that we don't have image ID for container scan.

The last commit is trying to refactor the code internally to separate the scanners into their own options. Also plumb the image-inspector to not be openscap centric (there is still some work there to be done ;-)

(the last commit can be a follow up, will have to plumb the tests...)

@simon3z @enoodle @pweil- added /results endpoint for the webdav server so you will be able to get the clam results and compacted openscap results in old fashion as well.

@mfojtik nice that's perfect!

Do we expect clamd to be running only on certain nodes and have these scans scheduled there?
Are we planning to use a tcp socket (if possilbe) so that clamav could be a regular service in OpenShift?

@simon3z i'm working on an example scanner pod with clam. The clamd can run inside the container and we can just point the image inspector to it's socket or we can have clamd on nodes and attach the socket via volume (maybe using the flex volume?)
service: we can run clam as http service (basically create proxy for the clamd socket).

@simon3z i'm working on an example scanner pod with clam. The clamd can run inside the container and we can just point the image inspector to it's socket or we can have clamd on nodes and attach the socket via volume (maybe using the flex volume?)
service: we can run clam as http service (basically create proxy for the clamd socket).

Nice. The two containers + flex-volume seems good (also to keep the image-inspector container slim), but the http service (proxy to unix socket) seems really cool.

OK, this is the example:
https://github.com/mfojtik/image-inspector/tree/clamav-example/examples/openshift/clamav
Basically:
[@dev] .../openshift/clamav # oc create -f clam-scan.yaml
pod "imagescan-1-rdrkz" created
[@dev] .../openshift/clamav # oc get pods
NAME                READY     STATUS              RESTARTS   AGE
imagescan-1-rdrkz   0/1       ContainerCreating   0          3s
[@dev] .../openshift/clamav # oc logs imagescan-1-rdrkz -f
LibClamAV Warning: **************************************************
LibClamAV Warning: ***  The virus database is older than 7 days!  ***
LibClamAV Warning: ***   Please update it as soon as possible.    ***
LibClamAV Warning: **************************************************
2017/06/05 12:02:23 clamd is ready to scan ...
2017/06/05 12:02:23 --> Waiting for scan results on <IP_ADDRESS>:8080 ...
2017/06/05 12:02:23 Pulling image docker.io/mfojtik/virus-test:latest
2017/06/05 12:02:26 Extracting image docker.io/mfojtik/virus-test:latest to /image-data
2017/06/05 12:02:45 ClamAv scan took 15s
2017/06/05 12:02:45 DEBUG: Posting scan results to "http://<IP_ADDRESS>:8080/" ...
2017/06/05 12:02:45 --> Scan report received, 1 problems found ...

(the binary can do whatever, annotate the image, trigger something, upload something... endless possibilities ;-)
This demonstrate that you can run clam scan in Pod, however, the pod has to be privileged and ran as root (to access docker socket) and you have to mount in service account secret and you have to grant the service account to pull any image in the cluster (image-auditor role). We can fiddle the security of this if needed, but without image-inspector switching to containers/image for pulling we will have to run as root IMHO).

This demonstrate that you can run clam scan in Pod, however, the pod has to be privileged and ran as root (to access docker socket) and you have to mount in service account secret and you have to grant the service account to pull any image in the cluster (image-auditor role).

@mfojtik really nice! Do you think you can keep the example in this repository? (As documentation)

@simon3z sure

@pweil- @simon3z @enoodle this is now ready :-)

@ilackarms @enoodle can you review?

@enoodle added clam tests (coverage: 95.8% of statements)

@enoodle @pweil- @simon3z changes:

addressed review comments
refactored the Scanner interface a little (Scan() can now return interface{} which can hold additional results (arf + html for openscap) so there is no need for scanImage() wrapper)
added clam unit test
remove --local/scan-local flag for now (it will be separate PR)
removed DEBUG: in download messages (can be discussed separately)

One remaining question here is the /results endpoint that serves the compacted JSON results. Do you think that URL path is ok?

@enoodle @pweil- @simon3z the last commit will cause that we will run the clamd as part of the scan if the -clam-socket is not specified. That allows to run this in container without doing complex process management. I would still like to have the -clam-socket option for complex scanning scenarios (custom config, shared clamd, clamd running directly on node, etc..)

@enoodle @pweil- @simon3z the last commit will cause that we will run the clamd as part of the scan if the -clam-socket is not specified.

@mfojtik any chance we can keep this decoupled in another PR? I am not sure about it... I am trying to keep the image as slim as possible (in fact it would be great to have OpenSCAP outside as well).
I have several comments on this new part but I'd keep them for the specific PR.

@simon3z np, I can keep it separate.

@simon3z @enoodle @pweil- are there any other comments for this PR or ok to merge?

none from me except....squash the clam related ones?

squashed, will wait for Federico

squashed, will wait for Federico

@mfojtik I'll try to review to details this in the next 1-2 days is that OK?

@simon3z perfectly ok, thanks!

@joelsmith FYI

OK LGTM :+1:
Waiting for @joelsmith review.

@joelsmith @pweil- @simon3z updated this to use the openshift clam-scanner package... couple follow ups:

openshift clam-scanner need to learn how to report versions
we should remove glog from the clam-scanner dependency or we asking for trouble (using two loggers)

PTAL

@joelsmith @pweil- @simon3z are we ok with this version? (not using go-clamd)
follow-ups:

report the clam version properly (should be handled upstream)
remove glog from clam scanner (to reduce num of dependencies)

no objections from me.  @joelsmith how do you feel about this direction?

Thanks for doing all of this work. 👍

@simon3z merge? or waiting in your queue? :)

@simon3z merge? or waiting in your queue? :)

@mfojtik I'd feel more comfortable merging if I had explicit approvals from @enoodle and @pweil-

The Clam part LGTM. I am afraid that something got broken with the openscap without me noticing. I wanted to build and run it but for some reason docker build fails on the yum commands. I will update when I figure it out.

update dockerfile? @mfojtik

@ilackarms which one? the image-inspector one? i don't think we want to install clamd there.


Dynamic data recibe on method ON

Hello, is there a way to generate an array if a string, JSON, or other types are received? I am creating a plugin for NativeScript and the data received is dynamic. Is it possible?

Can you describe your scenario in more detail? I haven't worked with NativeScript so don't understand how SignalR fits there.
In general this client doesn't provide access to the raw JSON received from the server (or parts of it). In SignalR, the handlers need to correspond to how the server invokes these handlers, so building something that would work for any server seems challenging.

@moozzyk  Hi, I am creating a plugin for NativeScript, and in Swift, it's almost ready. I still need to deserialize the data that arrives in on("my_event", data) and thus be able to generate an array if data is an array, a string if data is a string, and so on with each of the types. But I am using Swift, and from there I call the Swift functions from NativeScript, and I need to know how to convert that event data in Swift and pass it to a type compatible with TypeScript.

Do you control the server? If so, you could send your data as a string from the server as the argument and then decode it to a string on the client and then deal with it. Other than that I don't know of an easy way achieving what you want.

@moozzyk I don't have access to the server, and since this is a plugin, it should be possible to deserialize the received data.

Here are a couple of ideas (note I have not tried any of these so YMMV):

try using AnyDecodable type for your argument
do something similar yourself by using the ArgumentExtractor .getArgument method.
For the second option you wouldn't use the strongly typed on extensions but would have to create your own that has a customized callback that tries to decode to a different target type. For references here is how the extensions look like. If you look at the callback, it decodes for the target type. Your callback would have to try different types, similarly to what the AnyDecodable (option 1) does.

Using https://github.com/SwiftyJSON/SwiftyJSON, erasing primitive types

@jiasongs Does it work with an array of objects?

@jiasongs Does it work with an array of objects?

Yes, like this

thanks @moozzyk  and @jiasongs


Clean up of package.json scripts

Description

This PR

cleans up the scripts section of the package.json file for improved consistency and clearer hierarchy
increments the talkback port by one to prevent conflict with a macOS 12.3 service
corrects some documentation

Changes

prepare moved to the top (considering it's run first)
ver added as a generic replacement for playwright-version

dev split into predev and dev
dev:only added like build:only
build split into prebuild and build
build-only renamed to build:only (for consistency with the : notation)
prod removed (purpose unclear) restored, purpose discovered

storybook:build and storybook:smoke added as they are common actions

test:all renamed to test
test renamed to test:unit
test:watch renamed to test:unit:watch
ci:unit-test removed (can just call test:unit)
playwright-version removed (use ver)
pretest:playwright changed to pretest
test:playwright:recreate-tapes fixed by adding missing pnpm
generate-playwright-test renamed to test:playwright:gen

lintfix renamed to lint:fix
format:test renamed to format
format renamed to format:fix to match lint

i18n:get-translations reduced to just the middle portion (with i18n replacing it)
i18n added to run all i18n commands i18n:create-locales-list, i18n:get-translations and i18n:update-locales
i18n:no-get added to run i18n except for downloading the 200 translations, used by dev
vue-i18n-check renamed to i18n:check and updated to include ts files

Testing Instructions

Play around with some stuff you usually do, you should not experience any errors other than when you enter the old name from muscle memory.
Checklist

[x] My pull request has a descriptive title (not a vague title like Update index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[x] I added or updated tests for the changes I made (if applicable).
[x] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no visible errors.

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

This is the reason why this is a draft PR.

Turns out I do need reviews and repros so gotta undraft it.

@dhruvkb pre and post scripts are executed because of this .npmrc line: https://github.com/WordPress/openverse-frontend/blob/79c74e98842aa557fd809299fdd7dee7f4631a90/.npmrc#L2
It's a necessary override of the default pnpm behavior to get some of our packages to work which assume pre/post script existence.

Ahhh, thanks for pointing that out @sarayourfriend. That discrepancy between the docs and the actual behaviour was very weird!

While we await the second approval, it'll be nice to discuss whether we should and how we can trim the list of scripts down to the bare minimum.


Problem with AutoConfiguration Order

There is a problem related to ElasticsearchJestAWSAutoConfiguration with spring cloud.
Spring Cloud has many autoconfiguration classes:

ContextRegionProviderAutoConfiguration: Creates RegionProvider beanDefinition.
ContextCredentialsAutoConfiguration: Creates AWSCredentionProvider beanDefinition.
and many more.

Current implementation waits for ContextRegionProviderAutoConfiguration to define beanDefinition then starts configuring JestAWS.
@AutoConfigureAfter(name = "org.springframework.cloud.aws.autoconfigure.context.ContextRegionProviderAutoConfiguration")

But problem is in
@Bean
@ConditionalOnMissingBean(AWSCredentialsProvider.class)
public AWSCredentialsProvider awsCredentialsProvider() {
	return new DefaultAWSCredentialsProviderChain();
}

ElasticsearchJestAWSAutoConfiguration should also wait for ContextCredentialsAutoConfiguration to define AWSCredentialsProvider.
So my solution is to add ContextCredentialsAutoConfiguration to AutoConfigureAfter
@AutoConfigureAfter(name = {
        "org.springframework.cloud.aws.autoconfigure.context.ContextRegionProviderAutoConfiguration",
        "org.springframework.cloud.aws.autoconfigure.context.ContextCredentialsAutoConfiguration"
})

Thanks a lot.
It's available now on release 3.2.2.


UI and UX Enhancements | Implementation of Features Page | Responsive Features

I want to implement the complete features page because there is no page for this.

@CogitoKunal sir, Assign this to me under GSSOC Ext.

Assigned

@CogitoKunal sir, I am going to implement the Whole Feature page and even with responsiveness that supports small devices too.
So please increase the level to 3

@CogitoKunal Sir u have closed this issue, I think its done mistakenly

@CogitoKunal Sir ?

Just solve the Signin  issue also sign in is not working

Just solve the Signin issue also sign in is not working

Okay assign that to me.


Storage Implementation Bug

What version of the package are you using?

CertMagic 0.16.1
What are you trying to do?

I've forked an S3 Storage repository (you can find it here: https://github.com/edwardwc/better-s3). After implementing it into certmagic.Default.Storage, CertMagic seemed able to probe the bucket:
2022/07/23 14:55:24 Better S3: Storing for rw_test_3648290488268453755
2022/07/23 14:55:24 Better S3: Loading for rw_test_3648290488268453755
2022/07/23 14:55:24 Better S3: Deleting for rw_test_3648290488268453755

(the names correlate to the methods used)
However, there seems to be an issue when a certificate does not exist:
2022/07/23 14:55:24 Better S3: Locking for issue_cert_{domain}.com
2022/07/23 14:55:24 Better S3: Exists for certificates/acme-v02.api.letsencrypt.org-directory/{domain}/{domain}.crt
2022/07/23 14:55:24 Better S3: Loading for certificates/acme-v02.api.letsencrypt.org-directory/{domain}/{domain}.key

Now, after doing some debugging I was able to conclude the Load function in this case did, indeed, return an error. I am unsure as to why the Load function was called in the first place: when Exist returned false. The same program has no problem generating SSL certificates when using the default file storage.

Issue fixed! It turns out when Load is called and an object does not exist, you have to return fs.ErrNotExist. Have a great day!

Glad you figured it out! I'll keep an eye on your progress. What is better about your S3 implementation? (There are a few out there but they have trouble with locking because S3 does not provide atomic operations; I think one of them uses DynamoDB for that... but yeah. It's tricky.)

Have a great day!

Thank you :blush: You too


Investigate how ExEx WAL should work with reth db drop CLI

Describe the feature
Should we clear the WAL when the database is dropped?
Additional context
No response

Is WAL related to datadir, right?
So, if yes, we can add clean on reth db drop by  passing directory remove at WAL path on DbTool’s drop fn
Maybe also with reth db clear and Wal subcommand

This may to close the issue https://github.com/paradigmxyz/reth/pull/11855

@0xurb yep, that would work, but there's a catch that an ExEx can have its own state maintained in a separate DB, and it's unclear whether cleaning the WAL is expected on reth db drop. Because if an ExEx had a head block at height 10, and there's no notification in the WAL for such height, we would report an error about an inconsistent WAL.
So I think a proper solution here would be to address this https://github.com/paradigmxyz/reth/issues/11884.

Quick look to it, will be good to have smith like verbose canonical check error/result:
enum CanonicalCheckResult {
    Success,
    MissedNotification { block_hash: B256 },
    ExexIsAhead,
}

then add a logic for
/// If the head block is ahead of the node head,
/// we wait until the first notification with the tip greater than the ExEx head comes,
/// and do the canonical check.
if self.exex_head.block.number > self.node_head.number {
    return CanonicalCheckResult::ExexIsAhead;
}

and on ExExNotificationsWithHead's future polling just match this error/result enum and return Poll::Pending in case of this.


panic on 500 from googleapi

time="2017-07-24T05:49:46Z" level=info msg="Starting stackdriver_exporter (version=0.1.1, branch=HEAD, revision=b4cf43ee4f11fcfef7effcc66ca7aa862219128d)" source="stackdriver_exporter.go:120"
time="2017-07-24T05:49:46Z" level=info msg="Build context (go=go1.8.3, user=root@1cc1402df604, date=20170715-22:22:05)" source="stackdriver_exporter.go:121"
time="2017-07-24T05:49:46Z" level=info msg="Listening on :9255" source="stackdriver_exporter.go:147"
time="2017-07-24T05:56:16Z" level=error msg="Error while getting Google Stackdriver Monitoring metrics: googleapi: Error 500: An internal error occurred., backendError" source="monitoring_collector.go:123"
time="2017-07-24T06:01:12Z" level=error msg="Error while getting Google Stackdriver Monitoring metrics: googleapi: Error 500: An internal error occurred., backendError" source="monitoring_collector.go:123"
panic: send on closed channel

goroutine 4421 [running]:
github.com/frodenas/stackdriver_exporter/collectors.(*MonitoringCollector).reportTimeSeriesMetrics(0xc42001d2c0, 0xc42023e070, 0xc4201e7110, 0xc4204c2420, 0xc42023e070, 0x0)
        /go/src/github.com/frodenas/stackdriver_exporter/collectors/monitoring_collector.go:298 +0xd6b
github.com/frodenas/stackdriver_exporter/collectors.(*MonitoringCollector).reportMonitoringMetrics.func1.1(0xc420320ac0, 0xc42001d2c0, 0xed10782fc, 0xc411225411, 0x0, 0xed1078428, 0xc41122549d, 0x0, 0xc420494f60, 0xc4201e7110, ...)
        /go/src/github.com/frodenas/stackdriver_exporter/collectors/monitoring_collector.go:174 +0x5ef
created by github.com/frodenas/stackdriver_exporter/collectors.(*MonitoringCollector).reportMonitoringMetrics.func1
        /go/src/github.com/frodenas/stackdriver_exporter/collectors/monitoring_collector.go:183 +0x28b

In order to reproduce this, can you please comment which flags did you use when starting the exporter?

I use the following environment variables.
        - name: GOOGLE_APPLICATION_CREDENTIALS
          value: '/var/run/secrets/gcp/credentials.json'
        - name: STACKDRIVER_EXPORTER_GOOGLE_PROJECT_ID
          valueFrom:
            secretKeyRef:
              name: stackdriver-exporter
              key: project_id
        - name: STACKDRIVER_EXPORTER_MONITORING_METRICS_TYPE_PREFIXES
          value: 'bigtable.googleapis.com/disk,bigtable.googleapis.com/server,bigtable.googleapis.com/cluster'
        - name: STACKDRIVER_EXPORTER_MONITORING_METRICS_INTERVAL
          value: '5m'

Thanks @julianvmodesto. I'll try to do some tests tomorrow.

Let me know how I can help! We've seen this panic a few times, but I haven't been able to simulate it yet. I tried stubbing out the monitoring api and throwing 500s, but I haven't hit the panic yet.


Support for stateless components

This PR allows to emit a pure stateless function (see here) by placing the special tag <rt-stateless /> in the .rt file.
Example:
<rt-stateless />
<div>
   This is a stateless component showing {props.someProp} and {context.someContextData}
</div>

The difference is only in the signature of the emitted render function:
function (props, context) { ... }

instead of
function () { ... }

Sorry, looks like I beat you to it...
I'd already had all the code reviewed by the other admins when you mentioned you had it locally, so I just released my version.
Thanks for the contribution though! Could you please open PRs if there are any other features you've already implemented? I'd hate for this to happen again.

Sure no problem next I'll open the PR earlier.
Regarding this specific issue/PR, would you consider turning rt-stateless from attribute to a <rt-stateless /> tag? It seems to me a more coherent syntax that fits better with all the rest.

I don't know if I agree...
As opposed to tags like rt-virtual that signify an actual node, rt-stateless is an actual "attribute" of the node that tells the interpreter (in this case, React Templates) that it should be "rendered" differently, just like rt-repeat.

as attribute, 'rt-stateless' makes me think it's something related to the node where it appears and its children, that's why I think it's not so elegant.
As a tag, it makes me think to a "directive" (to be put on top of the file).
Another idea could be making a <rt-options> tag where to put flags for various customizations. In this case it could be something like:
<rt-options stateless="true" />

BTW, I think your PR is missing the "context" extra parameter that is passed in case of react context (see here). I mean we need:
function (props, context) {}

I think we'll keep this syntax for now and get some feedback.
A lot of us here at Wix are going to be using this new syntax soon, so that'll be a good chance to get people's opinions.
But you're right about the context part! Releasing a patch.


Update translations (zh-TW, zh-cn, en-us).

The "Dynamic Range" and "Surround" checkbox is not very clear in my opinion.
I've made a small change to the text to fix that.
(Maybe adding a detailed hint will help users to understand that unchecked means "Limited Dynamic Range" and "Stereo"?)

looks good to me

Heyo, just a heads up that the changes made will be force-synced into Crowdin under neon-nyan, the author of this repository. If you would like to get properly credited, please contribute through Crowdin. Thanks!

Heyo, just a heads up that the changes made will be force-synced into Crowdin under neon-nyan, the author of this repository. If you would like to get properly credited, please contribute through Crowdin. Thanks!

I see. To be honest, I forgot to read the Contribution Guidelines. I just cloned, edited the JSON, and built it to see if it works.
I'll look into the usage of Crowdin later. Thanks.


Use PycURL for fetching event streams

Earlier issues #35 and #114 made things better in this department. However,
when running marathon-lb with a large (400+ applications) marathon instance,
there are still problems.
These problems can be traced back to Python itself, unfortunately:
http://stackoverflow.com/questions/21797753/efficiently-reading-lines-from-compressed-chunked-http-stream-as-they-arrive
Python requests uses urllib under the covers, and there are implicit issues
when 7.5 MB of JSON comes back on a single line, as we're seeing when certain
events are emitted. These events are deployment_info and deployment_success at
a minimum, there may be more.
By switching to PycURL, as noted in the Stack Overflow post, we bypass this
whole issue. We use an HTTP library that handles this particular edge case
well, reducing CPU usage dramatically when a large event comes in. It also
handles gzip compression, which means any 7.5 MB JSON dumps should shrink
significantly.
One unsolved problem remains here: the addition of DC/OS authentication support
in #285 is extremely tightly coupled to internal implementation details of the
python requests module. This simply won't work with this code, and I have zero
ability to fix or test it as we don't use DC/OS.

Can one of the admins verify this patch?

I have also been experiencing this issue. While the upgrades to HAProxy (https://github.com/mesosphere/marathon-lb/pull/374) have resolved some of the 100% CPU issues, I see giant spikes from Python itself from these calls.

We're definitely going to need to support auth. It's probably worth dropping the other modes (polling & callback) as well.
It's probably okay to use PycURL for SSE, and requests for everything else. In which case, we can reuse the requests-based auth stuff, and just pass the token in the header when it's present.

@brndnmtthws awesome, thanks for the quick feedback! As long as the main thread of work here (switching to PycURL to work around issues in core python urllib) is acceptable to everyone, I can take some more time to get auth working again. I'll also remove the marked-as-deprecated old event subscription code to make this all a bit cleaner.
If I make a best-effort pass at restoring auth functionality, can I get someone with a DC/OS setup to test it? Your approach seems perfectly sane- continue to use requests to grab the token, but ensure the right token headers end up on the cURL-based request.
Here's the current situation, if it helps to show some real metrics on what this is trying to solve. The red background indicates deploy events in our environment; you can see these cause large CPU usage spikes by the marathon-lb docker process:

I'd be happy to test it and work with you on the PR. The auth code is here: https://github.com/mesosphere/marathon-lb/blob/master/common.py#L53-L90
It fetches a token periodically (approx. every hour), and sets the Authorization header with that token.

OK, @brndnmtthws - this PR has been updated to the best of my abilities. Changes since last time:

DC/OS JWT-based auth should now be supported. However, I can't test it against an actual implementation, because I don't have one. Testing help would be welcome!
The old event/listening mode has been dropped, and all code removed that was associated with it.
I kept 'poll' around because it wasn't much of a burden.

Testing against a non DC/OS stack, this seems to be working fine for us. Our JSON blobs have grown to 8 MB at this point, and this implementation works much smoother than the native Python/requests one did.

Ping? Thoughts here?
If marathon 1.4.x is going to make this problem go away with smaller event payloads, this can likely get dropped, but we're still experiencing this problem all the time with the current version due to huge payloads.

I apologize for taking so long to review this. I've got a lot of other stuff going on 🙂
I'll dig into it today, and test it w/ auth.

Thanks for taking a look and merging!


Why is acryl-datahub Python package uninstalled in datahub-ingestion Docker image?

Describe the bug
Layer 27 of datahub-ingestion image: link
requirements.txt is installed, and then acryl-datahub package is removed, forcing me to redownload it for my own custom image, as I am using datahub-ingestion image for programmatic metadata ingestion.
To Reproduce
Steps to reproduce the behavior:

Pull datahub-ingestion image
acryl-datahub Python package is uninstalled

@askarbozcan it is reinstalled in layer 31. We do this to speed up our docker build times.


MINOR: Improve usage of LogCaptureAppender

I started a "small" cleanup, and it ended up to become a "big" cleanup...

limit the usage of LogCaptureAppender to the class under test (if possible)
ensure that the registered appended is unregistered (via try-with-resource)
limit the "scope" of creating/unregistering the appended to a minimum
cleanup all warning of all classes touched
some code reformatting
some class renaming

Call for review @vvcephei

Did you consider using try with resources?

I did not consider using try-with-resource. Great idea!

Updated this. Call for review.

Java 8 passed.
Java11:  kafka.api.ConsumerBounceTest.testRollingBrokerRestartsWithSmallerMaxGroupSizeConfigDisruptsBigGroup
Java14 failed due to #8519 (already merged).
Retest this please.

Updated this PR. Will merge after Jenkins passe.

Java 8: org.apache.kafka.streams.integration.StandbyTaskEOSIntegrationTest.surviveWithOneTaskAsStandby[exactly_once_beta]
Java 11: org.apache.kafka.connect.mirror.MirrorConnectorsIntegrationTest.testReplication
Java 14: org.apache.kafka.streams.integration.StoreQueryIntegrationTest.shouldQuerySpecificActivePartitionStores
Retest this please.

Java 8 and Java 11 passed.
Java 14: failed with unknown error...


Support long text buttons

Hi @nartex thanks for your idea! But i don't think we're going to support long text buttons because the ActionButton would lose its meaning. What do you think about this issue?
Thanks again

I'd say It could be sometimes useful to use this menu as an action menu, feel free to pick some code from the version I just posted, commits are a bit messy so I won't create a pull request...


Forbid certain characters

Is it possible to forbid certain characters from generation algorithm?

This is a good question, given that I am using a language similar to Regex for string production rather than validation it should be possible to use the [^...] style notation to limit the allowed characters.
I'll see about adding it as a feature.


[stable/airflow] Values.redis.master.port  is compulsory

Workers where failing without Values.redis.master.port

Hi,
Could you elaborate more on the issue? Weren't you able to install it with the default values?

No it was a mistake with another issue with oassword having '/'  in it  which causes kombu to read charters before that '/' as port number but on cast to int it fails. Check kombu url_to_parts code for it


Envelope release problems

Hi Yotam,
We're experiencing a few issues with the Envelope release. Here's an example.
Noise connected to an AmplitudeEnvelope with the attack set to 2 seconds.
Trigger the attack and then change the attack value to 0.01 and release whilst still in the attack phase (within those 2 seconds)
You will hear a loud bang. _valueAtTime() is evaluating to a high number above 1 (sometimes in the 40's).
Also occasionally when i triggerAttack in release phase and triggerRelease in the attack phase the audio fails to ramp back to to 0.
Have you got any ideas?

Here is an example of what _linearInterpolate is evaluating to:
A normal release:
0.0001 + (1 - 0.0001) * ((86.07 - 85.74) / (86.74 - 85.74))
= 0.33
Loud volume increase:
0.0001 + (1 - 0.0001) * ((92.28 - 91.95) / (91.97 - 91.95))
= 16.63

And here's a video example
https://drive.google.com/file/d/0B-FXQg_VQV3JaDFGbDhqZDliaEE/view

Is it possible to get this in a jsfiddle or something i can play with? I want to check if this has been fixed with the updated Envelope implementation in r6.

I've created a mock example using a settimeout to replicate the user dragging the block as per the video. Hopefully this can shed some light.
http://jsfiddle.net/LwLny67u/3/

Thanks for the example. it was really helpful in debugging this situation. Here's your example with the new Tone.js build (r6-dev). No clicks or super loud pops.
Unlike in r5, the new envelope implementation keeps track of all scheduled events, so changing the attack time during the attack doesn't cause any discontinuities.

Ah cool! Which build are you using though? When I use r6-dev Tone.js it's not working. Sounds like this:
http://jsfiddle.net/prespcLb/9/
(In this example, I'm using in this: https://raw.githubusercontent.com/Tonejs/Tone.js/dev/build/Tone.js)

I don't often commit builds to the dev branch. The build i used in that example (and included in the html section of that jsfiddle) i created using the latest changes, but never committed. I just committed those files, so you should be able to make your own build and verify these changes.

Yes all working great now thanks!


How to Add some animation or effect when dropping or placing model?

I recently saw an Ikea app it is build in Unity and have very nice gravity effect which means when you drag the item and place it it shows some gravity effect. How can I achieve something similar to it?

Do you know in image detection android app when we place model on image after image detected how can we rotate and zoom that model using gestures in SceneView?This functionality is easily implemented when I had created surface detection android app using ArFragments but using SceneView or ArSceneView I do not find any solution

You can use ObjectAnimator in order to achieve that functionality.

@hardik-intellify thank you for your suggestion. I already have followed this and achieved the required results.


Mapbox Marker API no longer exists

See the line of code here for icon marker:
https://github.com/mapbox/tokml/blob/91c3fa5fb256ef74307847e21de53dbb1f043349/index.js#L203
https://api.tiles.mapbox.com/v3/marker/ no longer exists i think. Not sure if mapbox has another API for markers.

If anyone is experiencing this issue, my work around was just to set my geosjon simplestyle to have a white large marker which generates a url with that broken api that I replace with a pin from google map.
kml = kml
      .toString()
      .replace(
        'https://api.tiles.mapbox.com/v3/marker/pin-l+fff.png',
        'http://maps.google.com/mapfiles/kml/paddle/wht-circle.png',
      );

// then use xml-formatter

for other icons, refer to http://kml4earth.appspot.com/icons.html
This works for my use case because I only need this one white marker.


terraform-boot oauth with JWT fails

[ ] when authorization-token-type is set as JWT, the server boot fails.
[ ] Please also change the below parameter names to dot separated.
[ ] Make JWT the default token type
[ ] Update doc

APPLICATION FAILED TO START
***************************

Description:

Parameter 0 of method setFilterChains in org.springframework.security.config.annotation.web.configuration.WebSecurityConfiguration required a bean named 'jwtDecoder' that could not be found.

Action:

Consider defining a bean named 'jwtDecoder' in your configuration.```

authorization-token-type=OpaqueToken
authorization-server-endpoint=
authorization-api-client-id=
authorization-api-client-secret=
authorization-swagger-ui-client-id=```

let me do it


Adds _relay_metadata reserved parameter

Do you want me to merge these to the main branch? I will update the /next folder with these changes as well if you think they are ready to go, assuming of course these changes you have made for pre-alpha also apply to 0.1.0.

Yes, please do. These changes are already implemented in main branch. Thanks


Integrate flutter_tts

How to integrate https://pub.dev/packages/flutter_tts
Into this library?

sorry , but in this version TTS not supported


[WIP] fix(shared): Force disable swr devtools for organization hooks

Type of change

[ ] 🐛 Bug fix
[ ] 🌟 New feature
[ ] 🔨 Breaking change
[ ] 📖 Refactoring / dependency upgrade / documentation
[ ] other:

Packages affected

[ ] @clerk/clerk-js
[ ] @clerk/clerk-react
[ ] @clerk/nextjs
[ ] @clerk/remix
[ ] @clerk/types
[ ] @clerk/themes
[ ] @clerk/localizations
[ ] @clerk/clerk-expo
[ ] @clerk/backend
[ ] @clerk/clerk-sdk-node
[x] @clerk/shared
[ ] @clerk/fastify
[ ] @clerk/chrome-extension
[ ] gatsby-plugin-clerk
[ ] build/tooling/chore

Description

[ ] npm test runs as expected.
[ ] npm run build runs as expected.

TODO: write description

!snapshot

Hey @panteliselef - the snapshot version command generated the following package versions:

Package
Version

@clerk/chrome-extension
0.3.32-snapshot.965d3c8

@clerk/clerk-js
4.57.1-snapshot.965d3c8

@clerk/clerk-expo
0.19.2-snapshot.965d3c8

gatsby-plugin-clerk
4.4.8-snapshot.965d3c8

@clerk/nextjs
4.23.6-snapshot.965d3c8

@clerk/clerk-react
4.24.3-snapshot.965d3c8

@clerk/remix
2.10.3-snapshot.965d3c8

@clerk/shared
0.22.1-snapshot.965d3c8

Tip: use the snippet copy button below to quickly install the required packages.
# @clerk/chrome-extension
npm i<EMAIL_ADDRESS>
# @clerk/clerk-js
npm i<EMAIL_ADDRESS>
# @clerk/clerk-expo
npm i<EMAIL_ADDRESS>
# gatsby-plugin-clerk
npm i<EMAIL_ADDRESS>
# @clerk/nextjs
npm i<EMAIL_ADDRESS>
# @clerk/clerk-react
npm i<EMAIL_ADDRESS>
# @clerk/remix
npm i<EMAIL_ADDRESS>
# @clerk/shared
npm i<EMAIL_ADDRESS>

This PR has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.


Upgrade Grafana to v11.x

As a user I want to have the latest feature available for Grafana, therefore the boshrelease should include Grafana v11.X

Currently the plan is to create a new major version beginning of 2025 including Grafana v11.X


implement test-suite

Tests are scoped by service and type
i.e test:
service-name:
unit-tests
e2e-tests

This is in progress


Installation instructions should make clearer where c3 should be added

Based on installation instructions I'm not sure where "c3": "^0.7.20" should be added.

Hi, @arturojain
Yes, sorry, it's in the main OHIF project in package.json.
Inside package.json in dependencies.

Looking at the latest README, should yarn install be used?

@arturojain
Yes, you can add it manually in package.json and then do an npm install or install it with yarn, both options are valid. In the end it is an extension for OHIF Viewer, it works in the same way as the OHIF components, everything comes from the OHIF documentation.


Wasted space, console errors and list markers

While Drweb module is running the error counter runs madly.
Google chrome version 43.0.2342.2 dev (64-bit)

Please be kind reporting this issue to the module developers. I can't provide support for this third-party module. If they won't react now, they will defenetly react when the theme becomes default.
You can mention the fact that the theme in Webmin package already.

I don't insist, and of course you should not provide support to buggy modules, but experience tells me that such modules can be helpful stress-testing your product.
But the wasted space and strange list markers are still common issue.

Yes, I agree. I will take a look at .length issue. I can fix it, I think, you're right.

And 'wasted' space? Not sure.. It's intentional. Alright, I will make it optional next release.

Alright, will make a fix in 11.01.
Please check in couple of hours and tell me if your Dr.Web module is working better now?
For so called wasted space - well it's intentional. The blank space will go away on smaller screens (try to resize your browser window).
In case you want to have no blank space all the time, use the following code in your custom styles.css
.col-lg-10 {
    width: 100%;
  }

.col-lg-offset-1 {
    margin-left: 0;
  }
@media(min-width:1200px) {
    .container,
    .container-fluid {
        margin-top: 20px;
    }
}
@media(min-width:1600px) {
    .container,
    .container-fluid {
        margin-top: 20px;
    }
}


use caching for bench and test

Hmm, is this going to do that thing where clippy doesn't recheck because it's using a cached build?

I'll push a clippy break and see if it catches it.

What workflow is the clippy job on?

Clippy failed when I pushed an error so I think we're good.


Shader compilation error when trying to view assets

Trying to view any asset in Hide gives me the following error, I have tried on both Windows 10 and Linux, self-built and nightly builds with the latest code base. Haxe version is 4.1.5, and all the required haxelibs listed in README.md are installed using their latest git versions.
Error: An error occurred compiling the shaders:

#version 300 es
precision highp float;
in vec2 position;
in vec2 uv;
vec4 position_2;
out vec2 uv_2;
uniform vec4 vertexParams[1];

void main(void) {
    position_2 = vec4(position.x, position.y * vertexParams[0].x, 0., 1.);
    gl_Position = position_2;
    uv_2 = uv;
    gl_Position.z += gl_Position.z - gl_Position.w;
}

new haxe.ValueExpection.
$hxClasses.haxe.ValueException
Function.haxe.Exception.thrown
h3d.impl.GlDriver.compileShader
h3d.impl.GlDriver.selectShader
h3d.Engine.selectShader
h3d.pass.ScreenFX.render
Function.h3d.scene.Environment.computeIrradLut
Function.h3d.scene.Environment.getDefaultLUT
h3d.scene.pbr.Environment.get_lut
hide.PbrRenderer.beginPbr

This is still an issue by the way. I'm on windows 10 and just downloaded a nightly build and get the same issue. I haven't tested with a self build yet as I can't build the project without getting errors (even though I'm using the latest code and libraries).

I tried this on a laptop with Radeon Vega 6 Graphics with the latest drivers. It works as expected on my desktop though.

Could you check if it works better if you set isIntelGpu=true in h3d/impl/GlDriver.hx ?

having same issue on a laptop with linux, tried both discrete nvidia gpu(1050ti, akmod nvidia driver) and intel hd graphics

also setting isIntelGpu as true didnt help

i just turneed off pbr rendering in Project > Renderer and it started working

Huh... that fixed it for me.


security/acme-client: release 1.9

New features

update acme.sh to version 2.7.2 (https://github.com/opnsense/plugins/issues/210)

Bugfixes

Rename Certificate "Name" to "Common Name" for better clarity (https://github.com/opnsense/plugins/pull/214)
Fix title in "Rewew" and "Revoke" dialogs
Add dependency to BIND to fix nsupdate support

@fichtner Just to understand what has happened: was BIND actively removed during any of the last OPNsense upgrades?

bind, or specifically nsupdate was used by RFC-2136, which now resides in the plugins.