Panic in the compactor when writing to SlateDB from a DataFusion operator

Describe the bug
We are attempting to use SlateDb to checkpoint the state of our operators of our Stream Processing system built atop DataFusion.
On a high level here's how our system works -

We are executing a compute graph that is running on DataFusions tokio based runtime. DataFusion operators implement that ReceiverStream pattern and produce data asynchronously that is routed through the graph by the runtime. (See: https://github.com/apache/datafusion/blob/main/datafusion/physical-plan/src/stream.rs )

In some of our stateful operators, we want to checkpoint the state

    fn check_point_state_for_epoch(&mut self, epoch_ts: u128) {
        debug!(
            "initiating checkpoint for {} at epoch {}",
            self.channel_tag, epoch_ts
        );
        ...
        let serialized_checkpoint = bincode::serialize(&checkpointed_state).unwrap();
        let key = self.channel_tag.as_bytes();
        let _ = block_on(self.state_backend.put(key, &serialized_checkpoint));
        debug!(
            "finished checkpoint for {} at epoch {}",
            self.channel_tag, epoch_ts
        );
    }

The self.state_backend here is an Arc<Db> instance that is shared between the tasks.

Ever so often, we see a panic while checkpoint that ends up killing the runtime.
For purposes of testing we are using the LocalFilesystem
async fn get_slate_db_backend(path_str: &str) -> Db {
    let object_store: Arc<dyn ObjectStore> = Arc::new(LocalFileSystem::new());
    let options = DbOptions::default();
    debug!("initialising slate db at {}", path_str);
    Db::open_with_opts(Path::from(path_str), options, object_store)
        .await
        .unwrap()
}

And here is the backtrace
thread '<unnamed>' panicked at /Users/amey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/slatedb-0.2.0/src/compactor.rs:159:46:
fatal error loading manifest: ObjectStoreError(JoinError { source: JoinError::Cancelled(Id(109)) })
stack backtrace:
   0:        0x1068a0ef8 - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h14dcb4794e811c9f
   1:        0x1068b12d4 - core::fmt::write::h49a9cf68bfb4a321
   2:        0x1068a431c - std::io::Write::write_fmt::h00e4f1b82d71ad85
   3:        0x10688cf74 - std::panicking::default_hook::{{closure}}::hda52b8046522f561
   4:        0x10688cbb0 - std::panicking::default_hook::h9566cb73367aef3e
   5:        0x10688d92c - std::panicking::rust_panic_with_hook::hfa41181adbcf532d
   6:        0x1068a12dc - std::panicking::begin_panic_handler::{{closure}}::ha417e66bf9de7acf
   7:        0x1068a1120 - std::sys::backtrace::__rust_end_short_backtrace::hc7b890f013fa9efb
   8:        0x10688d068 - _rust_begin_unwind
   9:        0x1069eb128 - core::panicking::panic_fmt::h29609029db4d41f4
  10:        0x1069eb0f8 - core::result::unwrap_failed::h1a030347a5d7de67
  11:        0x102ad14d8 - core::result::Result<T,E>::expect::he3596e049c212a86
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/core/src/result.rs:1059:23
  12:        0x102bf6608 - slatedb::compactor::CompactorOrchestrator::run::he520b96bb88ca345
                               at /Users/amey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/slatedb-0.2.0/src/compactor.rs:159:25
  13:        0x1024d87d4 - slatedb::compactor::Compactor::new::{{closure}}::{{closure}}::h37fe734dfc21b1c4
                               at /Users/amey/.cargo/registry/src/index.crates.io-6f17d22bba15001f/slatedb-0.2.0/src/compactor.rs:67:13
  14:        0x1024dadc4 - std::sys::backtrace::__rust_begin_short_backtrace::h68fbd1990538055a
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/std/src/sys/backtrace.rs:152:18
  15:        0x10249b704 - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::hc04c79247f926931
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/std/src/thread/mod.rs:538:17
  16:        0x10249f2d0 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h666de1a3662eea42
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/core/src/panic/unwind_safe.rs:272:9
  17:        0x1024e7498 - std::panicking::try::do_call::hc5aad5f651f779b4
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/std/src/panicking.rs:557:40
  18:        0x10249b918 - ___rust_try
  19:        0x10249b3b4 - std::panicking::try::h50bcaf7edf39fa2a
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/std/src/panicking.rs:521:19
  20:        0x10249b3b4 - std::panic::catch_unwind::hcf164ee567674fd1
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/std/src/panic.rs:350:14
  21:        0x10249b3b4 - std::thread::Builder::spawn_unchecked_::{{closure}}::ha82d38afd7a6518f
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/std/src/thread/mod.rs:537:30
  22:        0x1024ab908 - core::ops::function::FnOnce::call_once{{vtable.shim}}::h134233046647c648
                               at /private/tmp/rust-20240905-9990-tmdsun/rustc-1.81.0-src/library/core/src/ops/function.rs:250:5
  23:        0x10688e484 - std::sys::pal::unix::thread::Thread::new::thread_start::h3d45bacb8043a502
  24:        0x18d622f94 - __pthread_joiner_wake

Additional context
Unclear if this is related to slatedb internals or particularly how SlateDB's threading is interacting with DataFusion's runtime.

fyi... trying this against InMemory and Localstack does not seem to reproduce the issue. Might be strictly related to LocalFilesystem impl of the ObjectStore.

Conducted a simple tracking based on the error report. It seems likely that there is something wrong with manifest_store::read_lateste_manifest
But continuing to track it lacks more context.

Perhapws it's this? https://github.com/apache/arrow-rs/issues/6375

Ah, nm. We sort the files in list_manifests

@ameyc Can you list the local filesystem folder (recursively) where the object store is writing to? I want to see what files are inside.

@ameyc Can you share your complete DbOptions config for the DB?

Using DBOptions::default()
Here they are -
[2024-10-08T19:34:20Z DEBUG denormalized::state_backend::slatedb] DBOptions DbOptions {
        flush_interval: 100ms,
        manifest_poll_interval: 1s,
        min_filter_keys: 1000,
        filter_bits_per_key: 10,
        l0_sst_size_bytes: 67108864,
        l0_max_ssts: 8,
        max_unflushed_memtable: 2,
        compactor_options: CompactorOptions { poll_interval: 5s, max_sst_size:<PHONE_NUMBER>, max_concurrent_compactions: 4, compaction_runtime: None }
        compression_codec: None
        object_store_cache_options: ObjectStoreCacheOptions { root_folder: None, part_size_bytes: 4194304 }
        block_cache: Some(Arc<dyn DbCache>)
        garbage_collector_options: GarbageCollectorOptions { manifest_options: Some(GarbageCollectorDirectoryOptions { poll_interval: 300s, min_age: 86400s }), wal_options: Some(GarbageCollectorDirectoryOptions { poll_interval: 60s, min_age: 60s }), compacted_options: Some(GarbageCollectorDirectoryOptions { poll_interval: 300s, min_age: 86400s }), gc_runtime: None }
    }

Added a few debug statements to list_manifests and read_manifest as suggested in the discord. Here is the trace
[2024-10-08T19:43:13Z DEBUG slatedb::manifest_store] entering read_latest_manifest
[2024-10-08T19:43:13Z DEBUG slatedb::manifest_store] entered list_manifests
[2024-10-08T19:43:13Z DEBUG slatedb::manifest_store] about to parse manifest files
thread '<unnamed>' panicked at /Users/amey/pnl/3rdparty/slatedb/src/compactor.rs:159:46:
fatal error loading manifest: ObjectStoreError(JoinError { source: JoinError::Cancelled(Id(576)) })
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

My debug patch is here - amey_debug_changes.patch

Accidentally opened a PR with my patch. But nevertheless in case it is easier to follow - https://github.com/slatedb/slatedb/pull/253/files

Based on the debug statements, it appears to be dying here:
        while let Some(file) = match files_stream.next().await.transpose() {
            Ok(file) => file,
            Err(e) => return Err(SlateDBError::ObjectStoreError(e)),
        } {

Which suggests a file_stream.next() is returning an error. The nested JoinError is a bit puzzling.
I suppose it could die inside the loop as well (in parse_id, for example). But given the SlateDBError::ObjectStoreError in the error message, I think it's happening in the files_stream.next().await.transpose() call.

Some useful tidbits from ChatGPT:

The error you're seeing, ObjectStoreError(JoinError { source: JoinError::Cancelled(Id(576)) }), indicates that a join operation (likely from futures or tokio) was canceled. In async Rust, a JoinError::Cancelled usually happens when a future is aborted before it completes. There are a few potential reasons for this:

Timeout or Cancelation: If you're running this in an environment where there's a timeout or some form of future cancelation mechanism, the future running the files_stream.next() could be getting canceled before it completes. Check if there are any timeouts in your code or external timeouts being enforced.

Task Abort due to Other Conditions: If the code calling list_manifests depends on other futures, they might be failing or aborting, which could cascade and cause the join error in your current task. For example, if there's another task that fails and causes the task running this function to be canceled, you would see this error.

Misuse of Streams: Ensure that the files_stream.next().await isn't causing a deadlock or other issue that could result in cancellation. If the stream implementation has bugs or interacts badly with the environment (such as exhausting system resources), it might cause an unexpected abort.

@ameyc What are you setting for path_str?

it is being set to /tmp/checkpoints/ . currently seeing if how i am trying to use blocking calls to slatedb from DafaFusion's streams is the issue but at loss as to why this seems to be an issue with using LFS specifically.

added a few debugs to the while loop in the list_manifests and heres the full trace. Funnily adding these debugs, make the failure far rarer (though it still fails if i leave the pipeline running long enough). I will try and produce a repro that isnt super embedded in our code base over the weekend.
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] entering read_latest_manifest
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] entered list_manifests
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] about to parse manifest files
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] parsing<PHONE_NUMBER>0000000002.manifest
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] pushing id 2 to manifests
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] parsing<PHONE_NUMBER>0000000003.manifest
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] pushing id 3 to manifests
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] parsing<PHONE_NUMBER>0000000001.manifest
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] pushing id 1 to manifests
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] manifests successfully loaded
[2024-10-10T00:03:20Z DEBUG slatedb::manifest_store] successfully read the manifest_metadatas_list

@ameyc thanks. Do you have a debug log for when it failed with the debug! statements?

@ameyc I think we should close this out since it sounds like the issue was with DataFusion. Can you provide a brief summary of the cause in case others hit this problem in the future?

@criccomini so the "bug" was caused by an issue elsewhere in our code that was intermittently erroring out on a kafka timeout that said, we did see some panics in the DF runtime even after the fix. Our solution was to Slate DB on a separate runtime and that seems to have taken care of all our issues for now.
I have been meaning to add a follow up PR to make initialising SlateDB with your own runtime a bit more convenient.


missing_scope

I created slack app. and send slack app token.
but response is missing_scope.
this api is only use legacy token??
so user legacy token is only use???
how to send message poll using slack app.
please help me.

This is mentioned in the readme.

Slack apps: Since the post scope is not available through the standard Oauth process they can not be directly used in a normal Slack app. In order to use them in a Slack app a legacy token for the respective workspace needs to always be manually create and added.

@jtatum, but the legacy token which is created for a workspace will be used only by user. How to use the legacy token from a slack app?

I think it is clearly stated:

they can not be directly used in a normal Slack app

I agree that this is very unfortunate, especially with the legacy tokens now being removed from Slack. The only other solution is to ask the Slack team for adding this feature to the normal methods. (which I already did btw). The more people requests this, the more likely it is to be included.


Update to Mono <IP_ADDRESS> seems to break E4k 0.4.0 (NuGet)

Just updated the whole Xamarin/MS stack (Mono, Xamarin.Mac, VS, etc.) and after the update my sample project (available on Github) doesn't build anymore.
Here are the errors I get:
/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/include/xamarin/runtime-generated.h(86,1): error G315394AC: must use 'enum' tag to refer to type 'MarshalManagedExceptionMode'
/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/include/xamarin/runtime-generated.h(89,1): error G315394AC: must use 'enum' tag to refer to type 'MarshalObjectiveCExceptionMode'
/Library/Frameworks/Xamarin.Mac.framework/Versions/Current/include/xamarin/runtime.h(88,18): error GF12A802B: pointer to non-const type 'const Protocol *' with no explicit ownership
/PATH/Embeddinator-4000.targets(3,3): error MSB3073: The command "/PATH/packages/Embeddinator-4<IP_ADDRESS>/tools/objcgen bin/Debug/LIB.dll --target=framework --platform=macOS-modern --abi=x86_64 --nativeexception --compile --debug --outdir=bin/Debug/" exited with code 1.

And the info from VS's about screen:
=== Visual Studio Professional 2017 for Mac ===

Version 7.5 (build 1254)
Installation UUID: 7c1f22f0-9e53-4c07-9bd3-4086f6c262d4
Runtime:
	Mono <IP_ADDRESS> (2017-12/8eb8f7d5e74) (64-bit)
	GTK+ 2.24.23 (Raleigh theme)
	Xamarin.Mac <IP_ADDRESS> (master / 0c7c49a6)

	Package version: 510010047

=== NuGet ===

Version: <IP_ADDRESS>45

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
	2.0.5
	2.0.3
	2.0.0
	1.1.1
	1.0.4
SDK: /usr/local/share/dotnet/sdk/2.1.4/Sdks
SDK Versions:
	2.1.4
	2.0.3
	2.0.2
	1.0.3
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.10.1/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

'/Applications/Xamarin Profiler.app' not found

=== Apple Developer Tools ===

Xcode 9.3 (14154)
Build 9E145

=== Xamarin.Mac ===

Version: <IP_ADDRESS> (Visual Studio Professional)

=== Xamarin.iOS ===

Version: <IP_ADDRESS> (Visual Studio Professional)
Hash: 7e782c1e
Branch: d15-7
Build date: 2018-04-25 15:27:13-0400

=== Xamarin.Android ===

Not Installed

=== Xamarin Inspector ===

Version: 1.4.0
Hash: b3f92f9
Branch: master
Build date: Fri, 19 Jan 2018 22:00:34 GMT
Client compatibility: 1

=== Build Information ===

Release ID: 705001254
Git revision: 498923ea36d2c7fe440c4e4b8cfb75bd50bbd748
Build date: 2018-05-05 10:35:24-04
Xamarin addins: 219f1c4943b4693b837b4173dd10ea982a47c852
Build lane: monodevelop-lion-d15-7

=== Operating System ===

Mac OS X 10.13.5
Darwin 17.6.0 Darwin Kernel Version 17.6.0
    Wed Apr 25 22:09:51 PDT 2018
    root:xnu-4570.60.20~7/RELEASE_X86_64 x86_64

=== Enabled user installed addins ===

Internet of Things (IoT) development (Preview) 7.1

Same thing happening for me after upgrading today:
Compiling binding code...
Found Mono <IP_ADDRESS> (between <IP_ADDRESS> and 5.2.99)
Found Xamarin.iOS <IP_ADDRESS> (between <IP_ADDRESS> and <IP_ADDRESS>)
	xcode-select -p
	xcrun clang -g -O0 -fobjc-arc -ObjC -Wall -arch armv7 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk -miphoneos-version-min=8.0 -I/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/SDKs/MonoTouch.iphoneos.sdk/usr/include -DXAMARIN_IOS -DMONO_EMBEDDINATOR_DLL_EXPORT -fembed-bitcode -c ./glib.c -o ./armv7/glib.o
	xcrun clang -g -O0 -fobjc-arc -ObjC -Wall -arch armv7 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS.sdk -miphoneos-version-min=8.0 -I/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/SDKs/MonoTouch.iphoneos.sdk/usr/include -DXAMARIN_IOS -DMONO_EMBEDDINATOR_DLL_EXPORT -fembed-bitcode -c ./mono_embeddinator.c -o ./armv7/mono_embeddinator.o
In file included from ./mono_embeddinator.c:29:
In file included from ./mono_embeddinator.h:32:
In file included from ./mono-support.h:32:
In file included from /Library/Frameworks/Xamarin.iOS.framework/Versions/Current/SDKs/MonoTouch.iphoneos.sdk/usr/include/xamarin/xamarin.h:14:
In file included from /Library/Frameworks/Xamarin.iOS.framework/Versions/Current/SDKs/MonoTouch.iphoneos.sdk/usr/include/xamarin/runtime.h:21:
/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/SDKs/MonoTouch.iphoneos.sdk/usr/include/xamarin/runtime-generated.h:86:1: error: must use 'enum' tag to refer to type 'MarshalManagedExceptionMode'
MarshalManagedExceptionMode
^
enum
/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/SDKs/MonoTouch.iphoneos.sdk/usr/include/xamarin/runtime-generated.h:89:1: error: must use 'enum' tag to refer to type 'MarshalObjectiveCExceptionMode'
MarshalObjectiveCExceptionMode
^
enum
In file included from ./mono_embeddinator.c:29:
In file included from ./mono_embeddinator.h:32:
In file included from ./mono-support.h:32:
In file included from /Library/Frameworks/Xamarin.iOS.framework/Versions/Current/SDKs/MonoTouch.iphoneos.sdk/usr/include/xamarin/xamarin.h:14:
/Library/Frameworks/Xamarin.iOS.framework/Versions/Current/SDKs/MonoTouch.iphoneos.sdk/usr/include/xamarin/runtime.h:88:18: error: pointer to non-const type 'const Protocol *' with no explicit ownership
        const Protocol **protocols; // the corresponding native protocols
                        ^
3 errors generated.

Done

Mine is broken in the same way. :(

Until this is sorted out, does anyone know if it is possible to download the previous version of VS Mac?

Unfortunately it seems like there's no way to download previous releases curently: https://forums.xamarin.com/discussion/123413/install-previous-version-of-visual-studio-for-mac

Awesome, thanks @chamons, back in business here with 15.6.

I've done some homework and can confirm that https://github.com/xamarin/xamarin-macios/commit/c9fac28a8f6840c069ad715b43f1a0192eb38ef6 needs to be added to 15-7 to un-break E4K. I will look at getting it added, but you can downgrade XI/XM to 15.6 or using master until then.
Mono/VSfM should be able to upgrade to 15.7 without issues (beyond the we didn't test mismatched version thing).

You can also work around this by installing XM/XI from the alpha channel, which has the fix from master already.

Thx for the hint, @chamons. I can confirm that it works with the bits from the alpha channel.

Confirming upgrading to the alpha channel fixes the issue

Ok, I'm going to close this as the fix will be pushed out more general to stable soon (it is in Alpha). Given how long it takes to push releases, I won't be able to get it out to stable sooner even if I worked on a service release.
Sorry for the trouble everyone!

I have the following (Latest) version, I am still having above errors, if I downgrade to Xamarin.iOS (<IP_ADDRESS>) and Xamarin.mac (<IP_ADDRESS>) it unbreaks the E4k. My question is, do we have a fix in the following version or still the fix exists in the alpha channel ?

Xamarin.Mac
Version: <IP_ADDRESS> (Visual Studio Community)

Xamarin.iOS
Version: <IP_ADDRESS> (Visual Studio Community)
Hash: 64fece5f
Branch: d15-7
Build date: 2018-05-29 20:00:44-0400

15-7 should have this fix @emysa341 - Could you open a new issue with a build log / information requested so we can figure out how this regressed.

Another data point:  I've been running the downgraded Xamarin.Mac version <IP_ADDRESS> for a few months now.  Today, 2018-08-29, I upgraded my Visual Studio for Mac using the stable branch, which included upgrades to Mono (<IP_ADDRESS> -> <IP_ADDRESS>1) and Xamarin.Mac (<IP_ADDRESS>).  I started to get this error from Embeddinator:
error MM2002: Failed to resolve "System.Diagnostics.PerformanceCounter" reference from "System, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=b77a5c561934e089"
I downgraded only Xamarin.Mac back to <IP_ADDRESS> to resolve the error.  Embeddinator outputs a valid framework bundle, which I'm still able to use in my Objective-C application.

@chamons I updated the visual studio again, it got fixed. Thanks though.

@jnajerasc That shouldn't have changed unless it in the mono bump.
Consider posting on the forum or raising and issue on xamarin-macios - https://github.com/xamarin/xamarin-macios/issues/new

@chamons Thanks for convert, I updated the mono, it got fixed.


Gossip trimming works only when Trimming is ON

https://steamuserimages-a.akamaihd.net/ugc/2057625097159429500/5F8E7E20B42AAA11191D3EABFDC15EB9963AFEAB/

Works both for ON and OFF, cannot replicate.

Well there is an issue. Appeared in Wilhelmina game, turn 265. But the trimming was ON.


Remove FAQ text about read-only resources without IDs

This seem to stem from an older version of the spec.

This seem to stem from an older version of the spec.

That's true. Thanks for catching this @wecc!


Firebase Analytics triggers a session_start event when app is launched in the background by the iOS from a push notification

Description
We have multiple iOS apps in production which are using Firebase Analytics and our own internal analytics system in parallel to log their sessions. Our internal analytics system is setup to log all app foreground launches & moves from background to the foreground. When comparing those numbers to the session counts produced by Firebase Analytics we noticed suspiciously high session counts on Firebase Analytics side and decided to investigate further.
When the app is launched in iOS simulator and the -FIRDebugEnabled launch argument is included, the session_start event is emitted by Firebase SDK when the simulator receives a push notification payload with content-available: 1 flag, causing the OS to launch the app to the background. This only happens when more than 30 minutes has passed since the last session start event.
Based on Firebase documentation this should not be happening:

a session initiates when a user either opens your app in the foreground or views a page or screen and no session is currently active (e.g. their previous session has timed out).

In the end, we were able to reduce our GA4 session counts considerably by removing the content-available: 1 field from our push payloads.
A potentially related issue which has been already closed: https://github.com/firebase/firebase-ios-sdk/issues/6161
Reproducing the issue

Have an iOS app with push notifications & Remote notifications background mode enabled
Add -FIRDebugEnabled launch argument in Xcode scheme editor
Select Launch: Wait for the executable to be launched in Xcode scheme editor
Select Product -> Run
Send a push notification to that app in simulator
Note that the app is launched into background by the iOS
Note that session_start event is triggered (even though the app is in background), if more than 30 minutes has passed since the last push

Firebase SDK Version
10.18.0
Xcode Version
15.1
Installation Method
Swift Package Manager
Firebase Product(s)
Analytics
Targeted Platforms
iOS

@nzhuk I'm seeing this exact issue (raised an issue with Firebase also https://github.com/firebase/firebase-ios-sdk/issues/12347)! I have been trying to resolve this for a while without having to alter the silent pushes being sent to the iOS app (because these are being sent by a third party for uninstall tracking). I haven't had any joy yet. I think this is a bug in the Firebase Analytics SDK.

We are experiencing the same. Any luck with it?

@javierservatepoq unfortunately not.
I don't think any engineers at Firebase have looked into it, and as the Analytics library is closed source it's not possible to debug our side.
I'm surprised that Firebase haven't noticed and addressed this before its quite a big issue 😅
Are all of your sessions skewed?

@peterfriese apologies for tagging you but is there any chance you could help get this prioritised.
As Firebase is now the source of truth for GA4 (can no longer use Segment -> GA3) this is quite a big issue 😅

This is happening also to us, we have false numbers. Any updates on this?

This is happening also to us, we have false numbers. Any updates on this?

Currently in communication with Firebase/Google on this. We are on a paid support plan so hoping to get some more traction on a fix.

I'm client that works closely with Google, they've replied with below:

We log the session start event during initialization,by that time UIApplicationDidBecomeActiveNotification] has not been received yet. That is, we don't know if it's a foreground or background launch.
This issue was already reported and our engineering teams are looking into it. However, we don't have any ETA for the fix right now and will keep you posted when I hear more on this.

Was there a fix for this in recent versions? We've updated our Analytics SDK to 11.0.0 in October 4 and we've experience a drop in sessions compared to the previous month.


[8/x] Add label truncation for long numbers

What does this PR do and why?
Adds a fancy function to display long numbers using K, M, B, T notation, ie 100K, 2M, 5B
Acceptance checklist

[ ] I have evaluated the Approval Guidelines for this PR.
[ ] I have tested this PR in all supported browsers.
[ ] I have tested this PR in dark mode and light mode (if applicable).

Current dependencies on/for this PR:

main

PR #1586

PR #1587

PR #1588

PR #1594

PR #1595

PR #1596

PR #1597

PR #1598   👈

This comment was auto-generated by Graphite.

Graphite Merge Job
Current status: ⏳ Queued to merge
This pull request is currently queued to be merged as part of a stack.
This comment was auto-generated by Graphite.

Job Reference: 5peKCVtRG7hIeC6BSrBI

Graphite Merge Job
Current status: ⏳ Queued to merge
This pull request is currently queued to be merged as part of a stack.
This comment was auto-generated by Graphite.

Job Reference: 0agyc4HkFZeecGjH4TTU

Graphite Merge Job
Current status: ⏳ Queued to merge
This pull request is currently queued to be merged as part of a stack.
This comment was auto-generated by Graphite.

Job Reference: I3QazDznUYZOKiV2bxLz


Add option to disable Uglify.

Uglify can't parse ES6 syntax, and the feature request to support ES6 syntax in Uglify is >3 years old (https://github.com/mishoo/UglifyJS2/issues/448).  For projects that feed ES6 source into Storybook (e.g., as output from tsc targetting ES6), Uglify gets in the way.
A configuration option to disable Uglify in Storybook's webpack config was suggested here: https://github.com/storybooks/storybook/issues/788#issuecomment-294299492, and would be a great help!

We can switch to babili plugin. I think that would solve both issues

@ndelangen I created a PR for this, however, I did not create any configurability for the minifier.

I am running into this issue as well. Right now our workaround is to remove the UglifyJS plugin manually through "full-control" mode, which is a bit of an ugly hack.
Hopefully this can be merged soon?

This will be in 3.2.0, right? Looking forward to that :)

@bigfish this is not a change about the be released with 3.2.0, if you want to remove Uglify, you'll need to do so using webpack full control mode.

+1000 on this.  Would very much rather pass a single flag to turn things off than adopt full-control mode.

I will give this on my high priority list to work on after the 3.3 release.
Related:
https://github.com/storybooks/storybook/issues/1570#issuecomment-345896597


Add security group, improve instance

Add security group, with acceptance test
Improve the mechanism for looking up instances
Works with https://gist.github.com/markfuller/95f216af719b7f54e2164beac8c91a62

Superseded by https://github.com/lyraproj/lyra/pull/33


Can't disable traefik in HA

Environmental Info:
K3s Version:  k3s version v1.19.3+k3s3 (0e4fbfef)
Node(s) CPU architecture, OS, and Version:
Linux 5.8.0-29-generic #31-Ubuntu SMP Fri Nov 6 12:37:59 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
Linux 5.8.0-1007-raspi #10-Ubuntu SMP PREEMPT Thu Nov 5 18:01:40 UTC 2020 armv7l armv7l armv7l GNU/Linux
Cluster Configuration: 3 masters, 1 worker
Describe the bug:
I installed a HA cluster on 2 Raspberry Pis and 2 x86_64 boxes, but I forgot to disable traefik when installing the server/master nodes.  I've gone through and purged whatever traces of traefik I can find according to the documentation, but it still randomly respawns.. often a few hours after I've finished removing it.
Steps To Reproduce:

Installed K3s, but forgot to disable traefik.  Oops.
Removed running traefik pods with kubectl -n kube-system delete helmcharts.helm.cattle.io traefik
Removed traefik manifest with rm /var/lib/rancher/k3s/server/manifests/traefik.yaml from all 3 master nodes.
Added --disable traefik to /etc/systemd/system/k3s.service on all 3 master nodes and ran systemctl daemon-reload.
Wait several hours.
Traefik rises from the ashes and starts running again, clobbering existing services I have running on port 80 and 443.

Expected behavior:
Traefik remains disabled.
Actual behavior:
Traefik respawns for an unknown reason.
Additional context / logs:

Added --disable traefik to /etc/systemd/system/k3s.service on all 3 master nodes and ran systemctl daemon-reload.

You need to restart k3s after reloading the systemd configuration. That should do it.

I went ahead and ran sudo systemctl daemon-reload && sudo systemctl restart k3s on all the master nodes. Within a few seconds of one of them coming back up, a fresh copy of traefik emerged.  This roughly matches my memory that I physically restarted some / all of the masters during my early troubleshooting steps.
Is there anywhere else it could be hiding?  My cluster is new enough that I can tear it down and start fresh with --disable traefik when installing but this feels like a very strange bug.
kube-system                    helm-install-traefik-7bpsh                       1/1     Running       0          35s

I've also tried purging the addons, since it sounds like that's how Helm charts are getting automagically added:
 kubectl delete addon traefik -n kube-system
 kubectl delete addon traefik-config -n kube-system

Will wait to see if it rises again.

I decided to tear down and rebuild the cluster with --disable traefik from the start, closing.


Improve test code

Currently the test code just prints the responses to console which has to be checked manually.
We need to improve it with assert modules.

Merged, closing.


API-1835: add mutation tracking to the fake client

Adds the ability to track mutations to our manifest client with unit tests.  We can build on this for comparing predicted versus actual mutations.

lgtm
let's give it a try.


Update metrics and property keys

Update the metrics and property keys in Alluxio docs.

@Xenorith i create this PR targetting branch-2.7 to prevent any deviation between branch-2.7 and master

Codecov Report

:exclamation: No coverage uploaded for pull request base (branch-2.7@766023b). Click here to learn what that means.
The diff coverage is n/a.

:exclamation: Current head f86ae28 differs from pull request most recent head a2cfa6e. Consider uploading reports for the commit a2cfa6e to get more accurate results

@@              Coverage Diff              @@
##             branch-2.7   #14391   +/-   ##
=============================================
  Coverage              ?   16.96%
  Complexity            ?     2707
=============================================
  Files                 ?     1495
  Lines                 ?    87476
  Branches              ?    10424
=============================================
  Hits                  ?    14842
  Misses                ?    71396
  Partials              ?     1238

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 766023b...a2cfa6e. Read the comment docs.

you will also create the corresponding one for master branch too? otherwise may cause merge conflicts if cherry picking code that changes lines next to these changes

https://github.com/Alluxio/alluxio/pull/14392 the master side PR

maybe need to regenerate since people seem to be updating lots of descriptions for metrics?

after nearly all PRs are merged, i can redo this one


Error while verifying the code

Hi @taurindb  I'm running this on Arduino 1.5.6-r2, after following the steps and trying to verify the code, I get this error:
Sketch uses 28,732 bytes (100%) of program storage space. Maximum is 28,672 bytes.
Global variables use 987 bytes (38%) of dynamic memory, leaving 1,573 bytes for local variables. Maximum is 2,560 bytes.
processing.app.debug.RunnerException: Sketch too big; see http://www.arduino.cc/en/Guide/Troubleshooting#size for tips on reducing it.
	at processing.app.Sketch.size(Sketch.java:1676)
	at processing.app.Sketch.build(Sketch.java:1590)
	at processing.app.Sketch.build(Sketch.java:1509)
	at processing.app.Editor$DefaultRunHandler.run(Editor.java:1915)
	at java.lang.Thread.run(Thread.java:695)

Anyone has the same issue?

I fixed it by using Arduino 1.5.8 with Java 7. I guess the core for the newer version for Arduino is smaller than the previous one.
Hope this will help in case someone get the same error.


Incorrect Namespace

Hi, I very much value your work on this gateway, however it shouldn't be using the Omnipay namespace. Can you please update it to use your own namespace?

Hi, no problem Kayla, that's fixed now. Cheers, Andy.


sideways fix

@voshch
sideways fix: replaced  velocity thread timout with normal ros timer … and added function for the robot to resume driving after stopping.

@voshch
done


Fix lib on flutter 3

It fixes a problem that was  Warning: Operand of
null-aware operation '!' has type 'WidgetsBinding' which excludes null.

Thanks for contributing!


Cannot be executed locally

When the RP site "https://connect-rp.herokuapp.com/" running on Heroku is used, the operation can be completed properly.  It is then believed that the OP site is working properly.
However, when the RP site is running locally (clone it from GitHub, and then set it up according to the given instructions), it returns the error message "Client Registration Failed".
When it is checked at the server console of the local RP site, the following cause of error is found.
{"error":"invalid_redirect_uri","error_description":"Value of one or more redirect_uris is invalid."}

As further information, the following is shown on the server console.
{"application_type":"web","client_name":"NOV RP","subject_type":"pairwise","redirect_uris":["http://[ip of the local machine]:3001/providers/4/open_id"]}

this is just a sample.
you can register any clients here.
moreover, you can run OP on your local.


Upgrade Curator to 5.4.0 [run-systemtest]

I don't like the last commit here, but it's the only way I found to avoid compiler warnings (which will fail the build) in controller-server module


Django 1.8 upgrade

@doctoryes @nedbat @symbolist

:+1:

:+1:

I'm guessing we don't need this any more?

@nedbat It looks like an optimization/cleanup so we never needed it?

@nedbat I'd say if the PR fixes a RemovedInDjango19Warning, it's worth merging. I've fixed a few of these in edx-platform recently. These warning spam the logs and stdout, making us miss any important messages.


Improve code coverage by testing error cases

Coverage increased (+4.92%) to 100.0% when pulling 13195a086673c1cf04a2e09a664ee12cafae87a1 on feature/improve-code-coverage into 6316ffde25d5204ded1048fe354f9f1f136afe08 on master.

Coverage increased (+4.92%) to 100.0% when pulling fe39b2ee69543f28956c8406df6cc42d15817035 on feature/improve-code-coverage into 6316ffde25d5204ded1048fe354f9f1f136afe08 on master.


Plugin makes Homebridge reboot

Hi
today I have experienced this issue. All was working fine until I finish to play a movie on Disnay+ app and I turned off the fire stick. At that point homebridge restarted diplaing this error:
[13/03/2021, 19:03:12] TypeError: Cannot read property 'split' of undefined
at /usr/local/lib/node_modules/homebridge-adb/index.js:609:27
at ChildProcess.exithandler (child_process.js:299:7)
at ChildProcess.emit (events.js:315:20)
at maybeClose (internal/child_process.js:1048:16)
at Socket. (internal/child_process.js:439:11)
at Socket.emit (events.js:315:20)
at Pipe. (net.js:673:12)
[13/03/2021, 19:03:12] Got SIGTERM, shutting down child bridge process...
[13/03/2021, 19:03:12] Got SIGTERM, shutting down child bridge process...
From that moment it was not possible to restart homebridge without disable homebridge-adb plugin.
So I disabled it and I installed the version 1.3.1 and I received the same problem.
While homebridge was stopping and restarting in a loop I turned on the fire stick and then I turned it off again.
After that homebridge ends the loop and started correctly. I update to version 1.4.0 and all continue to work fine.
So, now it is working. I solved the problem turning on and off the fire stick : it seems that it sends some bad signals to the plugin...
I don't know if my explanation can help...

Thank you, it's a very clear explanation, I'll try to fix this error in the next update.

The latest update might solve this problem, please try it and let me know the result.


Why does Google Cloud Buildpacks choose to use ubuntu:22.04 rather than "Distroless" Container Images as the build and run base image?

I wonder know what is the technical considerations behind this decision? Because "Distroless Container Images" project is also from Google and is well know for its security and performance,that aligns well with the security and efficiency goals of serverless  platforms.

Hey good question!  GCP's buildpacks are mostly used for Serverless use cases, things like App Engine, Functions, and Cloud Run. This lets us curate the runtime environment to support those developers, and it also means that nobody has to think about writing Dockerfiles! However, one of the tradeoffs of using Buildpacks is that it's not very straightforward for the end user to customize the build or run base images (aka the Stack). To support as many use cases as possible, we curate Stacks with system packages to capture as many use-cases as possible.
We've thought about supporting Distroless container images, but always struggle around the potential UX for the customers. There's open questions around how we would allow you to customize the build and run image, without telling you to just build your own stack.
I'd be curious to know how you'd want to use distroless + Buildpacks...maybe there's a use case that I'm not thinking of!


template out kube-rbac-proxy image

What this PR does / why we need it:
Templates out kube-rbac-proxy image in Helm charts so it can be overridden in values.yaml.
Type of changes

[x] New feature (non-breaking change which adds functionality)

Release note:
NONE.

/lgtm
/approve


add a 'closed and locked' connection state (and corresponding messages)

A closed connection can be reopened from both sides. Closing a connection prevents messaging spam but allows the remote end to reopen. Closing a reopened connection leads to the same state as before. Therefore, the current way to deal with this would be ignoring the reopened connection. That is difficult/cumbersom to implement as the client has to remember the decision to 'ignore' the reopened connection.
Solution: add the option to 'close and lock' the connection, in which state it can only be reopened by the side that locked it.

We'll make this change when we need it. For now, the issue does not help us.


[BUG]

Running it locally on localhost:3000 with nodejs at version 18.10 and trying any of the templates in the marketplace I am getting this error in the chat Error: Azure OpenAI API instance name not found and in the terminal I am getting
npx flowise start
Starting Flowise...
⚡[server]: Flowise Server is listening at 3000
📦[server]: Data Source has been initialized!
Error: Azure OpenAI API instance name not found
at new ChatOpenAI (/usr/local/lib/node_modules/flowise/node_modules/langchain/dist/chat_models/openai.cjs:239:23)
at ChatOpenAI_ChatModels.init (/usr/local/lib/node_modules/flowise/node_modules/flowise-components/dist/nodes/chatmodels/ChatOpenAI/ChatOpenAI.js:139:23)
at buildLangchain (/usr/local/lib/node_modules/flowise/dist/utils/index.js:212:72)
at async App.processPrediction (/usr/local/lib/node_modules/flowise/dist/index.js:611:44)
at async /usr/local/lib/node_modules/flowise/dist/index.js:362:13
Can you help with that please?

Sure

Git cloned the repo
In the flowise directory npm install -g flowise
After installing in the same directory npx flowise start
http://localhost:3000
From the Marketplace create a conversational AI but any other gave me the same error
Screenshot

you dont have to clone the repo if you are using npx flowise start.
just follow the quick guide - https://github.com/FlowiseAI/Flowise#quick-start and make sure you are above node >= 18.15.0

Sorry - but has the same error I completely delete the directory and didn't cloned the repo but it's the same thing in Azure Web app works perfect getting the image from docker

Correction now also in Azure I have the same problem

Correction now also in Azure I have the same problem

You have not entered the open AI API key in the embedding node.

Good call - Thank you but still not working in localhost.


Minor documentation, code bugs

The Documentation file README.md does not list one role defined in the code - Approver.
While the code does have this role defined on lines 1124-1128, the ValidateSet on line 79 does not have this role included.

Hi @bpgplus ,
Thanks for pointing that out
I have fixed the issue and will push to main shortly
Thanks,
Assaf


⚠ Fortnite has degraded performance

In 745f285, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1665 ms

Resolved: Fortnite performance has improved in 2d0097d.


GUI crash

1.19 version crashes when using /seed gui, but works with anything else

Maybe a mod incompatibility, try with only seedcrackerx


clustertemplate role binding namespace not found

hey guys i wanna to ask you is the template really valid ? because with that i cannot find my namespaces
namespace: c-m-{{ trunc 8 (sha256sum (printf "%s/%s" $root.Release.Namespace $root.Values.cluster.name)) }}

Hi!
Rancher 2.7 seems to have change to a random id since
https://github.com/rancher/rancher/commit/cc9f7eed4264a9805c69efcbd9b7856a24dbc6b1
Make it much harder to do project and fix permissions for the created clusters.

+1 still a problem in rancher v2.7.3

+1


[EventRequest] codeunit 11 "Gen. Jnl.-Check Line" OnBeforeCheckICPartner add parameter GenJnlLine

This request is linked to a previous one : https://github.com/microsoft/ALAppExtensions/issues/15714
We need to Check IC Partner depending of information in "Gen. Journal Line". Proposed solution cannot be done because GenJnlLine is not global and GlobalVarAccess = False
Can you add GenJnlLine: Record "Gen. Journal Line" in parameter to CheckICPartner and OnBeforeCheckICPartner
local procedure CheckICPartner(**GenJnlLine: Record "Gen. Journal Line"**; AccountType: Enum "Gen. Journal Account Type"; AccountNo: Code[20]; DocumentType: Enum "Gen. Journal Document Type")
    var
        Customer: Record Customer;
        Vendor: Record Vendor;
        ICPartner: Record "IC Partner";
        Employee: Record Employee;
        CheckDone: Boolean;
    begin
        OnBeforeCheckICPartner(**GenJnlLine** , AccountType, AccountNo, DocumentType.AsInteger(), CheckDone);
        if CheckDone then
            exit;

[IntegrationEvent(true, false)]
    local procedure OnBeforeCheckICPartner(**GenJnlLine: Record "Gen. Journal Line";** AccountType: Enum "Gen. Journal Account Type"; AccountNo: Code[20]; DocumentType: Option; var CheckDone: Boolean)
    begin
    end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. Please do not reply to this, as we do not monitor closed issues. If you have follow-up questions or requests, please create a new issue where you reference this one.
Build ID: 47191.


Add TAG storage metadata to Vitess

Pre-submission checklist:
Please check each of these after submitting your pull request:

[ ] Are you only including a repo_url if your project is 100% open source? If so, you need to pick the single best GitHub repository for your project, not a GitHub organization.
[ ] Is your project closed source or, if it is open source, does your project have at least 300 GitHub stars?
[ ] Have you picked the single best (existing) category for your project?
[ ] Does it follow the other guidelines from the new entries section?
[ ] Have you added your SVG to hosted_logos and referenced it there?
[ ] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[ ] Does your project/product name match the text on the logo?
[ ] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[ ] ~30 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

@castrojo should we just point to the github repo of the TAG?

Yeah that probably makes more sense, @cc @jeefy can I just change it here and in the other PR?

do-it.gif


Add missing fail() to QueryTests.

Some tests that expected the Observable to throw where missing a fail() statement.

@Test public void mapToListThrowsOnMapperNull() fails, since the call to employees.first(); doesn't actually throw. Searched for the expected message, and didn't find any. Does this test still reflect the expected behavior?

omg don't tell @swankjesse i forgot these

Does this test still reflect the expected behavior

no. see #121 for how to update the test.

omg don't tell @swankjesse i forgot these

I wont 😉

no. see #121 for how to update the test.

done

Perfect. Thanks for finding and fixing!


Core Encounter timing out in Athena

Workaround: use a table instead of view, drop postalcode which isn't in use.
`drop table core__count_encounter_month;
create table core__count_encounter_month as
with powerset as
(
select distinct
count(distinct P.subject_id) as cnt_subject,
count(distinct E.encounter_id) as cnt_encounter
, enc_class.code as enc_class_code
, start_month
, age_at_visit
, P.gender
, P.race_display
from core__encounter E, core__patient P
where E.subject_ref = P.subject_ref
group by CUBE (enc_class, start_month, age_at_visit, gender, race_display)
)
select  distinct cnt_encounter as cnt
, enc_class_code, start_month
, age_at_visit, gender, race_display
from    powerset
where    cnt_subject >= 10
order by start_month asc, enc_class_code, age_at_visit asc
;`

first reported by Regenstrief (david)

Resolved via #50


XamAR.Core.Exceptions.FeatureNotEnabledEx - Location not enabled.

Hello Everyone
I'm facing this error continuously, when after i have added this package and sample code.
I have also tried by downloading sample code from repo to my local machine and still facing same error

Attaching screenshot and stack trace
at XamAR.Core.Sensors.LocationMonitor.StartTracking () [0x0013d] in <4bb7dc889d9d4618ad82e826b8c91b87>:0
at (wrapper dynamic-method) Android.Runtime.DynamicMethodNameCounter.25(intptr,intptr)
at (wrapper native-to-managed) Android.Runtime.DynamicMethodNameCounter.25(intptr,intptr)
I have made sure to added permission in android manifest but still facing this same issue,

Hello, have you enabled Location on your Android device?


Could not find gem 'omniauth-open_id (~> 1.0.1) x86-mingw32'

My gemfile looks like this:
gem 'omniauth-google'
gem 'omniauth-twitter'
gem 'omniauth-facebook'
gem 'omniauth-open_id', '~> 1.0.1'```

But I get this error:

~/app> bundle install
Fetching source index for http://rubygems.org/
Could not find gem 'omniauth-open_id (~> 1.0.1) x86-mingw32' in any of the gem sources listed in your Gemfile.

Maybe an issue between omniauth-open_id and rubygems? All the other oauth providers installed fine.

Adding source 'http://gems.github.com' didn't help.

It's omniauth-openid not omniauth-open_id :)

I can see where I got confused, I think there's a symbol in the library that is :open_id:
http://forrst.com/posts/Sign_in_with_Google_OpenID_Facebook_OAuth_with-9nJ


expose version info

We could add this to upstream.

We could add this to upstream.

Yeh that's the aim


with_server helper should not use stubs for the stub_no_typechecker option

Description
stubs comes from Mocha, which is not a dependency of ruby-lsp. So if the addon doesn't use Mocha but sets stub_no_typechecker to true, this line will cause NoMethodError:
     NoMethodError:
       undefined method `stubs' for an instance of RubyLsp::GlobalState

We could probably refactor so that it's an option passed to RubyLsp::Server.new, then no stubbing would be needed.


Audubon Core Maintenance Group call 20:00 UTC, Wednesday, 2 February 2022

The Audubon Core Maintenance Group will have an online meeting at 20:00 UTC Wednesday, 2022-02-02. That corresponds to 3:00 PM Wednesday eastern (US) standard time (New York), 2:00 PM Wednesday central (US) standard time (Chicago), 10:00 AM Wednesday Hawaii standard time (Honolulu), 7:00 AM Thursday Australian eastern daylight time (Canberra), 8:00 PM Wednesday GMT (London), and 9:00 PM Wednesday central European time (Brussels).
We will discuss priorities for Audubon Core in 2022.
Meeting details:
Join Zoom meeting:
https://vanderbilt.zoom.us/j/93242348096
Find your local number: https://vanderbilt.zoom.us/u/abVZNA3Iqu

Meeting notes available at https://github.com/tdwg/ac/blob/master/historical/2022-02-02-meeting-notes.pdf


Convert s2orc pdf2text json output to txt

Convert the json output from s2orc-pdf2text extractor to a .txt file.
Get only "text" from the json output file, concatenate them and write to a .txt file. Upload the .txt file to same dataset in clowder.
This will be built on top of https://github.com/clowder-framework/extractors-s2orc-pdf2text/pull/7

PR merged. Closing this issue.


added autofocus support

PR to add autofocus support.
Any focusable element with the autofocus attribute set will be focused once the modal is shown. Also works for child modals.
<input type="text" autofocus />

Thanks @thermiteplasma 🙌


Bug: KeyError: 'centroid_lat'

Describe the bug
Traceback (most recent call last):
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3361, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'centroid_lat'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py", line 218, in <module>
    main()
  File "/home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py", line 203, in main
    datamodule = get_data_module()
  File "/home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py", line 162, in get_data_module
    data_module.prepare_data()
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pytorch_lightning/core/datamodule.py", line 428, in wrapped_fn
    fn(*args, **kwargs)
  File "/home/jack/dev/ocf/nowcasting_dataset/nowcasting_dataset/dataset/datamodule.py", line 146, in prepare_data
    self.gsp_data_source = GSPDataSource(
  File "<string>", line 14, in __init__
  File "/home/jack/dev/ocf/nowcasting_dataset/nowcasting_dataset/data_sources/gsp/gsp_data_source.py", line 69, in __post_init__
    self.load()
  File "/home/jack/dev/ocf/nowcasting_dataset/nowcasting_dataset/data_sources/gsp/gsp_data_source.py", line 84, in load
    lat=self.metadata["centroid_lat"], lon=self.metadata["centroid_lon"]
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/geopandas/geodataframe.py", line 1299, in __getitem__
    result = super(GeoDataFrame, self).__getitem__(key)
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/frame.py", line 3458, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
    raise KeyError(key) from err
KeyError: 'centroid_lat'

To Reproduce
Running prepare_ml_data.py on leonardo
Things I've tried to fix it
I tried changing centroid_lat to cen_lat in gsp_data_source.py but that didn't help :(

Any chance there are the logs before the error hits? in debug mode? That'll help a little bit

Sure, here's the log after I changed centroid_{lat,lon} to cen_{lat,lon}:
(nowcasting_dataset) jack@leonardo:~/dev/ocf/nowcasting_dataset/scripts$ /../prepare_ml_data.py

2021-10-04 08:08:31,779 INFO /home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py 96 Checking if all paths exist...
2021-10-04 08:08:32,036 INFO /home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py 112 Success!  All paths exist!
2021-10-04 08:08:32,106 INFO /home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py 161 prepare_data()
15 bad PV systems found and removed!
pv_power = 400.0 MB
/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/indexing.py:1773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_single_column(ilocs[0], value, pi)
/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/indexing.py:1773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_single_column(ilocs[0], value, pi)
/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/indexing.py:1773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_single_column(ilocs[0], value, pi)
/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/indexing.py:1773: SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame.
Try using .loc[row_indexer,col_indexer] = value instead

See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy
  self._setitem_single_column(ilocs[0], value, pi)
Traceback (most recent call last):
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3361, in get_loc
    return self._engine.get_loc(casted_key)
  File "pandas/_libs/index.pyx", line 76, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/index.pyx", line 108, in pandas._libs.index.IndexEngine.get_loc
  File "pandas/_libs/hashtable_class_helper.pxi", line 5198, in pandas._libs.hashtable.PyObjectHashTable.get_item
  File "pandas/_libs/hashtable_class_helper.pxi", line 5206, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'cen_lat'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py", line 218, in <module>
    main()
  File "/home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py", line 203, in main
    datamodule = get_data_module()
  File "/home/jack/dev/ocf/nowcasting_dataset/scripts/./prepare_ml_data.py", line 162, in get_data_module
    data_module.prepare_data()
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pytorch_lightning/core/datamodule.py", line 428, in wrapped_fn
    fn(*args, **kwargs)
  File "/home/jack/dev/ocf/nowcasting_dataset/nowcasting_dataset/dataset/datamodule.py", line 146, in prepare_data
    self.gsp_data_source = GSPDataSource(
  File "<string>", line 14, in __init__
  File "/home/jack/dev/ocf/nowcasting_dataset/nowcasting_dataset/data_sources/gsp/gsp_data_source.py", line 69, in __post_init__
    self.load()
  File "/home/jack/dev/ocf/nowcasting_dataset/nowcasting_dataset/data_sources/gsp/gsp_data_source.py", line 84, in load
    lat=self.metadata["cen_lat"], lon=self.metadata["cen_lon"]
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/geopandas/geodataframe.py", line 1299, in __getitem__
    result = super(GeoDataFrame, self).__getitem__(key)
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/frame.py", line 3458, in __getitem__
    indexer = self.columns.get_loc(key)
  File "/home/jack/miniconda3/envs/nowcasting_dataset/lib/python3.9/site-packages/pandas/core/indexes/base.py", line 3363, in get_loc
    raise KeyError(key) from err
KeyError: 'cen_lat'
(nowcasting_dataset) jack@leonardo:~/dev/ocf/nowcasting_dataset/scripts$

oh I might know what it is - can you delete the local file gsp_shape. It should be in the 'data_sources/gsp' folder. Then try it again?

if that doesnt work, could you cahgne the logging to debug, and I should be able to help more

Sure, thanks, I'll give that a go!
These files exist in the data_sources/gsp/gsp_shape folder:
gsp_shape.cpg  gsp_shape.dbf  gsp_shape.prj  gsp_shape.shp  gsp_shape.shx
Should I delete all of them?

yea delete them, they will reappear next time you run things, and hopefully with that variable

Nice, yup, I deleted the whole data_sources/gsp/gsp_shape folder and it seems to be running nicely now!  Thanks!
In the future, is there any way to automatically check if the data_sources/gsp/gsp_shape folder should be deleted?  If not, it's probably sufficient to write some documentation somewhere on how to delete nowcasting_dataset 'cache' of files.

Yea - its only if the data structure changes of that object. Could always add a try statement, and if it fails, force a new download, - or something like that

that sounds like a good plan :)


Update Trailer to version 1.3.0

This commit updates the version number to 1.3.0 and the download SHA

Looks good, thanks! :+1:


fix: delete bufs if cannot close window (#188)

not affiliated with the project, just a user, but I confirm that this PR seems to fix nvim-treesitter#188 for me. As the PR only adds 1 line, hopefully it should be quick to review and either accept/reject
NVIM v0.8.3
Build type: Release
LuaJIT 2.1.0-beta3
Compiled by builduser

Features: +acl +iconv +tui
See ":help feature-compile"

WSL2 on Win11, Arch Linux
Applied patch to lastest nvim-treesitter:master (959265a)

Ok, let's give it a go.


ran into some issues running the docker run command

I cloned the folder
git clone https://github.com/jpetazzo/dind.git
ran this...
tcruickshank@tom-build:~/test/dind$ docker build -t dind .
then....
tcruickshank@tom-build:~/test/dind$ docker run --privileged -t -i dind
ln: failed to create symbolic link '/sys/fs/cgroup/systemd/name=systemd': Operation not permitted
Command "daemon" is deprecated, and will be removed in Docker 1.16. Please run dockerd directly.
INFO[0000] libcontainerd: new containerd process, pid: 69
INFO[0001] Graph migration to content-addressability took 0.00 seconds
WARN[0001] Your kernel does not support swap memory limit.
WARN[0001] Your kernel does not support cgroup rt period
WARN[0001] Your kernel does not support cgroup rt runtime
INFO[0001] Loading containers: start.
WARN[0001] Running modprobe bridge br_netfilter failed with message: modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-57-generic/modules.dep.bin'
modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-57-generic/modules.dep.bin'
, error: exit status 1
WARN[0001] Running modprobe nf_nat failed with message: modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-57-generic/modules.dep.bin', error: exit status 1
WARN[0001] Running modprobe xt_conntrack failed with message: modprobe: ERROR: ../libkmod/libkmod.c:556 kmod_search_moddep() could not open moddep file '/lib/modules/4.4.0-57-generic/modules.dep.bin', error: exit status 1
INFO[0001] Default bridge (docker0) is assigned with an IP address <IP_ADDRESS>/16. Daemon option --bip can be used to set a preferred IP address
INFO[0001] Loading containers: done.
INFO[0001] Daemon has completed initialization
INFO[0001] Docker daemon                                 commit=49bf474 graphdriver=aufs version=1.13.0
INFO[0001] API listen on /var/run/docker.sock

Hi,
I would suggest that you try the official, upstream dind images. Check for the images tagged :dind in the official docker images on the Docker Hub!


Article may not account for global enterprises with multiple paths into Azure

The article assumes that the enterprise, regardless of size (global or not), has a single pathway for DNS resolution when it comes to Azure endpoints.
Consider adding a best practice for the below scenario for the Cloud Adoption Framework:
An enterprise that has multiple on-prem environments (that have connectivity to eachother), the on-prem environments have their own connectivity path to their own individual Azure Vnet (that don't connect to eachother within Azure).
Now if more than 1 VNet are using Private Endpoints, the Cloud Adoption Framework fails because DNS (at a basic design level) can only go 1 path for resolving a specific Private Link DNS zone, rather than going to the VNet hosting the specific endpoint for resolution.
Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 2bd28f7b-1764-83c9-9d82-d65ba75bcb94
Version Independent ID: e91679fb-9d61-7f89-5cb6-238e2eb91757
Content: Private Link and DNS integration at scale - Cloud Adoption Framework
Content Source: docs/ready/azure-best-practices/private-link-and-dns-integration-at-scale.md
Service: cloud-adoption-framework
Sub-service: ready
GitHub Login: @JefferyMitchell
Microsoft Alias: brblanch

Thanks for reporting - this issue is under review. This is a Microsoft Internal DevOps Bug ID # 414948


Improve options for quoting a message

We should:

Rename the "Quote and reply" option to "Quote message", in the app and any documentation (incl. help center). It's more accurate, as this option inserts the quoted text into whatever message you're composing or editing; it does not necessarily start a new reply.
Add a "Forward message" option just below "Quote message". This option should start a new message with the quoted context (message or selection). The compose box should be opened with the channel picker open, and no topic. This option should be documented in https://zulip.com/help/quote-and-reply#forward-a-message.
The "Quote and reply" help center page should be renamed "Quote or forward a message", and moved to the appropriate URL (with a redirect).

Notes:

As discussed on CZO, we may want to add a "Reply" option as well; let's leave that off for now.

Questions:

Should we keep the "Quote message" and "Forward message" documentation on one page, or split it into two pages?

CZO thread

@zulipbot claim

Hi @grehnen, I saw that you're assigned to this issue. Let me know if you find this difficult for you, as I'd like to raise a PR especially for 'Forward message' feature. Would it be okay if I proceed with that?

@imkurosaki Hi! I'm working on the issue. I just haven't had much time this week as I've been very busy in school. But my plan is to work on it now during this weekend!


Dry cases: qtg=0.0 leads to NaN reference profiles.

The way the initialization of the reference profiles is written, qtg=0.0 leads to NaN fields. This comes from logarithms of zero-valued variables when using eos and eos_first_guess_entropy.
I have for now avoided this problem in the only dry case we have (GABLS) by setting qtg to a very small value and the latent heat flux to zero for the surface conditions of GABLS.
Nevertheless, I think this issue should be resolved by modifying eos. Any thoughts @trontrytel @charleskawczynski @yairchn ?

So, this function is returning NaN for qt=0? I'm not sure why that would happen, there's no division by zero since qd=1-qt, and pd and pv should not be zero. Is this happening everywhere? Or just the ghost points?

we need to have another if command in Eos so that it can handle qt=0 , I am still not sure why moist entropy would be singular at the dry limit...


Handle change event for select option elements

I'm using MixItUp with select form elements and had to add a listener for change events, for option tags. If we don't add the change listener, the plugin works fine on Firefox (Mac) but is broken on Chrome (Mac) and Safari.
Solution found at: http://w3foverflow.com/question/how-to-fix-jquery-filtering-in-chrome/

Filtering via select options is now covered by the MultiFilter extension for MixItUp 3. See https://www.kunkalabs.com/mixitup-multifilter/ for more information.


container mtu size

Describe the bug
log into a container, ping -s 1500 www.cnn.com, it does not work.
To Reproduce
see as above
Expected
ping with any size works
Actual behavior
I do not see packet sent out on uplink, suspect some misconfiguration on ovs br-int. I did not dig deeper.
Versions:
Please provide the following information:

Antrea version (Docker image tag).
Kubernetes version (use kubectl version). If your Kubernetes components have different versions, please provide the version for all of them.
Container runtime: which runtime are you using (e.g. containerd, cri-o, docker) and which version are you using?
Linux kernel version on the Kubernetes Nodes (uname -r).
If you chose to compile the Open vSwitch kernel module manually instead of using the kernel module built into the Linux kernel, which version of the OVS kernel module are you using? Include the output of modinfo openvswitch for the Kubernetes Nodes.

Additional context
Add any other context about the problem here, such as Antrea logs, kubelet logs, etc.
(Please consider pasting long output into a GitHub gist or any other pastebin.)

Pinging a remote Node's gateway form a Pod (on a different Node) works fine for me:
abas-a01:antrea abas$ kubectl --kubeconfig test/e2e/infra/vagrant/playbook/kube/config exec -ti dnsutils -- ping -s 1500 <IP_ADDRESS>
PING <IP_ADDRESS> (<IP_ADDRESS>): 1500 data bytes
1508 bytes from <IP_ADDRESS>: seq=0 ttl=62 time=4.068 ms
1508 bytes from <IP_ADDRESS>: seq=1 ttl=62 time=3.431 ms
1508 bytes from <IP_ADDRESS>: seq=2 ttl=62 time=0.800 ms
1508 bytes from <IP_ADDRESS>: seq=3 ttl=62 time=1.596 ms
1508 bytes from <IP_ADDRESS>: seq=4 ttl=62 time=0.904 ms

I can see the fragmented tunneled packets leaving the Node:
01:33:20.116582 IP (tos 0x0, ttl 64, id 12560, offset 0, flags [DF], proto UDP (17), length 1494)
    k8s-node-worker-1.55215 > <IP_ADDRESS>.4789: [no cksum] VXLAN, flags [I] (0x08), vni 0
IP (tos 0x0, ttl 63, id 33969, offset 0, flags [+], proto ICMP (1), length 1444)
    <IP_ADDRESS> > <IP_ADDRESS>: ICMP echo request, id 36608, seq 8, length 1424
01:33:20.116641 IP (tos 0x0, ttl 64, id 12561, offset 0, flags [DF], proto UDP (17), length 154)
    k8s-node-worker-1.55215 > <IP_ADDRESS>.4789: [no cksum] VXLAN, flags [I] (0x08), vni 0
IP (tos 0x0, ttl 63, id 33969, offset 1424, flags [none], proto ICMP (1), length 104)

And I can confirm that ping -s 1500 www.cnn.com does not work either from my machine. When I ping www.google.com from my Mac, I get a "wrong total length" error if I try any length greater than 64 bytes. www.cnn.com is more forgiving :P

this perhaps not a big deal, I stumbled upon it while testing something else. As long as TCP/UDP works we should be fine.
should we close this?

I see no evidence that this is connected to Antrea, so yes I think we should close this.
The fragmented ICMP IP packets get sent out on the Node network correctly as far as I can tell. Since there is no legitimate reason to have large or fragmented ICMP packets, I imagine some router or other device is dropping them along the way.

That is interesting ...
as I recall, when I tcpdump on gw0, no packets came out, and ovs dp cache indicates dropping fragmented packets due to some rules in table 110 (some conntrack table), I did not dig too deep on this

Pinging from a Pod, I see this on the gw interface:
vagrant@k8s-node-worker-1:~$ sudo tcpdump -i gw0 -vvvv icmp
tcpdump: listening on gw0, link-type EN10MB (Ethernet), capture size 262144 bytes
02:50:31.717895 IP (tos 0x0, ttl 64, id 22886, offset 0, flags [+], proto ICMP (1), length 1444)
    <IP_ADDRESS> > sfo07s16-in-f68.1e100.net: ICMP echo request, id 59392, seq 11, length 1424
02:50:31.717895 IP (tos 0x0, ttl 64, id 22886, offset 1424, flags [none], proto ICMP (1), length 104)
    <IP_ADDRESS> > sfo07s16-in-f68.1e100.net: icmp

and this on the physical interface
vagrant@k8s-node-worker-1:~$ sudo tcpdump -i enp0s3 -vvvv icmp
tcpdump: listening on enp0s3, link-type EN10MB (Ethernet), capture size 262144 bytes
02:50:37.723948 IP (tos 0x0, ttl 63, id 23545, offset 0, flags [+], proto ICMP (1), length 1444)
    k8s-node-worker-1 > sfo07s16-in-f68.1e100.net: ICMP echo request, id 59392, seq 17, length 1424
02:50:37.724025 IP (tos 0x0, ttl 63, id 23545, offset 1424, flags [none], proto ICMP (1), length 104)
    k8s-node-worker-1 > sfo07s16-in-f68.1e100.net: icmp

Are you testing Antrea's master branch with the normal encap mode, or with some of your changes from #340 ?

with my change on kind.

Now that my interest is piqued, I dig deeper with ToT master Antrea
antrea/antrea-ubuntu      latest                     4370742c537d
On ovs table 30 , there is flow
cookie=0x1000000000000, duration=2896.101s, table=30, n_packets=10064, n_bytes=1865632, idle_age=0, priority=200,ip actions=ct(table=31,zone=65520)
basically it tells all packets go through conntrack, thereby generate conntrack connections, and coming back to table 31 for flows examine the state of packet coming through conntrack.
On my setup at least, (I guess) due to fragmented packet, conntrack did not generate a valid connection, and fragmented packets did not get a valid state, and subsequently getting dropped.
we can observe if an icmp conntrack connection is established by "conntrack -E -p icmp". On my kind setup, fragmented icmp package does not generate conntrack connections.
Hope this helps, thx su

Looks like it could be an issue specific to the netdev OVS datapath. I'll take a look.

yeah, it looks like that way. I am pretty sure ovs should send reassembled ip to conntrack ...;
feel free to close this issue
thx, su

I just noticed that the OVS release notes for 2.12.0 have the following:

Add v4/v6 fragmentation support for conntrack.

https://github.com/openvswitch/ovs/blob/master/NEWS#L76
That could solve the issue. We are planning to update the OVS userspace to 2.12, so let's keep this issue open for now, and see if updating resolves it.


Swap rest pose actions in the Skeleton2D editor

The texts were changed before, but the actions weren't swapped around. This led to unexpected behavior.
This closes https://github.com/godotengine/godot/issues/54793.
@irslutz Please double-check this, just in case :slightly_smiling_face:

Thanks!

Cherry-picked for 3.5.

Cherry-picked for 3.4.1.


LEGO: Pull request from lego/hb_04604851-bac4-4681-9f74-73de611d6e48_20210720093450684 to main

LEGO: Pull request from lego/hb_04604851-bac4-4681-9f74-73de611d6e48_20210720093450684 to main with localized lcls

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.003%) to 42.552%

Totals

Change from base Build<PHONE_NUMBER>:
0.003%

Covered Lines:
26727

Relevant Lines:
56924

💛  - Coveralls


Firebase utils : Scroll + Normalize

Hi,
I got a warning/error when I try to mix multiple things (scroll, normalize, $firebaseArray)
var fb = new Firebase(FIREBASE_URL)
var ref = fb.child("users/user1/posts");
var postsRef = fb.child("posts");
var scrollRef = new Firebase.util.Scroll(ref,'$priority');
var norm = new Firebase.util.NormalizedCollection(scrollRef, postsRef).select('posts.from', 'posts.content', 'posts.time');
var userPosts = $firebaseArray(norm.ref());
userPosts.scroll = scrollRef.scroll;
return userPosts;
with the following structure :
{
users: {
user1: {
name: "Alice",
posts: {
post1: true,
post3: true,
... (let's assume there are 100 other posts)
}
},
user2: {
name: "Bob",
posts: {
post2: true
}
}
},
posts: {
post1: {
content: "Hi"
from: user1,
time:<PHONE_NUMBER>010
},
post2: {
content: "Welcome",
from: user2,
time:<PHONE_NUMBER>011
},
post3: {
content: "Ya"
from: user1,
time:<PHONE_NUMBER>012
}
}
}
Here is the error/warning in JavaScript console : "this.off is not a function"

It was only a this which was not available (inside the once function) :
'once': function(event, callback, cancel, context) {
var that = this;
function fn(snap) {
/*jshint validthis:true */
that.off(event, fn, this);
callback.call(context, snap);
}
this.on(event, fn, cancel, this);
},
Line 3097/3098
Hope this helps :)

Nice! Will fix this up.

Great ! Thanks

Yeah I'm having the same issue!

@cameronbourke apply the fix provided by @mellinas; I'll add it into the next release.


Connection problem

hello
I use the rust code in relay mode, everything is correct and they connect to the server and the authentication process is done, but only data is sent to me in about a second and no more data is sent from the server side.
My camera's AP is dynamic, and my system's AP is also dynamic, and without using a relay, it is not able to connect directly to the camera, and it gives a timeout.
This one second of data is sent to me if I use the VLC player.

Sorry for the confusion, but I think I should re-quote my comment https://github.com/khoanguyen-3fc/dh-p2p/issues/3#issuecomment-1859388879

Keep in mind that it's currently an experimental feature as part of a proof of concept, so more effort and help are needed to finalize it.

Thanks for your understanding!


az container export to YAML fails if there is a user-assigned identity

Describe the bug
EDIT: Strikethrough for references to ACI creation from YAML, which now works after I copied an example.
az container does not handle UserAssignedIdentities in YAML during export. Errors occur both during container creation and container export. I will describe the container export issue as that is easier to reproduce.
Command Name
az container export
Errors:
The command failed with an unexpected error. Here is the traceback:
('cannot represent an object', <azure.mgmt.resource.resources.v2021_04_01.models._models_py3.IdentityUserAssignedIdentitiesValue object at 0x7f1a6a27bb20>)
Traceback (most recent call last):
  File "/opt/az/lib/python3.10/site-packages/knack/cli.py", line 233, in invoke
    cmd_result = self.invocation.execute(args)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 663, in execute
    raise ex
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 726, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 697, in _run_job
    result = cmd_copy(params)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/__init__.py", line 333, in __call__
    return self.handler(*args, **kwargs)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/core/commands/command_operation.py", line 121, in handler
    return op(**command_args)
  File "/opt/az/lib/python3.10/site-packages/azure/cli/command_modules/container/custom.py", line 633, in container_export
    yaml.safe_dump(resource, f, default_flow_style=False)
  File "/opt/az/lib/python3.10/site-packages/yaml/__init__.py", line 269, in safe_dump
    return dump_all([data], stream, Dumper=SafeDumper, **kwds)
  File "/opt/az/lib/python3.10/site-packages/yaml/__init__.py", line 241, in dump_all
    dumper.represent(data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 27, in represent
    node = self.represent_data(data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 48, in represent_data
    node = self.yaml_representers[data_types[0]](self, data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 207, in represent_dict
    return self.represent_mapping('tag:yaml.org,2002:map', data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 118, in represent_mapping
    node_value = self.represent_data(item_value)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 58, in represent_data
    node = self.yaml_representers[None](self, data)
  File "/opt/az/lib/python3.10/site-packages/yaml/representer.py", line 231, in represent_undefined
    raise RepresenterError("cannot represent an object", data)
yaml.representer.RepresenterError: ('cannot represent an object', <azure.mgmt.resource.resources.v2021_04_01.models._models_py3.IdentityUserAssignedIdentitiesValue object at 0x7f1a6a27bb20>)

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.
Note that userID is set following the instructions in the Deploy to Azure Container Instances from Azure Container Registry using a managed identity.

Create the container.

az container create \
	--name $aciName \
	--resource-group $resourceGroup \
	--image $privateACR/aci-helloworld:latest \
	--acr-identity $userID \
	--assign-identity $userID \
	--ports 80 \
	--dns-name-label $dnsLabelName

Export to YAML.

az container export --output yaml --name $aciName -g $resourceGroup --output.yaml

Expected Behavior
A YAML file that can be used to recreate the container should be successfully exported.
Environment Summary
Linux-<IP_ADDRESS>-microsoft-standard-WSL2-x86_64-with-glibc2.31, Ubuntu 20.04.5 LTS
Python 3.10.5
Installer: DEB

azure-cli 2.41.0

Extensions:
account 0.2.5
aks-preview 0.5.104
azure-devops 0.25.0
confcom 0.1.2
confidentialledger 1.0.0
subscription 0.1.5
confidentialledger 1.0.0 (dev) /home/lyshi/repos/azure-cli-extensions/src/confidentialledger

Dependencies:
msal 1.20.0b1
azure-mgmt-resource 21.1.0b1

Additional Context
Please disregard this section, preserving it so the error string is searchable.
As mentioned above, YAML does not work for az container create either. In a YAML file that contains a ContainerGroupIdentity object, I have the following:
identity:
  type: UserAssigned
  userAssignedIdentities:
  - "/subscriptions/<SUBSCRIPTION>/resourceGroups/<RESOURCE GROUP>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<IDENTITY NAME>"

After running az container create --resource-group $resourceGroup --file input.yaml, I got the following error:
The command failed with an unexpected error. Here is the traceback:
('Unable to build a model: ("Unable to deserialize response data. Data: [\'/subscriptions/<SUBSCRIPTION>/resourceGroups/<RESOURCE GROUP>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<IDENTITY NAME>\'], {IdentityUserAssignedIdentitiesValue}, TypeError: string indices must be integers", TypeError(\'string indices must be integers\')), DeserializationError: ("Unable to deserialize response data. Data: [\'/subscriptions/<SUBSCRIPTION>/resourceGroups/<RESOURCE GROUP>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<IDENTITY NAME>\'], {IdentityUserAssignedIdentitiesValue}, TypeError: string indices must be integers", TypeError(\'string indices must be integers\'))', DeserializationError("Unable to deserialize response data. Data: ['/subscriptions/<SUBSCRIPTION>/resourceGroups/<RESOURCE GROUP>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<IDENTITY NAME>'], {IdentityUserAssignedIdentitiesValue}, TypeError: string indices must be integers", TypeError('string indices must be integers')))
Traceback (most recent call last):
  File "/opt/az/lib/python3.10/site-packages/msrest/serialization.py", line 1626, in deserialize_data
    return self.deserialize_type[iter_type](data, data_type[1:-1])
  File "/opt/az/lib/python3.10/site-packages/msrest/serialization.py", line 1668, in deserialize_dict
    return {x['key']: self.deserialize_data(x['value'], dict_type) for x in attr}
  File "/opt/az/lib/python3.10/site-packages/msrest/serialization.py", line 1668, in <dictcomp>
    return {x['key']: self.deserialize_data(x['value'], dict_type) for x in attr}
TypeError: string indices must be integers

route to CXP team

Adding Service team to look into this.
@macolso Could you please look into this once you get a chance ?

@navba-MSFT looks like that person no longer works at Microsoft

Okay, I stumbled across an example of how to use userAssignedIdentities in Enable managed identity using YAML file.
identity:
  type: UserAssigned
  userAssignedIdentities:
    {
      '/subscriptions/<SUB ID>/resourcegroups/<RESOURCE GROUP>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<IDENTITY NAME>': {},
      "/subscriptions/<SUB ID>/resourceGroups/<RESOURCE GROUP>/providers/Microsoft.ManagedIdentity/userAssignedIdentities/<IDENTITY NAME>": {}
    }

So, I am at least able to create an ACI using user-assigned identity from a YAML file. However, this doesn't resolve the export command issue.

@yonzhan @navba-MSFT were you able to repro or find someone to assign this to?

@joseph-porter Could you please help take look at the container related issue?

Any update on this? Seems we still cannot export YAML for a container with a user assigned MI

@sam-cogan @lynshi  A PR has been created https://github.com/Azure/azure-cli/pull/24833 to fix this. It is currently under review. I am following up with the product owners to review that PR and I will update this thread.

This can be closed, is fixed in the latest versions


Fix remaining warnings

Removes a unused function and a deprecated Doxygen option.
Fixes #1185

could you also fix the following warning?
/root/ginkgo/include/ginkgo/core/solver/solver_base.hpp:90: warning: missing argument after \enum.
/root/ginkgo/include/ginkgo/core/solver/solver_base.hpp:104: warning: Documentation for undefined enum 'initial_guess_mode' found.

it's shown in make pdf

rebase!


Flesh out pubsub subscriptions

Uses #742 as a base.

@tseaver I'm going to hold off on this until #742 is through. Is that OK or do you prefer otherwise?

Holding off here is fine.

Coverage remained the same at 100.0% when pulling c64c0f2d6eb89c91151cacea07fa74c2186ac255 on tseaver:691-flesh_out_pubsub_subscription into aa684bebe247acd1f2ae48a00f4e88899accea2c on GoogleCloudPlatform:master.

Coverage remained the same at 100.0% when pulling 1d08a879056db6ab244524ffa858f732003ee1b5 on tseaver:691-flesh_out_pubsub_subscription into aa684bebe247acd1f2ae48a00f4e88899accea2c on GoogleCloudPlatform:master.

@dhermes rebased after merge of #742.

LGTM though you should get sign off from @tmatsuo to make sure you are making the correct API calls

Left few comments, but LGTM in terms of the correctness of API paths and methods.

Coverage remained the same at 100.0% when pulling ea1625cc6c708ae4931d64ad8e506e5df06aab00 on tseaver:691-flesh_out_pubsub_subscription into 01d697e1b3f77c6cb56690e138bb011428ff3101 on GoogleCloudPlatform:master.

still LGTM


input kmer quality check

The insert function has to check its input. If integer, it should be <= 2^p (where p is the no of hashbits. If string, it has to match the p/2 characters @mr-eyes

Only kDataframeMQF has to check for the integer value limit

Add warnings in the function documentation. Also add examples for abnormal behaviours in the FAQ


copr: metadata_ctor should return Result

Signed-off-by: TennyZhuang<EMAIL_ADDRESS>
What have you changed?
Please explain in detail what the changes are in this PR and why they are needed:
When use metadata in rpn framework, metadata_ctor may failed due to some expression error.
Now, metadata_ctor return the Metadata directly, It seems that return a Result is better.
What is the type of the changes?
Pick one of the following and delete the others:

Improvement (a change which is an improvement to an existing feature)

How is the PR tested?
Please select the tests that you ran to verify your changes:

Unit test

Does this PR affect documentation (docs) or should it be mentioned in the release notes?
Does this PR affect tidb-ansible?
Refer to a related PR or issue link (optional)
Benchmark result if necessary (optional)
Any examples? (optional)

/test

/test

/merge

@sre-bot /test


Remove knocking build flag and Synapse's experimental config option

As knocking has been stable for a while now (and is even in a released version of the spec), it is now certainly overdue to remove the build flag guarding Complement's knocking tests.
cc @matrix-org/dendrite-core
Recommended to review commit-by-commit.

I believe this is just waiting for an approval on your side @kegsay?

Oh -- let's request review from them then.


Add free ChatGPT API

Hello, i just find a free ChatGPT API, i don't know if it can be used in this project but anyways here's the link of the API

This is specifically for the OpenAI API, if your endpoint is compatible, it should also work


Move arrows that allow user to scroll through profiles to the bottom of the profile page

The reason for this is because it is annoying to scroll all the way down to look for a person and then have to scroll back up to switch pages. It would be a simple UI change.

Why not have them at the bottom and the top? I've seen sites with both!
—
Sent from Mailbox
On Tue, Sep 15, 2015 at 1:02 AM, vkr96<EMAIL_ADDRESS>wrote:

The reason for this is because it is annoying to scroll all the way down to look for a person and then have to scroll back up to switch pages. It would be a simple UI change.
Reply to this email directly or view it on GitHub:
https://github.com/rcos/Observatory3/issues/147

Fixed


Create F2_8.c

The application you will be writing for is intended to run on a small, inexpensive embedded device that does not have a floating point unit. This means that the processor can only do integer arithmetic. However, the device does have a display and needs to display the result of floating point arithmetic to the user.  Because this is a small inexpensive processor there is no support for strings or any mathematical library functions. You must write all the code yourselves performing all the floating point math with integers only and no strings! Character arrays, or C strings, can't tell you their size but they do end with a '\0' character.
The requirements for the code are to write one of the functions presented below. All the functions return a boolean value indicating the success or failure of the operation. Improper data is the most likely cause for a function to fail. You may add your own additional helper functions (I had at least a half dozen) but you must include them for the review. Do not change the interface to these functions.
These two functions take as parameters two sets of mantissa and characteristic and a char array to hold the result of the arithmetic operations. The result of the add or subtract should be converted into char's and placed in the result array. The array must end with a '\0'. The 'len' parameter tells how many characters can be placed on the result array. For these functions to return true you must at a minimum store the characteristic of the result. If the result is a non-integer place as many of the digits as will fit in the result after a decimal point.
bool add(int c1, int n1, int d1, int c2, int n2, int d2, char result[], int len);
bool subtract(int c1, int n1, int d1, int c2, int n2, int d2, char result[], int len);
...
...
char answer[100];
int c1, n1, d1;
int c2, n2, d2;

c1 = 1;
n1 = 1;
d1 = 2;

c2 = 2;
n2 = 2;
d2 = 3;

//if the C string could hold at least the characteristic if(add(c1, n1, d1, c2, n2, d2, answer, 100))
{
    //display string with answer 4.166666666...
}
else
{
    //display error message
}
'''

I am working on this pull request.

Potential Improvements

It will be easier for others to understand the goal and capabilities of each section of the code if there are more comments and documentation.
To avoid possible runtime errors, make sure edge cases like division by zero and overflow are properly handled.

Pros

Integer arithmetic was successfully used to implement the fundamental arithmetic operations, which complies with the specifications for operating on an embedded device without a floating point unit.
The code is well-organized into separate functions, improving readability and maintainability.

Line-Specific Feedback

Make sure the decimal point is positioned correctly and that the logic used to convert the results back into the character array is effective.
Verify the efficiency of the logic used to convert the results back into the character array and the proper placement of the decimal point.


TNO-1213: Create Razor Frontpages Report Template

TNO-1213: Create Razor Template for Frontpages Report

Can you change the migration number to 1.0.33?
Nice to see you build this one so quickly.

The first similar task might be a bit more challenging and time consuming, and after that things could be a bit smoother :)
I have just updated another PR (1209) with 1.0.33, so I guess we can keep the current one unchanged, thanks.


[FLINK-9163[e2e-tests] harden signal traps and config restoration

What is the purpose of the change
Signal traps on certain systems, e.g. Linux, may be called concurrently when the trap is caught during its own execution. In that case, our cleanup may just be wrong and may also overly eagerly delete flink-conf.yaml.
Brief change log

place pattern rm <file>; mv <file.bak> <file> with the more atomic mv -f <file.bak> <file>
reduce unnecessary error messages in the output during early (manual) test aborts
stop complaining about md5sum or md5 not being available if the error is from something else
stop complaining about not being able to delete non-existing log files
disable signal traps while in signal handling (needs to be done in each e2e test that was using this pattern)

Verifying this change
This change adapts the e2e tests and was tested with them also manually by aborting (CTRL-C) during different phases of the execution.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): no
The public API, i.e., is any changed class annotated with @Public(Evolving): no
The serializers: no
The runtime per-record code paths (performance sensitive): no
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: no
The S3 file system connector: no

Documentation

Does this pull request introduce a new feature? no
If yes, how is the feature documented? not applicable

Thanks for the contribution @NicoK!
These are nice fixes for the end-to-end tests.
+1, merging this ...