TextField (and others) - Error messages not read w/ NVDA/JAWS after another keypress

🐛 Bug Report
When an error message is triggered on a textField if there is a 2nd keypress after the message appears then the reading of the message is suppressed with JAWS and NVDA
🤔 Expected Behavior
The error message should be read to the user.
😯 Current Behavior
The user doesn't currently hear the message
💁 Possible Solution

🔦 Context

💻 Code Sample
https://codesandbox.io/s/errormessages-9skph5
Run the testcase and type "test" into the field. Note that JAWS/NVDA so not report an error to the user
Run the testcase again but this time just type "t" into the field. Note that the error is correctly reported here.
🌍 Your Environment

Software
Version(s)

react-spectrum
3

Browser
Chrome/FF w/ NVDA or JAWS

Operating System
Windows

🧢 Your Company/Team

🕷 Tracking Issue (optional)

A possible solution discussed by the team would be to add aria-live but only when field is focused to avoid multiple announcements wheen multiple form fields are marked as invalid at the same time. Any thoughts?


Can the Torch.Save support object?

Hi,
According to the official design, torch.save can save the object state.
Can TorchSharp support it?
https://pytorch.org/docs/stable/generated/torch.save.html
Because I need to save extra information.
https://github.com/DLR-RM/stable-baselines3/blob/f3abda5cbc9dab71fbda44d9feca7efd7bb09dfe/stable_baselines3/common/policies.py#L159

I know that because of the characteristics of C#, using Reflection to create instances will still encounter many problems.
Perhaps it is possible to design an interface that allows developers to choose which attributes need to be extracted

It cannot currently do so.

Will it be supported in the future?
Python has a pickle function library, which can easily save the state of the object
However, this seems to require support from the dotnet team. I think this feature is useful in the field of machine learning

Hi, According to the official design, torch.save can save the object state. Can TorchSharp support it? https://pytorch.org/docs/stable/generated/torch.save.html
Because I need to save extra information. https://github.com/DLR-RM/stable-baselines3/blob/f3abda5cbc9dab71fbda44d9feca7efd7bb09dfe/stable_baselines3/common/policies.py#L159

You could use the System.Text.JSON.JsonConverter  and when use public static void Save(this Tensor tensor, System.IO.BinaryWriter writer) as Nikolas explained. This would do the job of serialisation.

This should do the trick now you can make a class with Tensors in the Parameters and serialize this as Json.
       var options = new System.Text.Json.JsonSerializerOptions();
        options.Converters.Add(new TensorConverter());
        var data = System.Text.Json.JsonSerializer.Serialize(this, options);

using System.Text.Json;
using static TorchSharp.torch;
using TorchSharp;
using System.IO;
namespace Docker.Visual
{
public class JsonTensor
{
public long[] _shape { get; set; }
public  ScalarType _type { get; set; }
public byte[] data { get; set; }
public bool requires_grad { get; set; }
public string[] names { get; set; }
}
public class TensorConverter : System.Text.Json.Serialization.JsonConverter<Tensor>
{

    public override Tensor Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        var txt = reader.GetString();
        var jt = (JsonTensor) System.Text.Json.JsonSerializer.Deserialize(txt, typeof(JsonTensor));
        var tensor = torch.empty(jt._shape, dtype: jt._type, requires_grad: jt.requires_grad, names:jt.names);
        using (var s = new MemoryStream(jt.data))
        { tensor.Load(new BinaryReader(s));}
        return tensor;
    }

    public override void Write(Utf8JsonWriter writer, Tensor value, JsonSerializerOptions options)
    {
        using (var s = new MemoryStream())
        {
            using (var w = new BinaryWriter(s))
            { value.Save(w);}
            var jt =new JsonTensor() { data = s.ToArray(), _shape = value.shape, _type = value.dtype , names = value.names , requires_grad = value.requires_grad};
            var _o = new System.Text.Json.JsonSerializerOptions();
            var data = System.Text.Json.JsonSerializer.Serialize(jt, _o);
            writer.WriteStringValue(data);
        };
    }
}

}

There's also a 'bytes' read/write property on Tensor that allows you to get the byte array of the Tensor. That's what the Tensor.Save()/Load() methods use for serialization.

Ok Nicklas. Its updated.

Can the JsonConverter not take a Span as input? That's too bad, if that's the case, because the bytes.ToArray() call will make a copy.

Niklas. Ok i cannot serialize pointers right now. Maybe Microsoft knows how to do this. :-) Some cloud based rent a pointer business model.

Fair enough. I guess the performance of serialization isn't super-critical, it's not likely something you do for every batch, for example. Probably no more than once every epoch.

This should do the trick now you can make a class with tensors in the parameters and serialize this as Json.
` public class JsonTensor { public long[] _shape { get; set; } public ScalarType _type { get; set; } public byte[] data { get; set; } public bool requires_grad { get; set; } public string[] names { get; set; } }
public class TensorConverter : System.Text.Json.Serialization.JsonConverter<Tensor>
{

    public override Tensor Read(ref Utf8JsonReader reader, Type typeToConvert, JsonSerializerOptions options)
    {
        var txt = reader.GetString();
        var jt = (JsonTensor) System.Text.Json.JsonSerializer.Deserialize(txt, typeof(JsonTensor));
        var tensor = torch.empty(jt._shape, dtype: jt._type, requires_grad: jt.requires_grad, names:jt.names);
        tensor.bytes = jt.data;
        return tensor;
    }

    public override void Write(Utf8JsonWriter writer, Tensor value, JsonSerializerOptions options)
    {
            var jt =new JsonTensor() { data = value.bytes.ToArray(), _shape = value.shape, _type = value.dtype , names = value.names , requires_grad = value.requires_grad};
            var _o = new System.Text.Json.JsonSerializerOptions();
            var data = System.Text.Json.JsonSerializer.Serialize(jt, _o);
            writer.WriteStringValue(data);
    }
}`

If the property in the object is still an object. Then defines him as the parent class when writing the code.

Ok I maybe do not get the question. But now you can write something like this:
`
public string Serialize()
{
var options = new System.Text.Json.JsonSerializerOptions();
options.Converters.Add(new TensorConverter());
var data = System.Text.Json.JsonSerializer.Serialize(this, options);
return data;
}
    public IVisual Deserialize(string json)
    {
        var options = new System.Text.Json.JsonSerializerOptions();
        options.Converters.Add(new TensorConverter());
        return (IVisual)System.Text.Json.JsonSerializer.Deserialize(json, this.GetType(), options);
    }

`

@NiklasGustafsson @Sprinzl,
I think BinaryFormatter is the most suitable for my needs, he is very close to python's pickle, but he seems to be deprecated.
https://learn.microsoft.com/zh-tw/dotnet/api/system.runtime.serialization.formatters.binary.binaryformatter?view=net-7.0
https://github.com/dotnet/designs/blob/main/accepted/2020/better-obsoletion/binaryformatter-obsoletion.md

Hi @NiklasGustafsson,
I have my code serializable by system.text.json, but I need to pass JsonSerializerOptions to JsonSerializer.Serialize and JsonSerializer.Deserialize.

If you are using JSON (including BSON), I would suggest using the Newtonsoft,Json package: https://www.nuget.org/packages/Newtonsoft.Json

Why use Newtonsoft instead of system.text.json

Hi @NiklasGustafsson,
My code supports serialization using newtonsoft. But I need to pass JsonSerializerSettings to JsonSerializer.
        Newtonsoft.Json.JsonSerializer jsonSerializer = Newtonsoft.Json.JsonSerializer.Create(newtonsoftSerializerOptions);
        using MemoryStream ms = new();
        using BsonDataWriter writer = new(ms);
        jsonSerializer.Serialize(writer, space);
        ms.Position = 0;
        using BsonDataReader reader = new(ms);
        Box? box = jsonSerializer.Deserialize<Space>(reader) as Box;

Hi @NiklasGustafsson, My code supports serialization using newtonsoft. But I need to pass JsonSerializerSettings to JsonSerializer. Thank you.
Newtonsoft.Json.JsonSerializer jsonSerializer = Newtonsoft.Json.JsonSerializer.Create(newtonsoftSerializerOptions);
using MemoryStream ms = new();
using BsonDataWriter writer = new(ms);
jsonSerializer.Serialize(writer, space);
ms.Position = 0;
using BsonDataReader reader = new(ms);
Box? box = jsonSerializer.Deserialize<Space>(reader) as Box;

BsonDataWriter and BsonDataReader  moved to https://www.nuget.org/packages/Newtonsoft.Json.Bson/1.0.2

Ok just have a helper with an extension and you have no problem with always rewrite.
public static void Write(this string path, object object) ….
public static string Local(this string path)…
Now you can write @“Data/xxx.json”.Local().Write(object)
This is as simple as it gets.

What does it mean?

you have no problem with always rewrite.

Umm...
I still prefer BinaryFormatter, not all objects can be quickly serialized using Newtonsoft.Json


Map.Layers.LayerRemoved event doesn't fire when a layer is removed from a group

This issue was imported from CodePlex
jirikadlec2 wrote 2011-08-04 at 04:00
The Map.Layers.LayerRemoved event only fires when the removed layer is not in a group.
If the layer being removed is inside a group, then the Map.Layers.LayerRemoved event doesn't fire.
This makes the LayerRemoved event hard to use. As a plugin developer, I don't know how many groups (or nested groups) there are in the map.
vzavec wrote 2011-08-08 at 15:24
With below code there is no chance to remove layer from map. Layer is inside a group. Who can help?
            foreach (IMapLayer layer in map1.GetAllLayers())
            {
                IFeatureLayer fl = layer as IFeatureLayer;
                if (fl != null)
                {
                    if (fl.LegendText == "LOMOBJ")
                    {
                        map1.Layers.Remove(layer);
                        map1.MapFrame.ResumeEvents();
                        map1.MapFrame.Invalidate();
                    }
                }
            }

Not reproduced.


Aetherment causing other plugin issues

Not sure what the actual issue is.  Ex: Autoretainer and Artisan.  Disabling Aetherment allows those to run.

Some plugins seem to check the text color for some reason, v0.60.0 of Frost UI added text color support. Setting the foreground color to #EEE1C5 seems to fix it for other people, although the devs of these plugins should ideally find a better solution than to check the text color.


"expected none or () delimited group, but found different token tree (note: do not use the macros from bunt-macros directly, but only through bunt)"

Hi,
When macro expansion is activated in rust-analyser in VSCode, I get the following error, every time I use bunt::println!:

expected none or () delimited group, but found different token tree (note: do not use the macros from bunt-macros directly, but only through bunt)

Is this somehow fixable?
Macro expansion is useful for completion of types that are created via proc macros (e.g. typed_builder), but now my code is littered with error messages that the compiler itself doesn't report as errors.

I reproduced the problem locally. As to why this happens: good question! I don't know. I don't really know a lot about rust-analyzer and haven't used its proc macro expansion features yet. I would assume it's still not "finished" and that this is just a bug.
So what's happening with bunt is that bunt::println is actually a declarative macro that slightly transforms the input and passes it to the proc macro bunt_macros::writeln. See here. It seems like rust-analyzer ignores this transformation done by the declarative macro and just passes the original input token stream to the proc macro. This causes the error. But I have no idea why that would happen. Does rust-analyzer support declarative macro expansion yet?
Please excuse me pinging you directly here, but maybe @jonas-schievink could help us out here?

This certainly seems like a bug in rust-analyzer, but I'm not immediately sure what's causing it.

It also happens when calling the procedural macro directly:
    bunt_macros::writeln!(
        (bunt::termcolor::StandardStream::stdout(bunt::termcolor::ColorChoice::Auto))
        ["asdasd"]
    );

A closer manual invocation would be:
    bunt_macros::writeln!(
        (bunt::termcolor::StandardStream::stdout(bunt::termcolor::ColorChoice::Auto))
        [("hello")]
    );

It's a bit strange, but the [$($format_str)+] (in the definition of bunt::println!) will result in a token stream of:
Group {
    delimiter: Some('['),
    inner: Group {
       delimiter: None,
       inner: ...,
    }
}

Maybe rust-analyzer does not add this "dummy group"? I think the behavior changed recently: https://github.com/rust-lang/rust/pull/73084/
But I guess I could also change bunt-macros to optionally accept a None-delimited group there. That's probably the easiest fix.

Why is there an additional () around the format string? I only see that as part of the $(...)+ repetition operator, which should not end up in the output

I managed to get a fairly minimal example to reproduce this problem. You can see it on this branch. We have three crates: bunt-macros (exporting proc macro), bunt (exporting decl macro) and the simple example.
lib.rs of bunt-macros
use proc_macro::TokenStream;

#[proc_macro]
pub fn writeln(input: TokenStream) -> TokenStream {
    panic!("{:#?}", input);
}

lib.rs of bunt:
pub extern crate bunt_macros;

#[macro_export]
macro_rules! foo {
    ($target:expr) => {
        $crate::bunt_macros::writeln!($target)
    };
}

simple.rs
fn main() {
    bunt::foo!(peter);
}

If you cargo build --example simple, you get this output:
error: proc macro panicked
 --> examples/simple.rs:3:5
  |
3 |     bunt::foo!(peter);
  |     ^^^^^^^^^^^^^^^^^^
  |
  = help: message: TokenStream [
    Group {
        delimiter: None,
        stream: TokenStream [
            Ident {
                ident: "peter",
                span: #0 bytes(28..33),
            },
        ],
        span: #6 bytes(8793927..8793934),
    },
]

However, if I just save simple.rs in VScode with RA proc macro expansion activated, it shows this error (when hovering over the macro call):
proc macro returned error: proc-macro panicked: TokenStream [
    Ident {
        ident: "peter",
        span:<PHONE_NUMBER>,
    },
] rust-analyzermacro-error

RA seems to not add this "dummy group" when evaluating the decl macro.
But the most interesting part: if I rename writeln to bar (in both lib.rs), does not show the error anymore! It seems to give up on evaluating the macros. Instead, it only shows the output of cargo check, which also contains the dummy group.
Does RA treat writeln in a special way?

@jonas-schievink Just to be clear, I don't expect you to help out here. Also, if I should just open an issue at the RA repo, let me know!

But the most interesting part: if I rename writeln to bar (in both lib.rs), does not show the error anymore! It seems to give up on evaluating the macros. Instead, it only shows the output of cargo check, which also contains the dummy group.

This might just be because we don't reload proc. macros.
Thanks for finding that reproduction! It would be great if you could open a rust-analyzer issue with that.

This might just be because we don't reload proc. macros.

Ouch. I regularly restarted RA while building the minimal example, but of course I forgot to restart it after renaming that. So yeah, you seem to be right. Once RA is reloaded, I see the same behavior for bar.
I created the RA issue.

I checked the parsing code again and while some of the None delimited groups are not required, others are. So I decided not to implement a workaround in bunt. It would be a bit too much hassle for a fix like that. So we have to wait for the RA fix!

@LukasKalbertodt: Okay, thank you for investigating this. It looks like I should have reported this directly to RA.


Feature/auth

What Changed & Why
Added new Initializer option of passing in username and password (direct pass through to axios) for HTTP Basic Auth
ie:
var jsonApi = new JsonApi({apiUrl: 'https://example.com', auth: {username: 'secret', password: 'word'} })
And the HTTP_BASIC_AUTH middleware will pass this to axios and request should have http auth headers attached it it
Testing
npm test
People
@Emerson
@themarkappleby

Awsome

@Emerson Totally separate issue, but wondering if a changelog should be created, similar to: https://github.com/sequelize/sequelize/blob/master/changelog.md

Probably... I'd be fine adding that – I guess we just maintain that kinda stuff by hand?

Yep. Add a feature, prepend to the changelog.
It made sense with the sequelize library, since it is a very complex
library with a lot of moving parts. I wonder if it also makes sense here.
On 2016-05-12 11:14 AM, Emerson Lackey wrote:

Probably... I'd be fine adding that – I guess we just maintain that
kinda stuff by hand?
—
You are receiving this because you authored the thread.
Reply to this email directly or view it on GitHub
https://github.com/twg/devour/pull/25#issuecomment-218788889


"Channel page is broken. Please try again later" error message

Version
1.10.3
Platform
Windows 10 Pro
Steps to reproduce
Open YoutubeDownloader.exe, paste https://www.youtube.com/@HoracioFrazao/videos url in it, "Channel page is broken. Please try again later" error message show up.
Ins't a specific issue with this channel, as i'm unable to download anything with this program as same error messages show up.
Details
Expected behavior is to work and download videos, actual behavior is described in 'steps to reproduce' section here.
Checklist

[X] I have looked through existing issues to make sure that this bug has not been reported before
[X] I have provided a descriptive title for this issue
[X] I have made sure that that this bug is reproducible on the latest version of the application
[X] I have provided all the information needed to reproduce this bug as efficiently as possible
[ ] I have sponsored this project

Works now


slow inserts/saves

I am calling db.Save(&model) to save (or rather insert) two million objects.
This is incredibly slow. Is there a faster/better way of doing this with storm?

It's because each call to Save uses a transaction, which waits for disk synchronization for each object saved. You should save multiple objects using a single transaction.
Example:
const batchSize = 1000
const total = 1000000

for i := 0; i < total / batchSize; i++ {
  tx, err := db.Begin(true)
  if err != nil {
    return err
  }

  for j := 0; j < batchSize; j++ {
    model := ...
    err := tx.Save(&model)
    if err != nil {
      tx.Rollback()
      return err
    }
    ...
  }

  err = tx.Commit()
  ...
}

Thanks for the help! That said - even with a large batch size it's still quite slow.
Around 1000 Saves per second. I'd expect at least 5-10 times that.
But maybe that's just BoltDB not being very write speed friendly?

I'm just "guessing out loud here", but I would suspect that a lot of time in the above is spent in serializing to JSON and that it isn't the write to DB that is the bottle neck. I'm not sure if there are currently ways to make this faster with Storm by doing this in parallel, but one could imagine adding a tx.Prepare or something that could prepare 1000 objects with all CPUs running hot and then save them ... Just me thinking.

The same line of thinking let me try the serialization alone. A million serializations took around 14s.
Writing with storm I canceled after much much longer. This lets me to believe that the serialization isn't really the bottle neck.

Actually this is more related to how BoltDB synchronizes data on disk, the time spent on Storm functions is an order of magnitude lower than the disk sync time. This bbolt thread explains it well https://github.com/coreos/bbolt/issues/94

@prologic It seems that you are using too many indexes. Each index slows down the writes to speed up the reads when querying on that particular field. If you never or rarely query on a field, you should remove its index.

You can use the Batch() option
Then all the Storm write methods (Save, Update, DeleteStruct, etc.) will use the bolt Batch method.
If you only want that for one particular bucket, you can enable the option just for it using this

I guess that's what I was looking for. Why the Save in a go routine though?

It was just an example because Batch is usually used for concurrent writes accross multiple goroutines (ex: webservers, etc.)

@asdine I see; I'll give that a try. How do you explain the non-linear degradation in perf? Should I be thinking about using bleeve for indexing/search? Also how does batching work exactly in the context of web requests? Do multiple writes get "magically" batched together somehow?

@asdine For example:

Run #1: 1000 reqs, concurrency 10 ~200 req/s
Run #2: 1000 reqs, concurrency 10 ~150 req/s
Run #3: 1000 reqs, concurrency 10 ~130 req/s

I'm using hey here to do performance e2e benchmarks.
I did what you suggested and only have 2 indexes now (the only ones I search on)

I isolated as much as I could to just the BoltDB/Storm writes and I'm seeing perf like 285, 235, 180, 160 after successive load tests of 1000 reqs / concurrency 10 this is shockingly terrible and I must be doing something wrong; this kind of non-linear performance would kill you in production :)

Indeed that's terrible, can you share a code snippet ?

 If you want to do concurrency, Batch is the way to go

maybe try badger instead of bolt. when i compared them(write transactions), badger was 10x faster.

I'm also having linear slowdown issues with this struct (inserting 500 000 records in batches of 1 000 per Commit()):
type Blob struct {
	Ref                       BlobRef `storm:"id"` // type BlobRef []byte. stores sha256 so each ref is exactly 32 bytes
	Volumes                   []int
	VolumesPendingReplication []int
	IsPendingReplication      bool `storm:"index"`
	Referenced                bool

At first it inserts:

like three batches a second
then one batch per second
and one batch per four seconds when at 92 batch iterations (= 92 000 records in)

Things I've tried:

remove index from IsPendingReplication => did not considerably speed up
tried this both on Linux and Windows => issue persists

I'm also witnessing linear slowdowns with read latency (.One("Ref", ref, &record)) is actually taking 35 milliseconds which is... WTF I have 5 GB of RAM free and I'm on an SSD. The entire DB (containing other entities as well) is 500 MB.
Also worth noting is that my database is 524 MB (stored in heavily optimal Msgpack) while the exact same data EXPORTED IN JSON (Blob refs are base64 encoded instead of []byte which they are in the DB) is only 156 MB. I know that indices account for added overhead, but holy moly..

I verified that either the issue is in Storm or in the way I use it, proof:
I changed my import code to bypass Storm and use Bolt directly:
// withTx() to make commits every 1 000 calls
if err := withTx(func(tx *bbolt.Tx) error {
	blobs, err := tx.CreateBucketIfNotExists([]byte("blobs"))
	if err != nil {
		return err
	}

	blobRecord := record.(*varastotypes.Blob)

	recordInMsgPack, err := msgpackCodec.Marshal(blobRecord)
	if err != nil {
		return err
	}

	return blobs.Put([]byte(blobRecord.Ref), recordInMsgPack)
}); err != nil {
	return err
}

It absolutely chews through the ~ 600 k records in 12 seconds (previously it wouldn't finish in 15 minutes) without any slowdowns whatsoever, and I think I'm now being bottlenecked by JSON parsing. I also wrote a reader to verify that the data actually got persisted in the DB:
return db.Bolt.View(func(tx *bbolt.Tx) error {
	bucket := tx.Bucket([]byte("blobs"))

	recordRaw := bucket.Get(*ref)
	if len(recordRaw) == 0 {
		return errors.New("record not found")
	}

	record := &varastotypes.Blob{}
	if err := msgpackCodec.Unmarshal(recordRaw, record); err != nil {
		return err
	}

	jsonEnc := json.NewEncoder(os.Stdout)
	jsonEnc.SetIndent("", "  ")
	return jsonEnc.Encode(record)
})

And running it:
$ varasto server getblob d85db4b63d2b7a0f0a45db242c1cb5898d173a24e80fa451b40c4c18b35dd8fc
{
  "Ref": "2F20tj0reg8KRdskLBy1iY0XOiToD6RRtAxMGLNd2Pw=",
  "Volumes": [
    1
  ],
  "VolumesPendingReplication": [
    2
  ],
  "IsPendingReplication": true,
  "Referenced": true
}

(this code is ran as a separate process, so it definitely shows that the data got persisted properly)
My previous non-performing code in Storm was here: https://github.com/function61/varasto/blob/da3337fc/pkg/varastoserver/dbexportimport.go#L174
I'm running Storm 2.1.2 (5a6fe65e)

@joonas-fi Could you share a working example of what you wrote with Storm? I'll investigate tonight and see where is the bottleneck

@asdine, thanks for responding. :)
Sure, here's it distilled down program to reproduce this issue: https://gist.github.com/joonas-fi/c56c0f6fa23edac436609af3b22a5c46
By running it you will observe what I said above: the dots printing to the screen will become slower and slower.
Like I previously mentioned, the Ref attribute doesn't even need to be []byte, so I think you could just generate any random string there. Apologies for not being able to give you the export.log file I used, since they're hashes of all my personal files so I don't think it's wise to share that even though they are just hashes..
Let me know if you need help producing the input file (I didn't have time now, but might have in the evening).

I forgot to mention that during poor performance I observed that it was CPU bound

@asdine did you manage to look into this? Let me know if I can provide more information in diagnosing this! I replaced Storm with a homebuilt-but-crappy solution (I had to move forward fast because this issue is a show-stopper with big datasets), but find myself missing the better support for indices that Storm has.

Hello @asdine, I've been analyzing the problem of slowness and realized that the culprit is the call to the RemoveID (unique.go) method. To be more specific, the call to the next cursor is taking the longest.
I didn't quite understand what this code snippet does. Can you explain to me please? Thanks in advance.
I used the KCacheGrind tool to check the CPU usage.

Hello, @asdine . After thinking for a while, I managed to come up with a solution, at least, for part of the problem. I realized that the ID is being treated as a unique index, however, the ID is more than unique. The ID is also the primary key. In this sense, I decided to create my own type of index for the ID. This type, unlike the others, is not saved because the keys of the main bucket can serve as a primary index. I performed a test in a simple scenario and the result was very satisfactory.
In the test, I used a structure with some fields and only one index, the ID. For this scenario the solution is applicable. For scenarios where there are indexes beyond the ID, I will still have to work a few more days to adjust.
Please tell me what you think about my solution and I will send you a Pull Request. Thanks in advance.

I also found slowness due to RemoveID. Inserting 50k entries with a single storm:"id" field took tens of seconds, which definitely seemed wrong. After all, it's a primary key, so we can know it's unique; inserting into the index should be very cheap.
Here's where the insert removes from the index: https://github.com/asdine/storm/blob/23213e9525985cdd3f0ca079c19b78db7e67b76f/store.go#L235
For id/unique fields, that index interface is implemented by UniqueIndex. And, as you might guess, its RemoveID method loops over all index entries: https://github.com/asdine/storm/blob/23213e9525985cdd3f0ca079c19b78db7e67b76f/index/unique.go#L62-L71
So this can easily explain why inserting tens or hundreds of thousands of records takes a long time; with a large enough N, this O(N*N) behavior kicks in.
It seems to me like this is an oversight with a simple fix. For unique indexes, an insert should not be quadratic, as we only need to do one Put or Delete at the K/V store layer. Looping over many index entries should only be necessary for non-unique indexes - but even for non-unique indexes, quadratic time inserts feels like it has plenty of room for improvement.
@asdine do you intend to look at this soon, or accept patches? I'm afraid the library in its current state isn't usable for me, as trying to index more than a few thousand database entries simply burns CPU.

Hello @mvdan.
You are right.
The fix looks very simple. Indeed, I fixed it in a fork, called rainstorm.
https://github.com/AndersonBargas/rainstorm
Would be nice if you take a look at it to check if it makes sense, or not!
Thanks in advance.

@mvdan @AndersonBargas This library is not maintained these days because I'm putting all or my energy in another database which solves lots of Storm design issues (no reflection, SQL as primary API, works with Bolt and Badger), if you like Storm I definitely recommend taking a look at Genji.
Regarding Storm, even if I'm not actively working on it, I definitely accept PRs, especially regarding performance and stability


Feature Request: Support for mini vacuum K10+

Problem
No response
Solution
Add support for K10+ please
Alternatives
No response
Additional context
Logs say device: WoSweeper Mini

@MalachiteXD have you tried the plugin?

@donavanbecker yeah on my homebridge, and the code is what told me to submit a feature request

Can you enable debug logs so I can see what the api displays?

@donavanbecker like this?
device name is “Janet”
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] body: {"_readableState":{"objectMode":false,"highWaterMark":65536,"buffer":{"head":{"data":{"type":"Buffer","data":[123,34,115,116,97,116,117,115,67,111,100,101,34,58,49,48,48,44,34,98,111,100,121,34,58,123,34,100,101,118,105,99,101,76,105,115,116,34,58,91,123,34,100,101,118,105,99,101,73,100,34,58,34,51,54,48,84,89,52,50,48,55,48,51,48,50,51,54,48,52,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,74,97,110,101,116,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,87,111,83,119,101,101,112,101,114,77,105,110,105,34,44,34,101,110,97,98,108,101,67,108,111,117,100,83,101,114,118,105,99,101,34,58,116,114,117,101,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,68,53,69,48,53,70,66,49,65,53,53,69,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,72,117,98,32,77,105,110,105,32,107,105,116,99,104,101,110,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,69,56,66,54,48,50,67,68,68,56,49,52,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,66,101,100,114,111,111,109,32,72,117,98,32,77,105,110,105,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,70,51,69,65,53,67,49,50,57,69,51,67,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,72,117,98,32,77,105,110,105,32,76,105,118,105,110,103,32,82,111,111,109,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,93,44,34,105,110,102,114,97,114,101,100,82,101,109,111,116,101,76,105,115,116,34,58,91,123,34,100,101,118,105,99,101,73,100,34,58,34,48,50,45,50,48,50,49,49,48,49,54,50,48,50,57,45,57,54,51,51,51,48,52,48,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,67,97,115,116,108,101,34,44,34,114,101,109,111,116,101,84,121,112,101,34,58,34,68,73,89,32,76,105,103,104,116,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,70,51,69,65,53,67,49,50,57,69,51,67,34,125,93,125,44,34,109,101,115,115,97,103,101,34,58,34,115,117,99,99,101,115,115,34,125]},"next":null},"tail":{"data":{"type":"Buffer","data":[123,34,115,116,97,116,117,115,67,111,100,101,34,58,49,48,48,44,34,98,111,100,121,34,58,123,34,100,101,118,105,99,101,76,105,115,116,34,58,91,123,34,100,101,118,105,99,101,73,100,34,58,34,51,54,48,84,89,52,50,48,55,48,51,48,50,51,54,48,52,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,74,97,110,101,116,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,87,111,83,119,101,101,112,101,114,77,105,110,105,34,44,34,101,110,97,98,108,101,67,108,111,117,100,83,101,114,118,105,99,101,34,58,116,114,117,101,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,68,53,69,48,53,70,66,49,65,53,53,69,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,72,117,98,32,77,105,110,105,32,107,105,116,99,104,101,110,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,69,56,66,54,48,50,67,68,68,56,49,52,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,66,101,100,114,111,111,109,32,72,117,98,32,77,105,110,105,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,70,51,69,65,53,67,49,50,57,69,51,67,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,72,117,98,32,77,105,110,105,32,76,105,118,105,110,103,32,82,111,111,109,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,93,44,34,105,110,102,114,97,114,101,100,82,101,109,111,116,101,76,105,115,116,34,58,91,123,34,100,101,118,105,99,101,73,100,34,58,34,48,50,45,50,48,50,49,49,48,49,54,50,48,50,57,45,57,54,51,51,51,48,52,48,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,67,97,115,116,108,101,34,44,34,114,101,109,111,116,101,84,121,112,101,34,58,34,68,73,89,32,76,105,103,104,116,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,70,51,69,65,53,67,49,50,57,69,51,67,34,125,93,125,44,34,109,101,115,115,97,103,101,34,58,34,115,117,99,99,101,115,115,34,125]},"next":null},"length":1},"length":670,"pipes":[],"flowing":null,"ended":true,"endEmitted":false,"reading":false,"constructed":true,"sync":true,"needReadable":false,"emittedReadable":false,"readableListening":false,"resumeScheduled":false,"errorEmitted":false,"emitClose":true,"autoDestroy":true,"destroyed":false,"errored":null,"closed":false,"closeEmitted":false,"defaultEncoding":"utf8","awaitDrainWriters":null,"multiAwaitDrain":false,"readingMore":false,"dataEmitted":false,"decoder":null,"encoding":null},"_events":{},"_eventsCount":0}
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] statusCode: 200
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] headers: {"date":"Wed, 29 Nov 2023 19:15:30 GMT","content-type":"application/json","content-length":"670","connection":"keep-alive","x-amzn-requestid":"8724f4a7-5fd9-4d39-b3f8-00482117179f","x-amz-apigw-id":"PLMY5HzeoAMEJPQ=","x-amzn-trace-id":"Root=1-65678dd2-7cce1966595659bb7383b145;Sampled=0;lineage=c8c2b0f2:0|bf95bacf:0"}
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] devicesAPI: {"statusCode":100,"body":{"deviceList":[{"deviceId":"360TY420703023604","deviceName":"Janet","deviceType":"WoSweeperMini","enableCloudService":true,"hubDeviceId":""},{"deviceId":"D5E05FB1A55E","deviceName":"Hub Mini kitchen","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"E8B602CDD814","deviceName":"Bedroom Hub Mini","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"F3EA5C129E3C","deviceName":"Hub Mini Living Room","deviceType":"Hub Mini","hubDeviceId":"000000000000"}],"infraredRemoteList":[{"deviceId":"02-202110162029-96333040","deviceName":"Castle","remoteType":"DIY Light","hubDeviceId":"F3EA5C129E3C"}]},"message":"success"}
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] devicesAPI Body: {"deviceList":[{"deviceId":"360TY420703023604","deviceName":"Janet","deviceType":"WoSweeperMini","enableCloudService":true,"hubDeviceId":""},{"deviceId":"D5E05FB1A55E","deviceName":"Hub Mini kitchen","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"E8B602CDD814","deviceName":"Bedroom Hub Mini","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"F3EA5C129E3C","deviceName":"Hub Mini Living Room","deviceType":"Hub Mini","hubDeviceId":"000000000000"}],"infraredRemoteList":[{"deviceId":"02-202110162029-96333040","deviceName":"Castle","remoteType":"DIY Light","hubDeviceId":"F3EA5C129E3C"}]}
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] devicesAPI StatusCode: 100
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] statusCode: 200 & devicesAPI StatusCode: 100
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] DeviceLists: [{"deviceId":"360TY420703023604","deviceName":"Janet","deviceType":"WoSweeperMini","enableCloudService":true,"hubDeviceId":""},{"deviceId":"D5E05FB1A55E","deviceName":"Hub Mini kitchen","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"E8B602CDD814","deviceName":"Bedroom Hub Mini","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"F3EA5C129E3C","deviceName":"Hub Mini Living Room","deviceType":"Hub Mini","hubDeviceId":"000000000000"}]
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] DeviceLists Length: 4
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] SwitchBot Device Config Not Set: undefined
[29/11/2023, 11:15:30] [SwitchBot] Device: Janet with Device Type: WoSweeperMini, is currently not supported.
[29/11/2023, 11:15:30] [SwitchBot] Submit Feature Requests Here: https://tinyurl.com/SwitchBotFeatureRequest
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] Discovered Hub Mini: D5E05FB1A55E
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] Discovered Hub Mini: E8B602CDD814
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] Discovered Hub Mini: F3EA5C129E3C
[29/11/2023, 11:15:30] [SwitchBot] Total SwitchBot Devices Found: 4
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] IR Device Config Not Set: undefined
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] Discovered DIY Light: 02-202110162029-96333040
[29/11/2023, 11:15:30] [SwitchBot] Restoring existing accessory from cache: Castle DeviceID: 02-202110162029-96333040
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] DIY Light: Castle Using Platform Config Logging: debug
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] DIY Light: Castle Config: {"connectionType":"OpenAPI"}
[29/11/2023, 11:15:30] [SwitchBot] [DEBUG] DIY Light uuid: 02-202110162029-96333040-DIY Light, (4717bb26-263a-4fd6-87e9-78f7d28ac6dc)
[29/11/2023, 11:15:30] [SwitchBot] Total IR Devices Found: 1

Can you try the latest beta:

Click here to find out how to install Beta
I have added the deviceType: WoSweeperMini as part of the Robot Vacuum Cleaner that the OpenAPI talks about, we will see if that works for you.

Okay so it’s showing as a light (same if I add it as a device as a vacuum through the plugin settings), and I can turn it on but then later it shows off I’m not sure if that’s all I should be able to do, if anything this works since I mainly need it to start it! Here’s debug if you need anything
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] body: {"_readableState":{"objectMode":false,"highWaterMark":65536,"buffer":{"head":{"data":{"type":"Buffer","data":[123,34,115,116,97,116,117,115,67,111,100,101,34,58,49,48,48,44,34,98,111,100,121,34,58,123,34,100,101,118,105,99,101,76,105,115,116,34,58,91,123,34,100,101,118,105,99,101,73,100,34,58,34,51,54,48,84,89,52,50,48,55,48,51,48,50,51,54,48,52,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,74,97,110,101,116,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,87,111,83,119,101,101,112,101,114,77,105,110,105,34,44,34,101,110,97,98,108,101,67,108,111,117,100,83,101,114,118,105,99,101,34,58,116,114,117,101,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,68,53,69,48,53,70,66,49,65,53,53,69,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,72,117,98,32,77,105,110,105,32,107,105,116,99,104,101,110,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,69,56,66,54,48,50,67,68,68,56,49,52,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,66,101,100,114,111,111,109,32,72,117,98,32,77,105,110,105,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,70,51,69,65,53,67,49,50,57,69,51,67,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,72,117,98,32,77,105,110,105,32,76,105,118,105,110,103,32,82,111,111,109,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,93,44,34,105,110,102,114,97,114,101,100,82,101,109,111,116,101,76,105,115,116,34,58,91,123,34,100,101,118,105,99,101,73,100,34,58,34,48,50,45,50,48,50,49,49,48,49,54,50,48,50,57,45,57,54,51,51,51,48,52,48,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,67,97,115,116,108,101,34,44,34,114,101,109,111,116,101,84,121,112,101,34,58,34,68,73,89,32,76,105,103,104,116,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,70,51,69,65,53,67,49,50,57,69,51,67,34,125,93,125,44,34,109,101,115,115,97,103,101,34,58,34,115,117,99,99,101,115,115,34,125]},"next":null},"tail":{"data":{"type":"Buffer","data":[123,34,115,116,97,116,117,115,67,111,100,101,34,58,49,48,48,44,34,98,111,100,121,34,58,123,34,100,101,118,105,99,101,76,105,115,116,34,58,91,123,34,100,101,118,105,99,101,73,100,34,58,34,51,54,48,84,89,52,50,48,55,48,51,48,50,51,54,48,52,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,74,97,110,101,116,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,87,111,83,119,101,101,112,101,114,77,105,110,105,34,44,34,101,110,97,98,108,101,67,108,111,117,100,83,101,114,118,105,99,101,34,58,116,114,117,101,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,68,53,69,48,53,70,66,49,65,53,53,69,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,72,117,98,32,77,105,110,105,32,107,105,116,99,104,101,110,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,69,56,66,54,48,50,67,68,68,56,49,52,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,66,101,100,114,111,111,109,32,72,117,98,32,77,105,110,105,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,44,123,34,100,101,118,105,99,101,73,100,34,58,34,70,51,69,65,53,67,49,50,57,69,51,67,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,72,117,98,32,77,105,110,105,32,76,105,118,105,110,103,32,82,111,111,109,34,44,34,100,101,118,105,99,101,84,121,112,101,34,58,34,72,117,98,32,77,105,110,105,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,48,48,48,48,48,48,48,48,48,48,48,48,34,125,93,44,34,105,110,102,114,97,114,101,100,82,101,109,111,116,101,76,105,115,116,34,58,91,123,34,100,101,118,105,99,101,73,100,34,58,34,48,50,45,50,48,50,49,49,48,49,54,50,48,50,57,45,57,54,51,51,51,48,52,48,34,44,34,100,101,118,105,99,101,78,97,109,101,34,58,34,67,97,115,116,108,101,34,44,34,114,101,109,111,116,101,84,121,112,101,34,58,34,68,73,89,32,76,105,103,104,116,34,44,34,104,117,98,68,101,118,105,99,101,73,100,34,58,34,70,51,69,65,53,67,49,50,57,69,51,67,34,125,93,125,44,34,109,101,115,115,97,103,101,34,58,34,115,117,99,99,101,115,115,34,125]},"next":null},"length":1},"length":670,"pipes":[],"flowing":null,"ended":true,"endEmitted":false,"reading":false,"constructed":true,"sync":true,"needReadable":false,"emittedReadable":false,"readableListening":false,"resumeScheduled":false,"errorEmitted":false,"emitClose":true,"autoDestroy":true,"destroyed":false,"errored":null,"closed":false,"closeEmitted":false,"defaultEncoding":"utf8","awaitDrainWriters":null,"multiAwaitDrain":false,"readingMore":false,"dataEmitted":false,"decoder":null,"encoding":null},"_events":{},"_eventsCount":0}
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] statusCode: 200
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] headers: {"date":"Thu, 30 Nov 2023 02:09:09 GMT","content-type":"application/json","content-length":"670","connection":"keep-alive","x-amzn-requestid":"1e331dc8-3a80-42c6-8d59-5165af190cfe","x-amz-apigw-id":"PMI-4GQPIAMEFAA=","x-amzn-trace-id":"Root=1-6567eec5-3cff2e97761c8ae744a3c3dd;Sampled=0;lineage=c8c2b0f2:0|bf95bacf:0"}
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] devicesAPI: {"statusCode":100,"body":{"deviceList":[{"deviceId":"360TY420703023604","deviceName":"Janet","deviceType":"WoSweeperMini","enableCloudService":true,"hubDeviceId":""},{"deviceId":"D5E05FB1A55E","deviceName":"Hub Mini kitchen","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"E8B602CDD814","deviceName":"Bedroom Hub Mini","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"F3EA5C129E3C","deviceName":"Hub Mini Living Room","deviceType":"Hub Mini","hubDeviceId":"000000000000"}],"infraredRemoteList":[{"deviceId":"02-202110162029-96333040","deviceName":"Castle","remoteType":"DIY Light","hubDeviceId":"F3EA5C129E3C"}]},"message":"success"}
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] devicesAPI Body: {"deviceList":[{"deviceId":"360TY420703023604","deviceName":"Janet","deviceType":"WoSweeperMini","enableCloudService":true,"hubDeviceId":""},{"deviceId":"D5E05FB1A55E","deviceName":"Hub Mini kitchen","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"E8B602CDD814","deviceName":"Bedroom Hub Mini","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"F3EA5C129E3C","deviceName":"Hub Mini Living Room","deviceType":"Hub Mini","hubDeviceId":"000000000000"}],"infraredRemoteList":[{"deviceId":"02-202110162029-96333040","deviceName":"Castle","remoteType":"DIY Light","hubDeviceId":"F3EA5C129E3C"}]}
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] devicesAPI StatusCode: 100
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] statusCode: 200 & devicesAPI StatusCode: 100
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] DeviceLists: [{"deviceId":"360TY420703023604","deviceName":"Janet","deviceType":"WoSweeperMini","enableCloudService":true,"hubDeviceId":""},{"deviceId":"D5E05FB1A55E","deviceName":"Hub Mini kitchen","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"E8B602CDD814","deviceName":"Bedroom Hub Mini","deviceType":"Hub Mini","hubDeviceId":"000000000000"},{"deviceId":"F3EA5C129E3C","deviceName":"Hub Mini Living Room","deviceType":"Hub Mini","hubDeviceId":"000000000000"}]
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] DeviceLists Length: 4
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] SwitchBot Device Config Not Set: undefined
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] Discovered WoSweeperMini: 360TY420703023604
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] Discovered Hub Mini: D5E05FB1A55E
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] Discovered Hub Mini: E8B602CDD814
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] Discovered Hub Mini: F3EA5C129E3C
[29/11/2023, 18:09:09] [SwitchBot] Total SwitchBot Devices Found: 4
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] IR Device Config Not Set: undefined
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] Discovered DIY Light: 02-202110162029-96333040
[29/11/2023, 18:09:09] [SwitchBot] Restoring existing accessory from cache: Castle DeviceID: 02-202110162029-96333040
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] DIY Light: Castle Using Platform Config Logging: debug
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] DIY Light: Castle Config: {"connectionType":"OpenAPI"}
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] DIY Light uuid: 02-202110162029-96333040-DIY Light, (4717bb26-263a-4fd6-87e9-78f7d28ac6dc)
[29/11/2023, 18:09:09] [SwitchBot] Total IR Devices Found: 1
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] Device: Janet registerDevice: true
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] Device: Janet connectionType: OpenAPI, will display in HomeKit
[29/11/2023, 18:09:09] [SwitchBot] Restoring existing accessory from cache: Janet DeviceID: 360TY420703023604
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] WoSweeperMini: Janet Using Platform Config Logging: debug
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] WoSweeperMini: Janet Using Platform Config refreshRate: 120
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] WoSweeperMini: Janet Config: {"connectionType":"OpenAPI"}
[29/11/2023, 18:09:09] [SwitchBot] [DEBUG] WoSweeperMini: Janet openAPIRefreshStatus
[29/11/2023, 18:09:10] [@switchbot/homebridge-switchbot] This plugin generated a warning from the characteristic 'Firmware Revision': characteristic value expected string and received undefined.  I will look into this and let you know what I find.


Update kotlin version in gradle plugin tests

kotlin - 1.9.0, compose compiler plugin - 1.5.0

forgot about this thing,so a separate PR


Deploy lower gas Compound strategy

Code deployed, saving $450 three times a week, docs updated, and we are done here!


Add method to get unread inbox notifications count for rooms

Descriptions
Adds room.getUnreadInboxNotificationsCount
Usage:
  const room = useRoom();
  const count = await room.getUnreadInboxNotificationsCount();

  // OR

  const room = client.getRoom("my-room");
  const count = await room.getUnreadInboxNotificationsCount();

Closing, more thought needs to be put into this, with an API that likely reuses the project level one using queries


Add Sitemap

Interestingly there is some sitemap functionality. There's a configuration file in config/sitemap.rb and a few rake tasks: sitemap:create to generate a sitemap in public, and sitemap:refresh to re-generate the sitemap and "ping" search engines.


Audit record uses the exact value of: X-FHIR-FORWARDED-URL header as the URL called for the log entry #2473

Signed-off-by: Paul Bastide<EMAIL_ADDRESS>

I was going to suggest that we document that we'll use the request from the servlet context (not the original request url), but it looks like we never actually added the target to our CADF documentation.  Maybe it would be good to fix that omission as part of this PR?

I was going to suggest that we document that we'll use the request from the servlet context (and not the original request url), but it looks like we never actually added the target to our CADF documentation at https://ibm.github.io/FHIR/guides/FHIRServerUsersGuide#4111-cadf-audit-log-entry. Maybe it would be good to fix that omission as part of this PR?

Sure, once I have a clean workspace, I'll update.


Changing the UI of home page

Hello. I think the website has useful information but the website UI not looks good. I am interested in changing the UI to some card based style with logo like the below attached image. Kindly assign this to me. Thank you.

Hello @tharanithar-r, I think I can work on improving the CSS of this page and make the page more eye catching. So please assign this issue to me and label this issue as hacktoberfest2023.

I am assigned to both of you. Don't forget to tell the details about what are the changes you have made, create a screen recording too, after your added feat how the website is gonna look.


SECOAUTH-167 Token Service defaulting in authorization-server definition possibly incorrect

Patch to fix the defaulting of the token services.

Thanks for the patch.  Did you fill in the contributor agreement (see README.md: https://support.springsource.com/spring_committer_signup)?

It looks like your master branch has a load of other stuff in it now.  Some of that might be relevant to SECOAUTH-167, but most of it looks like it isn't and I can't tell the difference.  Can you pull the SECOAUTH-167 patch out into a separate branch, please so I can merge it?

Closing, as the SECOAUTH-167 change was merged in pull/4


[C#] Add complete pending to device API

So that we can ensure no outstanding IO when disposing allocator.

Closing - instead we will update the device to catch and ignore exceptions in the uncommon race condition of IO after allocator dispose.


Add update checker to turborepo

working on #178

@jaredpalmer where should I put the update message? i'm stuck on this

Closing this, but pick up this PR from here: #397

Continued in #397


Run benchmark in CI

Codecov Report

Merging #121 (b5bb520) into main (f5b3455) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #121   +/-   ##
=======================================
  Coverage   92.25%   92.25%
=======================================
  Files          59       59
  Lines        3785     3785
=======================================
  Hits         3492     3492
  Misses        225      225
  Partials       68       68

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f5b3455...b5bb520. Read the comment docs.


Client Credentials Support for Sharepoint

Summary
Add Client Credentials Support for Sharepoint
Work Item(s)
Fixes #524
Fixes AB#457003

@darjoo / @WaelAbuSeada, would you mind looking at this PR with your security glasses put on? 😎

How to test this new Autorization:

Create an App Registration in Azure with the following application permissions:

Create a suitable certifacte with password (see detailled guide below) + Upload the Certificate to your App Registration
Publish the System Application and the following sharepoint testing application (https://github.com/pri-kise/msdyn365bc-sharepointclient-test) in your docker environment
Open the Page Sharepoint Setup of the PTE App for manual testing
Upload the encrypted certifacte and set you password in the Setup Table
Test the connection with the suitable Actions

Detailled guide for creating a certifacte
Originally posted by @jwikman in https://github.com/microsoft/ALAppExtensions/issues/22261#issuecomment-1917426607

Create a KeyVault in the Azure Portal (or use an existing if you have one)
Generate a new certificate - KeyVault->Certificates->Generate/Import

Method: Generate
Name: Anything
Type: Self-signed
Subject: Any X.500 distinguished name
Content Type: PKCS #12

After it's been created, go to the certificate version and set Enabled: Yes
Then download the certificate as .pfx from the Certificate Version with the "Download in PFX/PEM format"

Now you've got a PKCS #12 certificate saved as a .pfx file, without password (no good).
To get a password protected .pfx file I did this:

Start Microsoft Management Console by running mmc.exe
Add Snap-In Certificates, select "My user account" when prompted
Expand certificate tree to Personal/Certificates
Import your passwordless .pfx file via Action/All Tasks/Import...
Then right click on your certificate that should be in the Personal/Certificates store and select All Tasks->Export...
Include the Private Key
Set a password when asked
Save to file
Remove the certificate from Personal/Certificates

Now you've got a self-signed PKCS #12 certificate saved as a .pfx file, with a password :)

@jwikman my PR is now ready to test.

@jwikman my PR is now ready to test.

Cool!
I'm choked with work that must be completed before the wave hits us, but a colleague of mine will try to test this out.

@pri-kise I think you would need to update your PR branch from microsoft:main, so we can access the new functions with SecretText parameters that Darrick added.

@jwikman @pri-kise There is a test failure which I'm not sure is related. I updated his PR with the code suggestions above :)
Build System Application Tests (Default) / System Application Tests (Default)
Function TestSendWithoutGetContent Assert.AreEqual failed. Expected:<502> (Integer). Actual:<200> (Integer). The response status code should be 200.

I update my branch from the main branch now.

Then let's see if that also solved the test failure. I don't think that the failure is related to this PR...

Is there any chance to get this backported to BC 24.x?
I orginally planned to get this addresse before the Deadline of BC24, but sadly I wasn't able to.

Is there any chance to get this backported to BC 24.x? I orginally planned to get this addresse before the Deadline of BC24, but sadly I wasn't able to.

Absolutely. Consider this shipping with 24.1 in May!

PR created: #840.

I was a few days late to this party... But now I finally got some time to test this out - and it works just as expected. 🥳
Thanks for doing this @pri-kise!

Since this is set to release with 24.1:
We'd need a feature details description with sample code. @pri-kise, could you provide a sentence or two incl. example - if possible?


IORING_MSG_SEND_FD

An issue to track our discussion and eventual PR to support sending a fixed fd (aka direct fd) to another uring.

We can decide how to model this functionality for this crate and even whether we only support a locked down version where the crate is responsible for all use cases or there is an open, unsafe, form too to let the user or higher level crate make their own flow.
At its core, the io_uring command allows sending a copy of the file descriptor stored in a fixed table's slot to another fixed table's slot, the fixed table of another uring. Both the source uring and destination uring have to be known, and the source table index has to be known. The destination table index can be chosen by the caller or can set to be auto selected by the destination ring.
If auto selected, the selected index is returned to the op caller as the result. It is not sent to the recipient.
The recipient is sent both a copy of the fixed file descriptor, which changes the destination's fixed file table, and the pair user_data and result, which get placed into the destination's completion queue.
Dest CQE
How to handle the CQE in the destination's completion queue is the most interesting part of this.
As it stands, this crate will simply drop the results of a CQE for a user_data value that it does not recognize. And what value should the sender be using for that dest data value?
One idea, the best I have at the moment, is to have the destination start a kind of fake operation Future. One where it doesn't create an SQE but does register a waker with the slab to get a user data value. And that user data value is reported to the caller in some fashion. Now that the user space knows what dest data value to use, then msg_send_fd can be performed.
That is some handshaking, but it would keep things safe.
One slight complication is the close of the dest ring might find a slab entry for a future that didn't get resolves and no amount of cancelling outstanding operations is going to help, on the sending side or the receiving side.
Perhaps this is a case where when the receiving future is dropped, it removes the entry from the slab entirely.

At its core, the io_uring command allows sending a copy of the file descriptor stored in a fixed table's slot to another fixed table's slot, the fixed table of another uring. Both the source uring and destination uring have to be known, and the source table index has to be known. The destination table index can be chosen by the caller or can set to be auto selected by the destination ring.

This is really cool! I didn't know this :)

This isn't documented in the liburing man pages yet but it's been promised. There is already the function and an interesting test acting as an example in the tip of io-uring.

Does this work for IPC?

Does this work for IPC?

I don't understand the question. The feature allows sending a fixed slot descriptor to any ring that you have a legal descriptor for. If you forked and therefore share a ring descriptor between two processes, then you can send from one process to another. I think they talk about that kind of security level downgrade, where a process running as root can accept connections and then pass the connection to another process running with lesser permissions. But you need a shared ring file descriptor. So general IPC over unix domain sockets - no.
Interestingly, Josh just committed kernel patches to let a uring file descriptor be stored in a fixed slot, so if you have the first level shared, you could send uring descriptors themselves with this mechanism. But that's for an unreleased kernel version.

Ah, that answers my question actually


Timestamp support for Kafka <IP_ADDRESS>

Hi,
Recently upgraded the Kafka server to <IP_ADDRESS>, and the message timestamp turns out to be a showstopper with this gem. We need a ruby-kafka gem for the producer app, so any word on what the status on adding the timestamp support is?
Any help on this issue is much appreciated!
(If you or anyone has any alternate ways to work around it, please advise! Thanks!

For now, you should be able to use the branch in https://github.com/zendesk/ruby-kafka/pull/279
This gem still primarily targets Kafka 0.10 - I expect to move it to 0.10 in the coming months, though.

Thanks Daniel. I tried the branch in #270 after reading up and down all the issues here.. :) That branch didn't work for me using Kafka broker 0.10.2 - the messages weren't getting processed at all on the consumer KStreams app, and even the useful error messages/exceptions that I get with this gem, I can't get using the other branch. Seemed like a lost case, and running out of time, I had to switch back to this gem for now, and roll back to Kafka 0.10.1. So.. I think you get the picture here: WE NEED this gem to get updated :) The compatibility wall with Kafka hits at Kafka 0.10.2.

Not sure if this will help you, but you can alter the message.timestamp.type config on a per topic basis to CreateTime. This will have the brokers use the time that a message is ingested into Kafka as the message's timestamp. For topics produced to by ruby-kafka, you can use this command
bin/kafka-configs --alter --entity-name test_topic --entity-type topics --add-config message.timestamp.type=CreateTime --zookeeper localhost:2181

@prestonwrthompson
Maybe you can use LogAppendTime, instead of CreateTime in TimestampType value.
bin/kafka-configs --alter --entity-name test_topic --entity-type topics --add-config message.timestamp.type=LogAppendTime --zookeeper localhost:2181

yeah, thanks for catching that. i meant to suggest changing the config to LogAppendTime, not CreateTime

Thanks all for your help.. We wound up rolling back to Kafka <IP_ADDRESS> due to incompatibility w/ this gem. Will upgrade when this gets upgraded.

@sophia-d it's likely that I'll work on upgrading ruby-kafka to Kafka protocol version 0.10 this month.

Hey @dasch do you have any update on the upgrade ? Thanks

I don't, I'm afraid - I don't have a production 0.10 cluster to test
against yet. Once I do I'll figure out a way to move to support that
version.
On Sun, 21 May 2017 at 16.23, Elyahou<EMAIL_ADDRESS>wrote:

Hey @dasch https://github.com/dasch do you have any update on the
upgrade ? Thanks
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/zendesk/ruby-kafka/issues/331#issuecomment-302939804,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAAYzzV_NW0AaZBVm7_ivi9DHrFtVOBLks5r8EjFgaJpZM4MjGxb
.

hey @dasch, is there any chance to get a support for 0.10.2 in the upcoming weeks? Thanks!

Duplicate of #270

Resolved by #376.

@sophia-d I've just released v0.5.0.beta1 with support for Kafka 0.10. Would you be willing to run this in production for some time and see if everything works as expected?

Hi @dasch, yes will give it a test run.. Mind that the biggest difference was the leap from Kafka 0.10.1 to 0.10.2. I already am using the gem as is for Kafka 0.10.1.

@sophia-d was there a change in the wire protocol in v0.10.2? Otherwise it should work just as well, unless there's a bug in that version of Kafka.
The new thing in ruby-kafka v0.5 is that it uses the Kafka 0.10 message format, which should improve performance but would also be the source of any breakage. Everything looks fine in my testing, though, but it's nice to validate against a real cluster.


Investigate stylelint highlighting in VSCode

What will this achieve?

Lint for everyone locally — figure out what settings are conflicting
How will success be measured?

Stylelint issues are shown in VS Code
Additional information

Designs
Existing code
etc

In about modal for instance if I change 43 to a px value, this should be highlighted in VSCode. Running the script lint:styles does report it.
padding: 0 20px 0 $spacing-05;

These are my personal notes condensed from vscode-stylelint's documentation
VSCode

Install VSCode's Stylelint extension vscode-stylelint
Open Settings page > find stylelint.validate, add "css" and "scss". Optionally include "less" and "postcss".

package.json, vscode-stylelint doesn't work with stylelint version 13 or less:

$ npm install stylelint@latest

If the above still doesn't work, it may be that global linting packages are interfering.
Optional, uninstall global linting packages
List global packages from anywhere using CLI

$ npm list -g --depth=0

Run this command to uninstall a global package

$ npm uninstall -g <package_name>

Global directories for reference

Mac: /usr/local/lib/node_modules
Windows 7+: %USERPROFILE%\AppData\Roaming\npm\node_modules

Does Lee's stylelint plugin solve for this?

Hey @paul-balchin-ibm, we're using "stylelint": "^15.x.x" so I'm not sure if this is still applicable for our library. Are you ok with closing this?


Fixing defaultProps Warning in Select Component

This fixes #306
The approach is to provide an initialValue prop for setting the
initial value rather than defaultValue since the component is not
uncontrolled. I then check the selectedIndex and initialValueIndex
(if provided) when setting the value prop of the HTML  element.

Try out https://joepuzzo.github.io/informed. I rewrote this library and will now be supporting informed instead.


Support for pushing content types to a CMS

I might have missed it in the docs, but it’s not clear to me if a content source can allow you to push the content types to a CMS as well?
If not, this is a feature we’d love to have. We’ve built our own wrapper around the different CMSes we use, letting us generate TypeScript types as well as push the content types to the CMS, but potentially replacing this with an open source solution would be very interesting.

Thanks for suggesting this feature. It's not yet possible with Contentlayer but I agree it would be very powerful. (However, quite complex to build so it'll take some time to build this.)


[io][mechanics] visualization of the initial state in siconos_vview

It is not possible to visualize the initial position in scions_vview. It need to simulate at least one to visualize the scene after this first step.
It seems also that the initial condition are taken into account in mechanics_io in the same for all the objects dynamic/static Convec shape/Primitive.

Fixed :-)
video


linux/bootstrap-tools: move libstdc++ out of default library search path

This change allows building new gcc during bootstrap without fear of pulling in outdated libstdc++.so after g++ switched from bootstrapTools to freshly built g++.
Noticed when tried to add early bootstrap stage to rebuild gcc before glibc is fully untangled from bootstrapTools as a failure to built binutils:
ld: dwp.o: in function `__gnu_cxx::new_allocator<gold::Dwp_output_file::Contribution>::allocate(unsigned long, void const*)':
/nix/store/...-gcc-11.3.0/include/c++/11.3.0/ext/new_allocator.h:116: undefined reference to `std::__throw_bad_array_new_length()'

The change survives existing bootstrap and unblockes early gcc bootstrap.
Description of changes

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

This regressed tests for bootstrap tarball building:
$ nix build --no-link -f ./pkgs/stdenv/linux/make-bootstrap-tools.nix
...
       > /nix/store/82k0qrddaa84i5nl3zv3f3bcadwwcxli-test-bootstrap-tools/bin/bar: error while loading shared libraries: libstdc++.so.6: cannot open shared object file: No such file or directory

I think it's a pure test issue. Test does not use any wrappers and assumes certain directory layout.
Looking.

Proposed the fix as https://github.com/NixOS/nixpkgs/pull/210752


Optimized  RecordBatch for constant columns

Is your feature request related to a problem or challenge? Please describe what you are trying to do.
Certain record batches have columns that entirely the same value -- for example the partitioning column in https://github.com/apache/arrow-datafusion/pull/1141
In that case the ListingProvider adds a column with a constant value based on the path in the file system
So a file in a path like /foo/date=2021-11-5/bar.parquet would end up producing one or more RecordBatches with a date column that had the value 2021-11-5.
At the moment, to represent such a column using an Arrow array we have to repeat the value 2021-11-5 over and over again which is inefficient both in space storage as well as in processing time.

date
other_col

2021-11-5
1

2021-11-5
2

2021-11-5
3

2021-11-5
4

...
...

2021-11-5
1000000000

Describe the solution you'd like
It seems that having a sort of RecordBatch that can contain constant columns represented by a scalar value (like the C++ ExecBatch) is a pretty nice abstraction that will help adding tons of optimizations. I didn't find an issue summarizing this discussion and conclusion, should we create one?
According to @westonpace  , in C++ there is ExecBatch in addition to RecordBatch for this purpose.  ExecBatch is used within the exec plan and it's pretty much identical to record batch except:

No schema
No field names
Column values can be scalars

Note DataFusion already has a Array/Scalar notion in ColumnarValue https://github.com/apache/arrow-datafusion/blob/a1c794cec233f7fe34f34c7d64f529625a507669/datafusion/src/physical_plan/mod.rs#L388-L395 but its use is confined to expression evaluation at the moment.
Perhaps we could build something like DFRecordBatch akin to DFSchema (aka wrap the arrow RecordBatch with methods that allow the columns to be ColumnarValue)
Describe alternatives you've considered
We could use a Dictionary(Int8, Utf8) array to do better (where each repeated string value only requires a single i8 (8 bytes) but it still requires ~ 8 bytes * the number of total rows in the RecordBatch
Another  classic approach I have seen for this kind of problem is to use RLE encoding so that these “almost entirely constant columns” become a very small number of RLE runs.
I think RLE has been proposed for Arrow several times before, but it doesn’t have the “access array[i]” in constant time” property required for arrays
Additional context
(this is my transcription of some information in an ASF slack thread: https://the-asf.slack.com/archives/C01QUFS30TD/p1634810772007000)
cc @houqp  @rdettai @Dandandan

Perhaps we could build something like DFRecordBatch akin to DFSchema (aka wrap the arrow RecordBatch with methods that allow the columns to be ColumnarValue)

I'm for this approach.
In fact we can allow several other types that are memory efficient:

sequence with increase or decrease direction and step size
repeated array that's n copies of underlying data
zoomed out sample of underlying data
etc.

Another thing to contemplate is support for RLE (aka store runs of repeated values)
A
A
A
B
B
B
B
-->
(A, 3)
(B, 4)

Tangentially related, since I have been investigating something related: the RecordBatch in arrow-rs is not defined in the arrow specification. Specifically, the term "RecordBatch" comes from the IPC specification RecordBatch message, but such a message does hold a schema, since the schema is written on a separate Schema message at the beginning of the file. The C data interface has no such concept at all. So, I do not think it is an issue to use something else that fits our needs better.
I think that the two requirements for something to be consistent with the arrow format are:

data and schema round-trips lossless over Arrow's IPC boundary
data and schema round-trips lossless over Arrow's C data interface boundary at zero cost

I do not think RLE and other encodings fulfill these requirements (not sure whether this is an issue, though).

👍  I agree and I think keeping a separation between what is generally useful for users of Arrow libraries and what is more likely to be more narrowly useful to DataFusion is important. For example, I think it would be fine to add specialized constant column or RLE support in DataFusion, but that likely doesn't belong in an arrow implementation

maybe something like this just to see how much change is needed

I had a further thought on this and believe having DFRecordBatch is approachable, with some inter steps:

[ ] create a trait DFRecordBatch, with basic functions defined such as fn is_emtpy(&self) -> bool, fn columns(&self) -> Vec<ArrayRef>, and fn column(&self, i: usize) -> ArrayRef defined, and also fn to_arrow(&self) -> RecordBatch
[ ] have a default DFRecordBatchImpl struct that wraps around arrow's RecordBatch and impl DFRecordBatch,
[ ] have fn to_df(&self) -> DFRecordBatch implemented for arrow's RecordBatch
[ ] change DataFrame to return DFRecordBatch instead and let this breaking change released to downstream
[ ] add other types of impls for DFRecordBatch or enhance DFRecordBatchImpl to have virtual columns

columns(&self) -> Vec, and fn column(&self, i: usize) -> ArrayRef

I think instead of returning ArrayRef, we should be returning ColumnarValue here?
RLE can be something we use to further compress arrow record batches, but I think it is an overkill for constant columns because this use-case can already be covered by ColumnarValue::Scalar. I can see us adding it in the future for other use-cases though, perhaps as another variant to ColumnarValue?. ColumnarValue::Scalar is effectively an special case for RLE, which maps nicely to arrow compute kernels that are specialized to scalar operands.

perhaps as another variant to ColumnarValue?
Sequence is one example

Recently I've tried to make a stab at this. As RecordBatch is widely used across the codebase, replacing them with DFRecordBatch makes a lot of changes. I'm wondering if this is expected or there is a way to limit the scope of change but I'm not aware of?

I would indeed expect it to be a substantial change across the codebase. I don't have any magic suggestions to reduce the potential impact


Fixes #2302 Possibility to create master with manager, but without agent

Signed-off-by: Nicolas De Loof<EMAIL_ADDRESS>

Nice
Usually changing the swarm options would require a config migration, but in this case it really is a very small change (and also around some code that already has some issues, so we're not breaking anything existing).  Still, it makes me slightly nervous.

FYI, just merged https://github.com/docker/machine/pull/3287 which will affect this.

rebased

@ndeloof Should I continue evaluating this for merge or wait until you've submitted the changes you suggest in the other thread?

@nathanleclaire rebased after #3387 has been merged

Nice.  Thanks for your persistence and good work @ndeloof.
LGTM


feat(json-rpc): add beacon network JSON-RPC endpoints

Add basic beacon network endpoints to JSON_RPC specs.
All of the endpoints are copy/paste from the history network except this one:

I'm proposing adding portal_beaconOptimisticStateRoot endpoint that returns the hex encoded recent known beacon state root.

I'm proposing adding portal_beaconOptimisticStateRoot endpoint that returns the hex encoded recent known beacon state root.

To confirm, this new one is just returning the beacon state root embedded in the optimisticUpdate that we have stored locally?

To confirm, this new one is just returning the beacon state root embedded in the optimisticUpdate that we have stored locally?

yes

Thinking about the state root endpoints more, we will want also similar endpoint for the finality updates: portal_beaconFinalityStateRoot and probably adding additional field to the response object,  the slot number?
Current use cases:

testing if a portal node is in sync - check if latest optimistic and finality roots/slots match the expected.
Glados monitoring - check if the updates are available on the network based on the expected slot number.


Histogram data fixes

Fixes two bugs in the Histogram constructor:

Updated old code assuming that Chart data is always arrays, fixing how Charts are cast to a Histogram.

Fixed code to compute bin edges from bin centers when centers are defined as integers.

Sounds good! Plan to add some unit tests before merging?

Yes, pushed, ready to merge once tests pass.

Test are green. Merging.


fix dropped test errors

This fixes two dropped test err variables.

This change is

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

The last comment says "Approved" as well as "Waiting on @alrs." What is it waiting for, @manishrjain?

Forgot to merge it. Merged now. Thanks for the PR!


GPU memory requirement

HI,
I run the test command in Readme, but meet out of memory problem. My computer has 8 GPUs  (TITAN X Pascal 12G).
What is the min GPU memory requirement?
python main.py --dir_data PATH_TO_KITTI_DC --data_name KITTIDC --split_json ../data_json/kitti_dc.json \
    --patch_height 240 --patch_width 1216 --gpus 0,1,2,3 --max_depth 90.0 --num_sample 0 \
    --test_only --pretrain ../results/NLSPN_KITTI_DC.pt --preserve_input --save NAME_TO_SAVE

There is also another small question. Is --top_crop option only used for training?

If you run the code with full-size KITTI, it may require about 12~13 GB GPU memory.
You can try with --test_crop argument to crop the top region for the test set.

@zzangjinsun  Thanks for your reply.
According to the paper, center-cropped 1216 * 240 patches with a batch size of 25 are used for training on KITTI DC and center-cropped 912 * 228 patches with a batch size of 12 are used for ablation study. What are the GPU requirements for these two setups?
As mentioned above, even testing with a batch size of 1 needs 12~13G memory, how to train the model with a batch size of 12 or 25? In README, the requirement is just 4x NVIDIA GTX 1080 TI / 4x NVIDIA Titan RTX GPUs

4x NVIDIA GTX 1080 TI / 4x NVIDIA Titan RTX GPUs are enough to train on NYUv2 or 912x228-sized KITTI dataset.
For the full-training, a powerful machine with 8 NVIDIA P40 GPUs is used. (Please refer to here)

@zzangjinsun Thanks for your reply.
If I train models on KITTI dataset with 4x NVIDIA GTX 1080 TI GPUs using a crop size of 912x228, is the largest batch size be 12? Could you please help supply the trained model of this setting?

Yes. You should be able to train with 912x228 KITTI data and batch size 12.
Please refer to the attached images.
Currently, there is no plan to release models trained with cropped images.
Please use the full-trained model I released.

Training parameters

GPU usage

@zzangjinsun
Thanks for your instructions.
For the setting above, I can only find its result on KITTI val set. The RMSE is 884.1 (Table 3 in the paper.) Canyou supply the full result (RMSE. MAE, iRMSE, iMAE)?

@zzangjinsun
On KITTI, I reproduced the result with RMSE=884.1 on val_selection_cropped set, but I can not reproduce the result of the released model with RMSE=771.8 on val_selection_cropped set. I followed the instructions in the 6.2 section of the paper.
Can you share all the arguments of this training setting? An image like the one above is appreciated.
Thanks!

@zzangjinsun
On KITTI, I reproduced the result with RMSE=884.1 on val_selection_cropped set, but I can not reproduce the result of the released model with RMSE=771.8 on val_selection_cropped set. I followed the instructions in the 6.2 section of the paper.
Can you share all the arguments of this training setting? An image like the one above is appreciated.
Thanks!

Please use the command provided in training with the following arguments:
--epochs 25 --gamma 1.0,0.4,0.16,0.064 --decay 10,15,20,25 --batch_size AS_MANY_AS_YOU_CAN --preserve_input
(Note that --preserve_input is optional.)

Please use the command provided in training with the following arguments:
--epochs 25 --gamma 1.0,0.4,0.16,0.064 --decay 10,15,20,25 --batch_size AS_MANY_AS_YOU_CAN --preserve_input
(Note that --preserve_input is optional.)

@zzangjinsun  Thanks,
When I enlarge the batch size, do I need to enlarge the learning rate correspondingly?

@zzangjinsun  Thanks,
When I enlarge the batch size, do I need to enlarge the learning rate correspondingly?

I think you can keep the original learning rate.

I think you can keep the original learning rate.


Update docs

Add counterterm documentation and density benchmark.

Codecov Report

Merging #117 (fc8db5c) into master (8b646a2) will increase coverage by 2.83%.
The diff coverage is n/a.

:exclamation: Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.
@@            Coverage Diff             @@
##           master     #117      +/-   ##
==========================================
+ Coverage   63.58%   66.42%   +2.83%
==========================================
  Files          29       32       +3
  Lines        1906     1930      +24
==========================================
+ Hits         1212     1282      +70
+ Misses        694      648      -46

see 13 files with indirect coverage changes


Add pickpockets to loot tracker

Suggested a few times in the past (e.g. https://github.com/runelite/runelite/issues/5033).
I thought this would be a good addition to the loot tracker. It was something I found myself wanting when thieving master farmers. It seems to have been a bit polarizing in the past, so I've put it behind a config flag by default.
Possibly extensible to cover loot from all types of thieving (stalls, chests, etc). Let me know if there are any improvements needed

Most of this was added in #10759. I kept the LootEvent out as it makes the diff a lot smaller. If you want to revisit that we could, though I am not sure I want to use it for pickpocketing since it requires having to hardcode in all of the possible pickpocket targets which is not so great. The other usages seemed good, though.

I am closing this PR for now, as the main objective (tracking pickpocket loots) has been implemented and merged separately. This PR, or a new PR, can be opened for the refactoring portion of this feature.


oauth.getRequestAccessURL requires 'redirect_uri' but doesn't validate that it is provided

oauth.getRequestAccessURL should validate that redirect_uri is provided. It's a wasted API call to Strava to have them tell us the value is required when we can check if it's required ourselves. Throwing an exception would be sensible, since it's a programming error if the value is not provided.
The documentation for oath.getRequestAccessURL should also document that it requires the client_id and redirect_uri. Most API calls just require the access_token, so this is exceptional.

I confirmed this bug still exists. The extra check is needed here: https://github.com/UnbounDev/node-strava-v3/blob/master/lib/oauth.js#L8

Confirmed today this is still an issue.

Hello @markstos, to fix it I created #120


Feat/support env var sdkconfig defaults

closes #17
closes #30

@igrr PTAL! the env var expansion is not used elsewhere, AFAIK, I think it's okay to remove it temporarily and add it on esp-idf-kconfig side.

Hi @hfudev with latest updates and my test branch https://github.com/espressif/esp-bsp/pull/142 I get following erros:

Find applications:

C:\github_repos\esp-bsp\examples> python -m idf_build_apps find -p display --config sdkconfig.bsp.* -t all --manifest-file .build-rules.yml --build-dir build_@w --check-warnings
Traceback (most recent call last):
  File "runpy.py", line 194, in _run_module_as_main
  File "runpy.py", line 87, in _run_code
  File "C:\github_repos\idf-build-apps\idf_build_apps\__main__.py", line 9, in <module>
    main()
  File "C:\github_repos\idf-build-apps\idf_build_apps\main.py", line 608, in main
    sdkconfig_defaults=args.sdkconfig_defaults,
AttributeError: 'Namespace' object has no attribute 'sdkconfig_defaults'

Build applications:

 python -m idf_build_apps build -p display --config sdkconfig.bsp.* -t all --manifest-file .build-rules.yml --build-dir build_@w --check-warnings
Executing action: all (aliases: build)
Running cmake in directory C:\github_repos\esp-bsp\examples\display\build_esp-box
Executing "cmake -G Ninja -DPYTHON_DEPS_CHECKED=1 -DPYTHON=C:\Espressif\python_env\idf5.1_py3.8_env\Scripts\python.exe -DESP_PLATFORM=1 -DIDF_TARGET=esp32 -DSDKCONFIG_DEFAULTS=; -DCCACHE_ENABLE=1 C:\github_repos\esp-bsp\examples\display"...

(The SDKCONFIG_DEFAULTS=;)

@igrr @tore-espressif Thanks for your review! I made some major changes today (revert the breaking change, change the find logic, ...), so it's not quite ready now. I'll add more tests on the sdkconfig_defaults related settings later this week.
@tore-espressif sorry still didn't fix the issue you reported last week, I'll ask for your review again once I got this PR ready.

@igrr @tore-espressif This PR is ready. PTAL again. Thanks!

Hi @hfudev thanks for the update! With the same test branch espressif/esp-bsp#142 I get following output:
python -m idf_build_apps find -p display --config sdkconfig.bsp.* -t all --manifest-file .build-rules.yml --build-dir build_@w
(cmake) App display, target esp32, sdkconfig display\sdkconfig.bsp.esp-box, build in C:\github_repos\esp-bsp\examples\display\build_esp-box
(cmake) App display, target esp32, sdkconfig display\sdkconfig.bsp.esp32_s2_kaluga_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp32_s2_kaluga_kit
(cmake) App display, target esp32, sdkconfig display\sdkconfig.bsp.esp_wrover_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp_wrover_kit
(cmake) App display, target esp32s2, sdkconfig display\sdkconfig.bsp.esp-box, build in C:\github_repos\esp-bsp\examples\display\build_esp-box
(cmake) App display, target esp32s2, sdkconfig display\sdkconfig.bsp.esp32_s2_kaluga_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp32_s2_kaluga_kit
(cmake) App display, target esp32s2, sdkconfig display\sdkconfig.bsp.esp_wrover_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp_wrover_kit
(cmake) App display, target esp32s3, sdkconfig display\sdkconfig.bsp.esp-box, build in C:\github_repos\esp-bsp\examples\display\build_esp-box
(cmake) App display, target esp32s3, sdkconfig display\sdkconfig.bsp.esp32_s2_kaluga_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp32_s2_kaluga_kit
(cmake) App display, target esp32s3, sdkconfig display\sdkconfig.bsp.esp_wrover_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp_wrover_kit

That is: the CONFIG_IDF_TARGET="espxx" from sdkconfig.bsp.someting is not honored

Hi @hfudev thanks for the update! With the same test branch espressif/esp-bsp#142 I get following output:
python -m idf_build_apps find -p display --config sdkconfig.bsp.* -t all --manifest-file .build-rules.yml --build-dir build_@w
(cmake) App display, target esp32, sdkconfig display\sdkconfig.bsp.esp-box, build in C:\github_repos\esp-bsp\examples\display\build_esp-box
(cmake) App display, target esp32, sdkconfig display\sdkconfig.bsp.esp32_s2_kaluga_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp32_s2_kaluga_kit
(cmake) App display, target esp32, sdkconfig display\sdkconfig.bsp.esp_wrover_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp_wrover_kit
(cmake) App display, target esp32s2, sdkconfig display\sdkconfig.bsp.esp-box, build in C:\github_repos\esp-bsp\examples\display\build_esp-box
(cmake) App display, target esp32s2, sdkconfig display\sdkconfig.bsp.esp32_s2_kaluga_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp32_s2_kaluga_kit
(cmake) App display, target esp32s2, sdkconfig display\sdkconfig.bsp.esp_wrover_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp_wrover_kit
(cmake) App display, target esp32s3, sdkconfig display\sdkconfig.bsp.esp-box, build in C:\github_repos\esp-bsp\examples\display\build_esp-box
(cmake) App display, target esp32s3, sdkconfig display\sdkconfig.bsp.esp32_s2_kaluga_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp32_s2_kaluga_kit
(cmake) App display, target esp32s3, sdkconfig display\sdkconfig.bsp.esp_wrover_kit, build in C:\github_repos\esp-bsp\examples\display\build_esp_wrover_kit

That is: the CONFIG_IDF_TARGET="espxx" from sdkconfig.bsp.someting is not honored.
I expect only three apps:

esp32 with wrove_kit
esp32s2 with kaluga_kit
esp32s3 with esp_box

Thanks for helping testing! Now it's fixed. Also added one relative path test case. @tore-espressif Thanks again!

Thanks for helping testing! Now it's fixed. Also added one relative path test case. @tore-espressif Thanks again!

Now for all the build jobs (same command, same test repository) I get the following:
CMake Error at C:/esp_repos/esp-idf/master/tools/cmake/project.cmake:446 (message):
  SDKCONFIG_DEFAULTS
  'C:/github_repos/esp-bsp/examples/display/display/sdkconfig.bsp.esp32_s2_kaluga_kit'
  does not exist.

Sorry for my poor debugging (Seems like the second line from the log should not be there?)

Thanks for helping testing! Now it's fixed. Also added one relative path test case. @tore-espressif Thanks again!

Find apps work OK now! Now for all the build jobs (same command, same test repository) I get the following:
CMake Error at C:/esp_repos/esp-idf/master/tools/cmake/project.cmake:446 (message):
  SDKCONFIG_DEFAULTS
  'C:/github_repos/esp-bsp/examples/display/display/sdkconfig.bsp.esp32_s2_kaluga_kit'
  does not exist.

Sorry for my poor debugging (Seems like the second line from the log should not be there?)
EDIT: OH! there is display/display in the path!

I get it. if I pass the sdkconfig_default as a relative path, ESP-IDF build system won't treat it as the relative path of current session pwd, but the relative path of project path. which makes it into two "displays".
thanks again! @tore-espressif


Inotify tail doesn't work when a file is quickly recreated

Inotify tail doesn't seem to properly work for files that are created and deleted quickly.
Using:
36ba8bccb91f119d56866e51ae0dd11ba2731ec2 github.com/ActiveState/tail (heads/master)
Using the tailing config:
o := tail.Config{
    Location: s,
    // Poll: true,
    ReOpen: true,
    Follow: true,
    MustExist: false,
}

Could you try again with latest master?

Experiencing this as well with 157f8ef

I have the same experience, setting Poll: true will be solve the issue.


Add timestamp to accelerometer record

Issue Description
When sending accelerometer data to the server, a timestamp should also be sent in order to properly graph events over time properly. The accelerometer cannot be relied upon to be consistently 60Hz.
Proposed Solution
Change data format from
x,y,z

to
timestamp,x,y,z

My original comment is a little hard to parse. Adding a time stamp means we don't have to worry about perfect timing or jitter when sending the data over websocket.


my.go is missing

the current master do not contain the my.go file which is linked through a symlink in the latest commit
linux/amd64
can't load package: package github.com/0leksandr/sshmirror: open /media/k/SSD2/ApacheServer/web/2021/mobymax/sshmirror/my.go: no such file or directory
Build failed for linux/amd64

building the version before this my.go change - works just fine

Hi! Thank you for the report, and sorry for the late reply.
It should be fixed now in release 1.4


Demo app for autoform 6

@aldeed I read awhile ago that you were working on a demo app for autoform 6. Is this almost complete?

The existing demo uses AF6 now:
http://autoform.meteorapp.com/
https://github.com/aldeed/autoform-demo/


Examples

Examples to Create
Basic

[x] Basic layout
[x] Wrapping
[x] A grid
[x] Active. A grid, clicking a tile makes it active and styled visually different from the others
[x] Disabled nodes
[x] Focus trap

Advanced

[x] Transitions using active states
[x] Transitions using isExiting
[x] Transitions with setting mount on focus to a particular node
[x] Modal using traps
[x] Basic TV app
[x] Pointer events
[x] Grid that maintains its position

The TV example is still a WIP, but I'm closing out this issue


gulp-bower breaks when bower.json does not exist

Bower can be replaced with npm / yarn. As such, dependencies can be listed in package.json instead of in a bower.json.
Running gulp build fails, as in my theme, bower.json does not exist. It should probably continue silently if the file is absent.
(Or, strip out gulp-bower completely)

I did notice this and I had to put an override in place for dependencies:
https://github.com/highrisedigital/wpgetstarted/blob/master/content/themes/the-ground-floor/gulpfile.js#L48


How to change AndroidTv Name, Logo, Launcher Logo

Hi, where do change/assign application name, logo, launcher, etc for AndroidTv and FireTv APPs ?

Im using renative.
On Thu, 21 Sep 2023 at 9:10 AM, Aurimas Mickys @.***>
wrote:

Are you using react native or renative?
—
Reply to this email directly, view it on GitHub
https://github.com/flexn-io/create/issues/172#issuecomment-1728981630,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACYOZ5RIH5ZJ7P5X4WRYZ7LX3PR7ZANCNFSM6AAAAAA5AHV3EY
.
You are receiving this because you authored the thread.Message ID:
@.***>

Im following below folder structure but images under /appConfigs/helloworld/asstes are not being transferred to apps under platformBuilds folders.

@saimirg things you described are editable in appConfigs/[APP_ID] under renative.json you can change app name, app id, used engines and so on. App icons are also under appConfigs, from example of create-template-starter https://github.com/flexn-io/create/tree/main/packages/template-starter/appConfigs/base/assets/runtime I see that icons are under runtime folder. Can read a bit more about project configuration here if that helps https://renative.org/docs/guides/advanced-configuration

hi @pauliusguzas i've bee doing several tests but the two files ic_log and ic_launcher get overwritten everytime.
I;ve also tried adding new files on respective folders as documentation says, but again they don't get copied into platformBuilds folder.

@saimirg what versions of rnv and create are you using? And node

Hi, these are the details. I need to understand where in the folder structure i need to place ic_logo and ic_launcher, or even other image names, so that that they are copied to /platformBuilds/... respective app.

@saimirg it's default folder structure where you should place your assets. As you can see there are multiple drawable folders difference is resolution. Do you see these folders in your file structure?

No, i dont see those folders, i only have base->assets. Shall i create those ?

The issue is that create template uses imported assets. Go to appConfigs/app/renative.json and add "assetSources": null  under common to not import assets and then you should be able to use your custom assets


Fixed setup error

Fixed error in type.less that caused error in LESS compiler, but had to
remove responsive headings.
Also moved from React.render to reactDOM.render, and a few other small
fixes.

Thought the db was required but removing it did not cause it to break, so it has been taken back out.
Should now be ready to pull.
This addresses issue #86

@JedWatson

Looks great - thanks @Noviny!


Compilation error in wasm demo code

Hi, Thanks for updating the example. I tried it on my PC and get the following compilation error:
error[E0277]: `*mut u8` cannot be shared between threads safely
  --> c:\Localdata\extra\rust\38_plotters\src\drawing\backend_impl\canvas.rs:62:6
   |
62 | impl DrawingBackend for CanvasBackend {
   |      ^^^^^^^^^^^^^^ `*mut u8` cannot be shared between threads safely

Thanks for reporting. I believe it's broken by a recent change. Already fixed in master.