Dashboard creation for data exploration

Create a dashboard to view and interact with different statistics about the data.
This could be built using Streamlit.
For example, input the annotations + results + more, then:

Show statistics about number of images in each class
Show examples of where model is failing/doing well
Be able to easily see what classes the model is performing on
Perhaps be able to compare across experiments the results a model has for different classes?

For example, select two experiments, show the lineage, then compare them across the performance of each class?

Should at all times be able to explore the data and see where the model is not performing well...

See an example here -- https://blog.streamlit.io/how-to-build-a-real-time-live-dashboard-with-streamlit/

How will the data transfer from said things to the dashboard? Is there an
API?
On Mon, 20 Feb 2023, 5:33 pm Daniel Bourke, @.***>
wrote:

See an example here --
https://blog.streamlit.io/how-to-build-a-real-time-live-dashboard-with-streamlit/
—
Reply to this email directly, view it on GitHub
https://github.com/mrdbourke/nutrify/issues/72#issuecomment-1436317698,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMGDQM64M5PUF5MYY2WMS4TWYLXZFANCNFSM6AAAAAAU23LN3E
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

@shivan-s, right now it's a static CSV but will be looking to make this via URL (e.g. straight from Google Storage or from Hugging Face Datasets etc)
"""
Streamlit dashboard for exploring data from FoodVision.

Basing off this: https://blog.streamlit.io/how-to-build-a-real-time-live-dashboard-with-streamlit/
"""
import pandas as pd
import streamlit as st

# Import the data
# TODO: change this to a URL that gets live tracked?/updated etc
dataset_url = "annotations.csv"

# TODO: cache this so it's saved: https://docs.streamlit.io/library/advanced-features/caching
def get_data() -> pd.DataFrame:
    """Get the data from a CSV file.
    """
    return pd.read_csv(dataset_url)

Cool stuff!
I'd love to have a go building this dashboard.
Just need to know how the data is managed and what you want to see on the
dashboard.
Shivan
On Mon, 20 Feb 2023, 5:48 pm Daniel Bourke, @.***>
wrote:

@shivan-s https://github.com/shivan-s, right now it's a static CSV but
will be looking to make this via URL (e.g. straight from Google Storage or
from Hugging Face Datasets etc)
"""Streamlit dashboard for exploring data from FoodVision.Basing off this: https://blog.streamlit.io/how-to-build-a-real-time-live-dashboard-with-streamlit/ """import pandas as pdimport streamlit as st
Import the data# TODO: change this to a URL that gets live tracked?/updated etcdataset_url = "annotations.csv"
TODO: cache this so it's saved: https://docs.streamlit.io/library/advanced-features/caching def get_data() -> pd.DataFrame:
"""Get the data from a CSV file.    """
return pd.read_csv(dataset_url)

—
Reply to this email directly, view it on GitHub
https://github.com/mrdbourke/nutrify/issues/72#issuecomment-1436324450,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMGDQM5X76KG4ONKNAWLLALWYLZSNANCNFSM6AAAAAAU23LN3E
.
You are receiving this because you were mentioned.Message ID:
@.***>

Epic! Just email you a sample dataset/labels.
Basically want an extensive EDA of the annotations for now.
So I know which labels need more work.
Can add image displays/model results later on.

I have your dashboard up now.
I'm trying to think of ways to give you value in the EDA process.
What do you need to see and look into?

Epic!
Basically I'd like several different ways to view label counts, for example, the dashboard should answer:

How many images are there per class?
How many images are there per specific label_source?
A quick way to view images that have under 100 manual_upload as the label_source (we're trying to get all classes above 100 manually uploaded images)

These are some of the main things we're looking for.
Perhaps a better looking way to view all the class names?
E.g. a map from class_name -> label ({0: "apple_red"}) so that all class names can be viewed easily


⚠ Bofrost.de has degraded performance

In f14db2b, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1132 ms

Resolved: Bofrost.de performance has improved in 2a41ea2 after 6 minutes.


Add support for image-rs 0.24

Since the latest version of image-rs library require Rust 1.56.0, I think that img_hash also should use Rust 2021 edition and bump minimal requirements.
For now I can't use image-rs 0.24.0, because img_hash crate cannot interoperate with this version and I have such errors(this works with 0.23)
error[E0277]: the trait bound `image::DynamicImage: Image` is not satisfied
   --> czkawka_core/src/similar_images.rs:606:50
    |
606 |                     let hash = hasher.hash_image(&image);
    |                                       ---------- ^^^^^^ the trait `Image` is not implemented for `image::DynamicImage`
    |                                       |
    |                                       required by a bound introduced by this call

For anyone wanting to use library with updated dependencies I created repo which use image 0.24.2 and have other libraries updated(logic is not changed)
https://github.com/qarmin/img_hash
https://crates.io/crates/image_hasher/

@qarmin : Is there any reason you chose to publish a new crate instead of contributing to this one ? I opened a pull request with your changes.

@lovasoa This crate is not very well maintained. I wish the author would add some co-maintainers. Though what worked out for me was simply using img_hash::image and then simply take the contents and construct a borrowed image from it. That way it's automatically compatible with all image versions.

@abonander , would you be open to adding some co-maintainers ?


Test jupyter notebooks

make sure jupyter notebooks run

@OskarElek the Jupyter notebook always fail because they have not been tested, this will make sure they are tested.
We expect all the ubuntu workflows to fail, however if the Jupiter notebooks run perfectly on ubuntu the tests will start to pass again.


Chroma

You can omit the base class implementation. Just one file where this works is enough, we'll highlight the interfaces a bit later.
You can use GPT-4 for initial implementation
Tests are encouraged but not required. It's better to do something minimal than nothing.

References:

https://api.python.langchain.com/en/latest/vectorstores/langchain.vectorstores.chroma.Chroma.html

Closed in #42


Unexpected token j in JSON at position 4787

Meet exception, how can I to solve it, thanks.
Loaded extension cutejs
Loaded platform lg
Loaded platform pc
Error: Failed to parse GCC output: Unexpected token j in JSON at position 4787
java.lang.NullPointerException: NAME default$$module$Users$dwbz$iwork$github$OrsayProjectTest$node_modules$zombiebox$zb$device$input$keys 56 [length: 4] [originalname: Keys] [source_file: /Users/dwbz/iwork/github/OrsayProjectTest/node_modules/zombiebox-platform-lg/lib/input.js] : ?
	at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:895)
	at com.google.javascript.jscomp.RemoveUnusedCode.getVarForNameNode(RemoveUnusedCode.java:719)
	at com.google.javascript.jscomp.RemoveUnusedCode.traverseNameNode(RemoveUnusedCode.java:574)
	at com.google.javascript.jscomp.RemoveUnusedCode.traverseNode(RemoveUnusedCode.java:420)
	at com.google.javascript.jscomp.RemoveUnusedCode.access$1300(RemoveUnusedCode.java:93)
	at com.google.javascript.jscomp.RemoveUnusedCode$Continuation.apply(RemoveUnusedCode.java:1588)
	at com.google.javascript.jscomp.RemoveUnusedCode.traverseAndRemoveUnusedReferences(RemoveUnusedCode.java:269)
	at com.google.javascript.jscomp.RemoveUnusedCode.process(RemoveUnusedCode.java:250)
	at com.google.javascript.jscomp.PhaseOptimizer$NamedPass.process(PhaseOptimizer.java:317)
	at com.google.javascript.jscomp.PhaseOptimizer.process(PhaseOptimizer.java:232)
	at com.google.javascript.jscomp.Compiler.performOptimizations(Compiler.java:2431)
	at com.google.javascript.jscomp.Compiler.lambda$stage2Passes$1(Compiler.java:801)
	at com.google.javascript.jscomp.CompilerExecutor$2.call(CompilerExecutor.java:102)
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)

The exception seems to arise from Google Closure Compiler failing to process some input and crashing.
Which I believe originates from zombiebox-platform-lg being out of sync with recent version of ZombieBox and trying to access an obsolete constant (Keys in device/input/keys).
A quick fix would be to downgrade zombiebox, 2.0.1 would probably be a good choice. We will however update zombiebox-platform-lgto be compatible with newer versions of ZombieBox shortly.
Do note however that Netcast platform is now being slowly phased out by LG and if I'm not mistaken, they are not even accepting new submissions to the app store, so developing an application for it may not be reasonable.

zombiebox-platform-lg version 2.1.0 was released to address this.


Make figures part of body.

Places image-figures directly in the body rather than using the include indirection. Fixes #41
I think we don't need the include indirection as we don't have externally managed resources anymore.

Merging this so Jure can test on edge tomorrow.


Refactoring Broadcast component

Refactoring the Broadcast as per the proposed style.
Note- Please make sure to follow all the guidelines mentioned in the proposed style properly!

@Kajol-Kumari Please assign me this issue. I assure you to complete this as I have completed working on one issue assigned to me.

Hey @sakshikhachane please go ahead with this issue :)


Refactor for new variable conversion and modularize sound speed

From here:
https://github.com/MFlowCode/MFC/issues/13#issuecomment-1340966703
We see that we compute the primitive variables "by hand" several times throughout the code.
This happens in m_data_output and some other places as well.
Now that @anshgupta1234 has made the routines nice and simple for this, we should call those instead.
Also, the speed of sound c is computed by hand in several places, though this could be moved out to common/. c is also sometimes called things like c_avg or c_L or c_R.

For example, see this
https://github.com/MFlowCode/MFC/blob/83528f12647f283cf0475abf17b635d0d73538c7/src/simulation/m_data_output.f90#L1645-L2048

and this
https://github.com/MFlowCode/MFC/blob/83528f12647f283cf0475abf17b635d0d73538c7/src/simulation/m_data_output.f90#L693-L756

Update: s_compute_speed_of_sound has now been implemented on anshgupta1234/fix_48. Now moving on to refactoring for new variable conversion

This should perhaps be closed now?

closed with #86


Get new trajectory id from sparse pose graph

Related to #409, #437
To split a trajectory in sparse pose graph level.

We decided not to continue in this direction.


Added link to repo in footer

Thanks @alienkitty . This issue was closed with https://github.com/arjunsinghy96/django-blogs/pull/10
Reasons I couldn't merge your PR:

Extra Merge commit. (You could have opened the PR from your branch itself)
Bad commit Message: Use imperative form example Add repo link to footer

Closing this PR as the issue is solved


[0-RTT] [RFC] Reuse mbedtls_ssl_write() for sending early data

This PR changes the API for sending early data on the client side. It deprecates the mbedtls_ssl_set_early_data() method and reuses mbedtls_ssl_write() method for sending early data. An alternative design is in #369.
Changes proposed

Add an error code MBEDTLS_ERR_SSL_HANDSHAKE_EARLY_RETURN for early return. Handshake returns with this error indicating the user can start sending early data. Calling mbedtls_ssl_handshake() again will continue the handshake. This pattern is similar to the handling of NewSessionTicket.
Rename MBEDTLS_SSL_EARLY_DATA_ON|OFF to MBEDTLS_SSL_EARLY_DATA_STATE_DISABLED |ON|OFF as these values represent the state of the client, rather than a configuration. When early data is enabled, the early_data state is initially ON, it will be turned to OFF if early data is rejected, a HRR is received, or max_early_data is reached (to add).

Status
DRAFT, solicit early feedback on design.
Requires Backporting
NO
Migrations
Additional comments
Any additional information that could be of interest
Todos

[ ] Tests
[ ] Documentation
[ ] Changelog updated
[ ] Backported

Steps to test or reproduce
Outline the steps to test or reproduce the PR here.
Start openssl server using
/usr/local/opt/openssl/bin/openssl s_server -key server5.key -cert server5.crt -accept 1234 -early_data -state

Then try
./ssl_client2 server_name=localhost server_port=1234 auth_mode=none reconnect=1 early_data=1 force_ciphersuite=TLS1-3-AES-256-GCM-SHA384  key_share_named_groups=secp384r1

Superseded by #369 after sync meeting.


[nxdata] bug when reading back unicode and testing for str type (PY2)

Found a bug when trying to run examples/hdf5Widget.py, NXdata/1D_spectrum. Single unicode string attr was not cast as list because I'm testing for type str.

It would be better to test strings using six.string_types, instead of doing that.
Anyway, let's merge it.


question about save figures to pdf

Your idea is very interesting. And thanks to your useful tutorial link, I've got my result by processing my scRNA-seq data. But cause I use R in my daily work , I'm unfamiliar about how to save the image to PDF after tg.plot_cell_annotation_sc. Hope you could give me a hand.
Thanks a lot!

Hi @Lena926, Thank you for using Tangram!
Here's some python code that will help you save the plot from tg.plot_cell_annotation_sc in pdf format:
fig, axs = plt.subplots(nrows=3, ncols=3, figsize=(10,10))
for ann,a in zip(annotation_list, axs.flat):
    tg.plot_cell_annotation_sc(ad_sp, [ann], spot_size=50, ax=a)
plt.tight_layout()
fig.savefig('annotation_plot.pdf')

you can change the number of figures in each row and column using nrows and ncols respectively and size of the figures using figsize.
Let me know if this works for you!
-Shreya

Hi Shreya,
I successfully saved the plot as pdf format with the code you gave.
Thanks for your help!!!
-Lena

completed


huggingface_hub.utils._errors.LocalEntryNotFoundError

Any idea how to resolve these errors?
(sd-api-server) PS A:\bc\source\repos\stable-diffusion-api-server> python server.py
Traceback (most recent call last):
  File "B:\home\anaconda3\envs\sd-api-server\lib\site-packages\diffusers\configuration_utils.py", line 324, in load_config
    config_file = hf_hub_download(
  File "B:\home\anaconda3\envs\sd-api-server\lib\site-packages\huggingface_hub\utils\_validators.py", line 124, in _inner_fn
    return fn(*args, **kwargs)
  File "B:\home\anaconda3\envs\sd-api-server\lib\site-packages\huggingface_hub\file_download.py", line 1169, in hf_hub_download
    raise LocalEntryNotFoundError(
huggingface_hub.utils._errors.LocalEntryNotFoundError: Connection error, and we cannot find the requested files in the disk cache. Please try again or make sure your Internet connection is on.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "server.py", line 131, in <module>
    manager.add_engine( 'txt2img', EngineStableDiffusion( diffusers.StableDiffusionPipeline,        sibling=None ) )
  File "server.py", line 62, in __init__
    self.engine = pipe.from_pretrained( 'runwayml/stable-diffusion-v1-5', use_auth_token=hf_token.strip() )
  File "B:\home\anaconda3\envs\sd-api-server\lib\site-packages\diffusers\pipeline_utils.py", line 446, in from_pretrained
    config_dict = cls.load_config(
  File "B:\home\anaconda3\envs\sd-api-server\lib\site-packages\diffusers\configuration_utils.py", line 352, in load_config
    raise EnvironmentError(
OSError: runwayml/stable-diffusion-v1-5 does not appear to have a file named model_index.json.

I created another environment from scratch and installed all the packages with conda instead of pip, and the downloads work now.


Add moon support to Node.js

This PR adds moon (https://moonrepo.dev/moon) support to Node.js projects.
Technically moon can be used for any language, but will simply start with Node.js and ensure it works.

Not sure who to ping about this.
@coffee-cup

Can you merge in main. It should have the latest lint fixes


Optional dump on fail.

With the current version I have (which I assume is the latest) when it does work, that's great. It works.
However in the few instances I try to launch it and it fails it proceeds to dump everything it needs.
Clearly it's supported (by successfully booting before and eventually booting again) and I'm also assuming you really have no use for dumps like these.
A small quality of life change would save time, headache, and confusion:

Prompt users to dump instead of dumping automatically.
Probably hint "if it's worked before don't dump".

If I'm wrong about anything let me know. Thanks!

Thanks for reporting, by the way, how often did you met this issue?

Not very often, in my case, as I typically boot it the first time and leave it (and the 3DS) running so I wouldn't need to run it again (until restart).


[EventHubs] Rename EventHubsEncryptionKeySource to EventHubsKeySource

Contributing to the Azure SDK
Please see our CONTRIBUTING.md if you are not familiar with contributing to this repository or have questions.
For specific information about pull request etiquette and best practices, see this section.

API change check
APIView has identified API level changes in this PR and created following API reviews.
Azure.ResourceManager.EventHubs


Document return type of recording.extract_embeddings()

What is expected here? A list?

It returns a dictionary with the image embedding for each 3 second frame of the spectrogram for the selected audio file


Remove setting __module on all items.

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Please tag this pr with at least one of these labels to make our release process faster: BREAKING, new feature, bug fix, other change, dev setup

At this point I don’t remember why these were needed: are these not needed anymore?

I can only speculate, but I think the idea is that things like trainer._base.Trainer are documented in sphinx as trainer.Trainer by re-setting the __module__ attribute on these entities.
However, this is normally done by listing the entities in __all__ of the re-exporting module, and sphinx will do the abbreviation itself, so there should be no need to write to __module__ just for sphinx. (Whether we should do it for serde is different question).
However, and more importantly, the to be removed code doesn't work, since it tries to set __module__ on the strings which represent the names and not the actual objects. In that sense, it didn't do anything.

However, and more importantly, the to be removed code doesn't work, since it tries to set module on the strings which represent the names and not the actual objects. In that sense, it didn't do anything.

Awesome! So maybe we should just make sure Sphinx is generating the API docs as we want it, before the next release: I’m not sure it’s doing the right thing in case it relies on the __module__ property


hare active set: create bootstrap data updater

Motivation
part of #4089
Changes
bootstrap data updater that

load the latest bootstrap data from disk at instantiation time
after app started, periodically query an URL for json update

validate data against json schema
validate data with app logic that cannot be enforced by schema
persist the latest update on disk and prune old ones
notify subscribes of a new update

note:

test with a memory filesystem from github.com/spf13/afero

Hey @countvonzero it would be useful for me if there was some package doc explaining what the purpose of the bootstrap package is, how its expected to be used and how it functions at a high level.

bors merge


Add CI configuration that would build ready-to-use containers for all major platforms

Add CI configuration (GithubActions or CloudBuild) that would build ready-to-use containers for all major platforms (e.g. linux/amd64, linux/arm, linux/arm64)

[x] Tag images with:

full version (e.g. 1.0.2)
minor version (e.g. 1.0)
major version (e.g. 1)
latest git ref (e.g. git rev-parse --short HEAD)
Build ID for debugging/tracking purposes

[ ] Cross compile images for linux/amd64, linux/arm, linux/arm64

Cross-platform images are WIP on a forked repository, but de-prioritised due to multiple errors in the build pipeline.
I attempted to build the images using buildx for linux/amd64, linux/arm, linux/arm64, but the arm platforms encountered the following issues:

Unsupported syscall errors on Alpine, but builds ok on Buster. This is undesired as using Buster may introduce additional vulnerabilities and make images unnecessarily large
takes a very long time to build (20+ minutes) when on default cloud build machine

We attempted to mitigate the above issues by changing the machine type to be of 8CPUs, but this resulted in segmentation faults for arm/v7 builds.
The current solution in the above branch only works for building linux/amd64 and linux/arm64 on Alpine with 8 CPUs. In the future, the segmentation fault will be fixed so that builds can be completed for other platforms.


[BUG] Getting an invalid MD5 Exception while uploading a Blob even when MD5 is valid

Describe the bug
Unable to set MD5 on an object during upload even though the MD5 is valid. The MD5 sent is Base64 encoded yet the error message says that the value isn't. The same value works in the v4.0.0 SDK but fails in the v12.7.0 SDK.
Is it that we no longer have to send the Base64 encoded version and should send the 16 byte array MD5 directly?
Exception or Stack Trace
Caused by: com.azure.storage.blob.models.BlobStorageException: Status code 400, "<?xml version="1.0" encoding="utf-8"?><Error>InvalidMd5<Message>The MD5 value specified in the request is invalid. MD5 value must be 128 bits and base64 encoded.

RequestId:8c9c9d76-c01e-006f-0638-7a98a2000000

Time:2020-08-24T17:02:06.1907490Z</Message></Error>"

To Reproduce
Upload a blob with an MD5 value using BlobHttpHeaders.
Code Snippet
md5 = "3b2e740dae9b434b6ec9922a54a6919f";
byte[] md5Bytes = Hex.decodeHex(md5.toCharArray());
byte[] md5Base64Bytes = Base64.encodeBase64(md5Bytes);

blob.uploadWithResponse(new BufferedInputStream(inputStream), size, /*parallelTransferOptions*/ null, new BlobHttpHeaders().setContentMd5(md5Base64Bytes),
                    metadata, /*tier*/ null, new BlobRequestConditions(), TIMEOUT, Context.NONE);

Expected behavior
The blob should be uploaded successfully.
Screenshots
N/A
Setup

OS: Ubuntu 18.04
IDE : IntelliJ 19.1.4
SDK:  azure-storage-blob v12.7.0

Additional context
This same Base64 MD5 works in the v4.0.0 SDK when we use the below lines
String md5base64 = new String(md5Base64Bytes);
properties.setContentMD5(md5base64);

I tried the above request WITHOUT the base64 encoding. And the request was successful.
byte[] md5Bytes = Hex.decodeHex(md5.toCharArray());
// byte[] md5Base64Bytes = Base64.encodeBase64(md5Bytes);  // This line was commented out

blob.uploadWithResponse(new BufferedInputStream(inputStream), size, /*parallelTransferOptions*/ null, new BlobHttpHeaders().setContentMd5(md5Bytes)),
                    metadata, /*tier*/ null, new BlobRequestConditions(), TIMEOUT, Context.NONE);

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

[x] Bug Description Added
[x] Repro Steps Added
[x] Setup information Added

Thanks for filing this issue @somanshreddy.  Someone from the storage team will follow up shortly.
/cc @gapra-msft @rickle-msft

Hi @somanshreddy, Thank you for posting this issue. We will work on investigating this issue and get back to you.

Hi @somanshreddy, I was able to reproduce your issue and it looks like the problem is md5Base64Bytes has more than 128 bits in it. I'm not too sure what v4 is doing with your md5 byte array (it could only be sending the first 16 bits or something like that).
Could you capture a fiddler trace or debug your program to see what is being sent to the service in v4 vs what is being sent in v12 when you don't base 64 the bytes?

So it was sending a 16 byte array before the Base64 encoding. And after Base encoding it is a 24 byte array.
This is the MD5 = 3b2e740dae9b434b6ec9922a54a6919f
Hex decoded array which is a 16 byte array :
byte[] md5Bytes = {59, 46, 116, 13, -82, -101, 67, 75, 110, -55, -110, 42, 84, -90, -111, -97}
Array after Base64 Encoding is 24 byes :
byte[] md5Base64Bytes = {79, 121, 53, 48, 68, 97, 54, 98, 81, 48, 116, 117, 90, 73, 121, 113, 86, 75, 97, 82, 110, 119, 61, 61}

Hi @somanshreddy, Took a more detailed look into the code and it looks like we internally take your byte array and base 64 encode it into a string before we send it to the service, so just passing in the md5 bytes is what the API expects.
I think this sufficiently resolves your issue. If you don't have any more concerns, feel free to close the issue.
Thanks

Yes, this is as suspected. So, this means that the error message is outdated, right?

I think the error message is correct since it mentions that "MD5 value must be 128 bits and base64 encoded."
Since the md5 passed in in the error case was not 128 bits, it did not pass the check.

But the MD5 wasn't base 64 encoded right?

So MD5 is always 128 bits. If we perform Base64 encoding, it would break the input into 6 bit groups. And each group would be 1 byte after base encoding. 21 such bytes would represent 21 x 6 == 126 bits. So we need a minimum of 22 bytes to store the MD5. 2 bytes are added as padding so we have a total of 24 bytes.
So I think the error message should be "MD5 value must be a 128 bits". Because the internal SDK performs the Base 64 encoding.

Hi @somanshreddy That error you are seeing is actually returned by the service, so it is correct to be more specific, in case people are developing against the REST endpoint.

Okay. Can the SDK documentation at least be updated with this requirement? Otherwise those who are upgrading from a previous version would have issues

Yes, absolutely. I will create an issue to update the documentation to address this. Thank you for the feedback!
Since your issue seems to have been resolved, I will go ahead and close this issue. Please feel free to reopen/create issues if you need further support.


Addition of devices introduced in 2024, minor clean up for iPad content

As per title.
Checklist:

[x] iPad Air M2
[x] iPad Pro M4
[ ] iPhone 16 series
[ ] Apple Watch Series 10
[ ] Update tests
[ ] Update CI

Supersedes #95, #96, #97

@vincentneo Looks great! I will fix CI later.

@vincentneo Ok I updated the CI settings and the build and tests are passing now!
We can check that off the list.

I think I'm done, @Nirma if you have some time, please do take a look, in case I miss something!

@vincentneo Looks great! I updated the CI settings in swift.yml mind rebasing with the latest develop?
That should get CI working.

@Nirma I think I messed up the rebasing, and somehow it's all on the main branch anyways. Is that alright? 😬

@vincentneo Sure!


Move all config translations from REST API to client side

The config labels and values should translated by client that an user can switch languages which also switch the config translations. So move all config translations to angular and translate it in JavaScript.
see also #2025

@matakuka Do you want to try this issue? Please ask me if you need help or more info.

You can make it as we do it for the permission strings, see:
https://github.com/OpenSlides/OpenSlides/blob/master/openslides/users/static/js/users/site.js#L966
Copy all config strings into core/site.js and remove the ugettext functions and imports from config strings in python files.

Maybe this example commit deea2224344ad8c72338cdcca888afd2cf11e5b2 helps you to understand it better what to do.

this issue is already merged in master (#2194), so it can be closed (I see no button to do it myself)


Expose Content-Length in IHttpResponseFeature

At the moment accessing content-length goes to string and back through parsing and the header collection. It is a long? on Response.
The string transitions are alloc heavy for 2000 requests at 108k meaning 54 Meg per 1M requests:

What I'd much prefer is to have the long? directly so I can output that straight to bytes without going via string or allocating.

Is the cost you're measuring for long -> string -> ascii bytes, and you're trying to shorten it to long -> ascii bytes?
Background: System.Web has had no end of problems special casing certain headers to bypass the header collection. Response cookies are a particularly bad example where it keeps a separate collection for them and only adds them to the headers at the end of the response. This causes conflicts for anyone using the headers directly. We don't want to repeat this mistake so we've kept the headers collection as the source of truth for all headers.

Trying to shorten it to long -> pre allocated byte buffer; so zero alloc.
Am ok with the other headers, as they come with a user generated cost, and where the user has already created the string; but this is generally always present except if chunked and is a hidden library cost that cannot be worked around.

For comparison its 30% of the alloc cost of DI: https://github.com/aspnet/DependencyInjection/pull/291

StaticFiles does set it but most of our other components don't, they rely on the server to chunk or to buffer and set it for them. In those cases the server can take shortcuts internally.
I can't think of any of our other libraries that set it by default.

MVC does in some extremely obscure scenarios, but not anything @benaadams or the average user is likely to have encountered.

With the current set up this is ok:
httpContext.Response.Headers["Content-Length"] = "fred";
But using the strongly typed ContentLength on HttpResponse as used by plaintext benchmark is an anti-pattern:
httpContext.Response.ContentLength = _helloWorldPayload.Length;
Which feels a bit wrong.
This is because it goes  ContentLength->new HeaderDictionary->ParsingHelpers.SetContentLength->value.ToString()->IDictionary[key]=
If IHttpResponseFeature implemented ContentLength it could chose to go via the headers; but it wouldn't have to.

The content length being set has to appear in the headers dictionary. It could be lazy be we want a single source of truth. Managing separate ways to set data is a PITA (we know from experience)

Managing separate ways to set data is a PITA (we know from experience)

There are already 2 ways; strongly typed on Request route and weakly typed via headers. They both resolve to headers.
You could change this so setting via headers goes to IHttpResponseFeature.ContentLength with a type check on setting that throws an exception if it can't be set to a long or null. Since there is a level of indirection on setting headers this would be possible.
For output it would follow a similar path to StatusCode and ReasonPhrase?

Also it means when deciding to chunk the check is: ContentLength.HasValue vs HeadersDictionary.Contains("Content-Length")+ignore-case

Content-Length can only be a numeric >= 0 http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.13
but you can set it to a whole bunch of weird via the headers, including multiple values; which if you try to read back via ContentLength will just say its not set
I think that's all I've got; will leave this bone alone now :)

To resolve this issue you'd need to convert it from the header back to a long https://github.com/aspnet/KestrelHttpServer/issues/175
Though kestrel does do the work of re-parsing from the header back to an int perhaps for this reason: https://github.com/aspnet/KestrelHttpServer/blob/c50aec17292b9f6d444da5f11637fd3c9146146d/src/Microsoft.AspNet.Server.Kestrel/Http/MessageBody.cs#L54

Final thing; have done some improvements to server and now find myself in the particular situation of setting content length being the big allocator in the plaintext middleware:

Ok, now am done...

Moving this to Backlog as we will be in RC2 ask mode very soon.  If you feel strongly about this issue, please ping me.

I feel strongly about it; but am ok with it being a post RTM discussion (although it would be a breaking change)

To clarify: Is breaking change for servers, not for end users as the attribute is already exposed - its just how its implemented that's emulated.

Did some measurements; the difference between these two lines
response.Headers["Content-Length"] = "13";

response.ContentLength = 13;

Is as follows
         Method |                 Type |        Mean |    StdDev |           RPS | Allocated |
--------------- |--------------------- |------------ |---------- |-------------- |---------- |
 BenchmarkAsync | ContentLengthNumeric | 115.8341 ns | 1.4149 ns |  8,633,038.71 |      32 B |
 BenchmarkAsync |  ContentLengthString |  80.2629 ns | 1.1965 ns | 12,459,057.14 |       0 B |

Benchmarks https://github.com/aspnet/KestrelHttpServer/pull/1256

2.0 Proposal:
Put long ContentLength { get; set; } on IHeaderDictionary, not IHttpResponseFeature.  The IHeaderDictionary implementation can maintain it's own consistency and optimize any lazy conversion between strings and longs as needed. ContentLength is an especially good candidate for IHeaderDictionary because it applies equally to requests and responses.
Are there any other headers that would benefit from direct IHeaderDictionary properties?

The Date header?

The Date header?

Any you'd want to though? People wouldn't normally read the date request header, and shouldn't be encouraged to write the date response header 😉

Thank you for resolving this 😁


jenkins: fix case sense on OverwriteConfig condition in config.yaml

.Values.master.overwriteConfig has the Master text represented with starting letter in Caps which is throwing error when helm update command is executed. error calling include: template: jenkins/templates/config.yaml:259:16: executing "jenkins/templates/config.yaml" at <.Values.Master.OverwriteConfig>: nil pointer evaluating interface {}.OverwriteConfig

Is this a new chart

NOTE: We're experiencing a high volume of PRs to this repo and reviews will be delayed. Please host your own chart repository and submit your repository to the Helm Hub instead of this repo to make them discoverable to the community. Here is how to submit new chart repositories to the Helm Hub.

What this PR does / why we need it:
Which issue this PR fixes
(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged)

fixes #

Special notes for your reviewer:
Checklist
[Place an '[x]' (no spaces) in all applicable fields. Please remove unrelated fields.]

[ ] DCO signed
[ ] Chart Version bumped
[ ] Variables are documented in the README.md
[ ] Title of the PR starts with chart name (e.g. [stable/mychartname])

/lgtm


[Project sync] Deleted forms, options, etc. not removed from local db when updating project

The current sync logic in RoomLocalDataStore only inserts or updates existing rows. We also need to delete items that are no longer found in the project. One way to do this would be delete the entire project and reinsert it when updating (preferably in a single transaction, e.g. using cascading deletes).

Layer deletes cascade to descendants, so this is now obsolete.


Fix Navigation Preset Issue

I created the explodePath method in the ConfigLaoder. Thank you for the suggestion! The update should work for you on windows. Let me know if it doesn't.
https://github.com/aw-studio/fjord/blob/master/src/Config/ConfigLoader.php


Remove coverage and coveralls for now

This is about fixing the SUPER-SCARY false positives we get from Travis
CI service. See #759. I see this as urgent, so if the tests fail for
this PR, that's good.
While we haven't had any incident yet, we were about to have two on
Spotify's side and apparently @daveFNbuck have had one already (#752).
Just now, I got some 2.6 error which is still green.
https://travis-ci.org/spotify/luigi/jobs/51362963

@Tarrasch The test that fails are test that I added. The issue is in the test itself. I didn't realize that this is valid in python2.7 but not valid in python2.6
with gzip.open('test.gz') as f:
    f.read()

It's solved in #769.
+1 for this PR, I wouldn't have introduced those failing tests at the first place if Travis gave red.

@gpoulin Thank you so much!!!!!

@Tarrasch I'm gonna open a different PR for those tests and to fix that too https://travis-ci.org/spotify/luigi/jobs/51822677#L1024

@gpoulin or @berkerpeksag Hmm, do you have any idea what the issue is here? It didn't help to remove coveralls and coverage. Scroll to the bottom on this page:
https://travis-ci.org/spotify/luigi/jobs/51825576

@Tarrasch if I'm not mistaken, the problem is here: https://github.com/spotify/luigi/blob/master/test/server_test.py#L48 Could you disable remote_scheduler_test, rpc_test and server_test tests in this PR to test my theory? I've posted a couple of comments about the subject: https://github.com/spotify/luigi/issues/759#issuecomment-74893179
I have a WIP branch to make the server tests isolate.
(I'm +1 to remove the coverage thing anyway :))

Anyway. I remove it, I think we definetly need to re-evaluate what value coveralls add. So far it has only made me waste time I think.

Everything else equals, I like coveralls a lot. If we had added it earlier it probably would have helped us a bit giving ideas of where to add unit tests.
 I'm not following why.

Well, Travis was still green. But I wasn't sure if coveralls was the issue. Make Travis red on failures first, and then re-add coveralls. I mean, it's just a button-click away in github to create a PR that re-adds it.


feat(extension): lw-9268 add stake pools grid view

Checklist

[x] JIRA
[ ] Proper tests implemented
[ ] Screenshots added.

Proposed solution
Use virtuoso lib to enable virtual scroll for staking table.
Create separate grid component by utilising VirtuosoGrid
Testing
General staking flows regression is highly recommended
Screenshots
Attach screenshots here if implementation involves some UI changes

Changes unknown
when pulling df91ed6d275ed5bc0b797f3ad1bb4bfebed380ec on feat/LW-9268-stake-pools-grid-view
into ** on feat/LW-9531-migrate-table-component**.


Added ability to configure background color

Thanks @ponces, I will look into it and will merge your PR

Could you please also update you dart package to the last version?

Yeah sure, I will do it!


Text position for certain font might not work

Documenting the issue at least, so people with similar goals with me would know it exists.
Basically when some fonts are decoded, it is analyzed character by character, however, all those characters would have the same position coordinates...See screenshot below.
I might dig into it and try to fix it. We'll see.

Dear @dayfine,
I experienced the same issue. Additionally, all white spaces were missing. From my point of view this happens when the Font-Object (pdf internals) is missing /FirstChar /LastChar and /Widths attributes.


Distance to a service

As a user viewing a provider, I want to see the distance from my location (entered during a search) to each service shown on the page

on futher thought ,we're not going to show this, because we don't have a dataset that can provide this with any accuracy


Fixing the regex for csharp

Fixes the regex for csharp to allow -rc* builds to also be available.
/cc @BanzaiMan @akoeplinger

Tested the regex changes and LGTM. Thanks for foxing this up!

@Joshua-Anderson not a problem, glad to help. I cannot tell why the build is failing, though...seems like some Docker issue?

Currently blocked by https://github.com/travis-ci/travis-ci/issues/7367. I've tried a few restarts, but it's still having problems. Will try again in a few hours.


Nautilus/pcmanfm without DE can't use GVFS

Issue description
If you install just nautilus/pcmanfm/probably others without a full DE, auto-mounting via GVFS doesn't work.
Steps to reproduce

Enable gvfs and udisks2 services (services.{gnome3.gvfs,udisks2}.enable = true;)
Install nautilus or pcmanfm
Insert a removable device
Notice that the device doesn't show up in the file manager

Technical details
The file managers need the environment variable GIO_EXTRA_MODULES to include ${pkgs.gvfs}/lib/gio/modules for gvfs integration. This variable is set correctly only if you get the full desktop environment:
https://github.com/NixOS/nixpkgs/blob/2e7105467b22292eca1fc37c2c8019e6aacc7051/nixos/modules/services/x11/desktop-managers/lxqt.nix#L62
https://github.com/NixOS/nixpkgs/blob/710973e354cc0f1c18a2d201f631d54420c6084a/nixos/modules/services/x11/desktop-managers/xfce.nix#L126
https://github.com/NixOS/nixpkgs/blob/bd545892332ede39cd5c1c0ba26e7101b0d2c971/nixos/modules/services/x11/desktop-managers/gnome3.nix#L181

System: NixOS unstable
Sandboxing enabled: no

I'm also facing this issue while trying pcmanfc-qt inside sway. Do you have/propose a workaround?
configuration.nix

Solution (for me):

Adding gvfs into packages of my user ;
Adding environment.variables.GIO_EXTRA_MODULES = [ "${pkgs.gvfs}/lib/gio/modules" ]; into my config. file ;

I removed services.gnome3.gvfs.enable = true; since I don't need full GNOME support.

services.gvfs.enable = true sets the GIO_EXTRA_MODULES variable since 4bb6625767b2428cbacb712165f5db63ebadcf66.


Support react-native > 0.40

Updated header imports to build on react-native > 0.40

Consider bumping the package version to 1.0, since it is a breaking change (react-native-video will no longer work on React Native <= 0.39).

Bumped version. Let me know if there is anything else I can do.

Are you guys having issues with the device events such as, onLoad, not triggering RN .40?

@joekim yup.. same here. This PR was intended just to make it build. But can take a look at this as well.

@joekim No, wait. It worked here. Just some confusing JS. The event is onLoad not onVideoLoad. When I called it onLoad it was called.

@olofd My onLoad does not seem to be triggering. But, I know through the debugger that onVideoLoad is being emitted on the native side. How does that event name map to onLoad? Any tips on debugging my issue?

set a debugger; on line 48 in Video.js in your node_modules/react-native-video folder, close the packager-terminal and relaunch. Mine hit that when setting  onLoad={() => console.log('load')} on the video element at least.

@olofd I added a breakpoint in _onLoad and it's not triggering. I added one in render just to see that it's working and that works fine. I also know that onVideoLoad is being emitted by native. Not sure what the issue is. Maybe it's something to do with my project. I'm going to try to create a new project from scratch.

Are you guys having issues with the device events such as, onLoad, not triggering RN .40?

To test this, I tried to see if the Example project would work using this PR branch. I couldn't even get it to build, running, I believe, into this issue: https://github.com/facebook/react-native/issues/11721. I tried upgrading RN a few different ways with the same result.
Can anyone get the Example project working in RN 0.40?  If we can get the native files upgraded, we could add that to this PR.

@hellogerard I'll take a look at the example-project tonight.

@hellogerard I've now upgraded the example project to run on 0.44. Ran react-native upgrade and did some manual fixes after that. Now runs. Also left a console.log for you and @joekim in onLoad. Would recommend that you wipe your node_module-folder in the example project and run npm install in the example project again after pulling my update.

@olofd That's awesome. No reason not to merge this now.
@joekim If you are still having issues with callbacks, I suggest creating a new issue. (BTW, is your video playing at all? If not, make sure you are using a https URL.)

@hellogerard I'll take a look. The video is playing but the call backs are triggering.

@olofd @hellogerard I retested the sample app and it triggers properly.

@mattapperson This PR is merged but react-native-video v1.0 is not published on npm. Can you also publish the latest master on npm.

published


Add and fix some toString

Closes #151

@AnatolyPopov I think it's better to keep as is, the commits are meaningful separately


Rust exception under iOS

Hi!
When I try to run the code under iOS I got an error
code
final mnemonic =
        await Mnemonic.fromString('puppy interest whip tonight dad never sudden response push zone pig patch');

error
[VERBOSE-2:dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Failed to lookup symbol 'store_dart_post_cobject': dlsym(RTLD_DEFAULT, store_dart_post_cobject): symbol not found
#0      DynamicLibrary.lookup (dart:ffi-patch/ffi_dynamic_library_patch.dart:34:70)
#1      RustWire._store_dart_post_cobjectPtr (package:bdk_flutter/src/generated/bindings.dart:1714:76)
#2      RustWire._store_dart_post_cobjectPtr (package:bdk_flutter/src/generated/bindings.dart)
#3      RustWire._store_dart_post_cobject (package:bdk_flutter/src/generated/bindings.dart:1716:41)
#4      RustWire._store_dart_post_cobject (package:bdk_flutter/src/generated/bindings.dart)
#5      RustWire.store_dart_post_cobject (package:bdk_flutter/src/generated/bindings.dart:1708:12)
#6      StoreDartPostCObjectExt.storeDartPostCObject (package:flutter_rust_bridge/src/ffi/io.dart:18:5)
#7      FlutterRustBridgeBase._setUpRustToDartComm (package:flutter_rust_bridge/src/basic.dart:60:11)
#8      new FlutterRustBridgeBase (packag<…>

Plugin version - bdk_flutter: ^0.3.1
Doctor
[✓] Flutter (Channel stable, 3.3.10, on macOS 13.0.1 22A400 darwin-arm, locale en-DE)
[✓] Android toolchain - develop for Android devices (Android SDK version 33.0.0)
[✓] Xcode - develop for iOS and macOS (Xcode 14.1)
[✓] Chrome - develop for the web
[✓] Android Studio (version 2021.3)
[✓] IntelliJ IDEA Community Edition (version 2022.2.2)
[✓] VS Code (version 1.74.0)
[✓] Connected device (5 available)
    ! Error: Aleksei’s Apple Watch needs to connect to determine its availability. Check the
      connection between the device and its companion iPhone, and the connection between the
      iPhone and Xcode. Both devices may also need to be restarted and unlocked. (code 1)
[✓] HTTP Host Availability

• No issues found!

Just tried this, not able to reproduce.
Let me know if you are using a simulator or device, and environment details.
You can also try flutter clean and then flutter pub get and run again

Hi!
Thanks, complete clean + pod update fixed the issue, thank you!

Nope, same error again.
Should I do some additional installation for the iOS?

I found the reason, how to reproduce the bug:

example app
create a scheme (flavor) under ios, two gifs, takes 3 mins:

https://miro.medium.com/max/1400/1*tctD2CuUZuCV4FcFIXXddw.gif
https://miro.medium.com/max/1400/1*XsDo_Kv4UPiS-Qeix9YEEA.gif

run this flavor flutter run --flavor dev

We got the error
[VERBOSE-2:dart_vm_initializer.cc(41)] Unhandled Exception: Invalid argument(s): Failed to lookup symbol 'store_dart_post_cobject': dlsym(RTLD_DEFAULT, store_dart_post_cobject): symbol not found
#0      DynamicLibrary.lookup (dart:ffi-patch/ffi_dynamic_library_patch.dart:33:70)
#1      RustWire._store_dart_post_cobjectPtr (package:bdk_flutter/src/generated/bindings.dart:1714:76)
#2      RustWire._store_dart_post_cobjectPtr (package:bdk_flutter/src/generated/bindings.dart)
#3      RustWire._store_dart_post_cobject (package:bdk_flutter/src/generated/bindings.dart:1716:41)
#4      RustWire._store_dart_post_cobject (package:bdk_flutter/src/generated/bindings.dart)
#5      RustWire.store_dart_post_cobject (package:bdk_flutter/src/generated/bindings.dart:1708:12)
#6      StoreDartPostCObjectExt.storeDartPostCObject (package:flutter_rust_bridge/src/ffi/io.dart:18:5)
#7      FlutterRustBridgeBase._setUpRustToDartComm (package:flutter_rust_bridge/src/basic.dart:60:11)
#8      new FlutterRustBridgeBase (package:flutter_rust_bridge/src/basic.dart:28:5)
#9      new RustPlatform (package:bdk_flutter/src/generated/bindings.dart:1248:44)
#10     new RustImpl (package:bdk_flutter/src/generated/bindings.dart:19:59)
#11     loaderApi (package:bdk_flutter/src/utils/loader.dart:9:19)
#12     loaderApi (package:bdk_flutter/src/utils/loader.dart)
#13     Mnemonic.fromString (package:bdk_flutter/src/root.dart:783:25)
#14     _MyAppState.createDescriptorSecret (package:first_app/main.dart:46:24)
#15     _MyAppState.restoreWallet (package:first_app/main.dart:36:11)
#16     _MyAppState.initState (package:first_app/main.dart:24:5)
#17     StatefulElement._firstBuild (package:flutter/src/widgets/framework.dart:5101:55)
#18     ComponentElement.mount (package:flutter/src/widgets/framework.dart:4944:5)
#19     Element.inflateWidget (package:flutter/src/widgets/framework.dart:3953:16)
#20     Element.updateChild (package:flutter/src/widgets/framework.dart:3682:18)
#21     RenderObjectToWidgetElement._rebuild (package:flutter/src/widgets/binding.dart:1176:16)
#22     RenderObjectToWidgetElement.mount (package:flutter/src/widgets/binding.dart:1145:5)
#23     RenderObjectToWidgetAdapter.attachToRenderTree.<anonymous closure> (package:flutter/src/widgets/binding.dart:1092:18)
#24     BuildOwner.buildScope (package:flutter/src/widgets/framework.dart:2682:19)
#25     RenderObjectToWidgetAdapter.attachToRenderTree (package:flutter/src/widgets/binding.dart:1091:13)
#26     WidgetsBinding.attachRootWidget (package:flutter/src/widgets/binding.dart:926:7)
#27     WidgetsBinding.scheduleAttachRootWidget.<anonymous closure> (package:flutter/src/widgets/binding.dart:906:7)
#28     Timer._createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:18:15)
#29     _Timer._runTimers (dart:isolate-patch/timer_impl.dart:398:19)
#30     _Timer._handleMessage (dart:isolate-patch/timer_impl.dart:429:5)
#31     _RawReceivePort._handleMessage (dart:isolate-patch/isolate_patch.dart:192:26)

@awaik Was this resolved?

Nope, for avoiding confusion because of the wrong first message, before I found the exact reason, I made a new issue https://github.com/LtbLightning/bdk-flutter/issues/52


Can't output summaries using tensorflow.contrib.slim

When I run my code, it gives the following warning:
WARNING:tensorflow:From C:\Program Files\Anaconda3\lib\site-packages\tensorflow\python\training\supervisor.py:344 in __init__.: SummaryWriter.__init__ (from tensorflow.python.training.summary_io) is deprecated and will be removed after 2016-11-30. Instructions for updating: Please switch to tf.summary.FileWriter. The interface and behavior is the same; this is just a rename.
And It can't output the summaries. Although I think it is training my net. Here's some of my code:
summaries = set(tf.get_collection(tf.GraphKeys.SUMMARIES)) for end_point in end:     x=end[end_point]     summaries.add(tf.summary.histogram('activations/' + end_point, x))     summaries.add(tf.summary.scalar('sparsity/' + end_point,  tf.nn.zero_fraction(x))) for loss in tf.get_collection(tf.GraphKeys.LOSSES):     summaries.add(tf.summary.scalar('losses/%s' % loss.op.name, loss)) for variable in slim.get_model_variables():      summaries.add(tf.summary.histogram(variable.op.name, variable)) summaries.add(tf.summary.scalar('learning_rate', learning_rate)) summaries.add(tf.summary.scalar('total_loss', total_loss)) #summaries |= set(tf.get_collection(tf.GraphKeys.SUMMARIES)) summary_op = tf.summary.merge(list(summaries)) slim.learning.train(train_op,      model_store_dir,      init_fn = init_fn,     summary_op=summary_op,     number_of_steps=100000,     save_summaries_secs=300,      save_interval_secs=600)

@mxmxlwlw As it said, SummaryWriter has renamed to tf.summary.FileWriter

@inflation I noticed that slim.learning.train has a parameter summary_writer. So I passed the tf.summary.FileWriter to it, and the warning didn't show again. However, the log still did't come up. I think the error may be in other place of my code.
Here's my code:
import pickle
import re
import random
import numpy as np
import tensorflow as tf
from vgg_preprocessing import preprocess_image
slim = tf.contrib.slim
_R_MEAN = 123.68
_G_MEAN = 116.78
_B_MEAN = 103.94
def vgg16Net(inputs,
num_classes=1000,
is_training=True,
dropout_keep_prob=0.5,
spatial_squeeze=True,
scope='vgg_16'):
with tf.variable_scope(scope, 'vgg_16', [inputs]) as sc:
end_points_collection = sc.name + '_end_points'
# Collect outputs for conv2d, fully_connected and max_pool2d
with slim.arg_scope([slim.conv2d, slim.fully_connected, slim.max_pool2d],
outputs_collections=end_points_collection):
net = slim.repeat(inputs, 2, slim.conv2d, 64, [3, 3], scope='conv1')
net = slim.max_pool2d(net, [2, 2], scope='pool1')
net = slim.repeat(net, 2, slim.conv2d, 128, [3, 3], scope='conv2')
net = slim.max_pool2d(net, [2, 2], scope='pool2')
net = slim.repeat(net, 3, slim.conv2d, 256, [3, 3], scope='conv3')
net = slim.max_pool2d(net, [2, 2], scope='pool3')
net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv4')
net = slim.max_pool2d(net, [2, 2], scope='pool4')
net = slim.repeat(net, 3, slim.conv2d, 512, [3, 3], scope='conv5')
net = slim.max_pool2d(net, [2, 2], scope='pool5')
# Use conv2d instead of fully_connected layers.
net = slim.conv2d(net, 4096, [7, 10], padding='VALID', scope='fc6')
net = slim.dropout(net, dropout_keep_prob, is_training=is_training,
scope='dropout6')
net = slim.conv2d(net, 4096, [1, 1], scope='fc7')
net = slim.dropout(net, dropout_keep_prob, is_training=is_training,
scope='dropout7')
net = slim.conv2d(net, num_classes, [1, 1],
activation_fn=None,
normalizer_fn=None,
scope='fc8')
# Convert end_points_collection into a end_point dict.
end_points = slim.utils.convert_collection_to_dict(end_points_collection)
if spatial_squeeze:
net = tf.squeeze(net, [1, 2], name='fc8/squeezed')
end_points[sc.name + '/fc8'] = net
return net, end_points
def getsamples():
rootpath = 'C:\Users\mx\Desktop\nextLevel\UCF-101'
with open('res.pickle', 'rb') as f:
pathdict = pickle.load(f)
with open('trainlist01.txt', 'r') as f:
lines=f.readlines()
samples_all=[]
labels_all=[]
for line in lines:
[videogroup, video, label]=re.split('/| |\n', line)[0:3]
samples = [rootpath + '\' + videogroup + '\' + video[:-4] + '\' + i for i in pathdict[videogroup][video]]
samples_all.extend(samples)
labels_all.extend([label]*len(samples))
return samples_all, labels_all
samples_all, labels_all = getsamples()
labels_one_hot_all = np.zeros((len(labels_all), 101))
index_offset = np.arange(len(labels_all))*101
ind = index_offset + np.array(labels_all, np.int32) - 1
labels_one_hot_all.flat[ind]=1
#samples_all = tf.constant(samples_all)
#labels_all = tf.constant(labels_all)
[sample, label] = tf.train.slice_input_producer([samples_all, labels_one_hot_all])
imagecontent = tf.read_file(sample)
image = tf.image.decode_jpeg(imagecontent, channels=3)
image = tf.cast(image, dtype = tf.float32)
channels = tf.split(2, 3, image)
channels[0] -= _R_MEAN
channels[1] -= _G_MEAN
channels[2] -= _B_MEAN
image=tf.concat(2, channels)
image=tf.reshape(image, [240, 320, 3])
images, labels = tf.train.batch([image, label], 16, 4, 64)
net, end = vgg16Net(images, num_classes = 101, is_training=True)
slim.losses.softmax_cross_entropy(net, labels)
total_loss = slim.losses.get_total_loss()
learning_rate = 0.001
optimizer=slim.train.GradientDescentOptimizer(learning_rate)
train_var = slim.get_variables_to_restore(exclude = ['vgg_16/conv1', 'vgg_16/conv2', 'vgg_16/conv2', 'vgg_16/conv3', 'vgg_16/conv4', 'vgg_16/conv5']);
train_op=slim.learning.create_train_op(total_loss, optimizer, variables_to_train = train_var)
model_store_dir = './log/'
init_var = slim.get_variables_to_restore(exclude = ['vgg_16/fc6', 'vgg_16/fc7', 'vgg_16/fc8'])
init_fn = slim.assign_from_checkpoint_fn('./vgg_16.ckpt', init_var)
summaries = set(tf.get_collection(tf.GraphKeys.SUMMARIES))
for end_point in end:
x=end[end_point]
summaries.add(tf.summary.histogram('activations/' + end_point, x))
summaries.add(tf.summary.scalar('sparsity/' + end_point, tf.nn.zero_fraction(x)))
for loss in tf.get_collection(tf.GraphKeys.LOSSES):
summaries.add(tf.summary.scalar('losses/%s' % loss.op.name, loss))
for variable in slim.get_model_variables():
summaries.add(tf.summary.histogram(variable.op.name, variable))
summaries.add(tf.summary.scalar('learning_rate', learning_rate))
summaries.add(tf.summary.scalar('total_loss', total_loss))
#summaries |= set(tf.get_collection(tf.GraphKeys.SUMMARIES))
summary_op = tf.summary.merge(list(summaries))
summary_writer = tf.summary.FileWriter(model_store_dir)
slim.learning.train(train_op,
model_store_dir,
init_fn = init_fn,
summary_op=summary_op,
summary_writer=summary_writer,
number_of_steps=100000,
save_summaries_secs=300,
save_interval_secs=600)

@inflation Here's the code.
doNet.zip

You can try to add tf.summary.FileWriter.add_summary before your training

@inflation It suggest that I can't pass tensor to the add_summary function...

Slim uses supervisor for training. Do you have the model checkpoint files?

@inflation Yes, I downloaded it from slim page in tensorflow/models(repository)

No, I mean when you trying to train your model, does it produce anything in the output directory?

@inflation No. So you mean it didn't train?

Yes, checkpoint files should be in the same folder. And they are automatically saved in some period when using supervisor if the path is valid.

I noticed that it's not the problem of summary. The training process can't even do one process step. Here's my new python code. Can you find what's the problem in that file? I really don't know how to fix it.
doNet3.zip

Problem solved. I loss the code tf.train.start_queue_runners(sess=sess). It needs to run queue manually.


Operation Inlining

We gradually want to incorporate some of the compilation steps and optimizations usually done for classical programming languages (opposed to quantum programming languages) into the Q# compiler. While our focus is naturally on different aspects of a program when compiling and optimizing for execution on a quantum processor, we still need some of the classical infrastructure as well.
In particular, we want some infrastructure for inlining operations.

Since this task is part of a larger piece containing the first steps to set up some infrastructure for a full compilation process, I set up a feature branch to PR again here: https://github.com/microsoft/qsharp-compiler/tree/features/classicalCompilerOptimizations
Such a feature branch may contain parts of a task and/or experimental implementations with the intent to eventually merge it back to master once it is in a reasonably good state.

Merged into master with https://github.com/microsoft/qsharp-compiler/pull/177


Prevent deps:sync from touching unmodified package.json

If deps:sync hasn't actually changed a package.json, it would be much nicer not to rewrite the file at all, just leave it as is, so git wouldn't see a change. The reason seems to be an added "0d" character at the end.

..which in turn is caused by having git config core.autocrlf=input on Windows. Whereas writing the parsed json did write LF characters at the end of each line, the last line break was CR/LF due to EOL being CRLF on Windows. So this is not in sync.


How to Make it work with Aws lambda

Hello
I read the Blog & some resource but i didn't get the idea , how to make nuclear pond work With AWS lambda ?
Best Regards

So this is just the client and you need to setup the backend. To get it to work with lambda you need to deploy these resources using terraform. Clone this repository, run terraform apply, and set the lambda function name that was deployed in this utility. Feel free to let me know if you have any issues.
https://github.com/DevSecOpsDocs/terraform-nuclear-pond


COOKED content error in Carla 0.9.8 on ubuntu 18.04

I have installed UnrealEngine_4.22 and Carla 0.9.8 on my ubuntu 18.04 system. During make I did not got any error. When I run "./Unreal/CarlaUE4/Binaries/Linux/CarlaUE4-Linux-Shipping" I got below error:-
The global shader cache file '/home/administartor/UnrealEngine_4.22/Engine/GlobalShaderCache-SF_VULKAN_SM5_NOUB.bin' is missing.
Your application is built to load COOKED content. No COOKED content was found; This usually means you did not cook content for this build.
It also may indicate missing cooked data for a shared plateform(e.g., OpenGL under Windows): Make sure your platform's packaging settings include this Targeted RHI.
Alternatively build and run the UNCOOKED version instead.
Please let me know what needs to be done in this. @sergi-e
Thanks in advance for helping!

Hi!
I would like to know more about your intentions. Why are you trying to run CarlaUE4-Linux-Shipping after the "make launch"? If you are trying to get a package you can "make package".

I want to start server without any display. For that I am trying to use below command:-
SDL_VIDEODRIVER=offscreen SDL_HINT_CUDA_DEVICE=0 /Unreal/CarlaUE4/Binaries/Linux/CarlaUE4-Linux-Shipping -carla-server
I am new to this carla. I have installed Carla on Linux system. I can access this system through MobaXterm using SSH. Can you suggest better way to connect to server without any gui?

Running CARLA with no display, for instance a GPU with no screen plugged, that is what we call "Off-screen mode": https://carla.readthedocs.io/en/latest/adv_rendering_options/#off-screen-mode
The recommended approach for the Off-screen mode is to run CARLA in a Docker: https://carla.readthedocs.io/en/latest/build_docker/


Create chainer.optimizer_hooks namespace and move hooks there.

This PR is continuation of #2943.

@delta2323 Cloud you resolve conflicts?

@okuta I have rabased.

There are still conflicts.

Sorry, I merged #4259, which made new conflicts. I resolved it.

Thanks!

flake8
.\tests\chainer_tests\test_optimizer.py:10:1: F401 'functions' imported but unused
.\tests\chainer_tests\test_optimizer.py:11:1: F401 'links' imported but unused

Is it OK to deprecate these classes?
I feel it is inconsistent with what we did for updaters. https://github.com/chainer/chainer/pull/3037

flake8
./tests/chainer_tests/test_optimizer.py:10:1: F401 'functions' imported but unused
./tests/chainer_tests/test_optimizer.py:11:1: F401 'links' imported but unused

@kmaehashi Cloud you tell me more detail?

We moved StandardUpdater and ParallelUpdater from chainer.training to chainer.training.updaters in #3037, but they does not raise DeprecationWarning (chainer.training.StandardUpdater is just a alias to chainer.training.updaters.StandardUpdater). The same rule applied for chainer.cuda v.s. chainer.backends.cuda.
I think we need to set policy regarding deprecations.
For me, immediately raising a DeprecationWarning just for internal refactoring (without explicit need to remove the API) is too aggressive.
How about leaving them as an alias and add comment like # Following functions are kept for backward compatibility?

@kmaehashi I agree with you in that raising a deprecation warning is inconsistent with changes we have done with StandardUpdater or chainer.backends.cuda. On the other hand, if we are to remove old APIs in future, it might be better to have some mechanisms that promote users to move to recommended new APIs.

The document test was failed.
/home/user/.local/lib/python3.5/site-packages/chainer-4.0.0b4-py3.5.egg/chainer/optimizer_hooks/gradient_clipping.py:docstring of chainer.optimizer_hooks.GradientClipping:None: WARNING: more than one target found for cross-reference 'GradientClipping.timing': chainer.optimizer_hooks.GradientClipping.timing, chainer.optimizer.GradientClipping.timing

@delta2323 One way to express our intension is to use Upgrade Guide, as done in StandardUpdater or chainer.backends.cuda.
https://docs.chainer.org/en/latest/upgrade.html

jenkins test this please

jenkins, test this please.

LGTM!


Improve collection creation workflow

IMHO, the current workflow involved in creating a collection does not match the real practice of the users.
It is rare for a developer to be able to set the mapping for a yet-to-be-created collection. This would be the case at deploy-time, but I can bet nobody deploys a Kuzzle in production using the back office (we use to make scripts for that).
On the other hand, for a newbie, only seeking to quick-start a project, the mapping of a collection is not something to be bothered with: just, as the projects gets into shape, the mapping is automatically detected by ES. We imagine that, once the mapping has a reasonable shape, it is exported, cleaned up and set to production environment via a script.
We should verify these processes by asking people who use Kuzzle and the back office on a daily basis, then find a UI design that would fit all the workflows.

Guys at CATS confirmed they never set the mapping upon collection creation. They do point out that it is vital to them to be able to eventually edit the auto-generated mapping, specially for search purposes (sometimes, the mapping proposed by ES does not match the search query they expect to create).

Next action :point_right: have a chat with the guys at Comexposium.

Creating a collection in v4 looks better


InvalidTypeIdException on getVoiceClient().listCalls()

Describe the bug
Exception when trying to get listCalls()
To Reproduce
Try to execute getVoiceClient().listCalls()
Expected behavior
No exception
Vonage SDK Version
7.2.0
Java SDK Version
openjdk version "17.0.1" 2021-10-19
OpenJDK Runtime Environment Temurin-17.0.1+12 (build 17.0.1+12)
OpenJDK 64-Bit Server VM Temurin-17.0.1+12 (build 17.0.1+12, mixed mode)
Additional context/ Error Log
Caused by: com.fasterxml.jackson.databind.exc.InvalidTypeIdException:
Could not resolve type id 'app' as a subtype of `com.vonage.client.voice.Endpoint`: known type ids = [phone, sip, websocket] (for POJO property 'from')
 at [Source: (String)"{"page_size":10,"record_index":0,"count":63,"_embedded":{"calls":[{"uuid":"xxx","status":"cancelled","direction":"inbound","rate":"0.00450000","price":"0.00000000","duration":"0","network":"US-VIRTUAL-BANDWIDTH","conversation_uuid":"xxx","start_time":"","end_time":"","to":{"type":"phone","number":"xxx"},"from":{"type":"phone","number":"xxx"},"_links":{"self":{"href":"/v1/calls/xxx"}}},{"[truncated 4081 chars]; line: 1, column: 4273] (through reference chain: com.vonage.client.voice.CallInfoPage["_embedded"]->com.vonage.client.voice.EmbeddedCalls["calls"]->java.lang.Object[][9]->com.vonage.client.voice.CallInfo["from"])

Same happens when I try to get voiceClient.getCallDetails(callId).

We are using "In-App Voice". Maybe that's related?

Thank you for reporting this and submitting a PR! I will add this fix to the v7.3.0 release close this issue once it's done.
Out of curiosity, how did you encounter this? I presume the call was made from another SDK or the REST API? I'm just trying to create a minimal reproducible example which would help with testing this in practice.

Should be fixed in #446.


The charset attribute of the query params are changed when it is going to the backend

Description:
The charset attribute of the query params(We set via SOAP UI) which has been changed from "Cp1252" to "US-ASCII"
The Gateway received
TID: [-1] [] [2021-10-21 12:51:04,676] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Listener I/O dispatcher-9 >> "Content-Type: text/plain; charset=Cp1252[\r][\n]"
TID: [-1] [] [2021-10-21 12:51:04,676] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Listener I/O dispatcher-9 >> "Content-Transfer-Encoding: quoted-printable[\r][\n]"
TID: [-1] [] [2021-10-21 12:51:04,676] DEBUG {org.apache.synapse.transport.http.wire} - HTTPS-Listener I/O dispatcher-9 >> "Content-Disposition: form-data; name="notification"[\r][\n]"

The gateway sends to the backend
TID: [-1] [] [2021-10-21 12:51:05,147] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-3 << "Content-Disposition: form-data; name="notification"[\r][\n]"
TID: [-1] [] [2021-10-21 12:51:05,147] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-3 << "Content-Type: text/plain; charset=US-ASCII[\r][\n]"
TID: [-1] [] [2021-10-21 12:51:05,147] DEBUG {org.apache.synapse.transport.http.wire} - HTTP-Sender I/O dispatcher-3 << "Content-Transfer-Encoding: 8bit[\r][\n]"

Steps to reproduce:

Create an API
Use dynamic endpoint configuration and upload a sequence as an inflow dynamic sequence.
Set a query param via the SOAP UI project
Enable wire logs and check the wire logs.

Affected Product Version:
APIM 3.2.0

Close the issue as per above commit


Feature Request - Black List

Creating blacklist for Domains and IPs.  This would alert on known badness no matter the techniques of the iframe or script for example.

Are there blacklists/feeds you recommend? This is something I'd like to put
in the dev pipeline for a later release.
On Tue, Jun 14, 2016 at 11:19 PM, r3comp1le<EMAIL_ADDRESS>wrote:

Creating blacklist for Domains and IPs. This would alert on known badness
no matter the techniques of the iframe or script for example.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/ciscocsirt/malspider/issues/5, or mute the thread
https://github.com/notifications/unsubscribe/AR0QEFzf4sKT1MAkqwBwIr4FoRDwSjdRks5qL29UgaJpZM4I19fD
.

I would generate them from OSINT collection and domains and ips that show up in reports, along with internal intel.

@r3comp1le - https://github.com/ciscocsirt/malspider/pull/9 added basic blacklist support!


Lock CakePHP version to 3.5.*

Lock CakePHP version to 3.5.* until all deprecation errors are fixed.

Codecov Report

Merging #530 into master will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff            @@
##             master     #530   +/-   ##
=========================================
  Coverage     28.18%   28.18%
  Complexity      952      952
=========================================
  Files            85       85
  Lines          3246     3246
=========================================
  Hits            915      915
  Misses         2331     2331

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c27d02d...9038bec. Read the comment docs.


fix: check that docker is installed

What does this PR do?
It checks that Docker is installed before run it.
Why is it important?
Some worker types do not have Docker installed this breaks the packer build.

:green_heart: Build Succeeded

Expand to view the summary

Build stats

Build Cause: [Pull request #863 opened]

Start Time: 2020-06-16T12:20:16.251+0000

Duration: 30 min 56 sec

Test stats :test_tube:

Test
Results

Failed
0

Passed
9277

Skipped
6796

Total
16073


bugifx:mock interface type

it didn't work for me by following

http://dev.apollodata.com/tools/graphql-tools/mocking.html#Mocking-interfaces

so I fixed the problem

hi @helfer  , can you please merge my PR as we need this in our project?
thanks

is this project still maintained? @helfer

@bochen2014 yes definitely - sorry, we've been traveling to some conferences recently and it's currently a Sunday. Let me take a look now.

Hmm, I just looked around and it's my mistake that this made it into the docs.
Looks like it started with this issue: https://github.com/apollographql/graphql-tools/issues/171
Then this PR, which I merged incorrectly even though it didn't work: https://github.com/apollographql/tools-docs/pull/53
The original docs content submitted in the PR had a resolver that looked at typename on the object to determine the type, but then that was removed even though it definitely doesn't work without that.

I've temporarily reverted that docs change: d243bf0da9743c3a9e2320b735f18dd06c852a71
I think the right implementation here should have the following properties:

Same behavior for interfaces and unions
Use a random member of the interface/union as a default, as before
If there is a __typename field on the object, uses that

I think using __typename instead of typename makes sense since it's closer to how GraphQL returns that information.

Let me know if the above suggestions seem good to you.

thanks @stubailo . yes your comments makes sense to me.
I've updated the PR with your suggestions (mostly typename->__typename)
all unit tests passed (including 2 new tests which I added as part of this PR). can you please review again? thanks

Can we make sure that Unions behave the same way? https://github.com/apollographql/graphql-tools/pull/332/files#diff-483b6e7e62e3109d7c87ae3c54425430R199
I think that means factoring out the type selection logic into a function.

will do it this morning... thanks

hi @stubailo I've updated my PR to address unionTypes explicit mock. please review it again..
plus, your updated doc

http://dev.apollodata.com/tools/graphql-tools/mocking.html#Mocking-interfaces

seems to be a bit misleading.. I think for most users, leaving the parameter  preserveResolvers  in addMockFunctionsToSchema to default false would be a much better choice because

it will trigger the random selection of possible implementation types for that interface/union and a typename (now renamed to __typename) will provided by the framework (so user don't have to care about it).
it also will tell the framework to call assignResolveType to inject a default resolveType function, which will look up the __typename property of the resolved result

@bochen2014 that's a good point.
I'm going to review and merge this today, do you mind if I make some code changes along the way?

feel free to change whatever you think is required ;-)

any updates?😏

hi @stubailo  have you got a chance to have a review? we actually need the change in our project . thanks

Thanks @bochen2014 for the great PR and sorry for the delay! I will talk with @stubailo today and we'll get the PR merged asap! 🙂

Ok, I think this change is pretty good, but it's also a breaking change (because we're changing typename to `__typename)., so if we merge it we'll have to release version 2.0

thanks @helfer  for merging my pr!


use arrayvec crate instead

Though the intention is to avoid using unsafe Rust, the current implementation of html5gum::arrayvec::ArrayVec needs much improvement and the there's no real unsafety if we just use the arrayvec crate.

Consider html5gum::arrayvec::ArrayVec::push():
    pub fn push(&mut self, item: T) {
        self.content[self.len] = item;
        self.len += 1;
    }

Keep pushing and you are guaranteed to get an runtime index out of bound error.
While arrayvec::ArrayVec::push() checks the capacity and there's no unsafety or run-time error.
    fn push(&mut self, element: Self::Item) {
        self.try_push(element).unwrap()
    }

    fn try_push(&mut self, element: Self::Item) -> Result<(), CapacityError<Self::Item>> {
        if self.len() < Self::CAPACITY {
            unsafe {
                self.push_unchecked(element);
            }
            Ok(())
        } else {
            Err(CapacityError::new(element))
        }
    }

Consider html5gum::arrayvec::ArrayVec::drain()
    pub fn drain(&mut self) -> &[T] {
        let rv = &self.content[..self.len];
        self.len = 0;
        rv
    }

It doesn't drain anything actually. It just gives you a view of the content.

I propose to use arrayvec::ArrayVec instead. If you wish, we can use new type pattern and only expose new(), push(), and drain() function.

Keep pushing and you are guaranteed to get an runtime index out of bound error.

this is the api contract for arrayvec as well, it will panic in push() by unwrapping the result of try_push
in this crate's impl, if self.len is one past the end of the array, the arrayvec is full, and the index operation will panic. this can be made more explicit, but then we may accidentally regress performance, and this is internal API so I didn't bother.

It doesn't drain anything actually

it resets len to 0, so it's equivalent to drain(..) (ranged drain is not supported)

right now the README says html5gum has no mandatory dependencies. I added jetscii because it offers significant performance gains that can't be replicated in safe code, but even that one is optional, and I believe the barrier for new dependencies should be similarly high
that said the library is still WIP, and many features I want (like tree builder or css selectors) will probably bloat the deptree, unless they are optional

@lebensterben i explained my dependency policy in a new CONTRIBUTING document in master in additional to the freaky test harness, does it make sense?

this can be made more explicit, but then we may accidentally regress performance, and this is internal API so I didn't bother.

The current implementation, i.e. html5gum::arrayvec::ArrayVec::push(), though as an internal API, still may cause panic at runtime. This would become a larger issue if you expect other people contribute to this project and use this function.
Suppose a library user encountered a panic due to this, the panic behaviour on his platform could be just aborting the program and not trying to rewind anything. If he has some data types that requires manual memory management, for example it doesn't implements Drop, then at panic there might be some memory with sensitive data not cleared. (Normally OS would prevent illegal access of memory but attacker may dump the memory to get its contents).
So at very least, for safety, we should eliminate the possibility of a panic.
Surely performance is important, and I would suggest to make html5gum::arrayvec::ArrayVec::push() unsafe. But that's against your principle here...

As for html5gum::arrayvec::ArrayVec::drain(), at very least that's a bad naming.
In STD drain() returns an iterator and regardless of whether the iterator is consumed, the original data is removed.
Currently, html5gum::arrayvec::ArrayVec::push() doesn't return an iterator and the original data is not removed.

In the linked PR, I've used the newtype pattern and only exposed the API this library used.
Though arraryvec crate is unsafe, but at least the three functions doesn't have unsafe usages.

ArrayVec is used instead of Vec precisely because we know the (static) upper bound of items the crate will push into the instance. If it pushes more than that, it is a logic bug in html5gum and must be fixed. It should not be possible regardless of how the crate's public API is used.
Please double-check your claims about drain() and push(). I am absolutely certain those methods work the same way as in the arrayvec crate, and your PR does not change their behavior. If you disagree please provide a code sample that actually does behave differently between your PR and master.

To challenge your claims directly:

Currently, html5gum::arrayvec::ArrayVec::push() doesn't return an iterator and the original data is not removed.

It is true that ArrayVec::drain() (sic?) does not return an iterator (but a slice), but it does correctly remove the old data by resetting self.len = 0. The effect is the same: drain() clears the array and returns the array's old items.

though as an internal API, still may cause panic at runtime.

This is exactly the same behavior as arrayvec::ArrayVec::push on crates.io. If you don't want a panic, use try_push, but I believe that would be a mistake here.

but it does correctly remove the old data by resetting self.len = 0

Setting the len field to zero makes it impossible inside Rust to access the data. But the memory is not cleared.
For arrayvec::ArrayVec::drain() or other drain() functions in STD.

drain() works in the way that it returns a Drain struct which holds a pointer to the original data and an iterator to the part of data to be removed.
Drain struct implements Drop trait in the way that it removes the data in the iterator, even if they are not used (i.e. the iterator is not consumed).
Note that Drain is dropped before the original data type. That's how it logically guarantees that the part of data will be removed.
That is to say, whenever you call drain(), it gives you the struct Drain which drops the data regardless of whether they're consumed.

For the current implementation of drain, see the playground example: https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=5a4a409ecd57ea191b65aee0a325ecfc

As for arrayvec::ArrayVec::push, I don't think we should use try_push for performance reason.
But we must note this in the documentation or comment about the potential panic and why we don't need to worry about it.
(Optionally, in debug build, we can debug_assert that the capacity is not exceeded.

I'm going to close this issue because I find this conversation/debate extremely frustrating. Originally you were claiming that the vendored arrayvec has fundamentally different behavior from the crate (and therefore we need to start using the crate), which I disproved, then tell me that drain() needs to return a Drain object + "clear" items from memory to be considered equivalent. I think this is unnecessarily pedantic and completely besides the point of a vendored subset of arrayvec that is supposed to be simpler to audit by being 10 lines instead of 100-1000 lines of code.
Both std vec and arrayvec will in fact not clear any memory: they will leave the memory as-is and reset the len. The fact that this memory is not visible in their respective debug implementations is 1) needed in their case because it would be undefined behavior to interpret that memory as item and use its Debug impl 2) a fair criticism of the Debug implementation of the vendored arrayvec, but in any case besides the point with regard to the API or behavioral correctness. Also clearing memory or executing item's Drop impls does not matter with Copy items.
Fundamentally two things remain true:

drain() "deletes" items in that they don't show up in successive drain() calls, but does not go through the effort of clearing them from memory. This is true across all sorts of vectors. The difference I see right now is that ours has a auto-derived debug impl that shows the old data.
push() panics if the bounds are exceeded, in both versions

...and that's all that matters.
Aside: I think it would be a good improvement to document why a potential panic cannot actually happen, but it would require documentation and linting effort to do so consistently across the codebase. in reader.rs you will find several potentially-panicking slice accesses that IMO are all not problematic, but for undocumented reasons.

arrayvec will in fact not clear any memory: they will leave the memory as-is and reset the len.

That's incorrect.
It's dropped here
https://docs.rs/arrayvec/latest/src/arrayvec/arrayvec.rs.html#973

I am talking about the memory of the array, not running destructors. I get that Drop needs to be executed, but all of our items are restricted to be Copy so it doesn't matter.

Concretely, if you build a Vec<u8>, and drain it, there is basically nothing that needs to run besides the set_len. you can leave the items where they are and overwrite them on the next push().
If you have a Vec<String>, you can do the same, but you need to go through each String struct, and deallocate the string data itself (for which you will find a pointer in the string struct). The string struct however still stays in the Vec, and as long as nobody accesses it or constructs a Rust reference to it, there is no memory unsafety.
In the case where all your items are Copy (like u8) you can do those sorts of shenanigans without unsafe Rust. Items that impl Copy cannot have a custom Drop.


DOCKER_HOST seems to be ignored by docker-compose

I have a weave network going (see https://github.com/weaveworks/weave/issues/2718), and have done
weave launch-proxy
eval $(weave env)

I do the following

root@app-docker11:~/pitchcrc# docker run -it --rm alpine ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:AC:11:00:02
          inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
          inet6 addr: fe80::42:acff:fe11:2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:3 errors:0 dropped:0 overruns:0 frame:0
          TX packets:3 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:258 (258.0 B)  TX bytes:258 (258.0 B)

ethwe     Link encap:Ethernet  HWaddr 2A:7E:DB:01:90:03
          inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
          inet6 addr: fe80::287e:dbff:fe01:9003/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1410  Metric:1
          RX packets:1 errors:0 dropped:0 overruns:0 frame:0
          TX packets:2 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:90 (90.0 B)  TX bytes:132 (132.0 B)

lo        Link encap:Local Loopback
          inet addr:<IP_ADDRESS>  Mask:<IP_ADDRESS>
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Next, I create the following compose file

root@app-docker11:~/pitchcrc# more docker-compose-alpine.yml
version: '2'

services:
 alpine:
  image: alpine
  command: sleep 3600

And I do the following
root@app-docker11:~/pitchcrc# docker-compose -f  docker-compose-alpine.yml up -d
Creating network "pitchcrc_default" with the default driver
Creating pitchcrc_alpine_1
root@app-docker11:~/pitchcrc# docker exec -it pitchcrc_alpine_1 ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:AC:12:00:02
          inet addr:<IP_ADDRESS>  Bcast:<IP_ADDRESS>  Mask:<IP_ADDRESS>
          inet6 addr: fe80::42:acff:fe12:2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0
          RX bytes:1296 (1.2 KiB)  TX bytes:648 (648.0 B)

lo        Link encap:Local Loopback
          inet addr:<IP_ADDRESS>  Mask:<IP_ADDRESS>
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

root@app-docker11:~/pitchcrc# echo $DOCKER_HOST
unix:///var/run/weave/weave.sock

Note sure where the problem is, but docker-compose appears not to use the DOCKER_HOST setting.
root@app-docker11:~/pitchcrc# docker-compose -v
docker-compose version 1.10.0, build 4bd6f1a

@bergtwvd I think what is happening is by default docker-compose creates a new network for your services, and this causes the Weave proxy to step back and not override that.
In #2402 you had a networks stanza in the docker-compose.yml file which requested to use the Weave network.  This would also cause the Weave proxy to step back - i.e. there would be no difference with or without the DOCKER_HOST setting.
I'm not sure if there is any way to request that docker-compose specify nothing about networking for the containers it creates, which would allow the Weave proxy to do its thing.

So, to make this work for docker-compose I need to use the launch-plugin option in https://github.com/weaveworks/weave/issues/2718, correct?

If you want the interface to be created as part of docker run you have two options:
Plugin: run weave launch-plugin then docker run --net=weave ...
Proxy: run weave launch-proxy then eval $(weave env) then docker run ...

I have not tried that option yet, since in that case I need to re-install my complete Kubernetes environment. I try to follow the standard steps as much as possible.

OK, so you are using Weave Net with both Kubernetes and docker-compose?
You don't need to change what you are doing with Kubernetes; there is only one Weave network underneath everything, whether you join to it via proxy, plugin or weave attach.  Just try changing your docker-compose.yml file.

I need to run a couple of containers outside Kubernetes (on WeaveNet) in order to use the mac-address option in docker-run.  I like to start those with with docker-compose (ultimately via a webbrowser UI).
What do you suggest that I change in the compose file?

networks:
  default:
    external:
      name: weave

I am still puzzling how to get this to work with docker-compose.
Above you refer to an external network; this needs to refer to the network that Kubernetes uses and that is created during the Kubernetes installation process. If I start my container on e.g. the  Kubernetes master node, I do not see a weave network listed with docker network ls, and this will therefore not work.
Using the following snippet in the docker-compose file
networks:
  default:
   driver: weave

does not work either: plugin not found. But this is what I actually expected since I did not install the plugin on this Kubernetes master node.
Thus, should I use the Weave plugin option?
Should I create a standard docker host node for this and somehow extend the weave network to that host, and not re-use one of the kubernetes nodes to run flat containers?

Sorry, I had forgotten some of the history here.  I think I'm now repeating https://github.com/weaveworks/weave/issues/2718#issuecomment-270381077
My understanding is that docker-compose in "version 2" mode can only attach networks via a plugin.
The plugin would refuse to run because the (non-Kubernetes) weave container is not running; you would need to start that via weave launch.  You have talked about re-installing Kubernetes, but that should not be necessary; just remove the weave-kube daemonset and reboot all hosts, then install Weave as per https://www.weave.works/docs/net/latest/cni-plugin/
One random thought: perhaps it would be an option to use the "version 1" format, then the Weave Net proxy (via DOCKER_HOST) should work?

I did the quick test with the "version 1" compose file: that works, a weave interface is created in each container.
I will try the other option later (I am constrained by time at the moment), which has my preference over v1 format compose files.

I think everything is now covered or duplicating #2718, so I'l close this issue.
If you think something is outstanding please re-open.

Yep, thanks!


Docs: Confusing Support information

Describe the bug
At https://external-secrets.io/latest/introduction/stability-support/ it says

we offer long-term support for our latest two (N, N-1) software releases

With the release of 0.10, N and N-1 releases are 0.10 and 0.9. The table on the page however depicts 0.8.x as supported too with the EOL date being "Release of 1.0"
I'm confused here as to whether 0.8.x is supported or not?
To Reproduce
Observe the compatibility table at external-secrets.io/latest/introduction/stability-support
Expected behavior
Not sure. Seeking guidance.
Screenshots

Additional context
None

I'm happy to create a PR to update the documentation once the confusion is resolved.

yeah - Its not supported anymore :smile: even 0.9 is probably poorly supported

(its just that we thought 0.10 wouldnt really exist and we would already be on 1.0 by now)


Compatibility with floating labels (remove empty tag upon closing)

I use this Material theme
The problem is that the empty tag returns after closing the picker even if there is data in the input field  so it results in the data and the floating label overlapping. I'm not that much into js so I don't know if it should be fixed from here or the theme author's side.

@gergalyb Do you have any update on this? I just hit the same problem :)

I'm having same issue. Any update on how to fix this?

Hi, at least there's an example on something like http://jsfiddle.net/ I have no idea what you're talking about, and (just in case that you ask) no I don't want to create and example by myself :)


Add precommit hooks running eslint and prettier

This PR aims to standardize and improve code quality by introducing pre-commit hooks for linting and formatting.
Problem Statement:
I noticed lots of unused variables, incorrect prop names. Similarly, prettier wasn't installed properly resulting in developer's default editor settings for formatting being applied.

Dependencies Added

Prettier: We had a .prettierrc file but the dependency was not installed properly
Husky: For pre-commit hooks
Lint-staged: For applying prettier and eslint checks on staged files before commit

@maskudo, Thanks for your contribution! Could you apply the suggested changes? After that, it’ll be good to go.

I have applied the suggested changes.
Furthermore, I have disabled the following rules:
{
    "react-refresh/only-export-components": "off",
    "react/prop-types": "off"
}

Notes
I have disabled eslint for the docusaurus directory because it uses a mix of commonjs and ecmascript modules which we dont have a clean way of resolving and trying to fix it doesnt feel worth the cost. We could probably have an eslint rule for the docusaurus page only inside its package.json if we need it in the future.
I'm assuming we would simply be using typescript or jsdocs if we were serious about typing our props so we probably dont need the prop-types rules.
The 'only export components' rule was previously set to warn which we dont really need. We export some hooks and constants along with some components which are good logical pairing that doesnt really make sense to split just for the sake of it.

Stills logs message are not changed!

Once again @maskudo , Thanks for your contribution


added my name to guestbook.md

Hi Sir,
Please merge my name to your guest book list.
Thank you.

Thank you!


fix: suggest reverting specific history on mismatch

What kind of change does this PR introduce?
Bug fix https://github.com/supabase/cli/issues/2063
What is the current behavior?
Please link any relevant issues here.
What is the new behavior?
Feel free to include screenshots if it includes visual changes.
Additional context
Add any other context or screenshots.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

12 of 12 (100.0%)  changed or added relevant lines in 1 file are covered.
5 unchanged lines in 1 file lost coverage.
Overall coverage decreased (-0.009%) to 58.992%

Files with Coverage Reduction
New Missed Lines
%

internal/gen/keys/keys.go
5
11.76%

Totals

Change from base Build<PHONE_NUMBER>:
-0.009%

Covered Lines:
6265

Relevant Lines:
10620

💛  - Coveralls


[v1.27.0] inputText failing on secureTextEntry

Describe the bug
inputText failing on secureTextEntry. The Input is showing a  "Strong Password" message and text entry just becomes disabled
To Reproduce
Steps to reproduce the behavior, e.g.:

Start the simulator
Run maestro test with - inputText: 'whatever" on a secure text entry Input
The Flow fails at the text input

Expected behavior
Enter the password
Screenshots

Environment information (please complete the following information):

Maestro version: v1.27.0
Platform: iOS 16.2
Device model and OS version: iPhone 11
Simulator
Host: Mac Mini

Workspace (if applicable)
N/A
Bug report output (if applicable)
N/A
Additional context
N/A

Facing the same issue. Android does not suffer from this. It also doesn't help to set the secureTextEntry property to false

Same here, any recommendation to fix it please ?

It's not great, but as a workaround I'm setting those fields to be normal text fields when I detect I'm in testing mode.

As a workaround, you could disable auto-filling in your simulator.
Inside your simulator: Settings → Passwords → Type in a random password → Password options → Disable toggle
In Maestro Cloud, it seems to be disabled by default, so there are no issues there.

@axelniklasson

For anyone else running into this, I have two solutions:
Maestro
- swipe:
    direction: RIGHT
- tapOn: "Settings"
# ^^ that could probably be done via some command to open Settings.app's app ID
- tapOn: "Passwords"
- inputText: "a" # anything works here as the password
- pressKey: Enter
- tapOn: "Password Options"
- runFlow:
    when:
      visible: "1" # Only toggle when currently ON. This isn't great, but I don't know of a way to be more explicit about which toggle it is. Luckily, there's just the one (for now...)
    commands:
      - tapOn: "AutoFill Passwords"

That's obviously not great.
However, here's how you can do it via the CLI beforehand:
plutil -replace restrictedBool.allowPasswordAutoFill.value -bool NO /Users/jesse/Library/Developer/CoreSimulator/Devices/5CF9D7C1-293E-4EB7-8FAB-B0B9A083B4DB/data/Library/UserConfigurationProfiles/EffectiveUserSettings.plist
Replace your simulator's device ID of course. Way better.

Hi 👋🏽  Any updates on this issue? I'm facing the same issue.

However, here's how you can do it via the CLI beforehand:

plutil -replace restrictedBool.allowPasswordAutoFill.value -bool NO /Users/jesse/Library/Developer/CoreSimulator/Devices/5CF9D7C1-293E-4EB7-8FAB-B0B9A083B4DB/data/Library/UserConfigurationProfiles/EffectiveUserSettings.plist

↑ For my case, just this command didn't solve the issue but running this command ↓ too solved it.
plutil -replace restrictedBool.allowPasswordAutoFill.value -bool NO ~/Library/Developer/CoreSimulator/Devices/YOUR_DEVICE_ID/data/Containers/Shared/SystemGroup/systemgroup.com.apple.configurationprofiles/Library/ConfigurationProfiles/UserSettings.plist

I managed to get it working with specifying the text in env variable and then using the variable:
appId:  someId
env:
  PASSWORD: "111111"
---

- tapOn:
    id: "inputId"
- inputText: ${PASSWORD}

@axelniklasson