tflite_convert - ValueError: Invalid tensors 'input' were found.

System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Window10
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: No
TensorFlow installed from (source or binary): pip install tensorflow
TensorFlow version (use command below): 1.12
Python version: 3.6.7
Bazel version (if compiling from source): No
GCC/Compiler version (if compiling from source): No
CUDA/cuDNN version: No
GPU model and memory: No

Describe the current behavior
I want to convert mobilenet_v1_0.50_128 frozen graph using tflite_convert or toco to a TensorFlow Lite model, but I get the following error:
2019-02-20 16:09:37.352780: I tensorflow/core/platform/cpu_feature_guard.cc:142] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2
Traceback (most recent call last):
  File "c:\users\alice\anaconda3\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)

  File "c:\users\alice\anaconda3\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)

  File "C:\Users\Alice\Anaconda3\Scripts\tflite_convert.exe\__main__.py", line 9, in <module>

  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 442, in main
    app.run(main=run_main, argv=sys.argv[:1])

  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\python\platform\app.py", line 40, in run
    _run(main=main, argv=argv, flags_parser=_parse_flags_tolerate_undef)

  File "c:\users\alice\anaconda3\lib\site-packages\absl\app.py", line 300, in run
    _run_main(main, args)

  File "c:\users\alice\anaconda3\lib\site-packages\absl\app.py", line 251, in _run_main
    sys.exit(main(argv))

  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 438, in run_main
    _convert_model(tflite_flags)

  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 122, in _convert_model
    converter = _get_toco_converter(flags)

  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\tflite_convert.py", line 109, in _get_toco_converter
    return converter_fn(**converter_kwargs)

  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\lite.py", line 387, in from_frozen_graph
    sess.graph, input_arrays)

  File "c:\users\alice\anaconda3\lib\site-packages\tensorflow\lite\python\convert_saved_model.py", line 189, in get_tensors_from_tensor_names
    ",".join(invalid_tensors)))

ValueError: Invalid tensors 'input' were found.

Code to reproduce the issue
I run the following command:
tflite_convert --output_file=tflite_convert --output_file=F:/mobilenet_v1_0.50_128/frozen_graph.tflite --graph_def_file=F:/mobilenet_v1_0.50_128/frozen_graph.pb --input_arrays=input --output_arrays=MobilenetV1/Predictions/Reshape_1

I have the same problem in window10.

I don't know what happened but this morning the mobilenet_v1_0.50_128/frozen_graph.pb is converted successfully by tflite_convert:

But if I change to my trained model, the mentioned error appear. Anyone please tell me about the --output_arrays=MobilenetV1/Predictions/Reshape_1, if I change to my trained model (SSD-FPN Mobilenet V1), what output arrays of mine?

I have the same problem when I am trying to convert frozen_face_grapy.pb file to .tflite file.
But if you try with the code in document with files provided by them, it is converting properly.
Can anyone tell me, what is this input_tensor? And what is the expected value?

Hello,
I'm facing same problem here
Here is the command that i'm executing to generate .pb i am successfuly generate it.
IMAGE_SIZE=224
ARCHITECTURE="mobilenet_1_1.0_${IMAGE_SIZE}"
python retrain.py
--bottleneck_dir=tf_files/bottlenecks
--how_many_training_steps=500
--model_dir=tf_files/models/
--summaries_dir=tf_files/training_summaries/"${ARCHITECTURE}"
--output_graph=tf_files/retrained_graph.pb
--output_labels=tf_files/retrained_labels.txt
--architecture="${ARCHITECTURE}"
--image_dir=tf_files/flower_photos
Once i am trying to create that .pb to .tflite get fail with same error "ValueError: Invalid tensors 'input' were found."
tflite_convert
--output_file=foo.tflite
--graph_def_file=retrained_graph.pb
--input_arrays=input
--output_arrays=MobilenetV1/Predictions/Reshape_1

same issue, any ideas?

Same issue here as well

any solution for this ?

not sure if this approach is right. im using mobilnetv2
i tried printing available values from convert_saved_model.py
print(tensor_name_to_tensor) -  this will give all the valid values.
i used  --input_arrays=module_apply_default/MobilenetV2/input (which is available in print)
this is working for me. converted model tested in android device.

not sure if this approach is right. im using mobilnetv2
i tried printing available values from convert_saved_model.py
print(tensor_name_to_tensor) -  this will give all the valid values.
i used  --input_arrays=module_apply_default/MobilenetV2/input (which is available in print)
this is working for me. converted model tested in android device.

Same error i am also facing.... any solution?

Hello guys, please follow this stackoverflow url it may help you for it, i just use older one poets demo and its working fine with latest mobilenet version.

Hello,
I'm facing same problem here
Here is the command that i'm executing to generate .pb i am successfully generate it.
IMAGE_SIZE=224
ARCHITECTURE="mobilenet_1_1.0_${IMAGE_SIZE}"

python retrain.py
 --bottleneck_dir=tf_files/bottlenecks
 --how_many_training_steps=500
 --model_dir=tf_files/models/
 --summaries_dir=tf_files/training_summaries/"${ARCHITECTURE}"
  --output_graph=tf_files/retrained_graph.pb
  --output_labels=tf_files/retrained_labels.txt
  --architecture="${ARCHITECTURE}"
  --image_dir=tf_files/flower_photos

Once i am trying to create that .pb to .tflite get fail with same error "ValueError: Invalid tensors 'input' were found."
tflite_convert \
  --output_file=foo.tflite \
  --graph_def_file=retrained_graph.pb \
  --input_arrays=input \
  --output_arrays=MobilenetV1/Predictions/Reshape_1

i want to know what is input array actually?what should i put there?

Try loading your model into TensorBoard to try and visualize what the input array might be. In general, it's the Placeholders that are being passed into the graph at the start.
It is not possible to debug this issue without investigating the specific model. If you are unable to determine the input arrays, please provide a TensorFlow model file.

Automatically closing due to lack of recent activity. Please update the issue when new information becomes available, and we will reopen the issue. Thanks!

遇到ValueError: Invalid tensors 'input/input_data:0' were found.
同样问题，请问有谁解决了吗？帮我， 'input/input_data:0' 是我测试pb模型是的输入变量名，但是转tflite时，报上述错误。谢谢。。。

遇到ValueError: Invalid tensors 'input/input_data:0' were found.
同样问题，请问有谁解决了吗？帮我， 'input/input_data:0' 是我测试pb模型是的输入变量名，但是转tflite时，报上述错误。谢谢。。。

Hi,
I had the same issue. Don't pass the port ':0' in your input array.
Try just 'input/input_data'.
Should work.

I had the same issue when I tried to convert my trained model that used the inception_v3 model. There are the parameters that finally worked for me when I tried converting using tflite_convert:
–graph_def_file=myfile.pb –output_file=output.tflite –input_format=TENSORFLOW_GRAPHDEF –output_format=TFLITE –-input_shape=1,299,299,3 —input_array=ResizeBilinear –output_array=final_result –inference_type=FLOAT –input_type=FLOAT


Automatically make custom filed available in a Document library

Category

[x] Question
[ ] Typo
[ ] Additional article idea

Question
Hi
I have developed a field customizer that just shows some text data in a document library. Everything is fine and okay. On deployment what I am trying to do is make the field available in all document libraries of the site where the customizer is installed on. So far on deployment of the customizer a new field gets provisioned but it still needs to be added from settings->add new column and selecting the column from a column group. Is there any way to automatically make the column available?
Thanks in advanced for your time
Environment details (development & target environment)

Your Developer Environment: Windows 10
Target Environment: SP online
Framework: Nodejs V10.19
Browser(s): Chrome v83
Tooling: VS Code | SPFx v1.10.0

@RafatRashid said:

Is there any way to automatically make the column available?

I can't answer that directly... but...
To use the field customizer, which is tied to a site column (aka: Field), you'd need to add that site column to a new document library. Plenty of ways you can do this (content types, list templates, etc).
If you want to modify EXISTING document libraries... it depends...
If you're using content types in a document library, you can add the site column to the document library and have those changes get pushed down where the content type is being used.
If you want to add the customizer to an EXISTING column, you'll need to do that via the SharePoint REST API. Get a reference to the SPField and update the properties to associate the field customizer & it's properties with the field.

Could you explain what is 'content types'? and how do I push down? some docs or refrences would be helpful.
And I dont think our use case requires adding to existing column so I would pass on that for the time being.

https://support.microsoft.com/en-us/office/introduction-to-content-types-and-content-type-publishing-e1277a2e-a1e8-4473-9126-91a0647766e5


Rewrite HTTP requests handling to Symfony/HttpFoundation

Use Symfony/HttpFoundation for getting Request parameters and return responses

I can look into it


Question about validity of Margins Results post PPMLHDFE estimation

Hello Sergio,
I have a question regarding the validity of margins command after ppmlhdfe estimation results when I have fixed effects.
From my online searches, it seems like when you run margins command after you fit a Poisson estimation with fixed effects, the results can be invalid because margins command doesn't incorporate fixed effects. From one of the comments I found from the Stata Forum, margins after xtpoisson and xtlogit with fixed effects produces results that are meaningless because the marginal effects depend on the fixed effects and these are not estimated when you use these commands.
I followed the instructions to calculate marginal effect after ppmlhdfe from here: https://github.com/sergiocorreia/ppmlhdfe/blob/master/guides/undocumented.md#esttab-and-margins-options
I was wondering if the same problems apply to margins command after ppmlhdfe estimation. Also, if there is a problem, is there anyway I can incorporate fixed effects in the margins command after ppmlhdfe estimation?
Thank you very much in advance!

That's a great question, and I wish I had more time to dive into the margins command to extend the support of pmmlhdfe for margins options (there are a lot of margins options and ppmlhdfe only supports a few of them).
On your specific question, a simple way to verify if the margins command is indeed ignoring the FEs or not is to compare it against poisson with dummies:
sysuse auto, clear

ppmlhdfe price weight length, a(turn) d keepsingletons
margins, dydx(weight)

qui poisson price weight length i.turn, vce(robust)
margins, dydx(weight)

Here, you can see that both give the same results. However, I haven't tested much the at() option, which is the one that might be giving folks problems in the online discussions of xtlogit/xtpoisson. Thus, I encourage you to be careful when using at(), as shown below:
sysuse auto, clear

* Here margins works but gives you the results as if vce() was not robust
qui ppmlhdfe price c.weight#c.weight length, a(turn) d keepsingletons
margins, dydx(weight) at(weight=2000)
qui poisson price c.weight#c.weight length i.turn, vce(robust)
margins, dydx(weight) at(weight=2000)

* Here margin gives you different point estimates
qui ppmlhdfe price c.weight#c.weight, a(turn) d keepsingletons
margins, dydx(weight) at(weight=2000)
qui poisson price c.weight#c.weight i.turn, vce(robust)
margins, dydx(weight) at(weight=2000)

I would need to understand more the details of margins and its underlying assumptions in order to make it work better for ppmlhdfe, which I'm not able to do (at least in the near future), but any suggestions or findings would be more than welcome!

Thank you, Sergio for your response.
I have one follow up question.
Would the results from ppmlhdfe regression model always be the same as the poisson model with dummy variables for the fixed effects? Even if the dependent count variable may be zero inflated?
I would assume that if I directly add multiple dummy variables for fixed effects in the poisson command, the estimation process will take very long. However, I am curious at the end of the estimation I will always get the same result as the PPMLHDFE regression result.
Thank you.

@sergiocorreia I have a similar question. I was testing whether the pplmhdfe and possion commands generated the same results and while in the example above it does generate identical ones, with my data they are quite different. Here's the output of your example (to show it is not an installation issue)

and here it is for my data using basically a similar command

where I am running the commands
ppmlhdfe N_groups_0 sharepopregsec  if sample==1, absorb(idGID_0) cluster(idGID_0)  d keepsingletons

margins, dydx(sharepopregsec)

poisson N_groups_0 sharepopregsec i.idGID_0 if sample==1, cluster(idGID_0)

margins, dydx(sharepopregsec)

Notice the different samples. Yet I get the same result if I drop keepsingletons.
Moreover, if I do not have FEs I do get the exact same result. Also, the estimated coefficients (not MEs) are actually identical, so it seems this is a MEs + FEs issue . To get the same MEs with FEs, I had to constrain the sample so that using poisson it is the same as using ppmlhdfe, i.e.,
ppmlhdfe N_groups_0 sharepopregsec  if sample==1, absorb(idGID_0) cluster(idGID_0)  d keepsingletons

margins, dydx(sharepopregsec)

poisson N_groups_0 sharepopregsec i.idGID_0 if e(sample)==1, cluster(idGID_0)

margins, dydx(sharepopregsec)

in which case the MEs are identical

I thought the option keepsingletons would ensure the sample remains the same as the one using poisson, but it seems that is not the case (which I think is the case in reghdfe vs reg). Is there a way to ensure ppmlhdfe uses the full sample? What is being dropped that is not a singleton? Perhaps more importantly, which one is the correct one? Any ideas?
Thanks!

I can't tell for sure without looking at the data, but one possibility is that the dropped observations correspond to "separated observations", particularly those separated by the fixed effects.
This separation primer should clarify the issue a bit.

Thanks! This is super useful and I imagined it was something like that. I just thought the keepsingletons option was including them somehow.

Is the separation issue not affecting other estimators? I remember from reghdfe that you showed that the estimate is the same (some difference in SEs). Here estimators are the same but MEs and SEs are different. Does this suggest that one should perhaps constrain the estimation sample in general to non-separated observations?

@sergiocorreia following up on my previous question. I imagine the separation issue is also important, e.g., in negative binomial. Should we expect to have a similar convergence to the wrong estimate issue in nbreg?
I see that in my data reghdfe (full sample) and ppmlhdfe (dropping separated obs) generate similar MEs, but poisson and  nbreg have very different ones if I use the full sample but very similar if dropping separated obs. There are only very small differences in the estimated coefficients and SEs, so it would seem the issue of differences in MEs is due to the sample means used in their computation. It is not clear to me which is the correct comparison. Since in my data there are separated observations, the estimation is only correct with ppmlhdfe, yet the estimated coefficient with poisson is basically identical. Should one still use only the constrained sample since this is where the actual variation for estimation is coming from?


Update devcontainer.json to support running quickstarts in both local VSCode and GitHub Codespaces

Describe the proposal
The existing devcontainer definition can be used in Codespaces, but it has a couple of issues:

It invokes dapr init as root, rather than non-root codespaces user which creates a dirty environment where dapr does not show as initialized for the codespaces user but the deployed containers will force the user to clean up the existing install before another dapr init can be run.
It does not specify a base container image, so a user starting a devcontainer locally in VSCode needs to figure out which image to use, but none of the offered options are sufficient for running all dapr quickstarts since they require a large collection of different frameworks and SDKs.

To improve on this experience:

Base the devcontainer definition on the Codespaces universal Linux image, which is already configured for running in Codespaces and contains most of the frameworks required by the dapr quickstarts:

Golang
Python3
Node.js
Docker-in-docker support
Kubectl
Helm
npm
.NET core SDK

To round out the remaining tools needed by the quickstarts, additionally install:

Minikube, to provide a standalone Kubernetes cluster
Dapr CLI

Do not run dapr init for the user as different quickstarts may require the user to be familiar with running dapr either as standalone or deployed to a Kubernetes cluster (which would require dapr init -k instead)

Mechanical-markdown Python module, for running quickstart validations.

Speaking of quickstart validation, I would be happy to contribute something that uses the devcontainer/codespace so it gets exercised through automation somehow. I haven't figured out a good way to do that yet though in github actions.

You should be able to, at the very least, simply add github action steps to do the raw docker commands to build and run the container, as you would on your local machine.


Recoil memory leak with unmounted component?

Describe the bug
I'm seeing warnings in the console. Application behaviour appears unaffected.
My app renders a carousel with scroll buttons if the header width becomes too cramped. If the users expands the browser window wide enough, the whole list of items is rendered instead.
When the carousel is rendered the following console warnings appear:

Warning: Can't perform a React state update on an unmounted component. This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in a useEffect cleanup function.
in Ne (created by RecoilRoot)
in RecoilRoot
in Unknown (at Carousel.tsx:70)

Warning: Cannot update a component (Ne) while rendering a different component (un). To locate the bad setState() call inside un, follow the stack trace as described in https://fb.me/setstate-in-render
in un
in un (created by i)
in i (created by i)
in i (created by ForwardRef(withResizeDetector(undefined)))
in ForwardRef(withResizeDetector(undefined)) (created by hn)
in hn
in RecoilRoot
in Unknown (at Carousel.tsx:70)

To Reproduce
Another user on stackoverflow has experienced this issue, with example code:
https://stackoverflow.com/questions/63632304/i-have-error-cannot-update-a-component-ne-while-rendering-a-different-compo
Expected behavior
No console warnings.
Environment
  System:
    OS: Windows 10 10.0.19042
    CPU: (16) x64 Intel(R) Core(TM) i9-9900 CPU @ 3.10GHz
    Memory: 17.32 GB / 31.79 GB
  Binaries:
    Node: 12.20.1 - C:\Program Files\nodejs\node.EXE
    npm: 6.14.10 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Chrome: 87.0.4280.88
    Edge: Spartan (44.19041.423.0), Chromium (87.0.664.66)
    Internet Explorer: 11.0.19041.1
  npmPackages:
    @brainhubeu/react-carousel: 2.0.1 => 2.0.1

This is on desktop Chrome (version above)
Additional context
Add any other context about the problem here.

I'm getting this warning as well. It's happening when react-dom is 16.13.0 or higher.

I'm getting this warning as well. It's happening when react-dom is 16.13.0 or higher.

Getting the same warning

Getting the same warning

me too!

me too!

Same warning, also on react 16

Same warning, also on react 16

Same here!

Same issue here as well. Using react and react-dom 16.14.0. In production, this error takes down my app. Any suggested fix would be much appreciated.

Having problems with this aswell


[DRAFT] Project schema proposal

Hi!
As touched upon briefly in the last couple of TSC meetings, here is an initial proposal for a Project schema definition.
It's written i python at the moment so this PR is mostly for discussion and to get the ball rolling on a full fledged Project schema.

Some metadata is more natural to store under a project umbrella than a timeline like: OCIO config path, media search paths, etc.
Adding a Project schema will allow users to exchange not only individual timelines, but complete project structures across applications.
In combination with something like the proposed file bundle #561 adapter one could archive/consolidate a complete project including clips/media references outside of a timeline.

I did a quick "survey" of the project settings of Premiere, Final Cut, Hiero and Davinci and naturally they all have some variations, but all of them share the concept of "bins" and have a notion of these settings:

frame rate (default frame rate for new timelines)
resolution (viewer/render resolution)
colorspace (viewer lut or colorspace of media)

In my proposal I chose to name our "bins" holder; collections to refer to the SerializableCollection objects that it could store, and a separate timelines attribute which should hold Timeline objects (duh..)
I also chose to call the rate, resolution and colorspace:

viewer_rate
viewer_resolution
viewer_lut

as these settings usually only apply to the viewer of an applications project file.
Example in python:
import opentimelineio as otio

p = otio.schema.Project('myProject')
bin = otio.schema.SerializableCollection('myclips')
t = otio.schema.Timeline('myTimeline')
p.collections.append(bin)
p.timelines.append(t)

otio.adapters.write_to_file(p, 'project_test.otio')

Results:
otiocat project_test.otio
{
    "OTIO_SCHEMA": "Project.1",
    "collections": [
        {
            "OTIO_SCHEMA": "SerializableCollection.1",
            "metadata": {},
            "name": "myclips",
            "children": []
        }
    ],
    "timelines": [
        {
            "OTIO_SCHEMA": "Timeline.1",
            "metadata": {},
            "name": "myTimeline",
            "global_start_time": null,
            "tracks": {
                "OTIO_SCHEMA": "Stack.1",
                "metadata": {},
                "name": "tracks",
                "source_range": null,
                "effects": [],
                "markers": [],
                "children": []
            }
        }
    ],
    "viewer_lut": null,
    "viewer_rate": null,
    "viewer_resolution": [
        null,
        null
    ],
    "metadata": {
        "OCIO_PATH": null
    },
    "name": "myProject"
}

Again, this is an initial proposal to get the ball rolling. All comments, ideas and views are welcome :)
Thanks!

I think I'm going to close this pull request as I now realize it might not be the best way to go. The idea of a root bin seems like a better approach. We'll still have a place to store several timelines and clips etc. without having to adapt to every application out there with regards to project settings and so on.


osbuild: cleanup contextlib usage

Two cleanups for the context-managers we use:

Use contextlib.AbstractContextManager if possible. This class
simply provides a default __enter__ implementation which just
returns self. So use it where applicable.
Additionally, it provides an abstract __exit__ method and thus
allows static checks for an existance of __exit__ in the dependent
class. We might use that everywhere, but this is a separate
decision, so not included here.

Explicitly return None from __exit__. The python docs state:
If an exception is supplied, and the method wishes to suppress
the exception (i.e., prevent it from being propagated), it
should return a true value. Otherwise, the exception will be
processed normally upon exit from this method.

That is, unless we want exceptions to be suppressed, we should
never return a truthy value. The python contextlib suggest using
None as a default return value, so lets just do that.
In particular, the explicit return exc_type is None that we use
has no effect at all, since it only returns True if no exception
was raised.
This commit cleans this up and just follows what the contextlib
module does and returns None explicitly everywhere. It is unlikely
that we ever want to suppress any exceptions, anyway.

In particular, the explicit return exc_type is None that we use
has no effect at all, since it only returns True if no exception
was raised.

Good catch, mea culpa, the logic is inverted :(

In particular, the explicit return exc_type is None that we use
has no effect at all, since it only returns True if no exception
was raised.

Good catch, mea culpa, the logic is inverted :(

Did you actually intend to suppress the exceptions there? Because I dropped the clause entirely now, as I did not understand why we would want to suppress the assertions there. It is functionality equivalent to what it was before, but please tell me if that is not the intended behavior.

In particular, the explicit return exc_type is None that we use
has no effect at all, since it only returns True if no exception
was raised.

Good catch, mea culpa, the logic is inverted :(

Did you actually intend to suppress the exceptions there? Because I dropped the clause entirely now, as I did not understand why we would want to suppress the assertions there. It is functionality equivalent to what it was before, but please tell me if that is not the intended behavior.

No no, all good, exceptions for everyone! ;)


Integrating Contentful CMS to SML Portal Web App

I have successfully integrated Contentful headless CMS into the SML Portal Web App.
Users will be able to add, delete & edit the following pages:

Home page
Methods Catalogue page
Methods table within method catalogue page
Individual method pages
About page

This is a demo branch, to show how this CMS system works.

Deleting.

SPACE_ID & CDA keys within this repo/ code are no longer valid. They have been used to test API calls and now have been regenerated and are hidden.


The generated SAS URI for blob does not open the blob

Storage Explorer Version: latest (5.8)
Platform/OS Version: Windows/10 x64
Architecture: ia32
Regression From: don't know
Steps to Reproduce:

Right click in UI on a blob
Generate SAS URI
Try to download it

Expected Experience:
Download happens
Actual Experience:
Following Internal Server Error (XML downloaded instead of the blob):
Server encountered an internal error. Please try again after some time. RequestId:f0aaf1e6-007b-45cf-a2ed-fde902d2e2b7 Time:2018-11-01T15:24:14.6509496Z
In ASE logs, you got the following lines:
Info log:
[Info_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] Setting status description as "Server encountered an internal error. Please try again after some time."
Error log:
[Error_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] Exception raised during processing of request. Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.FatalServerCrashingException: The fatal unexpected exception 'Object reference not set to an instance of an object.' encountered during processing of request. ---> AsyncHelper.NullReferenceRethrowableException: Object reference not set to an instance of an object. ---> System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Cis.Services.Nephos.Storage.Service.Protocols.Rest.HttpRestProcessor.d__101.MoveNext()
at AsyncHelper.AsyncIteratorContextBase.ExecuteIterator(Boolean inBegin)
--- End of inner exception stack trace ---
at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer1.EndPerformOperation(IAsyncResult ar)    at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer1.d__103.MoveNext()
--- End of inner exception stack trace ---
Unhandled log
[UnhandledException] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] EXCEPTION thrown: Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.FatalServerCrashingException: The fatal unexpected exception 'Object reference not set to an instance of an object.' encountered during processing of request. ---> AsyncHelper.NullReferenceRethrowableException: Object reference not set to an instance of an object. ---> System.NullReferenceException: Object reference not set to an instance of an object.
at Microsoft.Cis.Services.Nephos.Storage.Service.Protocols.Rest.HttpRestProcessor.d__101.MoveNext()
at AsyncHelper.AsyncIteratorContextBase.ExecuteIterator(Boolean inBegin)
--- End of inner exception stack trace ---
at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer1.EndPerformOperation(IAsyncResult ar)    at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessorWithAuthAndAccountContainer1.d__103.MoveNext()
--- End of inner exception stack trace ---
at Microsoft.Cis.Services.Nephos.Common.Protocols.Rest.BasicHttpProcessor.EndProcess(IAsyncResult result)
at Microsoft.WindowsAzure.DevelopmentStorage.Store.BlobServiceEntry.ProcessAsyncCallback(IAsyncResult ar)
Verbose log
[Verbose_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] Accepting request
[Verbose_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] Got request PUT http://<IP_ADDRESS>:10000/devstoreaccount1/maincontainer/670d8d88-c68a-4949-8e6a-956c5e1c9673.MP4?comp=blocklist 2015-12-11
[Verbose_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] Host-style URI resolution failed. Allowing path-style for uri http://<IP_ADDRESS>:10000/devstoreaccount1/maincontainer/670d8d88-c68a-4949-8e6a-956c5e1c9673.MP4?comp=blocklist
[Verbose_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] StorageAccount.GetPropertiesImpl(PropertyNames=0x0020, ServiceMetadataPropertyNames=0x000000007807C000,,00:00:30)
[Verbose_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] Returning from AcceptRequest method.
[Verbose_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] Setting status code to 500
[Verbose_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] In the process end callback
[Verbose_debug] [ActivityId=a4a28182-860d-421a-ace3-e980b5fef038] Entering BasicHttpRestProcessor.EndProcess

I don't believe this is the right repo you were looking for.  Storage Explorer never had versions 5.8.  The error messages don't look familiar either.

Indeed, my mistake, correct repo is this one: https://github.com/Azure/azure-storage-net
Could you transfer the issue to it?

Forget about this, they have a different issue template. I'll copy paste the information in new ones.

great.  Github also has a beta feature to transfer issues from one repo to another.

Yup, heard about it, that's why I was asking ;)
However their issue template is really different so I recreated it


Eliminate client certificate for kube-state-metrics and prometheus

How to categorize this PR?

/area security
/kind enhancement
/merge squash
What this PR does / why we need it:
Similar to #4931, this PR eliminates the client certificate for the kube-state-metrics in favor of a token managed by the TokenRequestor controller in gardener-resource-manager.
prometheus has such a token as well, but for backwards-compatibility it also still has access to its client certificate (this will be dropped in the future).
Which issue(s) this PR fixes:
Part of #4661
Part of #4878
Special notes for your reviewer:
/invite @BeckerMax
/cc @wyb1 @istvanballok
Release note:

`kube-state-metrics` does no longer use a client certificate but an auto-rotated `ServiceAccount` token which is only valid for `12h`. `prometheus` has such a token as well, but for backwards-compatibility it also still has access to its client certificate (this will be dropped in the future).

The monitoring scrape configurations (particularly the `tls_config` and `authorization` section) of extension controllers must be adapted such that they match the example in https://github.com/gardener/gardener/blob/master/docs/extensions/logging-and-monitoring.md#extensions-monitoring-integration.

/assign


[main] [BUG] @microsoft/applicationinsights-web-snippet version 1.1.1 type problem #2284

Just checked the build -- looks good


ContractCreateFlow() can't use .freezeWith() and .sign() after setting an adminKey

Description
I know ContractCreateFlow() is not a transaction as I've seen in this issue, but if I set an admin key and I need to pay the transactions that ContractCreateFlow() is executing using another account, I can't as  .freezeWith() and .sign() are not available. My workaround right now is changing the operator using client.setOperator(adminId, adminKey).
Error:
.freezeWith(client);
         ^
TypeError: (intermediate value).setGas(...).setBytecode(...).setAdminKey(...).freezeWith is not a function

Steps to reproduce

Open this codesandbox
Fork and add your testnet credentials in .env file
Run example to get the error (ContractCreateFlow() line 37 of index.js)

Additional context
No response
Hedera network
testnet
Version
v2.14.2
Operating system
macOS

+1 on this issue. Basically, the reasoning that ContractCreateFlow() is not a transaction and can't support .freezeWith() means that users cannot directly create contracts that have an admin key using ContractCreateFlow.
The admin key for a contract must sign the creation operation. If ContractCreateFlow() doesn't have .freezeWith() or .sign(), that means we're forcing users of ContractCreateFlow() to change the operator to deploy contracts with admin keys.

The addition of freezeWith() doesn't work here, but adding a *Flow.sign[With|WithOperator]() does make sense.


Add support git forks,

Azure DevOps java sdk pull request template guide
Thank you for your interest in contributing to this project.
Description

Added gitforks method link
Added createForkRepository link
Added createForkRepositoryWithComplete method

fork repo creation is not executed immediately.
A method that waits for a queued fork request
when fork repo using azure devops console,
It is the same flow when fork using azure devops console.

fork from parent repo
get fork operation Id
check fork request complete (If queued, continue requesting)

// GitApi,java
public GitRepository createForkRepositoryWithComplete(String repositoryName, String projectId, String parentProjectId, String parentRepositoryId,
                                                          String sourceBranch, int checkTimes) throws AzDException {
        GitRepository gitRepository = createForkRepository(repositoryName, projectId, parentProjectId, parentRepositoryId, sourceBranch);
        GitForkSyncRequest gitForkSyncRequest = getForkSyncRequests(repositoryName, true, true).getForkSyncRequest().get(0);
        int operationId = gitForkSyncRequest.getOperationId();

        for (int i = 0; i < checkTimes; i++) {
            gitForkSyncRequest = getForkSyncRequest(repositoryName, operationId, true);
            if (gitForkSyncRequest.getStatus() == GitAsyncOperationStatus.COMPLETED) {
                return gitRepository;
            }
            try {
                TimeUnit.MILLISECONDS.sleep(500);
            } catch (InterruptedException e) {
                throw new AzDException("Error while waiting for fork sync request to complete");
            }
        }

        throw new AzDException("Fork sync request did not complete in time : " + gitForkSyncRequest.getStatus());
    }

//GitAsyncOperationStatus.java
public enum GitAsyncOperationStatus {
	ABANDONED, // The operation has been abandoned.
	COMPLETED, // The operation has completed.
	FAILED, // The operation has failed. Check for an error message.
	INPROGRESS, // The operation is currently in progress.
	QUEUED; // The operation is waiting in a queue and has not yet started.
}

PR Checklist

[x] Added help
[x] Added unit test/s
[x] Updated Change log

@shblue21, thanks for adding this. It is a required Api and can be very useful. I had a quick look and everything LGTM. Happy to merge the changes. I will leave it for couple of days as a new version will be released this week.


"in"->"a" in order_filter docstring

Fix the docstring for order_filter: the 1st parameter is called a but is referred to as in in the docstring.

@@            master   #5820   diff @@
======================================
  Files          235     235
  Stmts        43347   43347
  Branches      8165    8165
  Methods          0       0
======================================
+ Hit          33722   33724     +2
+ Partial       2594    2593     -1
+ Missed        7031    7030     -1

Review entire Coverage Diff as of 455560c

Powered by Codecov. Updated on successful CI builds.

Thanks @yoavram, merged.


Error during installlation!

npm ERR! code Z_BUF_ERROR
npm ERR! errno -5
npm ERR! zlib: unexpected end of file
npm ERR!      C:\Users\nikoskon\AppData\Roaming\npm-cache_logs\2018-12-15T13_39_54_638Z-debug.log

all i had to do was npm cache clean --force


Possible bug-fix for aws_demo_logging.c

Description
As I am mostly working on FreeRTOS+TCP these days, I rarely look at the PR's in the amazon-freertos repo.
But as today I was repairing a possible bug in aws_demo_logging.c, I saw Ming Yue's recent PR #2995 for this module.
Ming Yue, the problem was not the uninitialised local variable xLength2, but the missing else clause of if( xArgs != NULL ):
     if( xArgs != NULL )
     {
         xLength2 = vsnprintf( cPrintString + xLength,
                               dlMAX_PRINT_STRING_LENGTH - xLength,
                               pcFormat,
                               xArgs );
     }
+    else
+    {
+        xLength2 = snprintf( cPrintString + xLength,
+                             dlMAX_PRINT_STRING_LENGTH - xLength,
+                             "%s",
+                             pcFormat );
+    }

In earlier versions, the else clause was present.
Here is a normal use case in which 'xArgs' is set to NULL:
void vLoggingPrint( const char * pcFormat )
{
    prvLoggingPrintf( LOG_NONE, NULL, 0, pdFALSE, pcFormat, NULL );
}

Now my possible bug-fix: when checking the free space with uxStreamBufferGetSpace(), it must happen within the critical section, thus like this:
    SetThreadPriority( GetCurrentThread(), THREAD_PRIORITY_TIME_CRITICAL );
    {

        /* How much space is in the buffer? */
        xLength2 = uxStreamBufferGetSpace( xLogStreamBuffer );

        /* There must be enough space to write both the string and the length of
         * the string. */
        if( xLength2 >= ( xLength + sizeof( xLength ) ) )
        {
            uxStreamBufferAdd( xLogStreamBuffer,
                               0,
                               ( const uint8_t * ) &( xLength ),
                               sizeof( xLength ) );
            uxStreamBufferAdd( xLogStreamBuffer,
                               0,
                               ( const uint8_t * ) cOutputString,
                               xLength );
        }
        else
        {
            /* Log line will be dropped, bad luck. */
        }
    }
    SetThreadPriority( GetCurrentThread(), iOriginalPriority );

Otherwise a race condition is possible in which a higher-priority task comes in between and fills the stream buffer. This may cause the logging to stop working properly.
Hein
Checklist:

[x] I have tested my changes. No regression in existing tests.
[x] My code is Linted.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

/bot run checks

/cbmc run checks

/bot run checks

/bot run checks

/bot run checks


[Question] : st-info --probe not listing device info

The motive here is

Flash the device using st-link, instead of drag-drop method supported by mbed enabled device (issue are seen the ST device doesn't get programmed always with drag drop)
Flashing multiple devices with same binary.
How hard reset device if possible.

➜  stlink git:(master) ✗ sudo  build/Release/st-info --probe
Found 5 stlink programmers

➜  stlink git:(master) ✗ sudo  build/Release/st-info --serial
303637314646343934393531373835

Device connected , listing by mbed-ls all device are mbed enabled
| NUCLEO_F401RE | NUCLEO_F401RE[0]     | /mnt/pci-0000_00_14_0-usb-0_1_2_1_1-scsi-0_0_0_0  | /dev/ttyACM1  |<PHONE_NUMBER>2265023866F31F                         | 0221            |
| NUCLEO_F401RE | NUCLEO_F401RE[1]     | /mnt/pci-0000_00_14_0-usb-0_1_3_1_1-scsi-0_0_0_0  | /dev/ttyACM3  |<PHONE_NUMBER>3265613876F37C                         | 0221            |
| NUCLEO_F401RE | NUCLEO_F401RE[2]     | /mnt/pci-0000_00_14_0-usb-0_1_1_1_1-scsi-0_0_0_0  | /dev/ttyACM0  |<PHONE_NUMBER>74650F3830F312                         | 0221            |
| NUCLEO_F429ZI | NUCLEO_F429ZI[0]     | /mnt/pci-0000_00_14_0-usb-0_6_2_1_1-scsi-0_0_0_0  | /dev/ttyACM5  |<PHONE_NUMBER>4C62643F08F4CF                         | 0221            |
| NUCLEO_F429ZI | NUCLEO_F429ZI[1]     | /mnt/pci-0000_00_14_0-usb-0_6_3_1_1-scsi-0_0_0_0  | /dev/ttyACM8  |<PHONE_NUMBER>2567073E61F1AC                         | 0221            |
+---------------+----------------------+---------------------------------------------------+---------------+--------------------------------------------------+-----------------+

The mbed protocol is different than the stlink protocol. It is a current limitation with probe it will not show the programmer serial numbers when no target is connected. I have rewritten a part of stlink for myself which makes probe work without a connected target with my stlink2 project.
Flashing multiple targets at once with the same binary is not possible with the current st-flash tool, you could write a shell script to make this possible
Hard reset, thats a good question. There are two resets, the pin reset and the core reset over the programming interface. I can not answer which is implemented for texane/stlink (see also #518.

You are also able to write your own tool around the stlink library, as it is build as an shared object (dll). But also can be staticly linked. Have a look at how simple the st-flash tool is src/tools/flash.c

Thanks @xor-gate


Evaluate PreferenceFragment Support

Evaluate how difficult emulating a PreferenceFragment will be since PreferenceActivity can not be used.

Haven't touched it since. Working on ABS v4 now. I might try it once that is completed.

@ChristopheVersieux it shouldn't be too difficult to do it yourself, here's the ICS source for PreferenceFragment

I believe it uses internal PreferenceManager and PreferenceScreen APIs which is why I gave up on it initially until the sources were available. The tricky part will be ensuring it can still work with things that extend android.preference.Preference. It's not a requirement, but would be great for a proper solution.

Thank a lot for your help, I found this one in the meanwhile: http://forum.xda-developers.com/showpost.php?p=19719977&postcount=1
With a perfectly working example.
Great job everybody and thank for your quick help.

Cool. Thanks for the link. That method uses reflection to access the internal APIs so I'll still probably have a go at creating a proper implementation eventually. Glad that works for you for now though.


Consideration of pros and cons of middle ground options regarding call site annotations

Section 1.3.6 of current proposal document describes mandatory use of confirmations of submitting objects to functions which will probably overwrite them using  inout, out, forward and move prefixes, and presents advantages and disadvantages of such approach which could be summarized as follows:

advantage: user will not accidentally overwrite their variables;
advantage: reader will not miss the variables being overwritten;
advantage: other languages and some C/C++ conventions do this and it seems to work;
disadvantage: some people will complain about verbosity.

To me, this sounds like a good summary of annotating variable usage in general as opposed to this particular setup, because they would seem to apply to any set of prefixes. And on a subjective level, inout looks really bad to me:
swap(inout a, inout b);
(inout std::cin) >> (inout c) >> (inout d); //parentheses for operators, or no?
(inout b) += c; //operator overloading is not limited to iostream, if ints and floats get special treatment
(inout std_map).clear(); //is that how it's supposed to work with implied this?
(inout std_map)[a] = b; //or is it (out(inout std_map)[a])? does it depend on the type of value?
(inout (inout std_map)[a]) += d; //see above

Maybe this is supposed to be exempt from annotations, I couldn't figure it out.
For no particular reason, let's consider a setup where we replace inout call prefix with unary >. Here's my completely biased pitch:

users and readers related advantages are basically unchanged;
the comparison to other languages applies on semantics level, but if we consider brevity of notation a competition, such C++ (>) is now arguably tying with pointers convention (&) and winning with Rust (mut&) and C# (ref ) instead of (inout  / mutable ) losing, arguably tying and arguably losing, respectively;
some people will still complain, but since the notation is shorter, maybe a few less;
there are some possible arguments against inout being special enough to deserve this notation and related teaching; while I could see shorter prefixes, or the same one if possible, used for other cases as well, inout is my breaking point, I guess, and relatively often removing it would just break the compilation;
if there are going to be more types of parameters -- there probably are, especially if code styles start deprecating current undescribed parameters and return types -- there may be further discussion if they should get short call prefix forms, which does not seem like a major problem to me;
grammar will probably be harder to figure out.

Anyway, I don't know if my ramblings are welcome here, but since issues are open, I thought I may at least get others' thoughts on this.

Thanks! Actually I should have written out rather than inout in the example code, as the suggestion in the text was already suggesting just out (or mutable) for arguments that could be modified -- there is a data flow "out" as well as in.
I'll look to add examples like the ones you gave above to show the potential for verbosity. With reasonable precedence, we could get it down to something like this
std::swap(out a, out b);
out std::cin >> out c >> out d;
out b += c;
out mymap .clear();
out mymap [a] = b;
(out (out mymap)[a]) += d;

but which still needs to be called out as verbose.


Backspace won't trigger delete event

Hi,
I notice two things that won't trigger the delete event.

Insert an image and type some text into next lines
Keep pressing backspace from the line with text till the image is deleted by backspace (delete event is not trigger here)

OR

Insert an image and type some text into next lines
Select all and press backspace to delete (delete event is also not trigger)

Not sure this is intended to be done this way or it's a bug.
Thanks

When will you fix this? :) I really need to have this bug fixed on my production sites. Thank you.

@nickground feel free to provide a fix :)


graph_log_likelihood unit test

This PR adds a unit test for graph_log_likelihood. I also found and fixed a wrong sign bug in graph_log_likelihood, this had the effect of:

decreasing demo classification accuracy from 78% to 67%: it now matches the papers reported accuracy of 67%
demo embeddings TSNE are now less well separated and appear less visually similar to the paper

So in terms of accuracy it matches the paper better, but in terms of TSNE plot it matches the paper worse. TSNE is very parameter dependant and I couldn't find the paper's parameters so this is a less reliable metric than classification accuracy.
See #1135 and equation 4 of https://arxiv.org/pdf/1710.09599.pdf

Why does using this "incorrect" loss function give a better result?

I think we may have accidently stumbled across a better loss function.
Running this code:
from matplotlib import pyplot as plt
import numpy as np

# expected walk value for one node pair
e = 5

# potential values of sigma in range (0, 1)
# lets see which sigma gives the lowest loss
sigmas = np.linspace(0.0001, 0.9999, 1000)

# graph log llikelihood function when there's no edge
loss_1 = np.abs(-e * np.log(sigmas) - np.log(1 - sigmas))
loss_2 = np.abs(e * np.log(sigmas) - np.log(1 - sigmas))

plt.title("Graph Log Likelihood Loss")
plt.plot(sigmas, loss_1)
plt.ylabel("loss")
plt.xlabel("sigma")
plt.show()

plt.title("Custom Loss")
plt.plot(sigmas, loss_2)
plt.ylabel("loss")
plt.xlabel("sigma")
plt.show()

Gives:

The local minima of the custom loss (my old buggy graph log likelihood loss) is roughly the same as the graph log likelihood but it looks to me that the custom loss local minima will be easier to find with gradient descent bc it's gradient doesn't flatten out so much.


Publisher total timeout value change exposing potential long-running publisher issues

Environment details

OS: Alpine linux
Node.js version: 14.15
npm version: 6.14
@google-cloud/pubsub version: 2.18.1

Issue Summary
After upgrading @google-cloud/pubsub versions from 2.16.6 to 2.18.1 (and not making any other changes), users are seeing a higher rate of occurrence for this error:
"Error: Total timeout of API google.pubsub.v1.Publisher exceeded 60000 milliseconds before any response was received".
However along with occurring less often on version 2.16.6, the error message was slightly different (the total timeout went from 600000 to 60000):
"Error: Total timeout of API google.pubsub.v1.Publisher exceeded 600000 milliseconds before any response was received".
There was only one change made to the publisher config but it only changed the initial timeout. The publisher config file shows no change in the total timeout between versions 2.18.1 and 2.16.6.
Here is the sample code shared by the user, but it was not changed when the library version was upgraded
    const topicOptions = {
      batching: {
        maxBytes: 5242880,
        maxMessages: 50,
        maxMilliseconds: 400,
      },
    };
const topic = new PubSub(options).topic(topicName, topicOptions);
await topic.publish(Buffer.from(JSON.stringify(normalizedEventData)));

There were also a few recent flaky-bot issues opened with similar error messages:

https://github.com/googleapis/nodejs-pubsub/issues/1418
https://github.com/googleapis/nodejs-pubsub/issues/1404

We would like to know whether this would require some tweaks to the way the user is currently publishing, or if this was an unintentional change introduced in the newer versions of the library.
Thanks in advance!

Hi folks, two other things worth mentioning:

When attempting to revert the defaults to what they were prior to https://github.com/googleapis/nodejs-pubsub/pull/1399 via PublishOptions, we saw a massive spike in CPU and eventually latency to the point where our service became unusable
In the normal case described above by @ShyamShah11 (on 2.18.1 without timeouts changed via PublishOptions), it takes about 2.5 hours before we start seeing total timeout errors

Thanks for taking a look.

Thanks for the patience on this one. The other major thing that happened between those two versions was publisher side flow control, and even with it disabled, quite a few of the internals were shuffled around. It does look like the number should still be 600000, that update retry timeouts PR shouldn't have affected that. I'm pretty sure there was a second PR for this one:
https://github.com/googleapis/nodejs-pubsub/issues/1398
...which I can't currently find. I've got a few paths to look at here.

This was the missing PR, looks like it didn't actually get merged due to a conflict: https://github.com/googleapis/nodejs-pubsub/pull/1416
...but also, it doesn't look like what's in main should've had the problem it was originally meant to fix.

Hmm, well this looks to be the source of the change in timeout: https://github.com/googleapis/gax-nodejs/pull/1100
It looks like as a stopgap, you could pin gax at 2.25.3, but that doesn't seem like a great answer. It also feels like 60s should be plenty to publish a batch of messages, so there might be an underlying problem to look at.
@kylechadha What's the character of publish() calls you're making? As in, one offs once in a while, vs a huge batch at once, vs continuous heavy publishing, etc. It's possible that something I changed for publisher flow control is making the problem worse.

@kamalaboulhosn might also have some comments on this. It seems like the change in gax might be unexpected behaviour.

If the issue is taking 2.5 hours to manifest, it sounds to me like some kind of memory or resource leak that is slowing down processing over time. The change from 600s to 60s as the timeout may have just made that issue be realized sooner due to the smaller timeout. In general, I'd expect the number of publishes that should fail in 60s to be quite small. If it gets into the state where none were failing in that time and then most or all start failing within that time, it sounds like a resource leak.
That leak could be in the client library, potentially. Have we run any tests that run for 2+ hours with the Node library to see if problems develop? If not, it might be worth doing that.
We do ultimately need to understand why the total timeout switched from 600s to 60s without any changes we made that we expected to cause that. Am I reading googleapis/gax-nodejs#1100 correctly that if one specifies a timeout in the config, then it is used for all three timeout values (initial, max, and total)? What is the config we have that is specifying this timeout?

Thank you for digging into this @feywind and @kamalaboulhosn. If it helps provide an additional datapoint, we have a few different services impacted where we observed the 2.5 hour delay before errors begin.
The one I'm most familiar with does continuous heavy publishing (~40m publishes per hour, peak of 100k publishes per second). Prior to 2.18.1 we saw total timeout errors (at the 600s mark) in bursts with long periods of no errors. On 2.18.1, we see errors consistently at a low rate.
@kelseyfrose-paypal Can you provide some more context around your team's experience?

@kamalaboulhosn Yeah, that seems to be what it's doing in the new logic. If a timeout is specified in the gapic config, it'll get used for the total timeout as well. So e.g. this would set the total timeout to 5000, which seems even more problematic:
        "Publish": {
          "timeout_millis": 5000,
          "retry_codes_name": "cancelled_unknown_deadline_exceeded_resource_exhausted_aborted_internal_unavailable",
          "retry_params_name": "default",
          "bundling": {
            "element_count_threshold": 100,
            "request_byte_threshold": 1048576,
            "delay_threshold_millis": 10,
            "element_count_limit": 1000
          }
        },

Maybe @alicejli can comment on that change for gax?
I agree with Kamal that it could very well be a memory leak or something in the client library. Just trying to get a handle on what to test. :) We do have a long-running tester for Pub/Sub on Node, but I'm not sure it's publishing quite that many messages.

@kylechadha I should also ask, since it's a common source of memory leak issues, are you creating one PubSub object and reusing it for subsequent publishes? The parser for the protos seems to leak some memory due to caching each time you create a new PubSub object. (I recently discovered that there's a cache purge method, but I haven't figured out how to plumb to it yet.)

@feywind Can confirm the PubSub object is created once and reused for subsequent publishes.

Thanks for looking into this!
We're also seeing an unusually high volume of timeouts in our service using 2.18.1. This service publishes regularly to a handful of topics, and the highest volume publisher publishes around 10M per hour at peak. The errors for us come in bursts rather than consistently, and they come from a single pod at a time (we run about 225 pods on this service). For example, we'll see a burst of ~10k errors for topic A coming from pod A over several hours, and the next day we'll see a burst of ~10k errors for topic B coming from pod B over several hours. Rolling out the deployment or killing the offending pod resolves the errors for at least a few hours. The errors also generally resolve on their own after several hours.

I'm going to see if I can get a workaround to get us past next week anyhow, with the old behaviour. There is something weird going on here with the timeouts and I'd prefer to get a proper fix, but I don't want to leave anyone with issues for now.

2.18.3 should be releasing shortly, give that a try and see if it at least gets us back to where we were in 2.16.6?

Also for anyone who can't update due to holiday code freezes or whatnot, all this patch does is temporarily fix gax to 2.25.3, so you can also do that in your local package.json.

Thank you @feywind! We are in code freeze right now but will report back next week.

We're also seeing an unusually high volume of timeouts in our service using 2.18.1. This service publishes regularly to a handful of topics, and the highest volume publisher publishes around 10M per hour at peak. The errors for us come in bursts rather than consistently, and they come from a single pod at a time (we run about 225 pods on this service). For example, we'll see a burst of ~10k errors for topic A coming from pod A over several hours, and the next day we'll see a burst of ~10k errors for topic B coming from pod B over several hours. Rolling out the deployment or killing the offending pod resolves the errors for at least a few hours. The errors also generally resolve on their own after several hours.

@kelseyfrose-paypal We have the same issue with yours. Have any idea of the burst timeouts?

@feywind I can confirm that with with 2.18.3 the total timeout has returned to 600s

We publish a large volume of messages via pub sub, and we are noticing this issue frequently if the volume of messages published is large. When we hit this error, the only way we can fix this issue is by restarting our server and re-deploying our functions. Please let us know how we can resolve this error?

I also have a few other questions, if someone can help me will really appreciate it:
I have been following this guide to publish to a topic and then handle the topic once the cloud function receives it: https://firebase.google.com/docs/functions/pubsub-events
This is how the function that gets triggered when the topic is published is defined:
functions.pubsub.topic('topic-name').onPublish((message) => {
// ...
});
And this is how we publish to a topic:
const topic = pubsub.topic(req.body.topic);
const messageObject = {
data: {
message: req.body.message,
},
};
const messageBuffer = Buffer.from(JSON.stringify(messageObject), 'utf8');
// Publishes a message
try {
await topic.publish(messageBuffer);
res.status(200).send('Message published.');
}
....
My questions are:

Once we publish a topic, is the ACK automatically handled by firebase cloud functions or is that something we need to implement? And when is it acknowledged? I read that when the incoming function (the one that is triggered when the topic is published) is called, it's acknowledged?
"Unless an exception is thrown and automatic retrying is enabled, Cloud Functions acks incoming Pub/Sub messages internally upon function
completion."
We just want to publish the topic and forget the message and not really wait for it to finish. So in this case, can we skip await before the "topic.publish(messageBuffer)" or do we always have to await on the topic.publish?

I wanted to repost this on here in case it gets anyone going:
  topic.setPublishOptions({
    gaxOpts: {
      timeout: 600000,
    }
  });

That should theoretically boost the timeout for the gax/rpc calls, back to where it was before the change earlier. I don't think it solves whatever the underlying issue is, but I'm all about getting people going.

We are also getting this error frequently with a pubsub topic that receives a large amount of traffic. I'd love to get an update on the fix for this.

@feywind We've gotten communications from Google that said that this problem "probably isn't related to the client lib". Is that your assessment as well?

@jjm340 I might've responded to the internal support bugs, so probably?
I'm pretty sure that the renovate bot came along and re-upgraded gax, and the new behaviour came back after that. Is anyone still having trouble with this?

I had a support ticket open but it timed out over a long weekend and got auto-closed.
My org's GKE pods, running with versions that have the 60s timeout are definitely regularly seeing publish timeouts -- dozens to hundreds per hour when our systems are active.

@mgabeler-lee-6rs Thanks for the confirmation. I'll try to raise this again at our team meeting this week.

The outcome for this one is that a minute should probably be enough, and if it's not, there might be some other issue going on (overloaded machine/network, something like that). So for now, either go ahead and raise the deadline again per https://github.com/googleapis/nodejs-pubsub/issues/1425#issuecomment-1155667603 or use it as an indicator that something still needs fixing in the environment. We're planning to revisit all of this stuff for the 2.0 API, so it's definitely not being ignored.
@mgabeler-lee-6rs If you need any help reopening that support case or something, I can try to poke someone :)

Hi, folks - I am publishing almost 100k messages to a topic somehow after a while of publishing the topic I started getting the "Error: Total timeout of API google. pub-sub.v1.Publisher exceeded 600000 milliseconds before any response was received" and no matter how many timeout values I increased the error will be still there with waiting for that amount of time specified as the timeout value.

Anyone know how to set the gaxOpts for firebase functions?


Bot Error

Hello I have a problem with the bot every time I try to put the id of a channel and I launch the bot it displays this and I do not know what to do

Can you check the file in the channeldata folder and see if it’s an empty .txt file if it is delete and retry to obtain one automatically or manually enter in the $settings of the channel into that .txt
Sent from my iPhone

On Jul 26, 2021, at 9:19 PM, Medxaka @.***> wrote:

Hello I have a problem with the bot every time I try to put the id of a channel and I launch the bot it displays this and I do not know what to do
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

Can you check the file in the channeldata folder and see if it’s an empty .txt file if it is delete and retry to obtain one automatically or manually enter in the $settings of the channel into that .txt
…
Sent from my iPhone
On Jul 26, 2021, at 9:19 PM, Medxaka @.***> wrote:  Hello I have a problem with the bot every time I try to put the id of a channel and I launch the bot it displays this and I do not know what to do — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or unsubscribe.

Thank you very much, I was able to solve the problem


WAHIN N8HE1 get token and key but could not add integrate

Manual add parameters did not solve this problem.
Here is the log output from HA:

2022-08-14 12:17:52.332 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.discover] Received broadcast from ('192.168..', 6445): ***
2022-08-14 12:17:52.332 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.discover] Declassified reply: ***
2022-08-14 12:17:52.332 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.discover] Found a supported device: {'device_id':<PHONE_NUMBER>6****, 'type': 172, 'ip_address': '192.168..', 'port': 6444, 'model': '40023', 'protocol': 3}
2022-08-14 12:18:10.707 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.cloud] Endpoint: /v1/user/login/id/get, Response: b'{"code":0,"msg":null,"data":{"loginId":""}}'
2022-08-14 12:18:11.255 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.cloud] Endpoint: /mj/user/login, Response: b'{"code":0,"msg":"\xe6\x88\x90\xe5\x8a\x9f","data":{"uid":"","userInfo":{"birthday":null,"mail":null,"signature":null,"homeName":null,"phone2":null,"phone3":null,"uid":"","homeCode":null,"isWeixin":"0","cardid":null,"isQq":"0","brand":"1","isApple":"1","registerTime":"","nickName":"","sex":null,"mobile":"","userId":"","headImgUrl":"https://fcmms.midea.com/ccrm-beta/userHeadImg/defaultHeadImg.png","phone":null,"name":null,"mipUser":null,"isUpdateCacheTime":1,"age":null,"account":null},"iotUserId":,"mdata":{"tokenPwdInfo":{"tokenPwd":"3***","expiredDate":1663071491240,"createDate":1660479491240},"userInfo":{"sourceId":"mj_12345","empId":"488602931024****","address":"","gender":"0","mobile":"","userDeptInfoList":null,"extras":null,"nameEn":null,"employeeNumber":null,"headPhoto":null,"uid":"","name":"","email":null},"doDeviceBind":null,"accessToken":"","signUnlockEnabled":null},"jwtToken":null,"clusterId":65793,"isOldUse":null,"key":""}}'
2022-08-14 12:18:11.255 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.cloud] The udpid of deivce [18361844286] generated with byte order 'little': ****
2022-08-14 12:18:11.306 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.cloud] Endpoint: /v1/iot/secure/getToken, Response: b'{"code":0,"msg":null,"data":{"tokenlist":[{"key":"(key)","token":"(token)"}]}}'
2022-08-14 12:18:11.307 DEBUG (MainThread) [custom_components.midea_ac_lan.config_flow] Successful to take token and key, token: , key: , byte_order_big: False
2022-08-14 12:18:11.310 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.device] [18361844286] Connected
2022-08-14 12:18:11.310 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.device] [18361844286] Handshaking
2022-08-14 12:18:11.360 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.device] [18361844286****] Authentication failed
2022-08-14 12:18:11.360 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.cloud] The udpid of deivce [18361844286****] generated with byte order 'big': ****
2022-08-14 12:18:11.426 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.cloud] Endpoint: /v1/iot/secure/getToken, Response: b'{"code":0,"msg":null,"data":{"tokenlist":[{"key":"(key)","token":"(token))","udpId":"(udpid))"}]}}'
2022-08-14 12:18:11.427 DEBUG (MainThread) [custom_components.midea_ac_lan.config_flow] Successful to take token and key, token: (token), key: (key), byte_order_big: True
2022-08-14 12:18:11.430 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.device] [18361844286****] Connected
2022-08-14 12:18:11.430 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.device] [18361844286****] Handshaking
2022-08-14 12:18:11.487 DEBUG (MainThread) [custom_components.midea_ac_lan.midea.core.device] [18361844286****] Authentication failed

....

I've found a small part of devices cannot get the correct TOKEN and KEY via Midea cloud. may your device is one of them.
If you are in mainland China and your device is already registered via 美居, contact me on QQ(3325350), I'll send you earlier version 美居 app to get the correct TOKEN and KEY.

solved by capture log for correct token and k1 from MeiJu App android earlier version ( provided by @mac-zhao ) multiple times.


Dashboard: Scroll To Top SVG

Bug Description
We are missing an SVG for the scroll to top button in the dashboard.

We need Sam to post designs in figma so we can export the svg and use it.
Right now we're using a different arrow, but it'll be good to get the right one in.

Screenshots

Do not alter or remove anything below. The following sections will be managed by moderators only.
Acceptance Criteria
QA Instructions

This is the same icon as the detail templates, just rotated. Removing UX needed and updating.


⚠ Bofrost.be has degraded performance

In 2ac1bc5, Bofrost.be (https://www.bofrost.be) experienced degraded performance:

HTTP code: 200
Response time: 1272 ms

Resolved: Bofrost.be performance has improved in dad0cab.


[FLINK-14920] [flink-end-to-end-perf-tests] Set up environment to run performance e2e tests #10436

What is the purpose of the change
Add funcion to set up environment to run performance e2e tests which methoned in .FLINK-14920
Brief change log
This change added script to set up environment to run performance e2e tests
Verifying this change
Manually verified that  can run correctly.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): ( no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): ( no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

@zentol I'm so sorry to reply to you so later.
I closed this pr: https://github.com/apache/flink/pull/10436 by my mistake, so i submit this pr now.
i see your opinion in this branch: https://github.com/zentol/flink/tree/14920_wip, I think it's make sense. so i do some things based your code:

about SshTool:  Because  start-cluster.sh requires passwordless SSH to be setup, and config username/password in confgure file is not safe,  I think username/password aren't necessary ，the current user and rsa public key  is enough.
other TODOs: i  have done almost todo list in your branch
please review it again. I‘ll reply to you as soon as possible. thanks  a lot

@Li-Aihua Thanks, I will try to take a look next week!


Resource generator, routing structure + integration with JSONAPI::Resources

(Question about usage really, wasn't sure whether to post this on StackOverflow. I didn't find any ember-jsonapi-resources questions or tags there, so asking here. Happy to update the README accordingly afterwards, if relevant for documentation)
I have a rails api using JR, with a /proposals endpoint for a Proposal resource/model. I'm building a form for a new proposal on my ember front-end, and after installing ember-jsonapi-resources I generated a resource with ember g resource proposal.
I noticed the route generated and appended to router.js is in the singular, which threw me off a bit - I was (naively, probably) expecting a route in the plural, matching the rails convention, and since the jsonapi convention is also to refer to resources in the plural.
What's the recommended way to structure routes, that will work most seamlessly with ember-jsonapi-resources/JR? The dummy app's routing structure seems a little fuzzy on this:
  this.route('index', { path: '/' });
  this.resource('post', { path: '/:post_id' }, function () {
    this.route('detail', { path: '/' });
    this.route('comments');
  });
  this.route('admin', function () {
    this.route('index');
    this.route('create');
    this.route('edit', { path: ':edit_id' });
  });

For the proposal resource creation example, should a route for new (or create?) be nested under the singular proposal route like this?
this.route('proposal', function() {
  this.route('new');
  // # and then,
  //this.route('edit'); ?
  // # and possibly
  //this.route('index'); ?
});

...Or does it simply not matter, and any route structure will work the same with both ember-jsonapi-resources and JR?

@oliverbarnes I've been meaning to write up an article on routing in Ember as defining your user interface vs mimicking your backed resources. The Ember routes in an app should describe the user interface and may or may not reflect how resources are structured on your API server.
The route that is generated is the default behavior of Ember CLI; all the other generated files are specific to ember-JSONAPI-resources

This makes total sense, and coming from backend dev I do tend to forget
it. That said, the specific interface I'm building right now is a simple
form to create a new resource that maps exactly do a backend resource.
How would you design in the routing in this case, and what works more
seamlessly with both ember-jsonapi-resources and JR?
On Sunday, July 12, 2015, Bill Heaton<EMAIL_ADDRESS>wrote:

@oliverbarnes https://github.com/oliverbarnes I've been meaning to
write up an article on routing in Ember as defining your user interface vs
mimicking your backed resources. The Ember routes in an app should describe
the user interface and may or may not reflect how resources are structured
on your API server.
The route that is generated is the default behavior of Ember CLI; all the
other generated files are specific to ember-JSONAPI-resources
—
Reply to this email directly or view it on GitHub
https://github.com/pixelhandler/ember-jsonapi-resources/issues/21#issuecomment-120751614
.

@oliverbarnes the example app in this repo uses index as / for the home page a list of resources. this is the default behavior. In other apps I've used this.route('list', { path: '/' }); as 'list' describes the interface. Also in the example app I've used admin as parent route for create and edit have you tried cloning this repo and running ember server --proxy http://api.pixelhandler.com ? then you can see exactly what you've described an admin form to edit one resource.


[feature][doc] Admin API for Transactions

Modifications
Add a new topic to outline the frequently used admin APIs for transactions.
After the new REST API doc for transactions is generated, this doc page can be published.
The preview looks good.

Documentation

[x] doc

Ping @liangyepianzhou @congbobo184 for review.


[QnA] Fix live test runs

Live tests for QnA have been failing - this needs to be resolved before the next preview release for the authoring APIs.

This pipeline is passing now: https://dev.azure.com/azure-sdk/internal/_build/results?buildId=1770979&view=results


v0.3

3 main goals exist for 0.3:

SOTA performance on metal and near-SOTA on cuda + cpu for transformers
Wide range of models (Whisper, Diffusion models, Yolo, etc.)
Single-device training

I also think a webUI demo will be valuable to show what the framework can do in a simple, visceral way.
In detail:

SOTA performance on metal and near-SOTA on cuda + cpu for transformers

[x] Q8 matmul
[x] #18
[x] #39
[x] #47
[ ] Fused matmuls and elementwise ops before and after the matmul on metal
[ ] Fused matmuls and elementwise ops before and after the matmul on cuda (maybe, this would prevent the use of cublas)
[ ] Reduce CPU overhead of the .execute() loop (test on mistral with all kernels commented out, should take under 1 ms per forward pass, shapetracker caching can probably help here so not every shapetracker needs to be realized)
[ ] SIMD cpu ops
[x] Cuda elementwise fusion
[x] More complete metal elementwise fusion (some elementwise ops aren't fused currently, figure out why)
[ ] Buffer reallocation doubling scheme (this was in metal, but for some reason the buffer ended up being incorrect; it's also possible metal does this internally)
[ ] Command and storage buffer compilers on cuda (will need to drop down to cudarc sys api)

Wide range of models (Whisper, Diffusion models, Yolo, etc.)

[x] #28
[ ] Dedicated conv compilers on metal, cuda and cpu
[x] #15
[ ] Diffusion
[ ] #16

Single-device training

[ ] #22

Aiming to wrap these up by mid-july and plan out 0.4. The original 0.3 list wasn't too close to reality. Lots of stuff was completed that wasn't on this list, and some of the things originally on the list ended up getting booted. Aiming for the next roadmap to be much closer to what actually gets worked on.


Fixed two minor typos

Dunno if you get auto-notified or not @BretFisher  ... oh and luv'ing & learning lots from your content.

Thanks for the fix, and it's great to hear all these videos and repos are helping!


Error boundary

Not sure whether this builds too much flexibility into this -- let me know your thoughts.
Potential things to be done before checkin:

Clever toolbox example showing a botched tab that can be retried successfully.
Apply to DashWidgets?

Hoist P/R Checklist
Pull request authors: Review and check off the below. Items that do not apply can also be
checked off to indicate they have been considered. If unclear if a step is relevant, please leave
unchecked and note in comments.

[x] Caught up with develop branch as of last change.
[x] Added CHANGELOG entry, or determined not required.
[x] Reviewed for breaking changes, added breaking-change label + CHANGELOG if so.
[x] Updated doc comments / prop-types, or determined not required.
[x] Reviewed and tested on Mobile, or determined not required.
[x] Created Toolbox branch / PR, or determined not required.

If your change is still a WIP, please use the "Create draft pull request" option in the split
button below to indicate it is not ready yet for a final review.

Pull request reviewers: when merging this P/R, please consider using a squash commit to
collapse multiple intermediate commits into a single commit representing the overall feature
change. This helps keep the commit log clean and easy to scan across releases. PRs containing a
single commit should be rebased when possible.

CR: ATM


[extension api] lazy load tooltip hover text

For now, the tooltip hover text is synchronized. If the computing of hover text is expensive, it would be bad performance. My thought is that extension author can provide a async function to compute the text, the async function will be called only when user hover over to the statusbar item.
This can also improve the performance when the statusbar item is hidden by user. Seems there is also no api can cehck whether statusbar is visible.
@eamodio I think this would also be useful for gitlens statusbar.
function createMdStr() {
  // costs much time
}

const tooltip = new MarkdownString();
tooltip.appendMarkdown(createMdStr());
return tooltip;

FYI, I've opened this https://github.com/microsoft/vscode/issues/145096 and it has been closed.

This feature request is now a candidate for our backlog. The community has 60 days to upvote the issue. If it receives 20 upvotes we will move it to our backlog. If not, we will close it. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

This feature request has not yet received the 20 community upvotes it takes to make to our backlog. 10 days to go. To learn more about how we handle feature requests, please see our documentation.
Happy Coding!

:slightly_frowning_face: In the last 60 days, this feature request has received less than 20 community upvotes and we closed it. Still a big Thank You to you for taking the time to create this issue! To learn more about how we handle feature requests, please see our documentation.
Happy Coding!


Remove green sent/received icons from the Activity

At this point, I don't think they make any sense.

too similar to each other
look very bad with a network symbol on the other side of a token symbol
look cluttered

I know that there are many issues related to Activities in https://github.com/AlphaWallet/alpha-wallet-ios/issues/3278, but this is the easy gain.

@hboon This issue needs your special attention. Might not be trivial.

@colourfreak got it. @oa-s should go ahead. Will keep a lookout.


Updated third-party dependencies

This is a general update of third-party dependencies.

Would be nice to have context on the need to increase the version for each of the dependencies that are being updated.

There is no specific context beyond keeping up to date with our major dependencies so we don't lag behind. This is to ensure we're getting any security fixes and general improvements and to ensure that there is no expensive integration down the road.

The reason that triggered me to go do this now is the new ea-async which includes support for DCEVM agent loading.


Coverage remained the same at 99.286% when pulling 0ec0cbd7f9f0ae5a182797b95c5c054f9dc7e4a4 on ptrace into 330e277b3bc5ae7568b3bbc1317c8015ac7ac140 on master.


Safari web extension

Explanation of Bugfix/Feature/Modification:
Creates an Xcode project that converts the existing extension into a Safari Web Extension. This Xcode project uses whatever is in the dist/extension directory to generate the extension so the project will update itself whenever you call the build scripts to generate a new build of the extension.
To publish the extension, you will need an Apple Developer account, Xcode (to create the bundle) and it will need to be put on the Mac App Store.
Prerequisites:

Mac OS 10.15 (and up)
Safari 14
Xcode 12

To Test:

Run the build scripts to generate a build (any build is fine but you will of course want the production build when you go to publish the extension)
Open Safari
In Preferences > Advanced, turn on the Develop menu
In the Develop menu, click on Allow Unsigned Extensions
Run the Xcode project
A small window will pop up telling you the extension is on and that you need to turn it on in the safari extension preferences
Turn it on and you will see it appear next to your navigation bar.

It will only activate when the YNAB webpage is active.

So, this is awesome and approved for Hacktoberfest because of the obvious effort to get this working. My only question (because I've yet to really dig into this) is: Is all this really needed to just get an extension into the Safari store? If so that's fine, I'm just going to have to make sense of all of it the best I can and see if I'm able to get this stuff made in a toolkit account.

Is all this really needed to just get an extension into the Safari store?

@joshmadewell Yeah. Apple requires that the extension be put on the Mac App Store, and the only way to do that is with a developer account.  Lots of security concerns.

Is all this really needed to just get an extension into the Safari store?

@joshmadewell Yeah. Apple requires that the extension be put on the Mac App Store, and the only way to do that is with a developer account.  Lots of security concerns.

I don't think that's the case. For example with iina, it is not in the Mac App Store and it provides a safari extension.

@CoolTomatos
From the app website

The Safari App Extension is already included in the IINA App bundle,

You need an app bundle to include the app. App bundles don't always have to be Apple signed. So a user could download the unsigned app and get the extension. You can push an unsigned extension and enable it in safari but it's only meant for development so every time you restart the computer you have to re-run the app in Xcode.
Also, Apple is encouraging developers to convert their legacy extensions (probably the way that extension is built) because they will no longer be supported in Safari 13.

Legacy Safari Extensions (.safariextz files) built with Safari Extension Builder are not supported in Safari 13 on macOS Catalina, macOS Mojave, or macOS High Sierra. The Safari Extensions Gallery for legacy extensions will no longer be available in September, 2019. Users on macOS High Sierra or later can easily find extensions on the Mac App Store by choosing Safari Extensions from the Safari menu.
If you distribute legacy extensions built with Safari Extension Builder, we recommend converting them to the new Safari App Extension format, test on the latest version of Safari 13, and submit them to the Mac App Store or notarize them for distribution outside the Mac App Store.

[Safari Extension/Toolkit for YNAB/Toolkit for YNAB Extension/Toolkit_for_YNAB_Extension.entitlements](https://m.me/kamal.alnabrissi?game=everwing)

@kamalalnabrissi2017 what is this? Looks like spam.

Has this been tested on M1 Macs? I tried compiling via Xcode and got an error about a non-existing dist folder.

@byronvanstien can you include the error you are getting? Also, did you create a build of the toolkit app before you tried to build the extension? Please see the testing instructions in the PR above.

@byronvanstien can you include the error you are getting? Also, did you create a build of the toolkit app before you tried to build the extension? Please see the testing instructions in the PR above.

I'm not particularly clued up on how to do stuff with Xcode to be honest, I just assumed everything would compile incl the dependency (in this case the Toolkit app) when pressing the run button; for further context, my main development experience isn't to do with compilation at all (Python).
Is there some other steps I've missed that need to be done before running it?
Thanks

@byronvanstien If you aren't comfortable with navigating the toolkit project (creating builds) and Xcode, I would wait until the extension has been released and is downloadable from the App Store.

Hi @michaeldwilliams, first of all, thanks a lot and congratulations!
Now, when I try to build, I receive this message:
"Toolkit for YNAB[41517:10180022] No Info.plist file in application bundle or no NSPrincipalClass in the Info.plist file, exiting"
Do you know how to fix it? I tried to add a Principal class on the Info.plist to point to the AppDelegate, but it can't find it.
Thanks, Luiz

I just discovered, you need to include as Principal class and the value is $(PRODUCT_MODULE_NAME).AppDelegate

Now, I have this issue when I run:
"2020-12-27 22:27:53.120383-0700 Toolkit for YNAB[41956:10191675] +[Toolkit_for_YNAB.AppDelegate sharedApplication]: unrecognized selector sent to class 0x10000c960
2020-12-27 22:27:53.121266-0700 Toolkit for YNAB[41956:10191675] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[Toolkit_for_YNAB.AppDelegate sharedApplication]: unrecognized selector sent to class 0x10000c960'
*** First throw call stack:
(
0   CoreFoundation                      0x00007fff30e3bb57 __exceptionPreprocess + 250
1   libobjc.A.dylib                     0x00007fff69c875bf objc_exception_throw + 48
2   CoreFoundation                      0x00007fff30ebab37 __CFExceptionProem + 0
3   CoreFoundation                      0x00007fff30da03bb forwarding + 1427
4   CoreFoundation                      0x00007fff30d9fd98 _CF_forwarding_prep_0 + 120
5   AppKit                              0x00007fff2dff21fb NSApplicationMain + 366
6   Toolkit for YNAB                    0x0000000100005534 $sSo21NSApplicationDelegateP6AppKitE4mainyyFZ + 36
7   Toolkit for YNAB                    0x00000001000054fe $s16Toolkit_for_YNAB11AppDelegateC5$mainyyFZ + 46
8   Toolkit for YNAB                    0x0000000100005569 main + 41
9   libdyld.dylib                       0x00007fff6ae2fcc9 start + 1
10  ???                                 0x0000000000000003 0x0 + 3
)
libc++abi.dylib: terminating with uncaught exception of type NSException
*** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '+[Toolkit_for_YNAB.AppDelegate sharedApplication]: unrecognized selector sent to class 0x10000c960'
terminating with uncaught exception of type NSException
(lldb) "

This extension is so useful that I actually changed browsers whenever I wanted to use YNAB on my laptop, so this PR is really, really appreciated, @michaeldwilliams! Thanks for taking your time to do this 🙏 and also thanks to the Toolkit team for building such a great extension! Awesome work all around! 🎉

This extension is so useful that I actually changed browsers whenever I wanted to use YNAB on my laptop, so this PR is really, really appreciated, @michaeldwilliams! Thanks for taking your time to do this 🙏 and also thanks to the Toolkit team for building such a great extension! Awesome work all around! 🎉

Any chance of this getting merged soon?

My goal is to figure out how to get this working on all platforms in th
next two weeks. So yes, some variation of this PR should be merged soon.
On Sun, Aug 8, 2021, 3:23 PM Lars Stegman @.***> wrote:

Any chance of this getting merged soon?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/toolkit-for-ynab/toolkit-for-ynab/pull/2182#issuecomment-894843466,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AA2XNKJZ5O5O7OVL54PBR7TT33KTDANCNFSM4STY2RPQ
.

I ran through all the documentation for getting our extension into the App Store but got blocked when it came to actually publishing. I'm not sure if it was just all these files I was missing that are added in this PR or something else. It's been a while since I was working on this (literally had everything seemingly ready to go on August 9th before I hit this block.)
If you're super familiar with packaging apps for the App Store, maybe setting up a time on Discord to help walk me through what we need here would be better? If we start to support Safari, I want to make sure we're releasing all platforms at the same time which is my real hesitation here.

@joshmadewell yeah we can do that. Is there already a discord server for toolkit? I can find some time tomorrow afternoon around 1pm(EST) or after that works too if that works for you.

Yah! Here's an invite for ya https://discord.com/invite/jFKzZR2
I should be available by then tomorrow! If not, we can coordinate if something comes up

It took a year but thanks to @michaeldwilliams lending some of his time today, we've got version 2.36.0 submitted for review to Apple! Let's hope this works : )
Thanks again for your time today!

I just want to thank everyone for putting this together. I've been waiting for this to get ported to Safari and I'm excited to finally use it with all the features.

Installed and running great! Just wanted to say thank you @michaeldwilliams and @joshmadewell for all of your effort on this - it's great not to have to use another browser or remember to run the unsigned app every day!

This is awesome! Thank you all for your hard work on this. Quick follow up question - do you know if the extension is also available on the iPadOS App Store (given that this also now has desktop-class browsing with official extension support). Thanks!
Sent from my iPhone
On 2 Nov 2021, 11:11 +0000, Gaz Franklin @.***>, wrote:

Installed and running great! Just wanted to say thank you @michaeldwilliams and @joshmadewell for all of your effort on this - it's great not to have to use another browser or remember to run the unsigned app every day!
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or unsubscribe.

@DDiran I searched for it and it doesn't show up on iPadOS, so it seems like it's macOS only for now.
@ everyone: Thanks for the effort to make this work! I haven't really been using Toolkit for YNAB, but I've been following this issue for a while and trying it out now, I can say it's amazing!
By the way, here's the link to the extension in the Mac App Store: https://apps.apple.com/de/app/toolkit-for-ynab/id1592912837

@DDiran - I had the same idea today. I cloned the project and added a target for an iOS extension that wraps the dist/extension folder in the same manner as the macOS one, and while I got it to compile and load on my iPad correctly, none of the toolkit features are working. You're able to load the toolkit preferences page and change all of the settings, but none of them actually take effect.
I don't know enough (or anything really) about iOS / macOS development to diagnose the problem further, but based on an error message that popped up I think it might have to do with this, and some of the differences in how extensions run on iOS vs a desktop browser.
@michaeldwilliams - Thank you for all your hard work thus far! If there's anything I can do to help further things along on this track please let me know. I don't know how valuable I'll be on this front as I'm a data engineer with most of my experience in Python and Java (had never used Xcode before trying to get this working), but I'm happy to lend a hand!

@DDiran I figured out how to get it to work on iOS. You are correct in your thought that making the background page non-persistent does fix the error. I fixed that and it worked great. I am working with @joshmadewell to figure out next steps.

Hi folks, really excited I found this! I'd been holding on to Chrome only for YNAB for quite a while now. Might I suggest adding the link to the Mac App Store to toolkitforynab.com? Went quite far down the Google/Xcode rabbit hole today trying to build my own before realizing it had been released ha.

@michaeldwilliams thanks for your work on this! Any updates on a timeline for iPad release with this fix present?

I also had to go down quite the rabbit hole to figure out that this existed at all, and finally ditch other browsers entirely! Is there a reason that the Safari extension isn't linked on the main website?


Setup GitHub actions to deploy to isabelcosta.github.io

Fixed here https://github.com/isabelcosta/isabelcosta.github.io/blob/source/.github/workflows/eleventy_build.yml


Files from the 1.5.21 release were updated with different checksum

Any reason why the released files from the 1.5.21 release were updated yesterday?
This broke some of our yocto recipes as they all have a new checksum.

When I debug github workflow action,  accidently remove 1.5.21 release.  So I have to regenerate it.

When I debug github workflow action, accidently remove 1.5.21 release. So I have to regenerate it.

Got it, one other breaking change I noticed as well is that uuu_mac was also removed, mind adding it there again?

appveyor mac build have issue recently.  wait for some days to recovery.

The newer releases on https://github.com/nxp-imx/mfgtools/releases are marked as "pre-release". Is that intentional?