Round out --use scenario (add "use" sub-command, add local file, add .bacpac / .mdf & .ldf / compressed file support)

This PR rounds out the "use" scenario:

--use now supports local and remote files
sqlcmd create mssql --use [<local file>|<remote file>]

The T/SQL database name by default becomes the --use file name (without extension).  This can be overridden with a ",", e.g.
sqlcmd create mssql --use https://aka.ms/AdventureWorksLT.bak,adventure_works

This will result in the restored database being named [adventure_works] instead of [AdventureWorksLT]

--use now supports .mdf/.bacpac and .bak files
sqlcmd create mssql --use [.mdf | .bacpac | *.bak]

--use now supports compressed file formats (this also allows a .mdf/.ldf file pair)
sqlcmd create mssql --use [.7z | .zip | .tar]  # containing any of the above (inc. a .mdf/.ldf pair)

the new sqlcmd use subcommand allows all of the above, to add a database to an already existing container
sqlcmd use [<local file>|<remote file>]

This PR also enables the open sub-command e.g. sqlcmd open ads to be leveraged directly in sqlcmd create, e.g.
sqlcmd create mssql --use https://aka.ms/AdventureWorksLT.bak --open ads --open-file https://aka.ms/AdventureWorksLT.sql

This enables a T/SQL repro (including database and T/SQL script file) to be shared in a single command line
Examples:
sqlcmd install mssql --accept-eula

# Restore a backup (override database name to be [adventure_works], instead of [AdventureWorksLT])
sqlcmd use https://aka.ms/AdventureWorksLT.bak,adventure_works
sqlcmd use c:\Users\alias\Downloads\AdventureWorksLT.bak,adventure_works

# Restore a Dacfx .bacpac
sqlcmd use https://github.com/Microsoft/sql-server-samples/releases/download/wide-world-importers-v1.0/WideWorldImportersDW-Full.bacpac

# Attach a compressed .mdf/.ldf
sqlcmd use https://downloads.brentozar.com/StackOverflow2010.7z

What about supporting non-public URLs (SAS etc)? Does it download the file then discover its type similar to how you'd have to examine a zip to know its content is?

Could you create a couple mermaid diagrams? One would show the relationship between components like ingest and extract and container, and one would show how the contents of the bak/ldf/dacpac file would move from their source to the database for each environment we support (sql in a container, sql on localhost, sql on a remote host, sql in the cloud).
a == label ==> b

What about supporting non-public URLs (SAS etc)? Does it download the file then discover its type similar to how you'd have to examine a zip to know its content?

Doesn't the non-pupil SAS URL contain the filename still (before the ?), e.g.:
https://blahblah.blob.core.windows.net/database/AdventureWorks.bak?sv=2021-10-04&st=2023-04-07T06%3A12%3A15Z&se=2023-04-08T06%3A12%3A15Z&sr=b&sp=r&sig=<REDACTED>

Are there URL types that don't container the filename (e.g .bak or .7z?)

A URL can be anything.  eg "aka.ms/ssmsfullsetup" is a pointer to an EXE.

In reply to:<PHONE_NUMBER>

This will work.  If the mechanism can't be determined from the file extension, then the --use-mechanism flag can be used (restore, dacfx, attach etc.).  I'll write unit tests around all these scenarios

In reply to:<PHONE_NUMBER>

sure

In reply to:<PHONE_NUMBER>

There is a --use-mechanism flag, that is required if we can't determine the mechanism from the file extension.

In reply to:<PHONE_NUMBER>


Add coq-tools / coq-bug-minimizer to the platform?

Does it make sense to add some version of the bug minimizer to the platform?  It's a collection of Python scripts (currently in the process of setting up a pip package) but I'm happy to set up some sort of standalone binary distribution if that'd be better for the platform.  (I don't think it makes sense to distribute on opam, though I'd be down to create an opam package that just builds the binary and installs it or something, depending on conf-python-3.7)

Now on PyPi (the documentation needs some work though).  And the CI builds standalone executables for Mac, Linux, and Windows across a variety of Python versions, which I can get the CI to add to releases or something if desired.
Let me know what needs to be done.

Releases on GitHub now contain standalone executables


fix: bug charmbracelet/lipgloss#236

add @prestidigitator-mt's test cases with some changes, mostly adding testcases for the center position.

Go's Round function rounds half away from zero, as a result of this, when the spacing is odd, the odd space goes towards the left/top instead of towards the right/bottom as before:
Before:

After:

Please let me know if this is acceptable, otherwise I can update the PR

I updated the PR to keep the odd spacing behavior


Vulnerability patch

Changelog

🛡 Patch vulnerability CVE-2023-26159 on follow-redirects

:tada: This PR is included in version 5.0.5 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


How to Use UFO to compile a project

Hi, I want to use UFO to test some open source softwares, but how can I compile them with UFO?
the usage example is only a single test file, could you please give some more examples ?

Exact usage will of course vary from project to project, but I can speak specifically for running UFO on the Google Chrome web browser. I will describe the instruction sequence for compiling Chrome with UFO, and hopefully you should be able modify the process to compile other open-source software.
(Note: this is not the complete build sequence. If you wish for more specifics/help, let me know here or email me at<EMAIL_ADDRESS>or simply refer to the Chromium documentation)
First, follow the steps at https://chromium.googlesource.com/chromium/src/+/lkcr/docs/linux_build_instructions.md/#setting-up-the-build to get the Chrome source code and set up for the build. Then, follow the instructions in this project to build LLVM containing the tsan outfitted with UFO. At this point, you should have /$your_llvm_build_dir$/bin/clang. Chrome uses GN and Ninja to build (you can research more into their build system independently) and allows a user to specify a custom clang binary to be used. Simply configure it to use our new clang binary by editing the args.gn file contained in the Chrome source directory (src/out/Default/) like this:
clang_base_path="/$your_llvm_build_dir$"
clang_use_chrome_plugins=false
is_tsan=true
enable_nacl=false
is_debug=false

After that, follow the documentation's steps to build the project. Then you can trace program execution by running chrome with the UFO flags. Ex.
UFO_ON=1 UFO_CALL=1 UFO_TL_BUF=512 ./src/out/Default/chrome --no-sandbox

Hope this helps.
Also, keep in mind that this tool is still in development. When I run it on Chrome it will crash after about 30 seconds. We are working to make it stable and more intuitive

Thanks, hope for your stable version.


invalid json file no such file or directory

I've been wrestling with this for hours trying every tutorial I could find to get a kubernetes cluster to work with this, including the official docs and I'm consistently getting the error:
invalid json file "/secrets/cloudsql/my-app-sa-key.json": open /var/run/secret/cloud.google.com/my-app-sa-key.json: no such file or directory.
Here's my yaml configuration (This is the full file verbatim, nothing changed!)
apiVersion: apps/v1beta2
kind: Deployment
metadata:
  name: loderunner-sql-proxy
  labels:
    sql-proxy: sql-proxy
spec:
  replicas: 1
  selector:
    matchLabels:
      sql-proxy: sql-proxy
  template:
    metadata:
      labels:
        sql-proxy: sql-proxy
    spec:
      volumes:
        - name: "my-app-sa-key"
          secret:
            secretName: "my-app-sa-key"
      containers:
        - name: cloudsql-proxy
          image: gcr.io/cloudsql-docker/gce-proxy:1.14
          command: [
              "/cloud_sql_proxy",
              "-instances=trezure-crawl:us-central1:pg1=tcp:3306",
              # If running on a VPC, the Cloud SQL proxy can connect via Private IP. See:
              # https://cloud.google.com/sql/docs/mysql/private-ip for more info.
              # "-ip_address_types=PRIVATE",
              "-credential_file=/secrets/cloudsql/my-app-sa-key.json",
            ]
          securityContext:
            runAsUser: 2 # non-root user
            allowPrivilegeEscalation: false
          volumeMounts:
            - name: "my-app-sa-key"
              mountPath: "/secrets/cloudsql"
              readOnly: true

I went and created a service account called sql-account with Cloud SQL Client privileges.
Steps:

gcloud iam service-accounts keys create --iam-account<EMAIL_ADDRESS>service-account.json
kubectl create secret generic my-app-sa-key --from-file service-account.json
kubectl create secret generic my-app-sa-key --from-file service-account.json
kubectl get secret

NAME                  TYPE                                  DATA   AGE
default-token-2g7q8   kubernetes.io/service-account-token   3      4h7m
my-app-sa-key         Opaque                                1      25m

kubectl apply -f ./sql-proxy-deployment.yaml

deployment.apps/loderunner-sql-proxy created

Name:               loderunner-sql-proxy-5c5f6cbb7c-9skb6
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               gke-standard-cluster-1-default-pool-aba0624a-3z33/<IP_ADDRESS>
Start Time:         Sat, 30 Nov 2019 22:18:17 -0800
Labels:             pod-template-hash=5c5f6cbb7c
                    sql-proxy=sql-proxy
Annotations:        kubernetes.io/limit-ranger: LimitRanger plugin set: cpu request for container cloudsql-proxy
Status:             Running
IP:                 <IP_ADDRESS>
Controlled By:      ReplicaSet/loderunner-sql-proxy-5c5f6cbb7c
Containers:
  cloudsql-proxy:
    Container ID:  docker://38a2e9709e1496eb32ac7d550ac6c90256fe2fe8a0c8213a8334dd5eac5cfec7
    Image:         gcr.io/cloudsql-docker/gce-proxy:1.14
    Image ID:      docker-pullable://gcr.io/cloudsql-docker/gce-proxy@sha256:96689ad665bffc521fc9ac3cbcaa90f7d543a3fc6f1c84f81e4148a22ffa66e0
    Port:          <none>
    Host Port:     <none>
    Command:
      /cloud_sql_proxy
      -instances=trezure-crawl:us-central1:pg1=tcp:3306
      -credential_file=/secrets/cloudsql/my-app-sa-key.json
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Error
      Exit Code:    1
      Started:      Sat, 30 Nov 2019 22:18:34 -0800
      Finished:     Sat, 30 Nov 2019 22:18:34 -0800
    Ready:          False
    Restart Count:  2
    Requests:
      cpu:        100m
    Environment:  <none>
    Mounts:
      /secrets/cloudsql from my-app-sa-key (ro)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-2g7q8 (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             False
  ContainersReady   False
  PodScheduled      True
Volumes:
  my-app-sa-key:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  my-app-sa-key
    Optional:    false
  default-token-2g7q8:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-2g7q8
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                From                                                        Message
  ----     ------     ----               ----                                                        -------
  Normal   Scheduled  27s                default-scheduler                                           Successfully assigned default/loderunner-sql-proxy-5c5f6cbb7c-9skb6 to gke-standard-cluster-1-default-pool-aba0624a-3z33
  Normal   Pulled     10s (x3 over 27s)  kubelet, gke-standard-cluster-1-default-pool-aba0624a-3z33  Container image "gcr.io/cloudsql-docker/gce-proxy:1.14" already present on machine
  Normal   Created    10s (x3 over 26s)  kubelet, gke-standard-cluster-1-default-pool-aba0624a-3z33  Created container
  Normal   Started    10s (x3 over 26s)  kubelet, gke-standard-cluster-1-default-pool-aba0624a-3z33  Started container
  Warning  BackOff    9s (x4 over 24s)   kubelet, gke-standard-cluster-1-default-pool-aba0624a-3z33  Back-off restarting failed container

kubectl logs loderunner-sql-proxy-5c5f6cbb7c-9skb6
2019/12/01 06:19:04 current FDs rlimit set to 1048576, wanted limit is 8500. Nothing to do here.
2019/12/01 06:19:04 invalid json file "/secrets/cloudsql/my-app-sa-key.json": open /secrets/cloudsql/my-app-sa-key.json: no such file or directory

No matter what I do it always gives this error. If I try to provide a bogus secret then it can't even get as far as creating the pod because it says the secret doesn't exist, which means that while kubernetes is finding the secret, cloudsql proxy isn't finding it for some reason.
I tried numerous variations of this. I tried using a different secret that was generated using the postgres literal credentials rather than using a service account. Same problem, the image isn't able to find the file.
Thank you for your patience and help

Just taking a quick look, but I think your problem is you've declared the secret using the filename service-account.json (IIRC, k8s uses the name of the file if no other name is provided).
Try doing this instead:
kubectl create secret generic my-app-sa-key --from-file=my-app-sa-key.json=service-account.json

There is also something funky going on because the first error you posted refers to  /var/run/secret/cloud.google.com/my-app-sa-key.json. There doesn't seem to be a reason for the proxy to be reading from that directory unless you modified your deployment.yaml from what you've posted above.

Correct, that was an error from an earlier configuration setting. As you might imagine it was the same thing regardless of the mounted directory.
Thank you so much! That worked :)


Building markdown failed while validating input.

I tried to compile the markdown example, but unfortunately compilation fails.
$ wasm-pack --version
wasm-pack 0.9.1
$ cargo --version
cargo 1.41.0 (626f0f40e 2019-12-03)

Here are the steps to reproduce. Not that I didn't copy full logs:
$ wasm-pack build
[INFO]: Checking for the Wasm target...
[INFO]: Checking for the Wasm target...
[INFO]: Compiling to Wasm...
[INFO]: Compiling to Wasm...
    Finished    Finished release [optimized] target(s) in 0.02s
 release [optimized] target(s) in 0.02s
[INFO]: Installing wasm-bindgen...
[INFO]: Installing wasm-bindgen...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[INFO]: Optimizing wasm binaries with `wasm-opt`...
[wasm-validator error in function render\20multivalue\20shim] (i32, i32) != i32: block with value and last element with value must match types, on
[(i32, i32)] [(i32, i32)] (block (result i32 i32)
 [none] (global.set $global$0
  [i32] (local.tee $2
   [i32] (i32.sub
    [i32] (global.get $global$0)
    [i32] (i32.const 16)
   )
  )
 )
 [none] (call $render
  [i32] (local.get $2)
  [i32] (local.get $0)
  [i32] (local.get $1)
 )
 [none] (local.set $3
  [i32] (i32.load
   [i32] (local.get $2)
  )
 )
 [none] (drop
  [i32] (i32.load offset=4
   [i32] (local.get $2)
  )
 )
 [none] (global.set $global$0
  [i32] (i32.add
   [i32] (local.get $2)
   [i32] (i32.const 16)
  )
 )
 [i32] (local.get $3)
) ;; end block
[...]
 (func $<T\20as\20core::any::Any>::type_id::ha8033cc8aaed9e29 (; 212 ;) (param $0 i32) (result i64)
  (i64.const<PHONE_NUMBER>891580772)
 )
 (func $<T\20as\20core::any::Any>::type_id::h26c6ec07af71e373 (; 213 ;) (param $0 i32) (result i64)
  (i64.const -225553400743110688)
 )
 (func $<T\20as\20core::any::Any>::type_id::hf03814fcf300737e (; 214 ;) (param $0 i32) (result i64)
  (i64.const<PHONE_NUMBER>228188672)
 )
 (func $std::process::abort::h83667d4dc61f8ff4 (; 215 ;)
  (unreachable)
 )
 (func $__rust_start_panic (; 216 ;) (param $0 i32) (result i32)
  (unreachable)
 )
 (func $dlmalloc::dlmalloc::Dlmalloc::malloc_alignment::hcee1c950df17a4f7 (; 217 ;) (param $0 i32) (result i32)
  (i32.const 8)
 )
 (func $<T\20as\20core::any::Any>::type_id::hcbe1e308a536e760 (; 218 ;) (param $0 i32) (result i64)
  (i64.const -225553400743110688)
 )
 (func $core::panic::Location::caller::h4586dda5eb624bd0 (; 219 ;) (param $0 i32) (result i32)
  (local.get $0)
 )
 (func $core::ptr::drop_in_place::hdf3ac4f0aee1b486 (; 220 ;) (param $0 i32)
 )
 (func $core::ptr::drop_in_place::ha08c4b3d9252f877 (; 221 ;) (param $0 i32)
 )
 (func $core::ptr::drop_in_place::he9358e90f45c44df (; 222 ;) (param $0 i32)
 )
 (func $core::ptr::drop_in_place::h01349a483699dc28 (; 223 ;) (param $0 i32)
 )
 (func $core::ptr::drop_in_place::h0293f9ecfaa56c65 (; 224 ;) (param $0 i32)
 )
 (func $core::ptr::drop_in_place::h86bf89378777bea4 (; 225 ;) (param $0 i32)
 )
 (func $core::ptr::drop_in_place::h3a3c6f3f877fe4f5 (; 226 ;) (param $0 i32)
 )
 (func $core::ptr::drop_in_place::h08414e5d2954751c (; 227 ;) (param $0 i32)
 )
 (func $std::alloc::default_alloc_error_hook::h9f234f4112b0e55e (; 228 ;) (param $0 i32) (param $1 i32)
 )
 (func $core::ptr::drop_in_place::h04478f7f402db464 (; 229 ;) (param $0 i32)
 )
 ;; custom section "producers", size 121
 ;; custom section "webidl-bindings", size 50
)
Fatal: error in validating input
Error: failed to execute `wasm-opt`: exited with exit code: 1
  full command: "/home/developer/.cache/.wasm-pack/wasm-opt-4d7a65327e9363b7/wasm-opt" "/home/developer/projects/wasmtime-demos/markdown/pkg/markdown.wasm" "-o" "/home/developer/projects/wasmtime-demos/markdown/pkg/markdown.wasm-opt.wasm" "-O"
To disable `wasm-opt`, add `wasm-opt = false` to your package metadata in your `Cargo.toml`.

I've tried the suggestion and modified the Cargo.toml and added the following lines:
[package.metadata.wasm-pack.profile.dev]
wasm-opt = false

But that didn't help

I have the same issue. Except that when I add
[package.metadata.wasm-pack.profile.dev]
wasm-opt = false

I get another error:
Error: multiple tables: tables count must be at most 1 (at offset 425)

It is also worth mentioning that nothing was modified besides the part mentioned above and updated crate versions.
Similar behavior occurs when trying to port simple modules with wasm_bindgen. Could anyone provide any insights?

I'm going to now that we've temporarily removed interface types support, but stay tuned to https://github.com/bytecodealliance/wasmtime/issues/677 for future updates!


Blocked prisma2 dev process also blocks prisma2 dev in other terminal

While investigating https://github.com/prisma/prisma2/issues/641 I encountered a thing that I was told would not be possible, so I created a gif to show it:

Tab 1: I use init to create a project using MySQL and run prisma2 dev to show it usually works.
Tab 2: Then I init a project using SQLite, which then fails on prisma2 dev because of #641
Tab 3: Then I init another project with MySQL, and this time prisma2 dev also hangs - probably because of the still running but broken dev from 2)

I can't reproduce this anymore after #641 was fixed. Closing this for now, reopen if you can still reproduce this.

The underlying problem still probably exists - one hanging process still will block the other one, we currently just don't have a reproduction to block a process. But as I was told this blocking would not be possible, I think this is still valid.


No way to free thread-local variables

XGboost allocates thread-local variables but doesn't provide any way to free their memory.
This discourages the use of xgboost inside long running frameworks that launch a thread per request.
I think the main problem is that the text dumps exported via the C API are kept in the thread-local store.
Using xgboost v0.60, I tested fitting and dumping a model with 10,000 nodes 51 times, one after another, each time in a separate thread, and found the process' working set increased by about 1.3 MB per iteration, with no end to the memory increase in sight.
I then tried adding a new C API method (XGBThreadLocalStoreFree) and calling that after each time I dumped the model, and I tried the same test again, and found that the memory usage stopped increasing after about 25 iterations.
See attached graph of my results.

I'm not a C++ developer, so there's probably a better way of solving this than my "XGBThreadLocalStoreFree" method, but I'll open a pull request with this change in it for comments and review anyway.

I think this is due to the problem of how we do thread local in DMLC. The old solution was mainly for portability for compilers which might not thread local object. We could try to patch it in a why that thread local actually get destructed when thread exit for compilers that supports it.

We can move discussion to https://github.com/dmlc/dmlc-core/issues/230

see if https://github.com/dmlc/xgboost/pull/2114 fixed the problem

This doesn't seem to have made any difference - I think because xgboost has its own copy of thread_local.h in src\common. I copied just the "static thread_local T inst; return &inst;" lines from the dmlc-core\include\thread_local.h into the src\common\thread_local.h and that did fix the problem though.
So does the thread_local.h in src\common need to be updated to match the one in dmlc-core\include?
Thanks for looking into this so quickly by the way!

oh i see, we should remove xgboost's own version of thread_local and switch to dmlc::ThreadLocalStore. Do you mind to submit a PR On this?

I can have a go at that, but it might have to wait until Monday now...

Pull request submitted!

Closing as pull request has now been merged :-)


Changelog and doc updates for next version

Please see https://github.com/apollographql/apollo-client/pull/10079 for changes suggested from @StephenBarlow  on our Web Client docs, we should we apply those same changes to these Kotlin docs. @martinbonnin

Port of https://github.com/apollographql/apollo-client/pull/10079 is here


Simple Game in rust

Guessing number game

Please Assign me thsi with hactoberfest tag

Hey ! @MohammadRehanPatel i want to contribute this issue


Training error with NaNs

Hello! Thanks for sharing the code.
I'm using a Tesla P100-PCIE-12GB GPU, which is not able to hold the model with BERT-large (raising CUDA out-of-memory error), so I have to switch to BERT-base. I also need to decrease the batch size from 16 to 8 and increase gradient accumulation steps from 2 to 4. No other changes are made to the model configuration.
When I train the model, I got the following error:
Traceback (most recent call last):
  File "/vault/.pyenv/versions/3.7.5/lib/python3.7/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/vault/.pyenv/versions/3.7.5/lib/python3.7/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/users/yshao/TabularSemanticParsing/src/experiments.py", line 407, in <module>
    run_experiment(args)
  File "/users/yshao/TabularSemanticParsing/src/experiments.py", line 392, in run_experiment
    train(sp)
  File "/users/yshao/TabularSemanticParsing/src/experiments.py", line 63, in train
    sp.run_train(train_data, dev_data)
  File "/users/yshao/TabularSemanticParsing/src/common/learn_framework.py", line 209, in run_train
    loss.backward()
  File "/vault/.pyenv/versions/py3.7.5-bridge/lib/python3.7/site-packages/torch/tensor.py", line 221, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/vault/.pyenv/versions/py3.7.5-bridge/lib/python3.7/site-packages/torch/autograd/__init__.py", line 132, in backward
    allow_unreachable=True)  # allow_unreachable flag
  File "/vault/.pyenv/versions/py3.7.5-bridge/lib/python3.7/site-packages/wandb/wandb_torch.py", line 256, in <lambda>
    handle = var.register_hook(lambda grad: _callback(grad, log_track))
  File "/vault/.pyenv/versions/py3.7.5-bridge/lib/python3.7/site-packages/wandb/wandb_torch.py", line 254, in _callback
    self.log_tensor_stats(grad.data, name)
  File "/vault/.pyenv/versions/py3.7.5-bridge/lib/python3.7/site-packages/wandb/wandb_torch.py", line 207, in log_tensor_stats
    tensor = flat.histc(bins=self._num_bins, min=tmin, max=tmax)
RuntimeError: range of [nan, nan] is not finite

I also tried decreasing grad_norm from 0.3 to 0.1, but the same error occurred.
Any suggestion on the reason of this or what I can try to get rid of it? Thank you!

Good catch. We run into this issue recently as well -- it appears to be an issue rooted in the wandb tool rather than the training algorithm (https://github.com/wandb/client/issues/800).
I upgraded wandb to the latest version (0.10.15)
python3 -m pip install wandb --upgrade

Currently trying to see if the problem goes away. You could try it as well.

Good catch. We run into this issue recently as well -- it appears to be an issue rooted in the wandb tool rather than the training algorithm (https://github.com/wandb/client/issues/800).
I upgraded wandb to the latest version (0.10.15)
python3 -m pip install wandb --upgrade

Currently trying to see if the problem goes away. You could try it as well.

Thank you! This seems to fix the error. Although I still get "loss = nan" in the training log, the performance of the trained model is reasonable.


Bump android-browser-helper billing library

Bump android-browser-helper billing library to 1.0.0-alpha11, which is updated to Play Billing v6

LGTM, thanks Oleksii!


[DynamoDBEnableServerSideEncryption] do not exist in the template

Describe the bug
I am not sure if this is AmplifyCLI issue or Console issue. I am trying to deploy using Amplify Console and getting the [DynamoDBEnableServerSideEncryption] do not exist in the template error.
It didn't happen until recently and we haven't change the cloud formation template generated by the AmplifyCLI.
amplify push is working fine.
To Reproduce
Steps to reproduce the behavior:

Go to 'Amplify Console'
Finish deploy setting
Hit 'deploy' and See error

                                 # Starting phase: build
                                 # Executing command: amplifyPush --simple
2019-08-07T16:41:00.824Z [INFO]: # Getting Amplify CLI Cloud-Formation stack info from environment cache
2019-08-07T16:41:00.853Z [INFO]: # Start initializing Amplify environment: dev
2019-08-07T16:41:00.855Z [INFO]: # Initializing new Amplify environment: dev (amplify init)
2019-08-07T16:41:14.816Z [INFO]: Note: It is recommended to run this command from the root of your app directory
2019-08-07T16:41:17.090Z [INFO]:
2019-08-07T16:41:17.092Z [INFO]: For more information on AWS Profiles, see:
                                 https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html
2019-08-07T16:41:17.162Z [WARNING]: - Initializing your environment: dev
2019-08-07T16:41:17.708Z [WARNING]: ✔ Initialized provider successfully.
2019-08-07T16:41:24.752Z [INFO]: Could not read cloudformation template at path: /codebuild/output/src150497733/src/handbook/amplify/backend/api/nhb/build/cloudformation-template.json
2019-08-07T16:41:24.863Z [WARNING]: - Updating resources in the cloud. This may take a few minutes...
2019-08-07T16:41:30.488Z [INFO]:
2019-08-07T16:41:30.492Z [INFO]: UPDATE_IN_PROGRESS nhb-dev-20190802090817 AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:25 GMT+0000 (Coordinated Universal Time) User Initiated
2019-08-07T16:41:30.610Z [INFO]:
2019-08-07T16:41:30.610Z [INFO]: UPDATE_IN_PROGRESS authnhbcognito AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:30 GMT+0000 (Coordinated Universal Time)
2019-08-07T16:41:35.408Z [INFO]:
2019-08-07T16:41:35.411Z [INFO]: UPDATE_COMPLETE    authnhbcognito       AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:31 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS apinhb               AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:33 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionnhbusersfunc AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:33 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_FAILED      apinhb               AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:33 GMT+0000 (Coordinated Universal Time) Parameters: [DynamoDBEnableServerSideEncryption] do not exist in the template
2019-08-07T16:41:35.593Z [INFO]:
2019-08-07T16:41:35.594Z [INFO]: UPDATE_IN_PROGRESS          nhb-dev-20190802090817-functionnhbusersfunc-1L6QD27RBKVX AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:33 GMT+0000 (Coordinated Universal Time) User Initiated
                                 UPDATE_ROLLBACK_IN_PROGRESS nhb-dev-20190802090817-functionnhbusersfunc-1L6QD27RBKVX AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:34 GMT+0000 (Coordinated Universal Time) Initiated by parent stack
2019-08-07T16:41:45.410Z [INFO]:
2019-08-07T16:41:45.412Z [INFO]: UPDATE_FAILED functionnhbusersfunc AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:44 GMT+0000 (Coordinated Universal Time) Resource update cancelled
2019-08-07T16:41:45.715Z [INFO]:
2019-08-07T16:41:45.716Z [INFO]: UPDATE_ROLLBACK_IN_PROGRESS nhb-dev-20190802090817 AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:45 GMT+0000 (Coordinated Universal Time) The following resource(s) failed to update: [functionnhbusersfunc, apinhb].
2019-08-07T16:41:55.422Z [INFO]:
2019-08-07T16:41:55.424Z [INFO]: UPDATE_IN_PROGRESS authnhbcognito       AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:51 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    authnhbcognito       AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:51 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    apinhb               AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:52 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionnhbusersfunc AWS::CloudFormation::Stack Wed Aug 07 2019 16:41:52 GMT+0000 (Coordinated Universal Time)
2019-08-07T16:42:25.483Z [INFO]:
2019-08-07T16:42:25.485Z [INFO]: UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS nhb-dev-20190802090817-functionnhbusersfunc-1L6QD27RBKVX AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:21 GMT+0000 (Coordinated Universal Time)
2019-08-07T16:42:30.432Z [INFO]:
2019-08-07T16:42:30.434Z [INFO]: UPDATE_COMPLETE                              functionnhbusersfunc   AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:26 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS nhb-dev-20190802090817 AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:27 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS                           apinhb                 AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:28 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE                              apinhb                 AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:29 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS                           functionnhbusersfunc   AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:29 GMT+0000 (Coordinated Universal Time)
2019-08-07T16:42:30.492Z [INFO]:
2019-08-07T16:42:30.494Z [INFO]: UPDATE_ROLLBACK_COMPLETE nhb-dev-20190802090817-functionnhbusersfunc-1L6QD27RBKVX AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:30 GMT+0000 (Coordinated Universal Time)
2019-08-07T16:42:40.438Z [INFO]:
2019-08-07T16:42:40.440Z [INFO]: UPDATE_COMPLETE functionnhbusersfunc AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:39 GMT+0000 (Coordinated Universal Time)
2019-08-07T16:42:45.441Z [INFO]:
2019-08-07T16:42:45.444Z [INFO]: UPDATE_IN_PROGRESS       authnhbcognito         AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:40 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          authnhbcognito         AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:41 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_ROLLBACK_COMPLETE nhb-dev-20190802090817 AWS::CloudFormation::Stack Wed Aug 07 2019 16:42:41 GMT+0000 (Coordinated Universal Time)
2019-08-07T16:42:55.505Z [WARNING]: Error updating cloudformation stack
2019-08-07T16:43:00.514Z [INFO]: Following resources failed
2019-08-07T16:43:00.517Z [INFO]: init failed
2019-08-07T16:43:00.518Z [WARNING]: ✖ An error occurred when pushing the resources to the cloud
                                    ✖ There was an error initializing your environment.
2019-08-07T16:43:00.520Z [INFO]: { ResourceNotReady: Resource is not in the state stackUpdateComplete
                                 at constructor.setError (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/resource_waiter.js:182:47)
                                 at Request.CHECK_ACCEPTORS (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/resource_waiter.js:44:12)
                                 at Request.callListeners (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
                                 at Request.emit (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
                                 at Request.emit (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:683:14)
                                 at Request.transition (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:22:10)
                                 at AcceptorStateMachine.runTo (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:14:12)
                                 at /root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:26:10
                                 at Request.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:38:9)
                                 at Request.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:685:12)
                                 message: 'Resource is not in the state stackUpdateComplete',
                                 code: 'ResourceNotReady',
                                 retryable: false,
                                 time: 2019-08-07T16:42:55.504Z,
                                 statusCode: 200,
                                 retryDelay: 30000 }
2019-08-07T16:43:00.598Z [ERROR]: !!! Build failed
2019-08-07T16:43:00.599Z [ERROR]: !!! Non-Zero Exit Code detected
2019-08-07T16:43:00.600Z [INFO]: # Starting environment caching...
2019-08-07T16:43:00.600Z [INFO]: # Environment caching completed
Terminating logging...

@monaye Amplify CLI released a new version recently. Could you try to install the latest version in pre-build stage.
        pre_build:             commands:                 - 'npm install -g<EMAIL_ADDRESS>

@garyleefight
1.10.0 is the version I am using.
mowin-9m:~ mowin$ amplify -v
1.10.0

@garyleefight
no~. still same.
                                 # Starting phase: preBuild
                                 # Executing command: npm install -g<EMAIL_ADDRESS>2019-08-07T19:46:24.483Z [WARNING]: npm WARN deprecated<EMAIL_ADDRESS>CircularJSON is in maintenance only, flatted is its successor.
2019-08-07T19:46:31.668Z [WARNING]: npm WARN deprecated<EMAIL_ADDRESS>use String.prototype.padStart()
2019-08-07T19:46:36.637Z [WARNING]: npm WARN deprecated<EMAIL_ADDRESS>Please upgrade to kleur@3 or migrate to 'ansi-colors' if you prefer the old syntax. Visit <https://github.com/lukeed/kleur/releases/tag/v3.0.0\> for migration path(s).
2019-08-07T19:46:40.580Z [WARNING]: npm WARN deprecated<EMAIL_ADDRESS>Please update to minimatch 3.0.2 or higher to avoid a RegExp DoS issue
2019-08-07T19:46:54.500Z [INFO]: /root/.nvm/versions/node/v10.16.0/bin/amplify -> /root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/bin/amplify
2019-08-07T19:46:54.964Z [WARNING]: npm WARN<EMAIL_ADDRESS>requires a peer of<EMAIL_ADDRESS>but none is installed. You must install peer dependencies yourself.
2019-08-07T19:46:54.964Z [WARNING]: npm WARN optional SKIPPING OPTIONAL DEPENDENCY<EMAIL_ADDRESS>(node_modules/@aws-amplify/cli/node_modules/fsevents):
                                    npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for<EMAIL_ADDRESS>wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
2019-08-07T19:46:54.964Z [WARNING]:
2019-08-07T19:46:54.966Z [INFO]: +<EMAIL_ADDRESS>                                 added 124 packages from 200 contributors, removed 170 packages, updated 141 packages and moved 26 packages in 43.474s
2019-08-07T19:46:54.982Z [INFO]: # Completed phase: preBuild
                                 # Starting phase: build
2019-08-07T19:46:54.983Z [INFO]: # Executing command: amplifyPush --simple
2019-08-07T19:46:54.987Z [INFO]: # Getting Amplify CLI Cloud-Formation stack info from environment cache
2019-08-07T19:46:55.013Z [INFO]: # Start initializing Amplify environment: dev
2019-08-07T19:46:55.013Z [INFO]: # Initializing new Amplify environment: dev (amplify init)
2019-08-07T19:47:01.351Z [INFO]: Note: It is recommended to run this command from the root of your app directory
2019-08-07T19:47:02.738Z [INFO]:
2019-08-07T19:47:02.739Z [INFO]: For more information on AWS Profiles, see:
                                 https://docs.aws.amazon.com/cli/latest/userguide/cli-multiple-profiles.html
2019-08-07T19:47:02.948Z [WARNING]: - Initializing your environment: dev
2019-08-07T19:47:03.293Z [WARNING]: ✔ Initialized provider successfully.
2019-08-07T19:47:09.156Z [INFO]: Could not read cloudformation template at path: /codebuild/output/src743284947/src/negotiators-handbook/amplify/backend/api/nhb/build/cloudformation-template.json
2019-08-07T19:47:09.204Z [WARNING]: - Updating resources in the cloud. This may take a few minutes...
2019-08-07T19:47:14.799Z [INFO]:
2019-08-07T19:47:14.802Z [INFO]: UPDATE_IN_PROGRESS nhb-dev-20190802090817 AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:09 GMT+0000 (Coordinated Universal Time) User Initiated
                                 UPDATE_IN_PROGRESS authnhbcognito         AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:14 GMT+0000 (Coordinated Universal Time)
2019-08-07T19:47:19.718Z [INFO]:
2019-08-07T19:47:19.720Z [INFO]: UPDATE_COMPLETE    authnhbcognito       AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:15 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionnhbusersfunc AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:18 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS apinhb               AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:18 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_FAILED      apinhb               AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:18 GMT+0000 (Coordinated Universal Time) Parameters: [DynamoDBEnableServerSideEncryption] do not exist in the template
2019-08-07T19:47:19.885Z [INFO]:
2019-08-07T19:47:19.885Z [INFO]: UPDATE_IN_PROGRESS          nhb-dev-20190802090817-functionnhbusersfunc-1L6QD27RBKVX AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:18 GMT+0000 (Coordinated Universal Time) User Initiated
                                 UPDATE_ROLLBACK_IN_PROGRESS nhb-dev-20190802090817-functionnhbusersfunc-1L6QD27RBKVX AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:19 GMT+0000 (Coordinated Universal Time) Initiated by parent stack
2019-08-07T19:47:34.723Z [INFO]:
2019-08-07T19:47:34.725Z [INFO]: UPDATE_FAILED               functionnhbusersfunc   AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:30 GMT+0000 (Coordinated Universal Time) Resource update cancelled
                                 UPDATE_ROLLBACK_IN_PROGRESS nhb-dev-20190802090817 AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:30 GMT+0000 (Coordinated Universal Time) The following resource(s) failed to update: [functionnhbusersfunc, apinhb].
2019-08-07T19:47:39.744Z [INFO]:
2019-08-07T19:47:39.746Z [INFO]: UPDATE_IN_PROGRESS authnhbcognito       AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:36 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    authnhbcognito       AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:36 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS functionnhbusersfunc AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:37 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE    apinhb               AWS::CloudFormation::Stack Wed Aug 07 2019 19:47:37 GMT+0000 (Coordinated Universal Time)
2019-08-07T19:48:09.942Z [INFO]:
2019-08-07T19:48:09.944Z [INFO]: UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS nhb-dev-20190802090817-functionnhbusersfunc-1L6QD27RBKVX AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:07 GMT+0000 (Coordinated Universal Time)
2019-08-07T19:48:14.750Z [INFO]:
2019-08-07T19:48:14.752Z [INFO]: UPDATE_COMPLETE                              functionnhbusersfunc   AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:11 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_ROLLBACK_COMPLETE_CLEANUP_IN_PROGRESS nhb-dev-20190802090817 AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:13 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS                           functionnhbusersfunc   AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:14 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS                           apinhb                 AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:14 GMT+0000 (Coordinated Universal Time)
2019-08-07T19:48:14.954Z [INFO]:
2019-08-07T19:48:14.955Z [INFO]: UPDATE_COMPLETE apinhb AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:14 GMT+0000 (Coordinated Universal Time)
2019-08-07T19:48:19.957Z [INFO]:
2019-08-07T19:48:19.959Z [INFO]: UPDATE_ROLLBACK_COMPLETE nhb-dev-20190802090817-functionnhbusersfunc-1L6QD27RBKVX AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:15 GMT+0000 (Coordinated Universal Time)
2019-08-07T19:48:29.775Z [INFO]:
2019-08-07T19:48:29.777Z [INFO]: UPDATE_COMPLETE          functionnhbusersfunc   AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:25 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_IN_PROGRESS       authnhbcognito         AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:25 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_COMPLETE          authnhbcognito         AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:26 GMT+0000 (Coordinated Universal Time)
                                 UPDATE_ROLLBACK_COMPLETE nhb-dev-20190802090817 AWS::CloudFormation::Stack Wed Aug 07 2019 19:48:26 GMT+0000 (Coordinated Universal Time)
2019-08-07T19:48:39.842Z [WARNING]: Error updating cloudformation stack
2019-08-07T19:48:44.851Z [INFO]: Following resources failed
2019-08-07T19:48:44.854Z [INFO]: init failed
2019-08-07T19:48:44.855Z [WARNING]: ✖ An error occurred when pushing the resources to the cloud
                                    ✖ There was an error initializing your environment.
2019-08-07T19:48:44.858Z [INFO]: { ResourceNotReady: Resource is not in the state stackUpdateComplete
                                 at constructor.setError (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/resource_waiter.js:182:47)
                                 at Request.CHECK_ACCEPTORS (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/resource_waiter.js:44:12)
                                 at Request.callListeners (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
                                 at Request.emit (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
                                 at Request.emit (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:683:14)
                                 at Request.transition (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:22:10)
                                 at AcceptorStateMachine.runTo (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:14:12)
                                 at /root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/state_machine.js:26:10
                                 at Request.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:38:9)
                                 at Request.<anonymous> (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/aws-sdk/lib/request.js:685:12)
                                 message: 'Resource is not in the state stackUpdateComplete',
                                 code: 'ResourceNotReady',
                                 retryable: false,
                                 time: 2019-08-07T19:48:39.842Z,
                                 statusCode: 200,
                                 retryDelay: 30000 }
2019-08-07T19:48:44.912Z [ERROR]: !!! Build failed
2019-08-07T19:48:45.055Z [ERROR]: !!! Non-Zero Exit Code detected
2019-08-07T19:48:45.055Z [INFO]: # Starting environment caching...
2019-08-07T19:48:45.055Z [INFO]: # Environment caching completed
Terminating logging...

@monaye Can you provide the local CLI version?

@garyleefight My local CLI version is also 1.11.0
The workaround on this had been introduced by Vinicius C. from AWS support.
Since I was having an issue with existing project option A worked for me.

Option A - in your existing project:
1 - Execute “amplify function add” with Hello World configuration and no access to other services;
2 - Execute "amplify push";
3 - Execute “amplify function remove” removing all related files to backend (option Y);
4 - Execute “amplify push”.

Option B - in a new project:
1 - Create a new project by running amplify init in a new folder;
2 - Once the init succeeds, copy everything in the amplify/#current-cloud-backend of your current project to backend of the new project you just created;
3 - Open amplify/backend/amplify-meta.json of the newly created project copy the json data under the key providers from amplify/#current-cloud-backend/amplify-meta.json from the newly created project and paste it on to amplify/backend /amplify-meta.jsons providers section
4 - Do an amplify push to update your CloudFormationStack.

For both options now you would be able to see the tag "DynamoDBEnableServerSideEncryption": "false" in your "amplify\backend\awscloudformation\nested-cloudformation-stack.yml" file. This error was caused by an old version of Amplify CLI that didn’t use this parameter to create the CloudFormationTemplate, which would cause the deployment to fail.

The suggested workaround works for first push but failed on second push and I had to change as follow to work :
-    "DynamoDBEnableServerSideEncryption": "false",
+    "DynamoDBEnableServerSideEncryption": false,

Having this same issue. Would this be fixed on the Amplify Console side by first checking if "DynamoDBEnableServerSideEncryption" exists in the template instead of just throwing a build error?

The suggested workaround works for first push but failed on second push and I had to change as follow to work :
-    "DynamoDBEnableServerSideEncryption": "false",
+    "DynamoDBEnableServerSideEncryption": false,

Where in the Cloudformation template does that flag get put in? The function push method is not working for me, I want to try manually adding it

I've tried that fix and didn't work for me.
Here's what I tried:
A. Add function, then remove into an existing project
B. Manually add "DynamoDBEnableServerSideEncryption": "false", into nested-cloudformation-stack.yml
While B looks like the workaround, it does get removed when I run amplify push
Amplify CLI version: 3.0.0

Got it working now and there was no need to add/remove function, but to change two files - Here's a quick summary.

Update API nested stack within the backend CloudFormation stack with DynamoDBEnableServerSideEncryption: true

In my case, my API is named apiawsserverlessairline - Excerpt from: amplify/backend/awscloudformation/nested-cloudformation-stack.yml
"apiawsserverlessairline": {
    "Type": "AWS::CloudFormation::Stack",
    "Properties": {
        "TemplateURL": "https://s3.amazonaws.com/awsserverlessairline-20190506121509-deployment/amplify-cfn-templates/api/cloudformation-template.json",
        "Parameters": {
            "AppSyncApiName": "awsserverlessairline",
            "DynamoDBBillingMode": "PAY_PER_REQUEST",
            "DynamoDBEnableServerSideEncryption": "true", <---
            ...
        }
    }
}

Update API parameters.json file with DynamoDBEnableServerSideEncryption: true

Excerpt from: amplify/backend/api/awsserverlessairline/parameters.json
{
    "AppSyncApiName": "awsserverlessairline",
    "DynamoDBBillingMode": "PAY_PER_REQUEST",
    "DynamoDBEnableServerSideEncryption": "true", <---
    "AuthCognitoUserPoolId": {
        "Fn::GetAtt": [
            "authcognitoa7d29d9d",
            "Outputs.UserPoolId"
        ]
    }
}

I am running into the same issue right now. Updated my schema and the amplify push now fails every time with:
UPDATE_FAILED               Answer                                               AWS::CloudFormation::Stack Fri May 22 2020 09:28:17 GMT+0100 (British Summer Time) Parameters: [CreateAPIKey, DynamoDBEnableServerSideEncryption] do not exist in the template
This error message is thrown for every single one of my models. I tried updating Amplify to the latest version, with no success.
I checked the build folder for the required values in cloudformation-template.json and I was able to find them both:
"DynamoDBEnableServerSideEncryption": {             "Type": "String",             "Description": "Enable server side encryption powered by KMS.",             "Default": "true",             "AllowedValues": [                 "true",                 "false"             ]  }
"CreateAPIKey": {             "Type": "Number",             "Description": "The boolean value to control if an API Key will be created or not. The value of the property is automatically set by the CLI. If the value is set to 0 no API Key will be created.",             "Default": 0,             "MinValue": 0,             "MaxValue": 1 }
One thing that I noticed is that I couldn't find any of the above values in the amplify/backend/awscloudformation/nested-cloudformation-stack.yml file
I've already tried the fixes suggested above, with no luck. Any insight into this would be appreciated as it's currently stopping us from pushing any updates to our back-end
Cheers!


KeyError

When i run python train.py i receive the following error.
python train.py
work_dir checkpoints/1 exists! Pls check it.
2019-03-11 19:12:17.115490: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
2019-03-11 19:12:17.137557: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency:<PHONE_NUMBER> Hz
2019-03-11 19:12:17.138025: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x561a1826ee40 executing computations on platform Host. Devices:
2019-03-11 19:12:17.138077: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): ,
Load data ...
Traceback (most recent call last):
File "train.py", line 541, in
train(options)
File "train.py", line 262, in train
data_provision = DataProvision(options)
File "/home/leena/mainproject/DenseVideoCaptioning/data_provider.py", line 23, in init
np.random.seed(options['random_seed'])
KeyError: 'random_seed'

I have corrected the error by adding "random_seed" in opt.py.

Thanks. That fixed it!


Filter all local packages out for the lockfile

Before, only the packages for which the dependencies are calculated were excluded from the lockfile. To avoid build errors due to duplicated packages, all packages in the workspace need to be excluded.

I also have a version that filters the packages that are in the workspace but not in the "project" (in the sense that you take them into account to compute dependencies) from the pin-depends, but not from the depends. But from our last conversation, @NathanReb , I think what we want is excluding them totally from the lockfile, right?

Should I add some test?


feat: Posted Alerts page

Asana task: [Screenplay] Basic Alert Card list
This adds a list of AlertCards to the Posted Alerts page. This is the first feature that will use the API, so there is a dependency on this task, which will add API keys for Screenplay. It looks for an ENV variable so local testing is not blocked (README has been updated).

Whoa, this diff is way big! I'm looking into the rebase situation; I don't think 40 file changes is quite right?

Whoa, this diff is way big! I'm looking into the rebase situation; I don't think 40 file changes is quite right?
EDIT: After rebasing alerts-feature, looks like this diff is correct. 40 files changed is too big for future PRs, though. Was it 40 changed files originally?

I agree, this PR kinda got away from me. Overall, there aren't a ton of new files added. I did move some OFT files to a new directory and some code needed to be added for interfacing with the API. Making this PR into one for frontend one for backend is what I should have done in the first place. I'm going to try to do that now.

@hannahpurcell @jzimbel-mbta Was able to get all of the backend logic moved to https://github.com/mbta/screenplay/pull/155. This PR will eventually just have frontend changes, but will close until it is ready.


New script removing contiguous duplicate keys from an input dataset

typo fixed! yes, i also look for a while for a one filter solution but
didn't come up with anything good enough.  the main problem i found is
not having in flatline any way of storing state as we traverse the
dataset, or to referring to previous values of the new fields.  sooner
or later we are going to have the former (the latter is much harder).

Merged!


fix(material): export datepicker and toggle types

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)

packaging
naming

What is the current behavior? (You can also link to an open issue here)

fix #3094 comments
fix #3116

What is the new behavior (if this is a feature change)?

exports addressed
new names comply with convention

Please check if the PR fulfills these requirements

[x] The commit messages follow our guidelines: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format
[ ] A unit test has been written for this change.
[x] Running npm run build produced a successful build. (Unit testing can be done by running npm test;)
[x] My code has been linted. (npm run lint to do this. npm run build will fail if there are files not linted.)

Thank you @vitale232!


Obsoletion:  GO:0036523 induction by symbiont of host cytokine production

This has no (0) annotations and is not a well thought-out term. What kind of cytokines? Proinflammatory or anti-inflammatory? Is the symbiont manipulating the host immune signaling pathways to "induce" this cytokine production? Or is the host reacting via pattern-recognition receptors (PRRs/PAMPs/MAMPs) to the presence of the parasite? If so, the symbiont is doing nothing and the host is doing its "job" so this doesn't involve the symbiont/parasite at all.

Right, this term needs to be obsoleted.

Dear all,
The proposal has been made to obsolete 'GO:0036523 induction by symbiont of host cytokine production'. The reason for obsoletion is that it is not an evolved function of an symbiont, to induce host cytokine production; this is a response by the host to the presence of a potential pathogen.
There is a single annotation to this term by BHL @RLovering
This term is not in any subsets. There are no mappings to this term.
You may comment on the ticket: https://github.com/geneontology/go-ontology/issues/21743
Thanks, Pascale

Hi
happy for this term to be removed, although looking at the abstract weirdly it does look like the parasite increases cytokine release to attract more of it's target cells
https://pubmed.ncbi.nlm.nih.gov/25063865/
Please confirm obsoletion still planned and I will remove the annotation
Ruth

Hey @RLovering,
You may be right about the paper, but this is the wrong term for that activity. And there should be an annotation at a lower (molecular pathway) level that gets at what GP63 is doing. Looking at the level of "cytokine production" is too high. There is a lot of signal transduction that has to occur--presumably this is where GP63 is operating, before IL-6 and TNF are made. There are lots of cases of parasites triggering (purposefully/mechanistically) a host inflammatory pathway. Inflammasome activation by a parasite protease is an example.
 What I think is happening is that the changes being wrought in the host macrophages by Leishmania GP63 and LPG are being detected by the host which is sensitive to membrane perturbations of the sort being perpetrated by the parasite. (This also happens with other intracellular denizens like Mycobacterium tuberculosis: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC7911501/.) Then the host is activating the inflammasome as a result and that is what results in TNF and IL-6 production. So it really is the host activating the pathway and not Leishmania. At least that's my story until they have better molecular evidence.

It looks like maybe Leishmania GP63 cleaves synaptotagmin, which trigger the host to activate signaling that leads to cytokine release?
Then we could perhaps consider a term in the branch GO:0052028 induction by symbiont of host signal transduction pathway?

good point I have replaced it with GO:0052028 induction by symbiont of host signal transduction pathway

There are a variety of activities that parasites do that are recognized as parasitic and activate what could be called parasite-recognition pathways. Microbial toxin activities that modify host small GTPases do this, changes in membrane dynamics do this. I suspect that in this case of Leishmania, that is what is happening. The researchers are NOT showing that GP63 or LPG are doing something to a host pro-inflammatory pathway that would be thought to activate it. But the activities of GP63 and LPG, including cleaving synaptogamin, are being detected by the host which itself activates inflammasomes leading to the production and secretion of IL-6 and TNF in the macrophages.
So what I've just described happens a lot and is a normal part of the innate immune system. Do we want to annotate every parasite protein that triggers these processes (in theory, I'm speaking) with GO:0052028 (induction by symbiont of host signal transduction pathway). This is just a passive thing. It's mainly about the host and innate recognition, not the parasite.

I grant you that your solution does get it out of this category. So that's good.

This is just a passive thing. It's mainly about the host and innate recognition, not the parasite.

Is this passive, as it cleaves the host receptor?

No, synaptogamin isn't the "host receptor" that I'm referring to. It isn't part of the innate system at all, to the best of my knowledge. My contention is that the cleavage or the results of the cleavage OR maybe something else entirely is what is detected by the host cell and that is what leads to inflammasome activation. The researchers have no mechanism of host inflammation in their paper. It's just hand-waving.

From: pgaudet @.>
Sent: Thursday, July 1, 2021 10:39 AM
To: geneontology/go-ontology @.>
Cc: Gene Godbold @.>; Author @.>
Subject: Re: [geneontology/go-ontology] Obsoletion: GO:0036523 induction by symbiont of host cytokine production (#21743)
[EXTERNAL EMAIL]
This is just a passive thing. It's mainly about the host and innate recognition, not the parasite.
Is this passive, as it cleaves the host receptor?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/geneontology/go-ontology/issues/21743#issuecomment-872303621, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ALEFOH4EWGNXBNITA273IRDTVR43VANCNFSM47RE34YQ.


Reporting errors across MPI processes

Originally reported by: Carsten Burstedde (Bitbucket: cburstedde, GitHub: cburstedde)

In a parallel application, some errors necessarily occur on all processors (command line parsing), others don't (inconsistency in some algorithm, serial file writing). When the program is to be aborted anyway, such as on file input and MPI errors, there is no need to resolve the situation. What would be nice though is to report the error up the call trace whenever possible, which might produce something like:
file_open: could not open foo
reading input data: could not process foo data file
program setup: data loading unsuccessful

Thus there should be a stack and a way to synchronize errors in parallel. Will think about an addition to libsc to do this.

Bitbucket: https://bitbucket.org/donnaaboise/forestclaw/issue/76

I've been thinking about such a stack framework, too. Very nice concept. Would fit well into libsc.
In the meantime, if just checking the flag is enough, I'd suggest an sc_MPI_Allgather with sc_MPI_LAND or sc_MPI_LOR. Code like this is in p4est_communication.c; see p4est_comm_sync_flag.


kube_addons - Adding variable with default for kubectl bin.

Added a way to specify the kubectl path. Issue seen using the CentOS yum version of Kubernetes installs the kubectl in /usr/bin.
Fixes issue  #9599

lgtm.

ok to test


Protect against int overflow

Fixes #9043
The previous solution to integer overflow was flawed and did not provide reliable protection. Now the overflow is tested before the sum is calculated and therefore it is guaranteed to protect against integer overflow in getchar.c

Hi,
I think there is a more elegant approach for safe_add function.
Also need to note that this function should not be used when the result is actually a -1, because we couldn't determine if there is a overflow occurred, for example: safe_add(0, -1).
here is my implementation:
#define SHIFT_BITS (sizeof(int) * 8 - 1)
int safe_add(int a, int b)
{
    int res = a + b;
    return ((a ^ res) & (1 << SHIFT_BITS)) && ((b ^ res) & (1 << SHIFT_BITS)) ? -1 : res;
}

That is true @JackyYin. I think your solution is a bit more difficult to read so I decided to change my previous solution slightly to avoid that -1 case. Thank you very much.

This looks much too complicated.

What exactly looks complicated? I believe it is a rather simple approach to solve the problem.

See my comment in https://github.com/vim/vim/issues/9043#issuecomment-955243186 for an alternative method to prevent overflow.  A range check that avoids undefined behavior can be done via:
  int OVERHEAD = 5 * (MAXMAPLEN + 4);
  if (typebuf.tb_len > MAXINT - OVERHEAD ||
      addlen > MAXINT - OVERHEAD - typebuf.tb_len) {
      // Overflow.
  }

I read your comment and I appreciate the detailed suggestions and comments.
Your solution is definitely an improvement to mine as it is just as readable but way more concise. Will add to the PR.
Thank you very much

Sometimes integer overflow is an expected part of the algorithm. For example, often pseudo-random number generators use integer overflow.

Sometimes integer overflow is an expected part of the algorithm.

It's true that integer overflow is desirable at times (though not for this case); however, in those situations it's critical to use unsigned integers, because signed integer overflow has undefined behavior in standard C.  @dpelle's comment https://github.com/vim/vim/issues/9043#issuecomment-950920727 has a good explanation:

Checking that the result is negative is undefined behavior, as signed overflows in C or C++ are undefined behavior (unlike unsigned overflow that wrap reliably). In practice, signed overflow wrap, but the problem is that the optimizer is allowed to assume that it does not, so it could very well remove checking if (newlen < 0) if the compiler knows that it multiplied positive values which can't possibly give a negative number. Compiler have been doing this kind of optimizations for years, so it's not a theoretical bug.  Such undefined behavior can cause to behave differently with -O0 and -O3.

I'm new to contributing to vim. Should I try and solve that failed check? It seems rather irrelevant given the small amount of code of the commit


add fluentd-gcp repo

adds fluentd-gcp from https://github.com/kubernetes/kubernetes/tree/master/cluster/addons/fluentd-gcp
fluentd-gcp will be remove in k/k in: https://github.com/kubernetes/kubernetes/pull/101132
fixes: kubernetes/kubernetes#79432

/verify-owners

/verify-owners

/verify-owners
should work now

/hold
to remove redundant OWNERS_ALIAS file

@ehashman errant file removed, thank you for sorting that out!


Configurable Transport

Non-breaking changes which allow for a configurable "Transport" (language which was already used in the project, really a *http.Client).
This allows for setting a custom RoundTripper or using github.com/hashicorp/go-retryablehttp:
transport := retryablehttp.NewClient().StandardClient()
c := gogpt.NewClientWithTransport(openAIAPIKey, transport)

With the recent stability issues and load of the OpenAI API this is necessary to use the module in a production environment.

Hey, thank you for that PR! It made me think about how we want to approach client initialization — there was a great article about different ways of doing that, but I could not find it. The closest is this one: https://asankov.dev/blog/2022/01/29/different-ways-to-initialize-go-structs/
I don't think the usage of .WithA().WithB().WithC() chaining is a way forward for us here. I would rather prefer an API akin to:
client := gogpt.NewClient()

// or

config := gogpt.DefaultConfig()
config.Transport = retryablehttp.NewClient().StandardClient()
client := gogpt.NewClientWithConfig(config)

@mwillfox would you have the bandwidth to do such a refactoring? It's totally cool if not — I can do it in a follow-up PR.

I should have time to refactor in the next few days and I'll update the PR. Thanks for the feedback!

Will follow up this work in a better-config branch


Should I fix @types/vscode's version in my extension?

I got a question regarding the new @types/vscode package in the helloworld-test-sample:
I do not understand why you can set the @types/vscode's version to ^1.32.0. Doesn't that bear the risk that the installed version of @types/vscode mismatches the specified engines version?
Therefore, "@types/vscode": "~1.32.0" makes more sense to me.

@types/vscode is package-locked to 1.32.0. If people remove lock file and do reinstall, indeed they might install a newer version. In that case if you are publishing an extension, you need to update engines.vscode as well. For development it doesn't matter.
Ofc that's just a starter template and you can do any change you wish. The only implication of engines.vscode version is minimum VS Code version where your extension can be launched, and the only check we do in vsce is @types/vscode version <= engines.vscode version.


feat: provide builds for both cjs and esm

added module to package.json
BREAKING CHANGE: default export removed. use import { LocationMock } from '@jedmao/location'.

Thanks!

:tada: This PR is included in version 3.0.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


created main page

@Abhilash107 Can you add a screenshot for the page you have created ?

@Abhilash107 Good Work!!

thanks sir

@Abhilash107 Do checkout new issues that are recently created.

sure sir


Implement OTservers?

It is possible to make it work on custom clients with IP changed and RSA key changed?

Possibly, if they use the official client. It depends on how many modifications there are and which version of Tibia they are running.

Basically they change IP and RSA key on every version.
El 31 oct. 2016 07:39, "Mark"<EMAIL_ADDRESS>escribió:

Possibly, if they use the official client. It depends on how many
modifications there are and which version of Tibia they are running.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Mytherin/Tibialyzer/issues/234#issuecomment-257262140,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AURCuEHCL10AQz8XURNO3zOp2yismZiUks5q5cVjgaJpZM4KkBDY
.

Does this solution not work? If not, you can manually try finding the addresses by following this tutorial.

It may but is too much to do... Could you create a feature to do all that
tutorial one button fix? Would be nice.
El 31 oct. 2016 11:30, "Mark"<EMAIL_ADDRESS>escribió:

Does this solution https://github.com/Mytherin/Tibialyzer/issues/226
not work? If not, you can manually try finding the addresses by following this
tutorial
https://github.com/Mytherin/Tibialyzer/wiki/Tibia-Updated,-What-Now%3F#how-do-i-update-the-addresses
.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/Mytherin/Tibialyzer/issues/234#issuecomment-257308806,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AURCuBveIgGnIHXgoAttJ_5MoTkZ-9R5ks5q5fuYgaJpZM4KkBDY
.


AttributeError: module 'keras.backend' has no attribute 'is_tensor'

Hi, this project looks great!
However, I didn't manage to launch it on my PC, even though I followed the instructions
I would appreciate it a lot, if you would help me fix the following problem
Global seed set to 42 Traceback (most recent call last):   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/gradio/routes.py", line 337, in run_predict     output = await app.get_blocks().process_api(   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/gradio/blocks.py", line 1015, in process_api     result = await self.call_function(   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/gradio/blocks.py", line 833, in call_function     prediction = await anyio.to_thread.run_sync(   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/anyio/to_thread.py", line 31, in run_sync     return await get_asynclib().run_sync_in_worker_thread(   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread     return await future   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/anyio/_backends/_asyncio.py", line 867, in run     result = context.run(func, *args)   File "src/test/test.py", line 124, in process     local_control = einops.rearrange(local_control, 'b h w c -> b c h w').clone()   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/einops/einops.py", line 424, in rearrange     return reduce(tensor, pattern, reduction='rearrange', **axes_lengths)   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/einops/einops.py", line 368, in reduce     return recipe.apply(tensor)   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/einops/einops.py", line 203, in apply     backend = get_backend(tensor)   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/einops/_backends.py", line 28, in get_backend     if backend.is_appropriate_type(tensor):   File "/home/rd289/anaconda3/envs/unicontrol/lib/python3.8/site-packages/einops/_backends.py", line 513, in is_appropriate_type     return self.K.is_tensor(tensor) and self.K.is_keras_tensor(tensor) AttributeError: module 'keras.backend' has no attribute 'is_tensor'
I did create conda venv and installed all the dependencies

Update:
Nevermind, issue was on my side, fixed it

@MrA2K2 Can you share how you fixed it? I am having the same issue.


SSL Issues

We have been experimenting with Skydive while securing the analyzer-agent connection over SSL and we found ourselves facing these following issues:

Even when the mode of operation is not all-in-one Skydive won't start with TLS support unless both certificates for the client and agent  are provided. E.g.:
tls:
client_cert: /path/to/client_cert.pem
client_key: /path/to/client_key.pem
server_cert: /path/to/server_cert.pem
server_key: /path/to/server_key.pem
ca_cert: /path/to/ca.pem
Note that when Skydive is started as an agent, we only need a client_cert and vice versa...

In etc/skydive.yml.default There are examples of certificate file names. These names end with .crt. However, our tests have revealed that .crt format is not supported and that Skydive only manages to parse .pem certificates.

this is probably the most important issue. We found out that there is a serious bug in the current implementation of the TLS protocol, as we use a function to translate the domain name representing the analyzer into its underlying IP address before verifying the certificate's authenticity.
However, when the analyzer is given by it's domain name and not a fixed IP, which is the common case for various reasons (+certificates are most likely to be generated with CNs that are domain names rather than IP addresses that change very often), the  authentication process will fail because the IP addess won't match the CN on the certificate which is the domain name. We found out that not translating the hostname into an IP and leaving this task for the DNS server to perform solves this issue and allows for both domain names and IP addresses for analyzers. I will be introducing a PR with a simple fix of my own very soon. Please let me know what you think.

Cheers
@eranra

Any news on this or can this be closed ?


Asynchronous API support

Hi @hootnot
Thanks for the API. I am writing a Pandas DataReader to get OANDA currency data and I am using your API to contact OANDA's endpoints.
The reader is currently multithreaded but its IO performance is limited by the requests dependency used in oanda-api-v20's implementation. Would you mind/considering/providing an asynchronous API ?
requests mentions this issue in the documentation and provide solutions.
Regards,
David

Hi @davidandreoletti
Can you provide some example code regarding the performance issue you have, the instruments you request, the frequency ... so the details that I can reproduce what you experience ?
Regards,
Feite

the instruments you request, the frequency ... so the details that I can reproduce what you experience ?

Simply run the OANDA tests here. Each thread can issue more than 2 requests/s at most [1]. My machine is in Asia and hitting a US server I believe. There is a bit of delay going on but that should not have much impact if requests were truly asynchronous. I am trying to get about 10-15 request/s as performance target.
[1] Uncomment this line to see the current number of requests/second

@davidandreoletti
I had some issues running the code you pointed to. I alse guess it is your fx-oanda-historical branch I need from your forked repo.
Nevertheless I did some experiments on oandapyV20 using https://github.com/ross/requests-futures.
I made some modifications to the example code of candle-data.py to test. It looks promissing. The async version completes in about half of the time.
I have created a branch async-requests-futures and added the modified candle-data example.
Please clone it and install it locally, maybe in a separate virtualenv to be sure.  Check the candle-data-par.py example for the details. Please let me know if this helps you tackling your problem.
I will see if/how/when to implement async. on oandapyV20.
(adrenv) hootnot@oatr:~/oanda-api-v20/examples $ time python src/candle-data.py --i EUR_USD --i EUR_JPY --i EUR_AUD --i EUR_GBP --i DE30_EUR --i EUR_ZAR --i US30_USD --i NAS100_USD --count 5000 --nice --gr M5 >out2

real	0m20.875s
user	0m4.636s
sys	0m0.540s
(adrenv) hootnot@oatr:~/oanda-api-v20/examples $ time python src/candle-data-par.py --i EUR_USD --i EUR_JPY --i EUR_AUD --i EUR_GBP --i DE30_EUR --i EUR_ZAR --i US30_USD --i NAS100_USD --count 5000 --nice --gr M5 >out

real	0m9.757s
user	0m4.768s
sys	0m0.696s
(adrenv) hootnot@oatr:~/examples $ wc out out2
  440048   760080  7825682 out
  440048   760072  7825650 out2
  880096  1520152 15651332 total

@hootnot I looked a bit more about the infamous Python GIL issue (especially IO/CPU issues) and how it affects IO/CPU bound multithreaded applications. Each pure python thread must acquire the GIL to run and a thread waiting for IO automatically release the GIL allowing another python thread to run. The thread owning GIL runs, every other threads waits.
Basically for me, it is important that each IO bound thread issue as many async requests as possible (up to max 15 reqs/s) before it releases the GIL once it begins blocking on the async requests to get the response with my_future.results().
I digged further down into the future_requests package and I found that it is basically doing I am already doing:

create a bunch of threads (via ThreadPool for example)
call session.request() on each thread
-- request() blocks until IO is available

So at first glance, it seems not to be a good fit.

@davidandreoletti
Yes the GIL issue. Everyone doing threading runs into it at some moment ...
The ratelimiting is set to 30 requests/second and is IP bound. So regardless the success of the threading solution you choose, this is the hard limit. Threading must respect this limit also.
It seems to me that you can only experience this problem when you retrieve a lot of instruments with a high frequent granularity . Can you share me some parameters ?
Have you taken a look at my other repo: oanda-trading-environment. It reads the pricestream and bakes candle-records of you choice. If offers 0MQ pub/sub. You could let it write candle records of you choice into https://redis.io/ and query REDIS with your application. Redis does also pub/sub. (redis has python bindings). Maybe oanda-trading-environment can be a solution for your problem

@hootnot
About the 30 reqs/s:
The official documentation says:

To provide equal resources to all clients, we recommend limiting both the number of new connections per second, and the number of requests per second made on a persistent connection (see above/below).

For new connections, we recommend you limit this to once per second (1/s). Establishing a new TCP and SSL connection is expensive for both client and server. To allow a better experience, using a persistent connection will allow more requests to be performed on an established connection.

For an established connection, we recommend limiting this to fifteen requests per second (30/s). Please see following section on persistent connections to learn how to maintain a connection once it is established.

As per the documentation, it seems possible to create several established connection (at a rate of 1/s) and have up to 30 req/s per established connection.
About 0MG pub/sub:
I am writing a datareader for pydata/pandas-datareader and unfortunately REDIS/0MQ are out of the picture. Nonetheless, I keep this good idea in mind :)

@davidandreoletti
It was not clear to me what your goal exactly was. The datareader could also be a spin-off of something you try to accomplish.
Maybe I will look into the async connection during X-max time


Re-adds maven wrapper downloader so users won't crash if wget is missing

Why submit this pull request?

[x] Bug fix

[ ] New feature provided

[ ] Improve performance

Related issues
#1574

Bug fix
#1574 removed the maven wrapper jar. The shell script will try wget and then this
downloader if missing. It is very likely in windows and small vms for
people to not have wget installed, so this will prevent support problems
(and is why the wrapper includes this)

If no wget, this java file will be compiled and run by script. Right?

yep


[5.2] - InvalidArgumentException - No handler registered for command [App\Jobs\TestJob]

I did not upgrade from 5.1 to 5.2, though it still seems that I am getting this error as well on v5.2.45.
I do have Illuminate\Bus\BusServiceProvider::class, in my app.php file.
InvalidArgumentException \vendor\laravelcollective\bus\src\Dispatcher.php:348
No handler registered for command [App\Jobs\TestJob]

I have attempted to call the job in multiple different ways.
$queue = Queue::later(\Carbon\Carbon::now()->addMinutes(1), (new \App\Jobs\TestJob(\Carbon\Carbon::now())));

$job = (new \App\Jobs\TestJob(\Carbon\Carbon::now()))->delay(10);
$queue = dispatch($job);

All attempts result in the same exception and fail.
[2016-08-29 09:22:22] Failed: App\Jobs\TestJob

My TestJob.php:
<?php

namespace App\Jobs;

use Carbon\Carbon;
use Illuminate\Queue\SerializesModels;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Contracts\Queue\ShouldQueue;
use Mail;

class TestJob extends Job implements ShouldQueue
{
    use InteractsWithQueue, SerializesModels;

    /**
     * @var Carbon
     */
    protected $date_scheduled;

    /**
     * Create a new job instance.
     *
     * @param Carbon $date_scheduled
     */
    public function __construct(Carbon $date_scheduled)
    {
        $this->date_scheduled = $date_scheduled;
    }

    /**
     * Execute the job.
     */
    public function handle()
    {
        $scheduled = $this->date_scheduled->format(DATE_RFC3339);
        $fired = Carbon::now()->format(DATE_RFC3339);
        Mail::raw("Scheduled at {$scheduled} and job fired at {$fired}", function ($m) {
<EMAIL_ADDRESS>'Test');
<EMAIL_ADDRESS>'Dak');
            $m->subject('TEST');
        });
    }
}

Replaced e-mail addresses for privacy reasons.
As you can see, it has a handle() method. I saw in the stack trace to confirm that it is looking for the handle() method.
// \vendor\laravelcollective\bus\src\Dispatcher.php:200 Collective\Bus\Dispatcher::Collective\Bus\{closure}
return $this->container->call([$command, 'handle']);

I just can't seem to figure out what is wrong. :disappointed:
Please help :cry:

We've removed the ability to have non-self handling commands in L5.2. This was partially restored in 5.3 though.

Please run composer show -i.

aftership/aftership-php-sdk           5.0.9              The PHP SDK of AfterShip API
barryvdh/laravel-cors                 v0.8.2             Adds CORS (Cross-Origin Resource Sharing) headers support in your Laravel application
barryvdh/laravel-dompdf               v0.6.1             A DOMPDF Wrapper for Laravel
bugsnag/bugsnag                       v3.3.0             Official Bugsnag notifier for PHP applications.
bugsnag/bugsnag-laravel               v2.3.0             Official Bugsnag notifier for Laravel applications.
bugsnag/bugsnag-psr-logger            v1.1.0             Official Bugsnag PHP PSR Logger.
chumper/zipper                        0.6.1              This is a little neat helper for the ZipArchive methods with handy functions
classpreloader/classpreloader         3.0.0              Helps class loading performance by generating a single PHP file containing all of the autoloaded files for a specific use case
composer/ca-bundle                    1.0.3              Lets you find a path to the system CA bundle, and includes a fallback to the Mozilla CA bundle.
dnoegel/php-xdg-base-dir              0.1                implementation of xdg base directory specification for php
doctrine/annotations                  v1.2.7             Docblock Annotations Parser
doctrine/cache                        v1.6.0             Caching library offering an object-oriented API for many cache backends
doctrine/collections                  v1.3.0             Collections Abstraction library
doctrine/common                       v2.6.1             Common Library for Doctrine projects
doctrine/dbal                         v2.5.4             Database Abstraction Layer
doctrine/inflector                    v1.1.0             Common String Manipulations with regard to casing and singular/plural rules.
doctrine/instantiator                 1.0.5              A small, lightweight utility to instantiate objects in PHP without invoking their constructors
doctrine/lexer                        v1.0.1             Base library for a lexer that can be used in Top-Down, Recursive Descent Parsers.
dompdf/dompdf                         v0.6.2             DOMPDF is a CSS 2.1 compliant HTML to PDF converter
drewm/mailchimp-api                   v2.2.4             Super-simple, minimum abstraction MailChimp API v3 wrapper
fabpot/goutte                         v3.1.2             A simple PHP Web Scraper
fzaninotto/faker                      v1.6.0             Faker is a PHP library that generates fake data for you.
guzzle/guzzle                         v3.9.3             PHP HTTP client. This library is deprecated in favor of https://packagist.org/packages/guzzlehttp/guzzle
guzzlehttp/guzzle                     6.2.1              Guzzle is a PHP HTTP client library
guzzlehttp/promises                   1.2.0              Guzzle promises library
guzzlehttp/psr7                       1.3.1              PSR-7 message implementation
hamcrest/hamcrest-php                 v1.2.2             This is the PHP port of Hamcrest Matchers
intervention/image                    2.3.7              Image handling and manipulation library with support for Laravel integration
ircmaxell/random-lib                  v1.1.0             A Library For Generating Secure Random Numbers
ircmaxell/security-lib                1.0.0              A Base Security Library
jakub-onderka/php-console-color       0.1
jakub-onderka/php-console-highlighter v0.3.2
jeremeamia/SuperClosure               2.2.0              Serialize Closure objects, including their context and binding
laravel/framework                     v5.2.45            The Laravel Framework.
laravelbook/ardent                    v3.4.2             Self-validating smart models for Laravel 5's Eloquent ORM
laravelcollective/bus                 v5.2               The Laravel Bus (5.1) package for use in Laravel 5.2.
laravelcollective/html                v5.2.4             HTML and Form Builders for the Laravel Framework
laurentbrieu/tcpdf                    dev-master 1352a46 Composer wrapper for TCPDF PHP library
league/flysystem                      1.0.27             Filesystem abstraction: Many filesystems, one API.
league/fractal                        0.14.0             Handle the output of complex data structures ready for API output.
league/oauth2-client                  1.4.2              OAuth 2.0 Client Library
maatwebsite/excel                     2.1.4              An eloquent way of importing and exporting Excel and CSV in Laravel 4 with the power of PHPExcel
mandrill/mandrill                     1.0.55             API client library for the Mandrill email as a service platform
mannysoft/surveymonkey                dev-master e93d747 PHP class for SurveyMonkey API.
mockery/mockery                       0.9.5              Mockery is a simple yet flexible PHP mock object framework for use in unit testing with PHPUnit, PHPSpec or any other testing framework. Its core goal is to offer a test double framework with a succ...
monolog/monolog                       1.21.0             Sends your logs to files, sockets, inboxes, databases and various web services
mtdowling/cron-expression             v1.1.0             CRON for PHP: Calculate the next or previous run date and determine if a CRON expression is due
nesbot/carbon                         1.21.0             A simple API extension for DateTime.
nikic/php-parser                      v2.1.0             A PHP parser written in PHP
pacely/mailchimp-apiv3                dev-master 859943e Simple API wrapper for Mailchimp API V3
paragonie/random_compat               v1.4.1             PHP 5.x polyfill for random_bytes() and random_int() from PHP 7
phenx/php-font-lib                    0.2.2              A library to read, parse, export and make subsets of different types of font files.
phpdocumentor/reflection-common       1.0                Common reflection classes used by phpdocumentor to reflect the code structure
phpdocumentor/reflection-docblock     3.1.0              With this component, a library can provide support for annotations via DocBlocks or otherwise retrieve information that is embedded in a DocBlock.
phpdocumentor/type-resolver           0.2
phpoffice/phpexcel                    1.8.1              PHPExcel - OpenXML - Read, Create and Write Spreadsheet documents in PHP - Spreadsheet engine
phpspec/prophecy                      v1.6.1             Highly opinionated mocking framework for PHP 5.3+
phpunit/php-code-coverage             2.2.4              Library that provides collection, processing, and rendering functionality for PHP code coverage information.
phpunit/php-file-iterator             1.4.1              FilterIterator implementation that filters files based on a list of suffixes.
phpunit/php-text-template             1.2.1              Simple template engine.
phpunit/php-timer                     1.0.8              Utility class for timing
phpunit/php-token-stream              1.4.8              Wrapper around PHP's tokenizer extension.
phpunit/phpunit                       4.8.27             The PHP Unit Testing framework.
phpunit/phpunit-mock-objects          2.3.8              Mock Object library for PHPUnit
psr/http-message                      1.0.1              Common interface for HTTP messages
psr/log                               1.0.0              Common interface for logging libraries
psy/psysh                             v0.7.2             An interactive shell for modern PHP.
sebastian/comparator                  1.2.0              Provides the functionality to compare PHP values for equality
sebastian/diff                        1.4.1              Diff implementation
sebastian/environment                 1.3.8              Provides functionality to handle HHVM/PHP environments
sebastian/exporter                    1.2.2              Provides the functionality to export PHP variables for visualization
sebastian/global-state                1.1.1              Snapshotting of global state
sebastian/recursion-context           1.0.2              Provides functionality to recursively process PHP variables
sebastian/version                     1.0.6              Library that helps with managing the version number of Git-hosted PHP projects
swiftmailer/swiftmailer               v5.4.3             Swiftmailer, free feature-rich PHP mailer
symfony/browser-kit                   v3.1.3             Symfony BrowserKit Component
symfony/console                       v3.0.9             Symfony Console Component
symfony/css-selector                  v3.0.9             Symfony CssSelector Component
symfony/debug                         v3.0.9             Symfony Debug Component
symfony/dom-crawler                   v3.0.9             Symfony DomCrawler Component
symfony/event-dispatcher              v2.8.9             Symfony EventDispatcher Component
symfony/finder                        v3.0.9             Symfony Finder Component
symfony/http-foundation               v3.0.9             Symfony HttpFoundation Component
symfony/http-kernel                   v3.0.9             Symfony HttpKernel Component
symfony/polyfill-mbstring             v1.2.0             Symfony polyfill for the Mbstring extension
symfony/polyfill-php56                v1.2.0             Symfony polyfill backporting some PHP 5.6+ features to lower PHP versions
symfony/polyfill-util                 v1.2.0             Symfony utilities for portability of PHP codes
symfony/process                       v3.0.9             Symfony Process Component
symfony/routing                       v3.0.9             Symfony Routing Component
symfony/translation                   v3.0.9             Symfony Translation Component
symfony/var-dumper                    v3.0.9             Symfony mechanism for exploring and dumping PHP variables
symfony/yaml                          v3.1.3             Symfony Yaml Component
tijsverkoyen/css-to-inline-styles     1.5.5              CssToInlineStyles is a class that enables you to convert HTML-pages/files into HTML-pages/files with inline styles. This is very useful when you're sending emails.
vlucas/phpdotenv                      v2.3.0             Loads environment variables from `.env` to `getenv()`, `$_ENV` and `$_SERVER` automagically.
weblee/mandrill                       dev-master 809dedb Laravel 5 Mandrill API Wrapper
webmozart/assert                      1.1.0              Assertions to validate method input/output with nice error messages.
yajra/laravel-datatables-oracle       v6.17.0            jQuery DataTables API for Laravel 4|5

This bug will be related to the collective package. Nothing to do with laravel.

You could just use 1.x of this package: https://github.com/AltThree/Bus/tree/v1.1.0. I know it works. I wrote it and use it myself.

v2 of that package works with L5.3 too.

Thank you!


feat[sailor-bridge]: support to add type config

Support to add custom type config for cases we need to add more type converters

Add document


Bug fix for the 'allOf' violation crash

Description
If an allOf constraint existed in a profile with multiple constraints and the violation mode was run an exception would be thrown. This worked as expected when running the generator without violation
Changes

In the ProfileDecisionTreeFactory a check is made against whether the Constraint is an AtomicConstraint. Previously it was presumed that it would always be.
Added test to cover bug fix
Fixed a previously existing test that was not being asserted.

Issue
Resolves #521

The test that failed, originally tested that
Violate(not(and(test, test2)))
would reduce to
or(not(test), not(test2))
which was incorrect. it should have actually reduced to
and(test, test2)


StochasticAlgorithms

Defines SAGA_Function as a subclass of  StochasticGradientAlgorithmFunction which is a subclass of SumFunction.
Requires to find appropriate name for StochasticGradientAlgorithmFunction. My point in using this name is that this function is suitable for use in gradient type algorithms making them work on stochastic subsets. Stochastic might be swapped for Subset as different ways of subset choice could be made.
https://github.com/TomographicImaging/CIL/blob/df207c69435a80c5bc64aac2111dbf0e5559032b/Wrappers/Python/cil/optimisation/functions/StochasticGradientAlgorithmFunction.py#L4-L46
The use of SAGA_Function is entirely the same as the one found in the Example.ipynb
Notice that, for whatever reason I had to reduce the step size of gradient descent otherwise it'd explode.

Regarding the function name:
https://github.com/TomographicImaging/CIL/blob/df207c69435a80c5bc64aac2111dbf0e5559032b/Wrappers/Python/cil/optimisation/functions/StochasticGradientAlgorithmFunction.py#L4
I would not call it "stochastic" as it isn't necessarily stochastic. I'd rather have "subset" instead of "stochastic".
I'm not convinced with having "algorithm" in the name.
I propose "GradientSubsetFunction".

Regarding SAGA:
subset_grad_old is defined here
https://github.com/TomographicImaging/CIL/blob/df207c69435a80c5bc64aac2111dbf0e5559032b/Wrappers/Python/cil/optimisation/functions/StochasticGradientAlgorithmFunction.py#L66
but not used here
https://github.com/TomographicImaging/CIL/blob/df207c69435a80c5bc64aac2111dbf0e5559032b/Wrappers/Python/cil/optimisation/functions/StochasticGradientAlgorithmFunction.py#L80
The following line updates subset_grad_old
https://github.com/TomographicImaging/CIL/blob/df207c69435a80c5bc64aac2111dbf0e5559032b/Wrappers/Python/cil/optimisation/functions/StochasticGradientAlgorithmFunction.py#L82
but it doesn't update self.subset_gradients[self.subset_num] which is the memory storage. So right now the memory of gradient subsets doesn't get updated after the first iteration.

However, the suggested docstring changes are part of #1093

subset_grad_old and full_grad_old are just two new references (pointers) to some objects in memory. As such, using subset_grad_old or self.subset_gradients[self.subset_num] doesn't make a difference.
This line updates the content of the memory pointed by subset_grad_old, i.e. the object in position self.subset_num in the list self.subset_gradients, which is what you want.
https://github.com/TomographicImaging/CIL/blob/df207c69435a80c5bc64aac2111dbf0e5559032b/Wrappers/Python/cil/optimisation/functions/StochasticGradientAlgorithmFunction.py#L82
This line updates self.full_gradient
https://github.com/TomographicImaging/CIL/blob/df207c69435a80c5bc64aac2111dbf0e5559032b/Wrappers/Python/cil/optimisation/functions/StochasticGradientAlgorithmFunction.py#L89
The update you had in your initial implementation, would just swap the pointer in the list, so really the list was containing an ever changing list of objects that got created during the update method.

self.subset_gradients[self.subset_num] = subset_grad

I propose "GradientSubsetFunction".

It makes sense to me, although the name gradient is a bit strange on its own and that's why I added algorithm

Notice that, for whatever reason I had to reduce the step size of gradient descent otherwise it'd explode.

This both in your and my implementation.

I am closing this PR as for some strange reason it fails. I replicated it in #1095

Closed
Continue at https://github.com/TomographicImaging/CIL/tree/StochasticDev


Add shutdown option

This PR adds a shutdown flag that can be set to true in order to send the ACPI power button signal to VMs when it gets the kill signal from Karma.
Closes #1.

This is perfect.  Thanks for adding this in.


Remove redundant key/value pairs from config.json

The slug is not used anywhere. We initialize a track based on knowing the Track ID.
Since the repository is always named after the track ID, this field, too, is redundant,
as it can be inferred.
See https://github.com/exercism/meta/issues/19

Merging, as this is failing on tests, not on the config.


chore: add commit message lint hook

:tada: This PR is included in version 1.1.0 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket: