Ambassador 0.50.0-ea3 header matching is broken

Describe the bug
Ambassador crashes after a mapping is applied attempting to use header matching (including method and host). If Ambassador is running fine before the mapping is applied, it will continue to run with the mappings it had before the header-matching mapping was applied; if you restart it at this point, Ambassador will not accept connections on any port.
To Reproduce
Apply httpbin service with this mapping:
      ---
      apiVersion: ambassador/v0
      kind:  Mapping
      name:  httpbin_mapping
      ambassador_id: [ "ambassador-notls" ]
      prefix: /httpbin/
      service: httpbin.org
      host_rewrite: httpbin.org
      cors:
        origins: "*"
        methods: POST, GET, OPTIONS
        headers: Content-Type,Content-Length
      timeout_ms: 300000

Test the mapping with cURL and checking the diagnostics page the mapping is seen.
Now apply this change (which uses a method match) to the service:
      ---
      apiVersion: ambassador/v0
      kind:  Mapping
      name:  httpbin_mapping
      ambassador_id: ambassador-notls
      prefix: /forms/post
      rewrite: ""
      service: httpbin.org
      host_rewrite: httpbin.org
      cors:
        origins: "*"
        methods: POST, GET, OPTIONS
        headers: Content-Type,Content-Length
      timeout_ms: 300000
      method: GET

I changed the prefix because it actually leads to interesting results. Ambassador acknowledges the mapping changed which you can see in the diagnostics page:

and the envoy config:
{
    "@type": "/envoy.config.bootstrap.v2.Bootstrap",
    "static_resources": {
        "clusters": [
            {
                "connect_timeout": "3s",
                "lb_policy": "ROUND_ROBIN",
                "load_assignment": {
                    "cluster_name": "cluster_127_0_0_1_8877",
                    "endpoints": [
                        {
                            "lb_endpoints": [
                                {
                                    "endpoint": {
                                        "address": {
                                            "socket_address": {
                                                "address": "<IP_ADDRESS>",
                                                "port_value": 8877,
                                                "protocol": "TCP"
                                            }
                                        }
                                    }
                                }
                            ]
                        }
                    ]
                },
                "name": "cluster_127_0_0_1_8877",
                "type": "STRICT_DNS"
            },
            {
                "connect_timeout": "3s",
                "lb_policy": "ROUND_ROBIN",
                "load_assignment": {
                    "cluster_name": "cluster_ambassador_admin_notls_8877",
                    "endpoints": [
                        {
                            "lb_endpoints": [
                                {
                                    "endpoint": {
                                        "address": {
                                            "socket_address": {
                                                "address": "ambassador-admin-notls",
                                                "port_value": 8877,
                                                "protocol": "TCP"
                                            }
                                        }
                                    }
                                }
                            ]
                        }
                    ]
                },
                "name": "cluster_ambassador_admin_notls_8877",
                "type": "STRICT_DNS"
            },
            {
                "connect_timeout": "3s",
                "lb_policy": "ROUND_ROBIN",
                "load_assignment": {
                    "cluster_name": "cluster_httpbin_org",
                    "endpoints": [
                        {
                            "lb_endpoints": [
                                {
                                    "endpoint": {
                                        "address": {
                                            "socket_address": {
                                                "address": "httpbin.org",
                                                "port_value": 80,
                                                "protocol": "TCP"
                                            }
                                        }
                                    }
                                }
                            ]
                        }
                    ]
                },
                "name": "cluster_httpbin_org",
                "type": "STRICT_DNS"
            }
        ],
        "listeners": [
            {
                "address": {
                    "socket_address": {
                        "address": "<IP_ADDRESS>",
                        "port_value": 80,
                        "protocol": "TCP"
                    }
                },
                "filter_chains": [
                    {
                        "filters": [
                            {
                                "config": {
                                    "access_log": [
                                        {
                                            "config": {
                                                "format": "ACCESS [%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\"\n",
                                                "path": "/dev/fd/1"
                                            },
                                            "name": "envoy.file_access_log"
                                        }
                                    ],
                                    "http_filters": [
                                        {
                                            "name": "envoy.cors"
                                        },
                                        {
                                            "name": "envoy.router"
                                        }
                                    ],
                                    "route_config": {
                                        "virtual_hosts": [
                                            {
                                                "domains": [
                                                    "*"
                                                ],
                                                "name": "backend",
                                                "routes": [
                                                    {
                                                        "match": {
                                                            "case_sensitive": true,
                                                            "headers": null,
                                                            "prefix": "/ambassador/v0/check_ready"
                                                        },
                                                        "route": {
                                                            "prefix_rewrite": "/ambassador/v0/check_ready",
                                                            "priority": null,
                                                            "weighted_clusters": {
                                                                "clusters": [
                                                                    {
                                                                        "name": "cluster_127_0_0_1_8877",
                                                                        "request_headers_to_add": null,
                                                                        "weight": 100.0
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    },
                                                    {
                                                        "match": {
                                                            "case_sensitive": true,
                                                            "headers": null,
                                                            "prefix": "/ambassador/v0/check_alive"
                                                        },
                                                        "route": {
                                                            "prefix_rewrite": "/ambassador/v0/check_alive",
                                                            "priority": null,
                                                            "weighted_clusters": {
                                                                "clusters": [
                                                                    {
                                                                        "name": "cluster_127_0_0_1_8877",
                                                                        "request_headers_to_add": null,
                                                                        "weight": 100.0
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    },
                                                    {
                                                        "match": {
                                                            "case_sensitive": true,
                                                            "headers": null,
                                                            "prefix": "/ambassador/v0/"
                                                        },
                                                        "route": {
                                                            "prefix_rewrite": "/ambassador/v0/",
                                                            "priority": null,
                                                            "weighted_clusters": {
                                                                "clusters": [
                                                                    {
                                                                        "name": "cluster_127_0_0_1_8877",
                                                                        "request_headers_to_add": null,
                                                                        "weight": 100.0
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    },
                                                    {
                                                        "match": {
                                                            "case_sensitive": true,
                                                            "headers": [
                                                                {
                                                                    "name": ":method",
                                                                    "regex": false,
                                                                    "value": "GET"
                                                                }
                                                            ],
                                                            "prefix": "/forms/post"
                                                        },
                                                        "route": {
                                                            "cors": {
                                                                "allow_headers": "Content-Type,Content-Length",
                                                                "allow_methods": "POST, GET, OPTIONS",
                                                                "allow_origin": [
                                                                    "*"
                                                                ],
                                                                "enabled": true
                                                            },
                                                            "host_rewrite": "httpbin.org",
                                                            "prefix_rewrite": "",
                                                            "priority": null,
                                                            "weighted_clusters": {
                                                                "clusters": [
                                                                    {
                                                                        "name": "cluster_httpbin_org",
                                                                        "request_headers_to_add": null,
                                                                        "weight": 100.0
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    },
                                                    {
                                                        "match": {
                                                            "case_sensitive": true,
                                                            "headers": null,
                                                            "prefix": "/diag/"
                                                        },
                                                        "route": {
                                                            "prefix_rewrite": "/ambassador/v0/diag/",
                                                            "priority": null,
                                                            "weighted_clusters": {
                                                                "clusters": [
                                                                    {
                                                                        "name": "cluster_ambassador_admin_notls_8877",
                                                                        "request_headers_to_add": null,
                                                                        "weight": 100.0
                                                                    }
                                                                ]
                                                            }
                                                        }
                                                    }
                                                ]
                                            }
                                        ]
                                    },
                                    "stat_prefix": "ingress_http"
                                },
                                "name": "envoy.http_connection_manager"
                            }
                        ],
                        "use_proxy_proto": false
                    }
                ],
                "name": "ir.listener"
            }
        ]
    }
}

But, if you send a curl to http://<amabassador-external-ip>/forms/post it does not resolve. However, a curl to http://<ambassador-external-ip>/httpbin/ still resolves fine.
curl -v http://<IP_ADDRESS>/httpbin/
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to <IP_ADDRESS> (<IP_ADDRESS>) port 80 (#0)
> GET /httpbin/ HTTP/1.1
> Host: <IP_ADDRESS>
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< server: envoy
< date: Wed, 24 Oct 2018 12:50:00 GMT
< content-type: text/html; charset=utf-8
< content-length: 10122
< access-control-allow-origin: *
< access-control-allow-credentials: true
< via: 1.1 vegur
< x-envoy-upstream-service-time: 37
<
<!DOCTYPE html>

I have verified that method: is the cause of the issue by removing it and restarting ambassador I am able to curl to http://<amabassador-external-ip>/forms/post and add a new service.
The ambassador log repeatedly output this error message:
[2018-10-24 14:02:57.521][13][warning][upstream] source/common/config/grpc_mux_impl.cc:226] gRPC config for type.googleapis.com/envoy.api.v2.Listener update rejected: Error adding/updating listener ir.listener: Unable to parse JSON as proto (INVALID_ARGUMENT:route_config.virtual_hosts[0].routes[3].match.headers[0]: invalid name value: Cannot find field.): {"http_filters":[{"name":"envoy.cors"},{"name":"envoy.router"}],"stat_prefix":"ingress_http","route_config":{"virtual_hosts":[{"routes":[{"route":{"weighted_clusters":{"clusters":[{"name":"cluster_127_0_0_1_8877","request_headers_to_add":null,"weight":100}]},"priority":null,"prefix_rewrite":"/ambassador/v0/check_ready"},"match":{"case_sensitive":true,"prefix":"/ambassador/v0/check_ready","headers":null}},{"match":{"prefix":"/ambassador/v0/check_alive","headers":null,"case_sensitive":true},"route":{"priority":null,"prefix_rewrite":"/ambassador/v0/check_alive","weighted_clusters":{"clusters":[{"name":"cluster_127_0_0_1_8877","request_headers_to_add":null,"weight":100}]}}},{"route":{"weighted_clusters":{"clusters":[{"name":"cluster_127_0_0_1_8877","request_headers_to_add":null,"weight":100}]},"priority":null,"prefix_rewrite":"/ambassador/v0/"},"match":{"prefix":"/ambassador/v0/","headers":null,"case_sensitive":true}},{"match":{"prefix":"/forms/post","headers":[{"regex":false,"value":"GET","name":":method"}],"case_sensitive":true},"route":{"weighted_clusters":{"clusters":[{"name":"cluster_httpbin_org","request_headers_to_add":null,"weight":100}]},"priority":null,"prefix_rewrite":"","cors":{"allow_origin":["*"],"allow_methods":"POST, GET, OPTIONS","allow_headers":"Content-Type,Content-Length","enabled":true},"host_rewrite":"httpbin.org"}},{"match":{"prefix":"/qotm/","headers":null,"case_sensitive":true},"route":{"priority":null,"prefix_rewrite":"/","weighted_clusters":{"clusters":[{"request_headers_to_add":null,"weight":100,"name":"cluster_qotm"}]}}},{"route":{"weighted_clusters":{"clusters":[{"weight":100,"name":"cluster_ambassador_admin_notls_8877","request_headers_to_add":null}]},"priority":null,"prefix_rewrite":"/ambassador/v0/diag/"},"match":{"prefix":"/diag/","headers":null,"case_sensitive":true}}],"domains":["*"],"name":"backend"}]},"access_log":[{"name":"envoy.file_access_log","config":{"path":"/dev/fd/1","format":"ACCESS [%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\"\n"}}]}
[2018-10-24 14:02:57.521][13][warning][config] bazel-out/k8-opt/bin/source/common/config/_virtual_includes/grpc_mux_subscription_lib/common/config/grpc_mux_subscription_impl.h:70] gRPC config for type.googleapis.com/envoy.api.v2.Listener rejected: Error adding/updating listener ir.listener: Unable to parse JSON as proto (INVALID_ARGUMENT:route_config.virtual_hosts[0].routes[3].match.headers[0]: invalid name value: Cannot find field.): {"http_filters":[{"name":"envoy.cors"},{"name":"envoy.router"}],"stat_prefix":"ingress_http","route_config":{"virtual_hosts":[{"routes":[{"route":{"weighted_clusters":{"clusters":[{"name":"cluster_127_0_0_1_8877","request_headers_to_add":null,"weight":100}]},"priority":null,"prefix_rewrite":"/ambassador/v0/check_ready"},"match":{"case_sensitive":true,"prefix":"/ambassador/v0/check_ready","headers":null}},{"match":{"prefix":"/ambassador/v0/check_alive","headers":null,"case_sensitive":true},"route":{"priority":null,"prefix_rewrite":"/ambassador/v0/check_alive","weighted_clusters":{"clusters":[{"name":"cluster_127_0_0_1_8877","request_headers_to_add":null,"weight":100}]}}},{"route":{"weighted_clusters":{"clusters":[{"name":"cluster_127_0_0_1_8877","request_headers_to_add":null,"weight":100}]},"priority":null,"prefix_rewrite":"/ambassador/v0/"},"match":{"prefix":"/ambassador/v0/","headers":null,"case_sensitive":true}},{"match":{"prefix":"/forms/post","headers":[{"regex":false,"value":"GET","name":":method"}],"case_sensitive":true},"route":{"weighted_clusters":{"clusters":[{"name":"cluster_httpbin_org","request_headers_to_add":null,"weight":100}]},"priority":null,"prefix_rewrite":"","cors":{"allow_origin":["*"],"allow_methods":"POST, GET, OPTIONS","allow_headers":"Content-Type,Content-Length","enabled":true},"host_rewrite":"httpbin.org"}},{"match":{"prefix":"/qotm/","headers":null,"case_sensitive":true},"route":{"priority":null,"prefix_rewrite":"/","weighted_clusters":{"clusters":[{"request_headers_to_add":null,"weight":100,"name":"cluster_qotm"}]}}},{"route":{"weighted_clusters":{"clusters":[{"weight":100,"name":"cluster_ambassador_admin_notls_8877","request_headers_to_add":null}]},"priority":null,"prefix_rewrite":"/ambassador/v0/diag/"},"match":{"prefix":"/diag/","headers":null,"case_sensitive":true}}],"domains":["*"],"name":"backend"}]},"access_log":[{"name":"envoy.file_access_log","config":{"path":"/dev/fd/1","format":"ACCESS [%START_TIME%] \"%REQ(:METHOD)% %REQ(X-ENVOY-ORIGINAL-PATH?:PATH)% %PROTOCOL%\" %RESPONSE_CODE% %RESPONSE_FLAGS% %BYTES_RECEIVED% %BYTES_SENT% %DURATION% %RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)% \"%REQ(X-FORWARDED-FOR)%\" \"%REQ(USER-AGENT)%\" \"%REQ(X-REQUEST-ID)%\" \"%REQ(:AUTHORITY)%\" \"%UPSTREAM_HOST%\"\n"}}]}

Versions (please complete the following information):

Ambassador: 0.50.0-ea3
Kubernetes environment Google Kubernetes Engine

Fixed by #956.


ARROW-15057: [R] [CI] Move where we install DuckDB from in CI

@github-actions crossbow submit -g r

@github-actions crossbow submit -g r


[BUG] !note undo  doesn't undo a warn

Expected behaviour
Works
Current behaviour
Doesn't work :/

!warn undo @ user exists, til


Single Page Mode returns 404 for IE

Actual behavior (the bug)
When the user uses a link that is not the main url (/) or index (/index.html) the request that goes from the browser (IE11) is not being captured as the one to be served by single page handler as the Accept header contains '/' but is not equal to just that. This results in the HTTP 404.
Expected behavior
Single page handler should recognize this request and serve the contents of index.html.
To Reproduce
Create sample single page app with page that is not the main url. Open the page address other than the main url and index in Internet Explorer 11 or lower and have HTTP 404.
Additional context
https://github.com/tipsy/javalin/issues/532 - the changes introduces in PR that solved this issue have improved things a bit but checking for exact value of "/" is not the perfect solution. I think the solution would be to check that Accept header contains "/"

Thanks @ligasgr ! Would you like to submit a PR with a fix ?

Hi David,
Sure, I'll be happy to prepare the change. Can you please let me know if there are any code style guidelines? Would prefer to follow those than having you to fix the code afterwards.
Thanks,
Grzegorz

There is an editorconfig file in the project. Sometimes I run the default intellij formatter, I don't mind having minor inconsistencies temporarily.


MachineKey's are they gone or aren't they?

Hi, i was under the impression that MachineKey's were going to be a thing of the past with ASP.Net 5... I was doing some research for a blog post I am writing and came across this comment:
https://github.com/aspnet/Security/blob/master/src/Microsoft.AspNet.Security.Cookies/CookieAuthenticationOptions.cs#L129
which says that the ISecureDataFormat will be based on a machine key.
Can anyone explain how cookie (and passwords for that matter) will actually be encrypted/decrypted and work between multiple servers and whether the machine key is still a thing or not?

http://docs.asp.net/en/latest/security/data-protection/index.html


resource/user.rb action :remove

In the following section:
action :remove do
  if user_exists?
    Chef::Log.info("#{new_resource} does not exist - skipping")
  else
    converge_by("Remove #{new_resource}") do
      remove
    end
  end
end

The user needs to be removed in case it exists and nothing needs to be done when it does not exist. Just reversing the if else.

Solved by #13


File Based Encryption

https://source.android.com/security/encryption/file-based

Implemented using ICE (Inline Crypto Engine).


Do librocclr really need libamd_comgr?

Related to https://github.com/ROCm-Developer-Tools/hipamd/issues/44
I'm maintaining hip and rocm-opencl-runtime package in Gentoo, and recently, I find although in wirtten cmake, rocclr is linked to libamd_comgr, in reality linker dropped it since there is not function from libamd_comgr called.
I browsed ROCclr source codes by ag amd::Comgr:: | grep -o -E "amd::Comgr::[^\(]*\(" | sort -u, finding these functions are defined in amd_comgr.h, not comgr .cpp files, so librocclr does not have to link libamd_comgr. Is this just a coincidence, or a design? For the later one, I can make hip and rocm-opencl-runtime  independent of rocm-comgr, since some tests of rocm-comgr are only available when hip is installed.

ROCclr loads libamd_comgr.so.2, see device/comgrctx.cpp. There's some compatibility code to allow this to also work on Windows, but there's no dynamic linking if you trace the code back. I believe it's done this way to make it more portable, but I'm not 100% sure as I'm not a developer.
If you package rocm-opencl or hipamd, which statically compile in ROCclr, you need to make sure you add an explicit requirement on comgr. I'm not sure about Gentoo, but on Fedora and Debian, runtime requirements are usually determined automatically via dynamic linking requirements. Since ROCclr does not dynamically link against comgr, a requirement must be specified explicitly.
E.g. from Fedora:
https://src.fedoraproject.org/rpms/rocm-opencl/blob/rawhide/f/rocm-opencl.spec#_37
ROCclr is not a stand alone library, and is designed to be a static lib within rocm's opencl/hip to act as an extraction layer of sorts (works against both Windows and Linux backends). As well, comgr is a compiler interface for both libraries, so I don't think it can be dropped.


Add smoke test after publishing oxlint

https://github.com/oxc-project/oxc/blob/2406e94c0d2bdf2159622b330add221c30099b88/.github/workflows/release_oxlint.yml#L186-L202


Cropped treeview items and troubles with launch

First of all, I would like to note that this theme looks really amazing. I like it. When I tried to launch an example, I stumbled upon two issues. First of all, script does not launch when invoked with Windows Python Launcher for some reason (but from terminal and IDLE it works fine, so I have no idea why). Something slightly similar happened to me previously. It may have something to do with a filesystem. Second, program looks blurry, just as any tkinter on my computer. I resolved that by standard method in this situation:
from ctypes import windll
windll.shcore.SetProcessDpiAwareness(1)

It resolves that problem, but unfortunately, introduces another: all items from treeview are cropped.  Screenshot attached.

Okay, it may have nothing to do with that particular theme then. Also, issue with Python Laucher fixed. It was indeed a filesystem problem. Unrelated to that theme also:
import os
path = os.path.dirname(__file__) + '\\azure.tcl'
root.tk.call('source', path)
# instead of
root.tk.call('source', 'azure.tcl')

Issue fixed. It was indeed a tkinter problem:
style = ttk.Style()
style.configure('Treeview', rowheight=40)

Thanks for hint, @oehhar.

Great !


The JAVA_OPTS conf does not seem to be taking effect in the CN nodes

The JAVA_OPTS conf does not seem to be taking effect in the CN nodes
It works fine in the FE nodes
Running in shared data mode with only FE and CN Nodes
below is the cn.conf
JAVA_OPTS_FOR_JDK_9_AND_LATER="-Djava.security.krb5.conf=/etc/krb5.conf"
JAVA_OPTS_FOR_JDK_9="-Djava.security.krb5.conf=/etc/krb5.conf"
JAVA_OPTS_FOR_JDK_11="-Djava.security.krb5.conf=/etc/krb5.conf"
JAVA_OPTS="-Djava.security.krb5.conf=/etc/krb5.conf"
sys_log_level = DEBUG
# ports for admin, web, heartbeat service
thrift_port = 9060
webserver_port = 8040
heartbeat_service_port = 9050
brpc_port = 8060
starlet_port = 9070
storage_root_path = ${STARROCKS_HOME}/storage

Running jps -lvm on CN Pods do not show above OPTS
FE picks it up fine with same command to check
StarRocks version (Required)

3.2.2 on K8s Operator

there is no java process running on be/cn node, BE/CN will launch JVM via jni. I guess that might be the reason jps doesn't show anything.

can you enter the cn pod and take a look at /proc/<cn_pid>/environ, check the environment variables such as JAVA_* and LIBHDFS_*.

can you enter the cn pod and take a look at /proc/<cn_pid>/environ, check the environment variables such as JAVA_* and LIBHDFS_*.

Thanks for the info. I found that JAVA options espcially for kerberos are not required for CN nodes.
But still here is what i did to add it to CN
Used a _JAVA_OPTIONS env variable in the CN Pods and it gets picked up.  No other variable works.
My ubuntu pod has JDK11


Software related biblatex entries are not recognized and imported as Journal Articles.

Report ID: RUG7Z4YB-euc
Full Bib(La)TeX item you are trying to import:
@SOFTWAREVERSION{delebecque:hal-02090402-condensed,
  ABSTRACT = {Software for Numerical Computation freely distributed.},
  AUTHOR = {Delebecque, Fran{ç}ois and Gomez, Claude and Goursat, Maurice and Nikoukhah, Ramine and Steer, Serge and Chancelier, Jean-Philippe},
  INSTITUTION = {Inria},
  LICENSE = {Scilab license},
  URL = {https://www.scilab.org/},
  DATE = {1994-01},
  FILE = {https://hal.inria.fr/hal-02090402/file/scilab-1.1.tar.gz},
  HAL_ID = {hal-02090402},
  HAL_VERSION = {v1},
  NOTE = {First Scilab version. It was distributed by anonymous ftp.},
  REPOSITORY = {https://github.com/scilab/scilab},
SWHID={swh:1:dir:1ba0b67b5d0c8f10961d878d91ae9d6e499d746a;origin=https://hal.archives-ouvertes.fr/hal-02090402},
  TITLE = {Scilab},
  VERSION = {1.1},
}

@SOFTWARE{delebecque:hal-02090402,
  ABSTRACT = {Software for Numerical Computation freely distributed.},
  AUTHOR = {Delebecque, Fran{ç}ois and Gomez, Claude and Goursat, Maurice and Nikoukhah, Ramine and Steer, Serge and Chancelier, Jean-Philippe},
  INSTITUTION = {Inria},
  LICENSE = {Scilab license},
  URL = {https://www.scilab.org/},
  DATE = {1994},
  HAL_ID = {hal-02090402},
  HAL_VERSION = {v1},
  REPOSITORY = {https://github.com/scilab/scilab},
  TITLE = {Scilab},
}

@SOFTWAREVERSION{delebecque:hal-02090402v1,
  CROSSREF = {delebecque:hal-02090402},
  DATE = {1994-01},
  FILE = {https://hal.inria.fr/hal-02090402/file/scilab-1.1.tar.gz},
  NOTE = {First Scilab version. It was distributed by anonymous ftp.},
SWHID={swh:1:dir:1ba0b67b5d0c8f10961d878d91ae9d6e499d746a;origin=https://hal.archives-ouvertes.fr/hal-02090402},
  VERSION = {1.1},
}

@SOFTWARE{cgal,
  AUTHOR = {{The CGAL Project}},
  EDITOR = {{CGAL Editorial Board}},
  URL = {https://cgal.org/},
  DATE = {1996},
  TITLE = {The Computational Geometry Algorithms Library},
}

@SOFTWAREVERSION{cgal:5-0-2,
  URL = {https://docs.cgal.org/5.02},
  CROSSREF = {cgal},
  DATE = {2020},
SWHID={swh:1:rel:636541bbf6c77863908eae744610a3d91fa58855;origin=https://github.com/CGAL/cgal/},
  VERSION = {{5.0.2}},
}

@SOFTWAREMODULE{cgal:lp-gi-20a,
  AUTHOR = {Karavelas, Menelaos},
  LICENSE = {GPL},
  URL = {https://doc.cgal.org/5.0.2/Manual/packages.html#PkgVoronoiDiagram2},
  CROSSREF = {cgal:5-0-2},
  INTRODUCEDIN = {cgal:3-1},
  SUBTITLE = {{2D} Voronoi Diagram Adaptor},
}

@SOFTWAREMODULE{cgal:lp-gi-20a-condensed,
  AUTHOR = {Karavelas, Menelaos},
  EDITOR = {{CGAL Editorial Board}},
  LICENSE = {GPL},
  URL = {https://doc.cgal.org/5.0.2/Manual/packages.html#PkgVoronoiDiagram2},
  DATE = {2020},
  INTRODUCEDIN = {cgal:3-1},
  SUBTITLE = {{2D} Voronoi Diagram Adaptor},
SWHID={swh:1:rel:636541bbf6c77863908eae744610a3d91fa58855;origin=https://github.com/CGAL/cgal/},
  TITLE = {The Computational Geometry Algorithms Library},
  VERSION = {{5.0.2}},
}

@SOFTWARE{parmap,
  AUTHOR = {Di Cosmo, Roberto and Danelutto, Marco},
  INSTITUTION = {Inria and University of Paris and University of Pisa},
  LICENSE = {LGPL-2.0},
  URL = {https://rdicosmo.github.io/parmap/},
  DATE = {2020},
  REPOSITORY = {https://github.com/rdicosmo/parmap},
  TITLE = {The Parmap library},
  VERSION = {1.1.1},
}

@CODEFRAGMENT{simplemapper,
  CROSSREF = {parmap},
  SUBTITLE = {Core mapping routine},
SWHID={swh:1:cnt:43a6b232768017b03da934ba22d9cc3f2726a6c5;lines=192-228;origin=https://github.com/rdicosmo/parmap},
}

Expected behavior:
Since Zotero has a Software category, these entries should be at least imported as Software, while waiting for a more refined Software taxonomy to find its way in Zotero
Actual behavior:
All entries are imported as "Journal Article", here are the import errors shown in Zotero
Import errors found:
Don't know what Zotero type to make of 'softwareversion' for @delebecque:hal-02090402-condensed, importing as journalArticle
Don't know what Zotero type to make of 'software' for @delebecque:hal-02090402, importing as journalArticle
Don't know what Zotero type to make of 'softwareversion' for @delebecque:hal-02090402v1, importing as journalArticle
Don't know what Zotero type to make of 'software' for @cgal, importing as journalArticle
Don't know what Zotero type to make of 'softwareversion' for @cgal:5-0-2, importing as journalArticle
Don't know what Zotero type to make of 'softwaremodule' for @cgal:lp-gi-20a, importing as journalArticle
Don't know what Zotero type to make of 'softwaremodule' for @cgal:lp-gi-20a-condensed, importing as journalArticle
Don't know what Zotero type to make of 'software' for @parmap, importing as journalArticle
Don't know what Zotero type to make of 'codefragment' for @simplemapper, importing as journalArticle

:robot: this is your friendly neighborhood build bot announcing test build <IP_ADDRESS>71 ("basic software import")
Install in Zotero by downloading test build <IP_ADDRESS>71, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

6671 will import these types as Software, but may not yet give the exact desired results. It'd be helpful if you could install 6671, import the items above, correct the imported items to what you think they should be, select these items and send a new debug log from the right-click menu.

Excellent, this version does the import. I submitted a suggestion for improvement via
UR99RWLV-euc

:robot: this is your friendly neighborhood build bot announcing test build <IP_ADDRESS>81 ("fixes #1544")
Install in Zotero by downloading test build <IP_ADDRESS>81, opening the Zotero "Tools" menu, selecting "Add-ons", open the gear menu in the top right, and select "Install Add-on From File...".

Does 6681 doe what you want?

Does 6681 do what you want?

Yes, that works quite fine!
I do not see how one could do better using the existing fields in CSL : it is really not clear where to put the extra software bib(la)tex fields like repository, swhid, etc.  so for the moment it's quite fine to keep them in the extra field as BBT does.
Thanks a lot for making this change.

Superb, I'll merge this into a release tonight. Anything else you need I'll be happy to discuss.


Check existing container for command and alter

At the moment existing container is not checked if matches requested spec.

Won't fix, use k3s:rm.


use userData

in the documentation (and also examples) you mention how to set userData
by
rpt.userdata({hi:4})
I noticed it is all lowercase 'userdata' and not camelCase 'userData'
I could not find an example where you were actually using this data in a report or a detail report.
I tried
report.userData.hi
report.userData().hi
...
I do have a use data where I would like to store some data in an object that is accessed later in generating links in the
.details(function(report,data){
var myUserData = report.userData; ??
})
I must be missing something...

Hmm, let me look I could have sworn I had an example.   But it is also possible that I broke this with the 1.0 release.  I'll take a look and see if I can get back to you shortly.

I'll test it but looking at the code you should have been:
details(report, data) {
  var myUserData = report.userData();
  // do something with it...
}

You are correct; I was able to spend a bit of time mocking up a report a couple days ago.  I have fixed the issue in the latest version.   Thanks for letting me know.


Add tests of different pint versions

In https://github.com/openscm/openscm-units/pull/40#discussion_r1194999636 this idea was raised. It turns out to be non-trivial because Pint doesn't support Python 3.7 for newer versions. Will require adding a couple of lines to our CI CD workflow.
pint-version: [0.18, 0.19, 0.20, 0.21]

pip install 'pint==${{ matrix.pint-version }}'

I think this is unblocked now since we support Python>=3.9


Change $userAgent parameter from null to empty string

For the error Argument 1 passed to UAParser\Parser::parse() must be of the type string, null given
I found the other 2 pull requests #493 and #500 a bit hacky and convoluted to fix this. My thinking here is that it doesn't make sense to have a null default value for the $userAgent parameter since the parse() function will never take it. So just changing it to an empty string will resolve this particular issue without affecting the !$userAgent condition which still resolves to true for an empty string.

Please merge it. I not, we are unable to install the package.
Thanks

Please add this. We are not able to use this package.


Rate Limit errors trigger full update

I use drive in this daily script to backup a directory tree on Linux mint 17.2 64 bit.
!/bin/sh -x
exec > /home/xxxx/edev/drive/Gdrivedate +%Y_%_%d_%H%M.log
exec 2>&1
at 1300 -f ~/edev/drive/Gdrive.sh

cd ~/Gdata
drive version
date
drive push -depth -1 -no-prompt --exclude-ops delete --verbose ~/Gdata
date
drive list -depth -1 --sort name -version
exit
No files have changed for several days so the last few log files looked like this....
warning: commands will be executed using /bin/sh
job 163 at Thu Nov 12 13:00:00 2015
drive version: 0.3.2
Commit Hash: <CURRENT_COMMIT>
Go Version: <GO_VERSION>
OS: <OS_INFO>
BuildTime:
Wed Nov 11 13:00:00 AEST 2015
Resolving...
Everything is up-to-date.
Wed Nov 11 13:00:54 AEST 2015
/0Archive  v47280
/Documents  v49511
/Pictures  v43220
[snip long list of files]
Today drive attempted to upload the entire dir tree...
warning: commands will be executed using /bin/sh
job 164 at Fri Nov 13 13:00:00 2015
drive version: 0.3.2
Commit Hash: <CURRENT_COMMIT>
Go Version: <GO_VERSION>
OS: <OS_INFO>
BuildTime:
Thu Nov 12 13:00:00 AEST 2015
Resolving...
googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
0 /<PHONE_NUMBER>  0.00 %
512 /<PHONE_NUMBER>  0.00 % 497h51m2s
131584 /<PHONE_NUMBER>  0.01 % 2h25m15s
164352 /<PHONE_NUMBER>  0.01 % 2h19m32s
263168 /<PHONE_NUMBER>  0.02 % 1h41m39s
[snip]
4726664 /<PHONE_NUMBER>  0.41 % 2h52m19s
4759432 /<PHONE_NUMBER>  0.42 % 2h51m56sremoteMod/remoteMkdirAll: /Pictures got googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
push: /Pictures/2009-05-05--15.03.58 err: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
4792200 /<PHONE_NUMBER>  0.42 % 2h51m33s
[snip several more rate errors]
Have the rate errors disrupted or aborted the resolving so the entire tree will be uploaded ?
regards CB

Hello again @CountryBumkin,
For starters: did the log show any lines such as "Trying again in <X.Y> seconds"? If not there might be a problem with your installation of packages because it should show you this information if at all you had been using go get -u to update and install drive and not just a git pull as the dependencies are installed differently.
drive will try its best to retry an upload, --verbose just shows you the steps. If it all fails, then the line that appeared in your log file
$ push: /Pictures/2009-05-05--15.03.58 err: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded

Drive is updated automatically from the PPA, I did not build it.
The text quoted was cut and paste from the log files, maybe some terminal messages are not captured by the redirections in the script file.

CC @shaggytwodope, if I may ask does the PPA have the latest of https://github.com/odeke-em/exponential-backoff? Thanks.

@odeke-em Certainly does, +Subproject commit 96e25d36ae36ad09ac02cbfe653b44c4043a8e09
Can see from the old commit here:
https://github.com/shaggytwodope/drivedebpackage/commit/a1f36e69bff72256e401d02298d910d16aa63d1d
I don't always update the commits for the ppa package. But the snapshot uploaded to the ppa SHOULD have the git repo info as well if need be for reference. If something is broken I can push an update parhaps using all current copys of stuff.

I started using drive some days ago in order to push local files to Google Drive.
I did it without problems until today. Now:
got googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
Local  o.s. and  drive binaries have not been updated since I started using it.
Ubuntu 12.04 LTS 64 bit
drive version: 0.3.1
Not Quota exceeded for the user. I'm using a Google Apps For Education account. No limits.
Account type:	UNLIMITED
Bytes Used:<PHONE_NUMBER>           (1.16GB)
Bytes Free:<PHONE_NUMBER>7760       (10.00TB)
Looking at Google for the error other applications such rclone have the same problem.
What to do?
Thanks,
Josep Pujadas-Jubany

There are other drive api projects with people reporting the same errors.  I just started seeing them the last couple days in my scripts at randoms times.  I think google changed something.

Is the upload (push) limited to 2 GiB per day?
I'm wondering it because other users that I totally uploaded to Gdrive had less than 2 GiB.
And now I'm seeing that for the "problematic" user I had 2 GiB uploaded and my total to upload is about 7 GiB.

Hello @bellera and @matthewlenz, welcome to drive and thanks for that!
Yeah I have the exact same situation as @bellera, same Education account Unlimited and yet when I just drive stat file I got a rateLimitExceeded error too. I'll ping them shortly.

Hi again, I'm currently running into the same error. Have not done any updating recently, but I am running the latest version I guess (0.3.2). I also don't know about any mass-syncs I did, so I see no reason why I would have exceeded any limits. Also I don't use an education account, so maybe this is a global bug "upstream" in the drive protocol / servers atm?

For a second I thought my account had been hacked since a single drive stat on a lone file can't be that heavy. However, I think maybe they cut down the qps offered to each project and given that drive's daily limit was 1,000,000,000 requests daily(1 billion) and 1000 requests per second (qps) per user, so they probably changed something. However I don't have access to that API project as Google owns it but @rakyll does have access to it. I've emailed a Google Developer to help me contact someone.

I started uploading another user with 11 GiB to push.
After some uploading (about 45 MiB) began the error.

Just jumping in to report that I have the same issue about googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded. I hope you guys find a solution quick, but if Google lowered number of requests per day per user then I guess we are out of luck :(

@odeke-em said:
maybe they cut down the qps offered to each project...
So, @JernejJerin, the problem is not "per user". It's for "all users" using this project!
Now I'm at home, with my @gmail.com and tried to pull mi Google Drive as I did the 29-october for testing this application.  I got:
$ drive pull
Resolving...
googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded
So, it seems that, for the moment, this solution has been unusable!
Terrible! I wanted to upload all my school users!

@bellera:
Drive API has daily quota allowance which is 1 billion requests per day. The project is currently using ~0.6%. The API also has a QPS limit per user which is 10. I am going to talk to the team to increase the per user quota.

This is being investigated now, thanks for the report.

Sorry for the late reply y'all. Was out all day for a practical class assignment.
@bellera it is unfortunate but hopefully this can be resolved soon.
@rakyll thanks for advocating for us and for getting this looked at!

drive push -fix-clashes -quiet
Resolving...
4158358499 /<PHONE_NUMBER> [============================================================>-------------------------------------------------------] 52.16 % 2h41m52s
Working! Thanks!

We are also experiencing this issue, which started on Wednesday, Nov 11.
We have a script which runs nightly on a number of servers and all of them started having this error on the same night.
I went in and deleted some old files in one directory and tried to push them with "drive push ." on each of the servers. 5 out of 7 servers reported the error but were able to sync. 2 of them failed to do anything.

Coming back an hour later it seems to be OK now. In the interim I had also deleted some of the files directly in the Google Drive folder. My main concern was, as mentioned above, that sometime it never succeeded at all.

Looks like this was solved, thanks everyone for your patience. Thanks to the advocacy and help from @rakyll we got our quota bumped up quite a bit so we shouldn't be getting such problems anymore
@rakyll you the real MVP

@odeke-em awesome, thank god is wasnt something I did lol.

Ahaha nah, my bad if it came off that way, I was just trying to diagnose the problem. It wasn't anything in our direct control that caused it.

Not at all mate, just being silly.

@odeke-em aww, thanks :)
Please update this thread if anyone happens to come across an unlikely rate limit error. They bumped the quota a bit, but not sure if the issue is permanently resolved.

@rakyll Ah I see, for sure. It has been working alright for me for the past two days but we'll def keep vigilant for any errors. Whenever you get the confirmation, please let me know :)

It is good to solve the particular problem but it would be useful to address to fail unsafe aspect of the resolving. It is common to run drive with unattended scripts and the present fail unsafe ( for whatever reason) aspect of resolving to push (or probably pull) the entire folder tree contents would be quite disruptive.
It would be more appropriate that if there is any uncertainty in resolving to do nothing and exit with a suitable status.

I confirm that I don't have more "rate limit errors".
However, migrating users from local server storage (Ubuntu 12.04 LTS 64 bit) to Google Drive I have to use
drive push
three times before receiving the message "everything is to date"
I'm pushing 10-15 GiB for each user. Anybody has access to hte user folder when I'm pushing it. So, no changes are made since the migration process is started.
If you think I have to open a new thread for this, let me know. Thanks!

@CountryBumkin good points right there. Currently drive will attempt a best try with an exponential back off then finally fail. If am thinking what I think you are thinking (correct me if am wrong) is you'd like a failure asap and the appropriate error code. Sure, but a mode to fail on first try is going to cause you a lot of headache as rateLimitExceeded errors are common and expb just attempts to alleviate the problem, you can examine this by passing in flag --verbose during a push/pull. In this issue's case, the max retries were exhausted on every single attempt because of a problem with quota allocations/limiting on the remote end. As mentioned in #479, I'd be happy to implement the bash error codes but time isn't something I have a lot of since it will require defining a new error conformer centric to drive then going through each usage of error. I'd review  a PR though or give guidance if you'd like :) End of Semester + Exams are fast approaching for me that's why I can only promise that in a month and a half or two.
@beller good to know it is working again for you too. Before you open an issue, please make sure you are using the latest at tip because of https://github.com/odeke-em/drive/issues/477

The most important aspect is to ensure the integrity of the overall resolving process. There should be some examination of how this error corrupted the resolving process but the program still tried to proceed.
If any error occurs in the resolving process whose retry success cannot be guaranteed then the entire resolving process should be aborted, re-initialized and started again, up to some reasonable number of re-tries.
If rate limit errors are potentially unrecoverable then the exponential back off parameters of the overall resolving process could include conservative rate limiting inside drive that becomes more severe as the retry count rises to minimize the chance of errors.
Only after multiple failed tries of repeating the entire resolving process should drive flag a fatal error and abort.

I, like the OP, am using the drive command in a script to backup some files to Google Drive. Our scripts run at night via cron, but for the last few days it fails every night. If I go into each server and run the effective command (drive push -quiet -no-clobber *.gz) it succeeds. I changed our scripts to execute at staggered intervals (10 min apart) and that did not fix it.
If I look at the Google Console for the Drive API project I set up for this, it doesn't show an excessive amount of requests or traffic. We have a Google Apps for Business account and are nowhere near our storage limit either.
This is what the log shows:

...
Pushing all backup files to Google Drive
Resolving...
googleapi: Error 403: User Rate Limit Exceeded, userRateLimitExceeded
Pushed successfully.
...

Any ideas what to look for to resolve this issue for me?

Hello @miketotman-sd so I guess for other projects the quota hasn't yet fixed. As mentioned, @rakyll hasn't yet received a confirmation that it is fixed so for now please use drive's config which has a higher quota and should be working alright for now.

What exactly do you mean by "use drive's config" -- do you mean I should use drive's default client ID & secret and re-initialize my drive folders? Couldn't I just ask for a quota increase for our own project credentials?
Also, I don't understand why this fails when run automatically but succeeds when run manually. Presumably it should be doing the same thing, right? Why the difference?

@miketotman-sd yes in deed, the default client Id and a reinit. I don't know about asking for a quota increase, that might work but the actual quota rationing problem as mentioned previously hasn't yet been resolved on Google Drive's backend -- it might just a higher quota for drive might be masking the problems or this project got a different configuration set up, this is just my speculation as I wasn't there when they were trying to mitigate the problems.
About cron, I don't run drive on cron jobs, I just do everything manually as that's my motivation for this project :) However, please try with drive's default credentials and let me know what happens.

Using the default drive config worked for all our servers. I'd still prefer to use our own config but this will do for now.

I just got a bunch of these errors again, like the user in the referenced issue (#513), when calculating the update for a push after deleting a bunch of files.

This seems to be related to bandwidth for me, if I am on a connection with low upload bandwidth it does not happen. I just got the following error as well:
after a lots of:
googleapi: Error 403: User Rate Limit Exceeded, userRateLimitExceeded

during resolve while pushing a directory (using -no-clobber):
unexpected fault address 0x0
fatal error: fault
[signal 0x7 code=0x80 addr=0x0 pc=0x51af05]

goroutine 1 [running]:
runtime.throw(0x8fd638, 0x5)
	/usr/lib/go/src/runtime/panic.go:527 +0x90 fp=0xc82004d738 sp=0xc82004d720
runtime.sigpanic()
	/usr/lib/go/src/runtime/sigpanic_unix.go:21 +0x10c fp=0xc82004d788 sp=0xc82004d738
github.com/odeke-em/drive/src.(*Change).op(0xc821962100, 0x2)
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/src/types.go:437 +0x65 fp=0xc82004d7b0 sp=0xc82004d788
github.com/odeke-em/drive/src.(*Change).Op(0xc821962100, 0x2)
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/src/types.go:525 +0x3a fp=0xc82004d7c8 sp=0xc82004d7b0
github.com/odeke-em/drive/src.sift(0xc822496000, 0x5563, 0x5800, 0xc821bc6000, 0x10, 0x10, 0x0, 0x0, 0x0)
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/src/changes.go:581 +0xa9 fp=0xc82004d848 sp=0xc82004d7c8
github.com/odeke-em/drive/src.(*Commands).resolveConflicts(0xc82000b100, 0xc822496000, 0x5563, 0x5800, 0xc822d2a001, 0x5563, 0x5800)
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/src/push.go:163 +0xdd fp=0xc82004d978 sp=0xc82004d848
github.com/odeke-em/drive/src.(*Commands).Push(0xc82000b100, 0x0, 0x0)
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/src/push.go:102 +0xbd1 fp=0xc82004dc98 sp=0xc82004d978
main.(*pushCmd).Run(0xc8200d48c0, 0xc82000a2f0, 0x1, 0x1, 0xc8200c3710)
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/cmd/drive/main.go:793 +0x237 fp=0xc82004dd50 sp=0xc82004dc98
github.com/odeke-em/command.Run()
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/command/command.go:202 +0xaa fp=0xc82004dd90 sp=0xc82004dd50
github.com/odeke-em/command.ParseAndRun()
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/command/command.go:209 +0x19 fp=0xc82004dd98 sp=0xc82004dd90
main.main()
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/cmd/drive/main.go:120 +0x1ffe fp=0xc82004df50 sp=0xc82004dd98
runtime.main()
	/usr/lib/go/src/runtime/proc.go:111 +0x2b0 fp=0xc82004dfa0 sp=0xc82004df50
runtime.goexit()
	/usr/lib/go/src/runtime/asm_amd64.s:1696 +0x1 fp=0xc82004dfa8 sp=0xc82004dfa0

goroutine 17 [syscall, 5 minutes, locked to thread]:
runtime.goexit()
	/usr/lib/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 5 [syscall, 5 minutes]:
os/signal.loop()
	/usr/lib/go/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
	/usr/lib/go/src/os/signal/signal_unix.go:28 +0x37

goroutine 6 [select, 5 minutes, locked to thread]:
runtime.gopark(0x9d27b0, 0xc82002e728, 0x8ff058, 0x6, 0x440018, 0x2)
	/usr/lib/go/src/runtime/proc.go:185 +0x163
runtime.selectgoImpl(0xc82002e728, 0x0, 0x18)
	/usr/lib/go/src/runtime/select.go:392 +0xa64
runtime.selectgo(0xc82002e728)
	/usr/lib/go/src/runtime/select.go:212 +0x12
runtime.ensureSigM.func1()
	/usr/lib/go/src/runtime/signal1_unix.go:227 +0x353
runtime.goexit()
	/usr/lib/go/src/runtime/asm_amd64.s:1696 +0x1

goroutine 7 [runnable]:
syscall.Syscall(0x1, 0x2, 0xc821fe2cb0, 0x4, 0x4, 0x4, 0x0)
	/usr/lib/go/src/syscall/asm_linux_amd64.s:18 +0x5
syscall.write(0x2, 0xc821fe2cb0, 0x4, 0x8, 0xc820070f70, 0x0, 0x0)
	/usr/lib/go/src/syscall/zsyscall_linux_amd64.go:1064 +0x5f
syscall.Write(0x2, 0xc821fe2cb0, 0x4, 0x8, 0x1, 0x0, 0x0)
	/usr/lib/go/src/syscall/syscall_unix.go:176 +0x4d
os.(*File).write(0xc820030018, 0xc821fe2cb0, 0x4, 0x8, 0x0, 0x0, 0x0)
	/usr/lib/go/src/os/file_unix.go:232 +0xaa
os.(*File).Write(0xc820030018, 0xc821fe2cb0, 0x4, 0x8, 0x1, 0x0, 0x0)
	/usr/lib/go/src/os/file.go:139 +0x8a
fmt.Fprintf(0x7fe9d8fbb1e8, 0xc820030018, 0x8fa408, 0x3, 0xc820041f20, 0x1, 0x1, 0x4, 0x0, 0x0)
	/usr/lib/go/src/fmt/print.go:189 +0xb2
github.com/odeke-em/cli-spinner.(*Spinner).spin.func1(0xc82000b180)
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/cli-spinner/spinner.go:99 +0x351
created by github.com/odeke-em/cli-spinner.(*Spinner).spin
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/cli-spinner/spinner.go:103 +0xce

goroutine 9 [chan receive, 5 minutes]:
github.com/odeke-em/drive/src.(*Commands).Push.func1(0xc8200d62a0, 0xc820011da0, 0xc82000b100)
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/src/push.go:51 +0x43
created by github.com/odeke-em/drive/src.(*Commands).Push
	/tmp/yaourt-tmp-gaute/aur-drive/src/drive-0.3.3/src/github.com/odeke-em/drive/src/push.go:55 +0x335

goroutine 16390 [IO wait]:
net.runtime_pollWait(0x7fe9d595d6c0, 0x72, 0xc8200122b0)
	/usr/lib/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc821508920, 0x72, 0x0, 0x0)
	/usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc821508920, 0x0, 0x0)
	/usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc8215088c0, 0xc821bfa000, 0x1000, 0x1000, 0x0, 0x7fe9d8fb7050, 0xc8200122b0)
	/usr/lib/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc82127a668, 0xc821bfa000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/lib/go/src/net/net.go:172 +0xe4
crypto/tls.(*block).readFromUntil(0xc821bcd170, 0x7fe9d59441b8, 0xc82127a668, 0x5, 0x0, 0x0)
	/usr/lib/go/src/crypto/tls/conn.go:455 +0xcc
crypto/tls.(*Conn).readRecord(0xc822830580, 0x9d2817, 0x0, 0x0)
	/usr/lib/go/src/crypto/tls/conn.go:540 +0x2d1
crypto/tls.(*Conn).Read(0xc822830580, 0xc821100000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/lib/go/src/crypto/tls/conn.go:901 +0x167
net/http.noteEOFReader.Read(0x7fe9d594cc28, 0xc822830580, 0xc8225e6c08, 0xc821100000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/lib/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc8219be240, 0xc821100000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	<autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc8216302a0)
	/usr/lib/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc8216302a0, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/lib/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc8225e6bb0)
	/usr/lib/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
	/usr/lib/go/src/net/http/transport.go:685 +0xc78

goroutine 16256 [select]:
net/http.(*persistConn).writeLoop(0xc821905550)
	/usr/lib/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
	/usr/lib/go/src/net/http/transport.go:686 +0xc9d

goroutine 16391 [select]:
net/http.(*persistConn).writeLoop(0xc8225e6bb0)
	/usr/lib/go/src/net/http/transport.go:1009 +0x40c
created by net/http.(*Transport).dialConn
	/usr/lib/go/src/net/http/transport.go:686 +0xc9d

goroutine 16255 [IO wait]:
net.runtime_pollWait(0x7fe9d5961ae0, 0x72, 0xc8200122b0)
	/usr/lib/go/src/runtime/netpoll.go:157 +0x60
net.(*pollDesc).Wait(0xc821a89090, 0x72, 0x0, 0x0)
	/usr/lib/go/src/net/fd_poll_runtime.go:73 +0x3a
net.(*pollDesc).WaitRead(0xc821a89090, 0x0, 0x0)
	/usr/lib/go/src/net/fd_poll_runtime.go:78 +0x36
net.(*netFD).Read(0xc821a89030, 0xc8214f2000, 0x2000, 0x2000, 0x0, 0x7fe9d8fb7050, 0xc8200122b0)
	/usr/lib/go/src/net/fd_unix.go:232 +0x23a
net.(*conn).Read(0xc821f9c020, 0xc8214f2000, 0x2000, 0x2000, 0x0, 0x0, 0x0)
	/usr/lib/go/src/net/net.go:172 +0xe4
crypto/tls.(*block).readFromUntil(0xc8223606f0, 0x7fe9d59441b8, 0xc821f9c020, 0x5, 0x0, 0x0)
	/usr/lib/go/src/crypto/tls/conn.go:455 +0xcc
crypto/tls.(*Conn).readRecord(0xc8225ee840, 0x9d2817, 0x0, 0x0)
	/usr/lib/go/src/crypto/tls/conn.go:540 +0x2d1
crypto/tls.(*Conn).Read(0xc8225ee840, 0xc8210e3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/lib/go/src/crypto/tls/conn.go:901 +0x167
net/http.noteEOFReader.Read(0x7fe9d594cc28, 0xc8225ee840, 0xc8219055a8, 0xc8210e3000, 0x1000, 0x1000, 0x0, 0x0, 0x0)
	/usr/lib/go/src/net/http/transport.go:1370 +0x67
net/http.(*noteEOFReader).Read(0xc82180e500, 0xc8210e3000, 0x1000, 0x1000, 0xc8223387d0, 0x0, 0x0)
	<autogenerated>:126 +0xd0
bufio.(*Reader).fill(0xc821b37e00)
	/usr/lib/go/src/bufio/bufio.go:97 +0x1e9
bufio.(*Reader).Peek(0xc821b37e00, 0x1, 0x0, 0x0, 0x0, 0x0, 0x0)
	/usr/lib/go/src/bufio/bufio.go:132 +0xcc
net/http.(*persistConn).readLoop(0xc821905550)
	/usr/lib/go/src/net/http/transport.go:876 +0xf7
created by net/http.(*Transport).dialConn
	/usr/lib/go/src/net/http/transport.go:685 +0xc78

What happens when there is an rateLimitError during push? does drive re-try?

Haven't used drive for a while, updated it today to 0.3.5 and did a drive pull on a dir and got this:
Resolving...
googleapi: Error 429: Rate Limit Exceeded, rateLimitExceeded
 |
clashes detected. Use `ignore-name-clashes` to override this behavior or `fix-clashes` to try fixing this

I have the same issue with GoogleDrive. Cannot transfer the files.
Error 403: Rate Limit Exceeded, rateLimitExceeded - low level retry 3/10

I found out howto fix it :)

Open https://console.developers.google.com
In the section "Google Apps APIs" go to "Drive API"
Push enable
Credentials --> Create Credentials --> OAuth Client ID
Application type --> other
Reconfigure the account in rclone and enter the just created ClientID and secret
Have Fun :)

Hey y'all, please get the latest. Thank you very much for all the patience and the bug reports.
Finally fixed this problem that was caused by not properly relaying remote resolution errors back to the callers. The fix is in https://github.com/odeke-em/drive/pull/741. Please get the latest code and even in these rate limit/resolutions problems drive will now be able to relay encountered errors and properly abort instead of performing the wrong behavior.

I see next now. Question about strategy. I see when one file failed, it goes to next. Should it stop on one file and to exp retry on it instead of going to next?
And here we have potentially infinite operation, so it would be great for noobs users show CTRL+C to abort as in other basic tools. May be some other hotkey as Esc to abort?
push: /courseera/progfun-002/hello-project/bin/greeter/LearningSuite.class err: googleapi: Error 403: User rate limit exceeded, userRateLimitExceeded

push: /courseera/progfun-002/hello-project/bin/greeter/TestRepl.sc err: googleapi: Error 403: User rate limit exceeded, userRateLimitExceeded

push: /courseera/progfun-002/hello-project/bin/week1 err: googleapi: Error 403: User rate limit exceeded, userRateLimitExceeded

push: /courseera/neuralnets err: googleapi: Error 403: User rate limit exceeded, userRateLimitExceeded
dlahoda@dlahodad2:/stuff/dlahoda/gdrive/courseera$ drive version
drive version: <IP_ADDRESS>
Commit Hash: '9ce18db69196f53da028fab0ba9621dc33a23656'
Go Version: go1.6.2
OS: linux/amd64
BuildTime: 2016-11-20 21:31:48.2487148 +0300 +03
dlahoda@dlahodad2:/stuff/dlahoda/gdrive/courseera$

I get
Error 403: User rate limit exceeded, userRateLimitExceeded
doing mass touching
drive touch  -matches
on a fast 1gig/s symetric fiber connection.

I also get a "got googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded"
I have tried increasing the chunk size to 64MiB to see if it is data rate issue or a "number of API requests" issue. The number of 403 instances increases when the chunk size increases. So it seems to be a data rate issue rather than a number of requests issue.
Running Master branch as on 17/12/2018.

My script faced this problem on the 22nd of February. It had been running smoothly until then for ~2 months uploading ~80 MB of data each day.
First it was the error like Failed to determine free space: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded, the next day it was like Failed to find root dir: googleapi: Error 403: Rate Limit Exceeded, rateLimitExceeded...
Less than 1GB is used at Google Drive.


az acs is missing kubernetes support

e.g. (for an ACS created via the web interface)
$ az acs list -g foo
u'Kubernetes' is not valid value for enum <enum 'ContainerServiceOchestratorTypes'>

adding the type to azure/mgmt/compute/models/compute_management_client_enums.py seemed to be sufficient to display the service, haven't tried creating one yet

Same thing like #1282


Add OpenPhenom benchmarking notebook

What?

update the benchmarking notebook to use pca-centerscale instead of TVN
show example run for OpenPhenom model

We might want a separate folder, notebooks/rxrx3-core, specifically for this notebook.
See this conversation.

I don't really think we need a separate folder for the notebook since it's self-contained. Instead I added a description of the benchmark to the readme @fedecomitani


The time sources docs are inconsistent, benchmark doesn't match speed

I noticed something minor here: https://github.com/NLog/NLog/wiki/Time-Source
FastUTC is listed as being the fastest, but the benchmark shows FastLocal is faster. Could that be because whoever ran the benchmark has set their local time zone to match UTC?

The difference between 5.7 ns and 5.8 ns is none compared to benchmark noise.
Guess one should say they have the same speed.
It would be more interesting to have comparison of:

32 bit net472
64 bit net472
64 bit netcore22

Closing this due to inactivity. Please let us know if this still an issue and please provide the requested info -
thanks!

Have updated the wiki-page: https://github.com/NLog/NLog/wiki/Time-Source
With the benchmark numbers from #3378


Network rules do not block sometimes on Android nightly 3.1.53

Steps to reproduce on Android

Add ||graph.facebook.com to the content blocking user filter
Install any app that uses Facebook SDK (for instance, Yandex Maps)
Run it
Check the filtering log

Expected behavior on Android

Connections to graph.facebook.com must be blocked.

Actual behavior

Connections aren't blocked

https://app.zenhub.com/files/74596049/f767be37-7035-4328-a513-b21c4ea17e8c/download

core/pull-requests/1366

The issue is still present.
Steps to reproduce

Go to cnn.com
Open menu, open US politics
Inspect the filtering log

Expected result
No tcp:// requests from Chrome
Actual result

tcp://code.jquery.com/


Document TorchIndexSelectOp

MLIR-HLO commit: https://github.com/tensorflow/mlir-hlo/commit/6353126c8a99d20d177a78f0fdd3ded1d924b6cd.

This has been implemented recently.


feat(simple): make usage simpler, remove model in config

Codecov Report

Merging #2 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master     #2   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           3      2    -1
  Lines           8     12    +4
=====================================
+ Hits            8     12    +4

Impacted Files
Coverage Δ

app.js
100% <100%> (ø)
:arrow_up:

config/config.default.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c0fd29b...f8f1183. Read the comment docs.


unused variable detection not working?

def main() -> int:
    input = parse_input()

    smallest = -1 as long
    for i = 0; i < input->n_seed_numbers; i++:
        n = input->seed_numbers[i]
        for k = 0; k < input->nmaps; k++:
            old_n = n
            n = input->maps[k].map_number(n)

        if smallest == -1 or n < smallest:
            smallest = n

    printf("%lld\n", smallest)
    return 0

I would expect warning message on line old_n = n

Minimal example
def main() -> int:
    n = 7
    return 0


Update readme to reflect ShareX implementation of Up1

Up1 has made it into the official ShareX repo and 10.0 release. Instead of linking to our fork, it should link directly to the official ShareX itself.

8c3417018527b4ea2051e75089f67341e7bd6236


SSH_KEY is an invalid value for 'category'

Hello !
I'm trying to get an SSH key from 1Password Connect Server with the API. But I'm getting the following issue :
ValueError: Invalid value for `category` (SSH_KEY), must be one of ['LOGIN', 'PASSWORD', 'SERVER', 'DATABASE', 'CREDIT_CARD', 'MEMBERSHIP', 'PASSPORT', 'SOFTWARE_LICENSE', 'OUTDOOR_LICENSE', 'SECURE_NOTE', 'WIRELESS_ROUTER', 'BANK_ACCOUNT', 'DRIVER_LICENSE', 'IDENTITY', 'REWARD_PROGRAM', 'DOCUMENT', 'EMAIL_ACCOUNT', 'SOCIAL_SECURITY_NUMBER', 'API_CREDENTIAL', 'CUSTOM']

Do I only need to submit a new PR ? including the new category type "SSH_KEY"

I get the same issue when I try to get items in a given vault id client.get_items(str(vault.id))

Hey @floriandaniel,
As @ftasbasi suggested, updating the Connect Python SDK to the latest version (1.3.0) should address the issue you're running to.
That release contains https://github.com/1Password/connect-sdk-python/pull/54, which solves the unexpected error for new categories that are added to Connect.
Closing the issue for now. Feel free to re-open if you still run into this issue with the latest version of the SDK.


Broker can't detect R interpreters

I have Microsoft R Client installed, but it can't see it.
Now listening on: http://<IP_ADDRESS>:5000
Application started. Press Ctrl+C to shut down.
dbug: Microsoft.AspNetCore.Server.Kestrel[1]
      Connection id "0HKUN7H0DKCP4" started.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 PUT http://ddhuvalo071:5000/sessions/REPL application/json; charset=utf-8 46
info: Odachi.AspNetCore.Authentication.Basic.BasicMiddleware[7]
      Basic was not authenticated. Failure message: No authorization header.
info: Odachi.AspNetCore.Authentication.Basic.BasicMiddleware[12]
      AuthenticationScheme: Basic was challenged.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 320.7822ms 401
dbug: Microsoft.AspNetCore.Server.Kestrel[9]
      Connection id "0HKUN7H0DKCP4" completed keep alive response.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
      Request starting HTTP/1.1 PUT http://ddhuvalo071:5000/sessions/REPL application/json; charset=utf-8 46
info: Odachi.AspNetCore.Authentication.Basic.BasicMiddleware[3]
      HttpContext.User merged via AutomaticAuthentication from authenticationScheme: Basic.
dbug: Microsoft.AspNetCore.Routing.Tree.TreeRouter[1]
      Request successfully matched the route with name '' and template 'sessions/{id}'.
dbug: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
      Executing action Microsoft.R.Host.Broker.Sessions.SessionsController.PutAsync (Microsoft.R.Host.Broker)
info: Microsoft.AspNetCore.Authorization.DefaultAuthorizationService[1]
      Authorization was successful for user: huvalo.
info: Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker[1]
      Executing action method Microsoft.R.Host.Broker.Sessions.SessionsController.PutAsync (Microsoft.R.Host.Broker) with arguments (REPL, Microsoft.R.Host.Protocol.SessionCreateRequest) - ModelState is Valid
fail: Microsoft.AspNetCore.Server.Kestrel[13]
      Connection id "0HKUN7H0DKCP4": An unhandled exception was thrown by the application.
System.InvalidOperationException: Sequence contains no elements
   at System.Linq.Enumerable.First[TSource](IEnumerable`1 source)
   at Microsoft.R.Host.Broker.Sessions.SessionsController.PutAsync(String id, SessionCreateRequest request)
   at lambda_method(Closure , Object , Object[] )
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.<InvokeActionFilterAsync>d__28.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Mvc.Internal.ControllerActionInvoker.<InvokeAsync>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Builder.RouterMiddleware.<Invoke>d__4.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware`1.<Invoke>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware`1.<Invoke>d__18.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Hosting.Internal.RequestServicesContainerMiddleware.<Invoke>d__3.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Microsoft.AspNetCore.Server.Kestrel.Internal.Http.Frame`1.<RequestProcessingAsync>d__2.MoveNext()
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
      Request finished in 3932.114ms 200

Here's the output if I specify path to interpreter in cmd line args:
C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\R Tools for Visual Studio>Microsoft.R.Host.Broker --server.urls http://<IP_ADDRESS>:5000 --R:interpreters:1:basePath "C:\Program Files\Microsoft\R Client\R_SERVER" --security:secret foo
trce: Microsoft.R.Host.Broker.Interpreters.InterpreterManager[0]
      Auto-detecting R ...
warn: Microsoft.R.Host.Broker.Interpreters.InterpreterManager[0]
      No R interpreters auto-detected.
info: Microsoft.R.Host.Broker.Interpreters.InterpreterManager[0]
      1 interpreters configured:
      '1': 3.2.2 at "C:\Program Files\Microsoft\R Client\R_SERVER"
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[3]
      Hosting starting
dbug: Microsoft.AspNetCore.Hosting.Internal.WebHost[4]
      Hosting started
Hosting environment: Production
Content root path: C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\Microsoft\R Tools for Visual Studio
Now listening on: http://<IP_ADDRESS>:5000
Application started. Press Ctrl+C to shut down.

This means RInstallation().GetCompatibleEngines() returned nothing, but there was something in _options.Interpreters that looked valid.

For the second log, that is the expected behavior (and should work when you connect, provided that you also specify the interpreter ID as a fragment URL, e.g. http://localhost:5000#1). The first log is where there were no manually configured interpreters, and auto-detect didn't find any either.

https://github.com/Microsoft/RTVS/commit/2e62ba12bb572cfadfafa58536ed607d89cc51f5


Update bytes to 0.6

This change updates the bytes crate dependencies to version 0.6.
Fixes https://github.com/danburkert/prost/issues/380

I don't have much context on the prior implementations to be confident that the ported version is absolutely correct, but at least the tests all pass locally.

This CI failure appears to be unrelated to my change?
error[E0658]: the `#[non_exhaustive]` attribute is an experimental feature
224
Error:    --> /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/curl-0.4.34/src/easy/handler.rs:395:1
225
    |
226
395 | #[non_exhaustive]
227
    | ^^^^^^^^^^^^^^^^^
228
    |
229
    = note: for more information, see https://github.com/rust-lang/rust/issues/44109
230

CI failure is indeed unrelated; #382 should fix.

Took a look, this unfortunately results in a very significant perf hit during encoding.  I switched encode_varint back to something closer to the master version:
    unsafe {
        let mut i;
        'outer: loop {
            i = 0;

            let uninit_slice = buf.bytes_mut();
            let ptr = uninit_slice.as_mut_ptr();
            for offset in 0..uninit_slice.len() {
                i += 1;
                let ptr = ptr.offset(offset as isize);
                if value < 0x80 {
                    ptr.write(value as u8);
                    break 'outer;
                } else {
                    ptr.write(((value & 0x7F) | 0x80) as u8);
                    value >>= 7;
                }
            }

            buf.advance_mut(i);
            debug_assert!(buf.has_remaining_mut());
        }

        buf.advance_mut(i);
    }

but that still is ~40-50% slower in the in-tree benchmarks (both the synthetic varint and the more real world protobuf benchmarks).  Will keep looking into this as I have time, #279 may be a good way to sidestep the issue.  My early thoughts are that this is related to slowness in the new UninitSlice API, perhaps there is an #[inline] tag missing, or it just doesn't optimize as well.

It was indeed missing #[inline] tags in bytes.  Closing in favor of #387.  Thanks @olix0r !


Rate a charger

There's a problem with the popup window for the rating of a charger.
It still says: "Yes", "Maybe", and "No", in the translation.
Some strings may be missing in the translation files, or a bug in the code.

The language doesn't matter, same issue in German.

The language doesn't matter, same issue in French

I will add those to the translation files for the next release

fixed, so the translations can be added now

Ok last french translation 4.5.0 done


Touch support on internet explorer / microsoft surface

I faced an issue where drag & drop did not appear to work on microsoft surface (internet explorer). The solution to this problem is fairly simple, just add:
.jstree li {
    -ms-touch-action: none;
    touch-action: none;
}

to your css and it works. It would make sense to update your demos with this css and add a comment to inform developers about it.

The problem with this 'fix' is that you now set touch-action: none which means nodes do not respond to the 'click' event anymore !


⚠ Slack has degraded performance

In b1b9f6a, Slack (https://status.slack.com/api/v2.0.0/current) experienced degraded performance:

HTTP code: 200
Response time: 42 ms

Resolved: Slack performance has improved in dd491e6.


Cityscape Dataset Error

Hello I have tried to run your code but I have several problem:

Which cityscapes dataset that you are download?
I have download from this link and download leftImg8bit_trainvaltest.zip (11GB) and it's GT gtFine_trainvaltest.zip (241MB)
I got error and realize that in your train.txt and val.txt must change GT path from labelTrainIds to labelIds.
After I change it again I got error said that Some problem with labels. Please check. Again I find that the dataset have bigger class. Therefore in your paper did you only have 20 class?, and could you share a link for cityscape dataset that you download for your paper?

Labels can take value between 0 and number of classes.
Some problem with labels. Please check.
Label Image ID: ./city/gtFine/train/zurich/zurich_000104_000019_gtFine_labelIds.png
Self class 20
Max Val 26
Min_Val 0
Labels can take value between 0 and number of classes.
Some problem with labels. Please check.
Label Image ID: ./city/gtFine/train/zurich/zurich_000098_000019_gtFine_labelIds.png
Self class 20
Max Val 33
Min_Val 1

Why you change the labelTrainIds to labelIds ？ The path in  train.txt is labelTrainIds. @herleeyandi

@MrLinNing Hello I can't find that file in the dataset. Here is the image.

I have the labelTrainIds file, but I met the same problem as you. I don't know how to use it

@MrLinNing I just found this problem, but I think its not the problem from the data anymore but maybe our pytorch version. I still working on it.

@herleeyandi You can see this question, but I dont understand the specific operation.

@sacmehta I still have the same error. I just confuse with your code in the loss part I try to print the size :
#set the grad to zero
optimizer.zero_grad()
print(output.size())
print(target.size())
print(target_var.size())

Then I got this:
torch.Size([12, 20, 96, 192])
torch.Size([12, 96, 192])
torch.Size([12, 96, 192])

So I just conclude that maybe because the output and the target class is not same, it should has 20 in the channel right? Because I have tried to do what you said and I still have this error.
/opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THCUNN/SpatialClassNLLCriterion.cu:99: void cunn_SpatialClassNLLCriterion_updateOutput_kernel(T *, T *, T *, long *, T *, int, int, int, int, int, long) [with T = float, AccumT = float]: block: [5,0,0], thread: [606,0,0] Assertion t >= 0 && t < n_classes failed.

Loss print func fix. Add loss_values = loss.item() after optimizer.step() (107 line in main.py)
Change epoch_loss.append(loss_values) and print('[%d/%d] loss: %.3f time:%.2f' % (i, total_batches, loss_values, time_taken))

The following repo support PyTorch 1.0+:
https://github.com/sacmehta/EdgeNets
I would suggest you to use this repo.

Loss print func fix.
Add loss_values = loss.item() after optimizer.step() (104 line in main.py)
Change epoch_loss.append(loss.data[0]) to epoch_loss.append(loss_values) (105 line)
Change print('[%d/%d] loss: %.3f time:%.2f' % (i, total_batches, loss.data[0], time_taken))  to print('[%d/%d] loss: %.3f time:%.2f' % (i, total_batches, loss_values, time_taken)) (111 line)
Also change loss print in val function (line 23)
Add loss_values = loss.item() after optimizer.step() (50 line)
Change epoch_loss.append(loss.data[0]) to epoch_loss.append(loss_values) (51 line)
Change print('[%d/%d] loss: %.3f time:%.2f' % (i, total_batches, loss.data[0], time_taken))  to print('[%d/%d] loss: %.3f time:%.2f' % (i, total_batches, loss_values, time_taken)) (58 line)

Thanks! I met cudnn problem too , so i changes the code with your suggestion, and finally it worked


App is requesting sparse data from the cloud when it should ask for dense data

One of my Ruuvi Gateways got whitelisted about an hour ago and its data is now stored to the cloud. If I clear local data on the Android app and then sync with the cloud, I get datapoints every ~15 minutes. On station.ruuvi.com I see dense data (~1min interval). App version 1.5.27

After a clear&new sync (sparse data and some dense data):

And after another clear&sync (only sparse data):

And another (sparse data and one extra dense datapoint):

(Note that I'm not in range of Bluetooth so this is cloud data only)

fixed in 1.5.28

QA passed both clean install fresh network sync and clear chart + network sync now showing 1 min interval data.


fix scale, rename transform to translate

@timsueberkrueb

This doesn't claim to implement rotation. rotation would have to be even more special handling in the renderer

This also doesn't fix event handling. I'll do that in another PR

This is great, thank you @jrmoulton! It fixes font rendering when scaling.

This doesn't claim to implement rotation. rotation would have to be even more special handling in the renderer

Maybe this gets easier if all transformations are consistently handled as a single affine transformation matrix (which is already partially the case). At the moment, with the vger renderer, we extract the individual properties (like translation and scale) from the transformation matrix instead of applying it directly. Maybe it makes sense to change this when we switch to Vello? I've seen that its API has first-class support for passing transformation matrices.
Anyways, I don't need rotation for my use case right now, but I think it would be nice to have eventually (for animations, spinners, etc).

This can sometimes trigger some strange effects when animating scale.
It doesn't happen consistently.
I've watched all of the scale values that affect the different properties while animating and they look correct. I now believe this is some bug in vger
https://github.com/user-attachments/assets/55eeda36-5079-49bb-8d15-4a778a939acc

. Maybe it makes sense to change this when we switch to Vello? I've seen that its API has first-class support for passing transformation matrices.

yeah in my branch for vello this should already be handled.

https://github.com/user-attachments/assets/9d67747c-ec4c-4390-8bab-2d924189bee5


Fix verible formula

Update GitHub workflows:
Fix Verible Homebrew formula:

Rebase after merging #32.

LGTM. Thanks Johan!


Create zh_CN.lang

I see that you refer to Nordic mythology, ancient Greek mythology, Finnish mythology, Japanese mythology, etc. I checked the information in detail and translated the most appropriate name

1.7.10 is no longer in development, I will close this but won't make a release that includes this changes for that version
I will however use the translation for the newly updated version i am going to release soon for 1.20.1


Unit tests for artifacts produced by GenAI-Perf

This PR is a part of unit test epic for genai-perf and handles unit tests for artifacts produced by genai-perf.
https://jirasw.nvidia.com/browse/TPA-83
Folder structure
genai-perf
|__ genai-perf
|__ tests
|__ artifacts
|__ file1
|__ file 2
|__ plots
Artifacts are created inside genai-perf directory. Plots directory is created inside artifacts.
Ran the test with wrong path (used /genai-perf instead of DEFAULT_ARTIFACT_DIR) to confirm the test fails.

When given the correct path, the test runs successfully.

Could you actually break this out into two tests:

One not specifying an arg (i.e. it should use the default path)
One specifying an arg (i.e. it should use that path)

You can break this into another PR or ticket, if easier. I forgot that --artifact-dir was added, so we'd want to test both the default and non-default behavior.

Sure, I will work on your comment @dyastremsky

Could you actually break this out into two tests:

One not specifying an arg (i.e. it should use the default path)
One specifying an arg (i.e. it should use that path)

You can break this into another PR or ticket, if easier. I forgot that --artifact-dir was added, so we'd want to test both the default and non-default behavior.

Discussed offline. Since this function uses the arg passed from the parser, it doesn't make sense to unit test the default path. If we want to unit test the default, we would do that in test_cli.py (though we'd be mostly replicating testing ArgParse defaults) or do an integration/end-to-end test (probably not worth the resources unless we find a low-cost way to run those tests). We should use the artifact directories in some of our end-to-end testing, so hopefully that'd provide some code coverage.


Bug: Not detecting all broadcasting APs

I'm using an M1 Pro Macbook Pro with WiFi explorer Pro 3.6.3. When using my WLAN Pi R4 as a sensor it correctly detects 3 APs for my 5GHz and 6 GHz BSSIDs. However, for the 2.4 GHz SSIDs WiFi explorer is only showing 2 BSSIDs. If I disconnect the WLAN Pi R4 and just use the Mac's native WiFi all 3 2.4 GHz BSSIDs appear. This also briefly happened on the 6 GHz band, but cleared up.

What Wi-Fi adapter are you using?

What regulatory domain?

What WLAN Pi OS release?

How strong is the RSSI for those undetected 2.4 GHz BSSIDs?

COMFRST AX1800 (came with the WLAN Pi R4 kit)
US
3.2.0
-69

That potentially sounds like a MediaTek firmware issue we have fixed recently fixed.
Please check for latest package versions and update this package for me:
sudo apt update && sudo apt install wlanpi-misc-firmware -y

Didn't help. It's missing 2x 2.4 GHz BSSID, and for a while two 6 GHz BSSIDs. All BSSIDs appear on my M3 MacBook Air. Interestingly, all 3 5 GHz BSSIDs ARE listed. All the 6 GHz BSSIDs were slow to come up, but eventually did make it. Not so lucky for 2.4 GHz.

Did you reboot after upgrading the firmware package?
Can you please take an over the air packet capture on the WLAN Pi on one of the 2.4 GHz channels where the missing AP is present? 5 second length is fine. Feel free to e-mail me<EMAIL_ADDRESS>if you don't want to share the PCAP publicly.

Yes I did a full reboot, and also sudo apt-get upgrade -y, then another reboot. Probably have to wait until this weekend to figure out how to do a capture.

@jiribrejcha I emailed you a pcap.

Any update on this? It's still happening.

@Coder84619 did you try the latest version of WFE? What channels are your 6 GHz APs on?


Do not upload orin image on master

fixes #1856

dont wait for tests for this IMHO

good call, thanks


Fail creating load balancer

Hi,
I deploy on ubuntu kuberneted cluster. I have received this error in deploying
"Failed to ensure load balancer for service longhorn-system/longhorn-frontend: Unsupported load balancer affinity: ClientIP"

Which version of Kubernetes is that?

1.8

Can you try to remove line: https://github.com/rancher/longhorn/blob/a917063ebc97f949e87153afa1960667e2d77454/deploy/example.yaml#L276


Can train normally, but run out of memory during val.

Hello，I encountered a problem during training. The following is my command.
 python SGG-Benchmark/tools/relation_train_net.py --task sgdet --save-best --config-file "SGG-Benchmark\configs\VG150\e2e_relation_X_101_32_8_FPN_1x.yaml" MODEL.ROI_RELATION_HEAD.PREDICTOR VCTreePredictor SOLVER.IMS_PER_BATCH 1 TEST.IMS_PER_BATCH 1 DTYPE "float16" SOLVER.MAX_EPOCH 20 MODEL.PRETRAINED_DETECTOR_CKPT epoch_60.pth OUTPUT_DIR output/
I can train normally, but I run out of memory during validation.
After my investigation, I found that the problem occurs .in this file

At line 123 of this file, it creates a very large result vector, which leads to running out of GPU memory. At this point, the size of outsize is as high as two hundred thousand. This should not be normal, but I don’t know how to solve this problem.

Hello @Quan-zzx,
I don't have this problem on my machine (I use one Nvidia A6000 GPU with 42GB of memory). I know the VCTree model is more memory-hungry than the others but this should not happen in the pooling function which creates tensors only by batches.
Which tensor exactly has a size of two hundred thousand? Can you post the logs of your experiments?
As a workaround, you can reduce the number of proposals used during training with the variable MODEL.ROI_HEADS.DETECTIONS_PER_IMG. You can reduce this down to 30-40 and you should not see a big drop in performance (if you want more details I have drawn a scaling law in the ablation studies of my paper here).
Using YOLOV8 instead of Faster-RCNN may also significantly reduce memory usage (especially in the pooler).

Hello @Quan-zzx, I don't have this problem on my machine (I use one Nvidia A6000 GPU with 42GB of memory). I know the VCTree model is more memory-hungry than the others but this should not happen in the pooling function which creates tensors only by batches. Which tensor exactly has a size of two hundred thousand? Can you post the logs of your experiments?
As a workaround, you can reduce the number of proposals used during training with the variable MODEL.ROI_HEADS.DETECTIONS_PER_IMG. You can reduce this down to 30-40 and you should not see a big drop in performance (if you want more details I have drawn a scaling law in the ablation studies of my paper here). Using YOLOV8 instead of Faster-RCNN may also significantly reduce memory usage (especially in the pooler).

Thank you for your reply. As shown in the picture below, I have printed the shape of the result that I want to set. The num_rois is very large，

The second image shows that it wanted to create a tensor of 263682 * 1024 * 7 * 7, but unfortunately failed because it exceeded the GPU memory of over 40GB.
But this only happens during validation, I can train normally。As shown in the figure below, I printed the situation during training, and the number of ROIs is only a few thousand

Hum that's really weird, can you give me your e2e_relation_X_101_32_8_FPN_1x.yaml file so I can try to reproduce the problem? For the pre-trained Faster-RCNN detector, are you using your own weights?

Maybe there's a problem with my pre trained model. My previous model was trained by MMDetection, and I just tried your code to train it, so the problem I mentioned won't occur anymore

Yes if you trained the backbone with another framework, it is likely to fail to work here. It only works for YOLOV8 because I am using the ultralytics implementation directly.
If you want to load your backbone with the MMDet implementation correctly, you will need to add a new backbone class here and modify the default arguments. For Faster-RCNN you could register for instance a "R-101-FPN-MMDet" class. You will then have to modify the overall structure to fit it to your new backbone structure (with appropriate names).

Yes if you trained the backbone with another framework, it is likely to fail to work here. It only works for YOLOV8 because I am using the ultralytics implementation directly. If you want to load your backbone with the MMDet implementation correctly, you will need to add a new backbone class here and modify the default arguments. For Faster-RCNN you could register for instance a "R-101-FPN-MMDet" class. You will then have to modify the overall structure to fit it to your new backbone structure (with appropriate names).

OK,Tranks. I have used your code to train my object detection network and the results are good.

Closing as resolved


.pull right doesnt supporting any more?

I see now only
.pull-right>.dropdown-menu{
    right:0;left:auto
}

But simple .pull-right doesnt exists any more?

Correct. See http://v4-alpha.getbootstrap.com/migration/#utilities and http://v4-alpha.getbootstrap.com/components/utilities/#responsive-floats


Improve readability / hierarchy of View Data

Description
Recent View Data mocks include several improvements to the readability and information / CTA hierarchy on that page.
This task implements those improvements.
Acceptance Criteria

[ ] allow user to clear location search bar (see #445 );
[ ] remove "Nearby" at top;
[ ] filter chips in Collisions, Studies use closeable X icon instead of checkmark;
[ ] "Request Study" uses secondary button treatment, rather than primary;
[ ] remove background shading on Collisions;
[ ] switch from data table treatment to heading / value treatment on collision summary statistics;
[ ] when filters active, show unfiltered summary statistics as denominator for reference;
[ ] grey box treatment for study counts in Studies table.

Additional Notes
We should probably make the same filter chips change in Track Requests.

Closing this as wontfix - we should instead create tasks against the new designs in Figma, as those handle both multi-location and single-location cases.


[Bug] Save container name button enabled

Save container name button enabled if you add 1 char to the end and delete it,
it would be enabled but dont make any changes

Any change in that field prompts for a possible change - Adding extra functionality to make sure you didn't change anything seems un-necessary.
Word does the same thing; if you add a letter and then delete it, you still have the option to 'save'


refactor(datepicker): remove deprecated APIs for v11

Removes the APIs that were marked as deprecated for v11 in the datepicker module.
BREAKING CHANGES:

The value emitted by MatCalendar.selectedChange can now be null.
MatDatepickerInput.getPopupConnectionElementRef has been removed. Use getConnectedOverlayOrigin instead.
The _changeDetectorRef, _model, _dateAdapter and _rangeSelectionStrategy parameters of the MatDatepickerContent constructor are now required.

@crisbeto needs rebase

Rebased.


Upgrade PyLD to 2.0.3

load_document signature changed
load_document return type schema changed

Description
A described above
Motivation and Context
PyLD was outdated (1.0.4)
How Has This Been Tested?
Api and Core unit tests are passing
Not sure if manual testing is required

Checklist

[ ] I have updated the documentation accordingly.
[x] All new and existing tests passed.

@jonathangreen Forgot to add you as reviewer


Freenas jail tutorial

This adds a quick rundown of how to install and configure Minio inside a FreeBSD jail on FreeNAS.

Thank you :)


All links in your repo seem to be invalid, please update for accessing, thanks!

I have updated the links. Thank you for letting me know. I will change the links to the permanent ones soon, as the OneDrive links are limited to only three months.


Please augment all dependency repo to add a valid license file

Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:

Go to github repo 'insights'
Click on 'dependency graph'
click on dtcookie/dyntrace
See the repo has no license file in dtcookie/dynatrace

Expected behavior
All dependency repo should have a valid license file.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: [e.g. iOS]
Browser [e.g. chrome, safari]
Version [e.g. 22]

Smartphone (please complete the following information):

Device: [e.g. iPhone6]
OS: [e.g. iOS8.1]
Browser [e.g. stock browser, safari]
Version [e.g. 22]

Additional context
Add any other context about the problem here.

without a valide license file in dependency repo prevents the usage of this awesome open source!

Hello @xcheng75
The dependency repo meanwhile contains a license.
With the upcoming release that's irrelevant anyways. All the relevant code will be contained within this repository from now on.


Add rubocop

Based off of Jekyll's Rubocop config.

Thank you! :heart: a couple comments above if you wouldn't mind clarifying.