"no API" Error if upstream is "http://app"

Summary
We've tested this with multiple variations and it looks like having an upstream of "http://app" breaks routing.  We get an error {"message":"no API found with those values"}
Steps To Reproduce

We've tested changing the API resource name, changing the URI path and switching to host based routing.  If the upstream is http://app we get the no API Found error
We were using a custom docker image, so we switched to the Mashape kong image and we experienced the same problem
We were running this in a docker-compose environment so we ran this locally with a host file entry for app and experienced the same problem.
An upstream of http://apps and any other variant we could come up with works perfectly fine.

Additional Details & Logs

Kong version 0.10.1
Operating System = official kong:0.10.1 (tested on OS X when it was tested outside of docker)

Examples of working vs non-working setups
# Working examples
curl -XPOST localhost:8001/apis --data name=app --data uris=/app --data upstream_url=http://apps
curl -XPOST localhost:8001/apis --data name=app --data hosts=app.example.com --data upstream_url=http://apps

# Non-Working examples
curl -XPOST localhost:8001/apis --data name=app --data uris=/app --data upstream_url=http://app
curl -XPOST localhost:8001/apis --data name=apps --data uris=/app --data upstream_url=http://app
curl -XPOST localhost:8001/apis --data name=apps --data uris=/apps --data upstream_url=http://app
curl -XPOST localhost:8001/apis --data name=app --data hosts=app.example.com --data upstream_url=http://app
curl -XPOST localhost:8001/apis --data name=apps --data hosts=app.example.com --data upstream_url=http://app
curl -XPOST localhost:8001/apis --data name=apps --data hosts=apps.example.com --data upstream_url=http://app

Based on the Admin API output everything is setup properly.
{
  "uris": [
    "/app"
  ],
  "id": "41c15c65-dc6d-48c1-a990-8b0b6a862f0e",
  "upstream_read_timeout": 60000,
  "preserve_host": false,
  "created_at":<PHONE_NUMBER>000,
  "upstream_connect_timeout": 60000,
  "upstream_url": "http://app",
  "strip_uri": true,
  "https_only": false,
  "name": "app",
  "http_if_terminated": true,
  "upstream_send_timeout": 60000,
  "retries": 5
}

Please let me know if you need anything else to debug this.

Thanks for the report!
@bungle Do you want to take a look at this?

@erutherford,
Can you also show us the failing and succesful requests (curls or urls)?

I did wrote a small test suite, and it worked fine (I tested against latest master branch):

  describe("exec()", function()
    local spy_stub = {
      nop = function() end
    }

    local function mock_ngx(method, request_uri, headers)
      local _ngx
      _ngx = {
        re = ngx.re,
        var = setmetatable({
          request_uri = request_uri,
          http_kong_debug = headers.kong_debug
        }, {
          __index = function(_, key)
            if key == "http_host" then
              spy_stub.nop()
              return headers.host
            end
          end
        }),
        req = {
          set_uri = function(request_uri)
            _ngx.var.request_uri = request_uri
          end,
          get_method = function()
            return method
          end,
          get_headers = function()
            return headers
          end
        }
      }

      return _ngx
    end
    describe("finds an API", function()
      local checks = {
        -- name     upstream url    uris       hosts
        {  "app",   "http://apps",  "/app",   "",                 },
        {  "app",   "http://apps",  "",       "app.example.com"   },
        {  "app",   "http://app",   "/app",   ""                  },
        {  "apps",  "http://app",   "/app",   ""                  },
        {  "apps",  "http://app",   "/apps",  ""                  },
        {  "app",   "http://app",   "",       "app.example.com"   },
        {  "apps",  "http://app",   "",       "app.example.com"   },
        {  "apps",  "http://app",   "",       "apps.example.com"  },
      }

      for i, args in ipairs(checks) do

        it("[" .. i .. "]", function()

          local use_case_apis = {
            {
              name         = args[1],
              upstream_url = args[2],
            }
          }

          if args[3] ~= "" then
            use_case_apis[1]["uris"] = { args[3] }
          end

          local headers = {}

          if args[4] ~= "" then
            use_case_apis[1]["headers"] = {
              ["host"] = { args[4] }
            }

            headers["host"] = args[4]
          end

          local router = assert(Router.new(use_case_apis) )
          local uri    = args[3] ~= "" and args[3] or "/"

          local _ngx = mock_ngx("GET", uri, headers)
          local api, upstream, host_header = router.exec(_ngx)

          assert.same(use_case_apis[1], api)

        end)
      end
    end)

@bungle - I've stripped this down to the bare bones and I'm still able to reproduce the issue.  I have failed and successful requests below.  I've also created a gist for a docker-compose file using the mashape official docker image where this is easily reproducible.  I should have noted in the original ticket that we're also using postgres 9.4.
https://gist.github.com/erutherford/d60f4dc8b5f2935d934853a291bffe18
Failed request:
$ curl localhost:8001/apis
{"data":[],"total":0}
$ curl -XPOST localhost:8001/apis --data name=app --data uris=/app --data upstream_url=http://app
{"http_if_terminated":true,"id":"0d0544c9-6eff-48f5-b6df-e33ccbd8e700","retries":5,"preserve_host":false,"created_at":1493127909000,"upstream_connect_timeout":60000,"upstream_url":"http:\/\/app","upstream_read_timeout":60000,"https_only":false,"upstream_send_timeout":60000,"strip_uri":true,"name":"app","uris":["\/app"]}
$ curl -v localhost/app
*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 80 failed: Connection refused
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to localhost (<IP_ADDRESS>) port 80 (#0)
> GET /app HTTP/1.1
> Host: localhost
> User-Agent: curl/7.51.0
> Accept: */*
>
< HTTP/1.1 404 Not Found
< Date: Tue, 25 Apr 2017 13:45:30 GMT
< Content-Type: application/json; charset=utf-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Server: kong/0.10.1
< X-Kong-Upstream-Latency: 1
< X-Kong-Proxy-Latency: 5
< Via: kong/0.10.1
<
{"message":"no API found with those values"}
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

Successful Request:
$ curl localhost:8001/apis
{"data":[],"total":0}
$ curl -XPOST localhost:8001/apis --data name=app --data uris=/app --data upstream_url=http://apps
{"http_if_terminated":true,"id":"ec65d25f-3663-4af6-9668-6e77964d31c8","retries":5,"preserve_host":false,"created_at":1493128075000,"upstream_connect_timeout":60000,"upstream_url":"http:\/\/apps","upstream_read_timeout":60000,"https_only":false,"upstream_send_timeout":60000,"strip_uri":true,"name":"app","uris":["\/app"]}
$ curl -v localhost/apps
*   Trying ::1...
* TCP_NODELAY set
* Connection failed
* connect to ::1 port 80 failed: Connection refused
*   Trying <IP_ADDRESS>...
* TCP_NODELAY set
* Connected to localhost (<IP_ADDRESS>) port 80 (#0)
> GET /apps HTTP/1.1
> Host: localhost
> User-Agent: curl/7.51.0
> Accept: */*
>
< HTTP/1.1 503 Service Temporarily Unavailable
< Date: Tue, 25 Apr 2017 13:48:23 GMT
< Content-Type: text/plain; charset=UTF-8
< Transfer-Encoding: chunked
< Connection: keep-alive
< Server: kong/0.10.1
<
The upstream server is currently unavailable
* Curl_http_done: called premature == 0
* Connection #0 to host localhost left intact

@erutherford, thanks a lot, I will continue on that.

@erutherford I feel this is some kind of DNS related error as app seems to resolve to <IP_ADDRESS>. But that doesn't explain why you get
{"message":"no API found with those values"}

yeah, even if kong can't resolve the upstream record, we shouldn't get the "no API found" error

@erutherford do you get anything on Kong error.log (.../kong/logs/error)

@erutherford if I provide you tomorrow a special version of 0.10.1``router.lua with added debug statements, would you like test the problem with that. We are still unable to reproduce this.

@erutherford one thing I'm a bit concerned is a fact that you provided this in your failed curl output:
X-Kong-Upstream-Latency: 1
X-Kong-Proxy-Latency: 5
Via: kong/0.10.1

I now believe that your Kong rerouted the request just fine, but that apppointed to the same Kong proxy. I mean it looks like your Kong proxied the request to same or another Kong proxy (I believe same).

you call this localhost/app
it goes to your Kong that strips the /app making the request to app/
now this app points to the same host as localhost and it gets the request, but there is no such API as the only API you have was for /app

I'm quite confident that this is the reason.

@bungle I'm using docker for mac, so the localhost in my curl isn't the same localhost that kong would see in docker.
I've created an upgraded test environment and documented steps and results in the README file.  It now has a simple node hello world express app so that I can actually test with a functioning upstream "app" server.  I'm more than willing to run code with debugging statements in it, but I'm curious to know what might be different in our testing scenarios since you're not seeing the same issue as I am.  I'd think it's highly likely that's where the issue lies.
Have you had a chance to test with the docker-compose environments I've provided?  I'm able to consistently reproduce the same results.

Please remember that Kong's DNS resolver doesn't neccessarily work the same as the system. Kong doesn't read hosts file. Can you show your /etc/resolv.conf inside your Kong? And if you connect to a name server (IPv4) listed there inside your Kong docker how does it resolver app?

Kong doesn't read hosts file.

(It does (or should) as per the dns_hostsfile config property)

the {"message":"no API found with those values"} error is generated by the router, not by the dns module.
That said, we've had an issue before where a failure in setting the upstream values (from Lua land into nginx) caused a recursive loop back into Kong. In that case it kept looping and finally errored out. My guess here is that if it is dns related, that it does a similar thing, but then somewhere in the loop fails to match an api. Only problem is how to replicate this.

@erutherford can you share the contents of the resolv.conf and hosts files?

@erutherford what if you change the port of the app upstream uri:
curl -XPOST localhost:8001/apis --data name=app --data uris=/app --data upstream_url=http://app:2007

This way there won't be a loopback to Kong.

The example I've provided you with uses docker-compose v3 and isn't using links/host file entries, it's using docker's DNS based discovery.  Also, while the examples I'm providing you with are docker, we were also able to reproduce this issue running kong directly on a mac...
Here's the proof that it's not using docker links/hosts file entries
$ docker-compose exec kong /bin/bash
[root@37c983700bdb /]# cat /etc/host
cat: /etc/host: No such file or directory
[root@37c983700bdb /]# cat /etc/hosts
<IP_ADDRESS>	localhost
::1	localhost ip6-localhost ip6-loopback
fe00::0	ip6-localnet
ff00::0	ip6-mcastprefix
ff02::1	ip6-allnodes
ff02::2	ip6-allrouters
<IP_ADDRESS>	37c983700bdb
[root@37c983700bdb /]# cat /etc/resolv.conf
nameserver <IP_ADDRESS>
options ndots:0

I've provided you with a full docker-compose environment where this is extremely reproducible.  At this point we've already worked around this issue, I was trying to help resolve what looks to me like a bug.

Remember that .app is a gTLD, so root servers (and, pretty much everyone) is responding with an A record for app queries (remembering that this is technically a FQDN app.):
$ dig app @<IP_ADDRESS> +short
<IP_ADDRESS>

So when Kong receives a request for an API with the upstream_url of http://app, and it does not find any upstream objects with the name app, it reverts to DNS resolution. Kong then sends the request on to <IP_ADDRESS>, and, depending on the environment (as @bungle and I suspect is happening here), Kong will handle this query as a loopback (remembering that by default, the strip_uri param is true, so Kong forwards the query to <IP_ADDRESS> as just /, which is then not found by the router).
@erutherford if you create a second API with uris=/, I'd be willing to bet you'll see that API handle the request as expected.
Thus, this is not a bug; this is simply Kong behaving as expected (and designed) as a result of DNS resolution on the upstream_url.

@erutherford, well just what I though. I installed your docker environment and was able to reproduce this. Now the DNS resolution inside that Kong repository is the root cause for the problem as I suggested.
bungle@docker:~/kong-test$ sudo docker-compose exec kong bash
PING app (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from app.kongtest_default (<IP_ADDRESS>): icmp_seq=1 ttl=64 time=0.090 ms

Now outside to Kong container:
bungle@docker:~/kong-test$ http <IP_ADDRESS>:8001/apis
HTTP/1.1 200 OK
Access-Control-Allow-Credentials: false
Access-Control-Allow-Origin: *
Connection: keep-alive
Content-Type: application/json; charset=utf-8
Date: Wed, 26 Apr 2017 20:42:00 GMT
Server: kong/0.10.1
Transfer-Encoding: chunked

{
    "data": [
        {
            "created_at":<PHONE_NUMBER>000,
            "http_if_terminated": true,
            "https_only": false,
            "id": "64163a61-5ba6-4cda-b34e-7b27a6dee649",
            "name": "app",
            "preserve_host": false,
            "retries": 5,
            "strip_uri": true,
            "upstream_connect_timeout": 60000,
            "upstream_read_timeout": 60000,
            "upstream_send_timeout": 60000,
            "upstream_url": "http://app",
            "uris": [
                "/app"
            ]
        },
        {
            "created_at":<PHONE_NUMBER>000,
            "http_if_terminated": true,
            "https_only": false,
            "id": "a3dfbf6c-9072-4a0d-95c8-c0806cd6aacd",
            "name": "apps",
            "preserve_host": false,
            "retries": 5,
            "strip_uri": true,
            "upstream_connect_timeout": 60000,
            "upstream_read_timeout": 60000,
            "upstream_send_timeout": 60000,
            "upstream_url": "http://apps",
            "uris": [
                "/apps"
            ]
        }
    ],
    "total": 2
}

Umm. I made a mistake here. I need to investigate this a bit more.

@p0pr0ck5 this was indeed a flaw in the original docker-compose file I provided and was why I created a new one that actually spins up a service named app and thus docker will create a DNS record for app.  Since docker is providing the DNS for this container, it should in theory take precedence over the tld of .app since it can service that request locally instead of hitting the loopback address and going back to kong.

What happens there is really Kong proxying to same Kong:
<IP_ADDRESS> - - [26/Apr/2017:21:01:51 +0000] "GET / HTTP/1.1" 404 56 "-" "curl/7.29.0"
<IP_ADDRESS> - - [26/Apr/2017:21:01:51 +0000] "GET /app HTTP/1.1" 404 56 "-" "curl/7.29.0"

So it seems like app is resolving to <IP_ADDRESS> inside Nginx / Kong. Outside Nginx / Kong it seems to resolve to <IP_ADDRESS>. When requesting localhost/appsI do get only a single access log entry:
<IP_ADDRESS> - - [26/Apr/2017:21:15:22 +0000] "GET /apps HTTP/1.1" 200 12 "-" "curl/7.29.0"

Kong resolves dns via the nameserver in resolve.conf correct? https://github.com/Mashape/kong/blob/master/kong/tools/dns.lua#L23-L30
[root@8406c4abe004 /]# cat /etc/resolv.conf
nameserver <IP_ADDRESS>
options ndots:0
[root@8406c4abe004 /]# nslookup
> server <IP_ADDRESS>
Default server: <IP_ADDRESS>
Address: <IP_ADDRESS>#53
> app
Server:		<IP_ADDRESS>
Address:	<IP_ADDRESS>#53

Non-authoritative answer:
Name:	app
Address: <IP_ADDRESS>

I understand that if there wasn't an app record, the .app tld record is set to a loopback address and that was causing kong to proxy to kong.  How is kong still resolving it to the loopback address if the nameserver it's supposed to be using is resolving it correctly?

Ok here is the full answer:

Kong reads resolv.conf
Inside your container the app is actually app.kongtest_default and that means that Docker DNS has a search options added (?)
Now Kong balancer actually resolves app and gets <IP_ADDRESS> and it seems like your docker Kong image is listening all 127.x.x.x.

There is a quite new commit here:
https://github.com/Mashape/lua-resty-dns-client/commit/0116f08ee823fcc10bcd77388836814975950dee
But I'm not sure if it resolves this.

@Tieske mind you check this out as this may be related to the recent work you have done with lua-resty-dns-client. I can test it on my machine as I now have this working on a VM / Docker.

I think I am having kind of the same error - I published a reproducible example here
https://github.com/XVincentX/kong-node-playground
The APIs are both reachable from the kong container
[root@b31ab0295c74 /]# ping customers
PING customers (<IP_ADDRESS>) 56(84) bytes of data.
64 bytes from customers.kongnode_default (<IP_ADDRESS>): icmp_seq=1 ttl=64 time=0.074 ms
64 bytes from customers.kongnode_default (<IP_ADDRESS>): icmp_seq=2 ttl=64 time=0.199 ms
^C
--- customers ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1028ms
rtt min/avg/max/mdev = 0.074/0.136/0.199/0.063 ms
[root@b31ab0295c74 /]# curl http://customers:3000
{"name":"Vincenzo","surname":"Chianese"}[root@b31ab0295c74 /]#

but if I try to forward a request to Kong, I'm getting an error:
vchianese@Vincenzos-MacBook-Pro:~/dev/kong_node$ curl http://localhost:8000/invoices
{"message":"Service unavailable"}

This is my API object
{
  "data": [
    {
      "uris": [
        "/invoices"
      ],
      "id": "2bf6a0e7-6417-4f16-8dd8-ff3f89446b60",
      "upstream_read_timeout": 60000,
      "preserve_host": false,
      "created_at":<PHONE_NUMBER>000,
      "upstream_connect_timeout": 60000,
      "upstream_url": "http://invoices:3000",
      "strip_uri": true,
      "https_only": false,
      "name": "invoices",
      "http_if_terminated": true,
      "upstream_send_timeout": 60000,
      "retries": 5
    }
  ],
  "total": 1
}

Whoops, sorry for the late reply, I missed this somehow 😞
@bungle I think you are right, Kong tries to resolve app as it requires fully qualified names.
@erutherford what's the behaviour with the new Kong 0.11 RC releases? That version will honour the search and ndots directives from /etc/resolv.conf

Hmm closing this and considering it a "stale" issue, I am afraid... As @Tieske mentioned, we do have support for the search and ndots directive in Kong 0.11+, so I would advise giving this another try, or consderring it solved.
@erutherford If you have upgraded to our latest release (0.11.1) and still are encountering what you believe is a bug, please do open a new issue on this repository. Thank you!


Contest pages

This PR:

Updates the contests page to work with the new public_id
Connects the individual contest page to postgres

Missing from this is connecting the overview table which will likely require us to first make some changes to the validations table.

LGTM


improve development setup docs and remove redundant quotes

What was the end-user problem that led to this PR?
Some distrobutions of linux do not ship with the col command which ronn requires. And there are also some code blocks that don't need to quoted.
What is your fix for the problem, implemented in this PR?
Update the developer docs to include instruction to install the package that has the col command and remove the code block quotes that don't need to be there.

Ah, sorry @segiddins, I really should have double checked before i pushed 😞

@bundlerbot retry


Chisels & Bits can't chisel Glaxx blocks

I was going to submit a PR, using the IMC fix mentioned here in the same vein as the Chisel compatibility code you've already got, but I'm not familiar with forge, and couldn't seem to get it working. Hardwiring BlockXtone#isFullBlock to always return true lets C&B recognize the blocks as modifiable, since that's what it's actually checking, but I'm not sure that that really counts as a solution...

Reopening until there is an actual release with this fix in it. Hopefully, it'll be available soon.

Fixed in 1.1.0


Add linting tasks as part of Circle CI

Add linting tasks as part of Circle CI
Description of what's changing
Linting as a separate step when a PR is made in ./.circleci/config.yml
...
What else might be impacted?
The build step is dependent on the success of the linting task.
...
Resolves #66
...
Checklist
[ ] Tests are written and maintain or improve code coverage
[ ] I've tested this in my application using yarn link (if applicable)
[x] You consent to and are confident this change can be released with no regression

Hey @vivek32ta , are you still working on this? I'm going to try to solve the same issue

@fguery You go ahead and do it. I'm currently stuck on personal issue. And thanks!  :)


Have to add domain to Service Account Username

Rancher Version: v1.1.0
Steps to Reproduce:

Go to Active Directory and fill out all information except don't add the domain to service account user

Results: Error message Cannot talk to ldap
Expected:  Until recently I didn't need to have the domain added to that field

Version - master 8/25
Verified fixed


nothing to see here yet

Closes TURBO-1943

Does this mean support for tty/virtual keyboard is getting added?

@weyert I'd love for you to add your use cases for tty/virtual keyboard in #1235! (this PR might get thrown away, so I want to make sure I don't lose your comments!)


Update ReverseInteger.cpp

Done, pls merge the pr #184 @ela2003 @divya520357 @abhinavcodedeveloper

please attach a ss of your output


Node-driver based provisioning does not respect NO_PROXY environment variable

What kind of request is this (question/bug/enhancement/feature request):
Bug
Steps to reproduce (least amount of steps as possible):

Create an environment that only allows internet access via an HTTP proxy and has access to a vSphere API endpoint in the local network.
Deploy Rancher server with the HTTP_PROXY and HTTPS_PROXY environment variables set to use the proxy and NO_PROXY set to exclude all private networks as documented here.
Try to create a vSphere cluster according to docs. Note that the vCenter IP address falls within the subnet CIDR specified in the NO_PROXY env variable.

Result:
Nodes fail to be provisioned because the machine driver attempts to connect to the vCenter via the internet proxy
Expected Result:
Nodes are provisioned. Machine driver respects the NO_PROXY CIDR values that exclude the vCenter IP from being proxied.
Other details that may be helpful:

Rancher supports CIDR notation ranges in this list

See docs

Rancher docker-machine is build using Go 1.9.x which doesn't support CIDR notation in the NO_PROXY env var (Go 1.10.x or later does).

Environment information

Rancher version (rancher/rancher/rancher/server image tag or shown bottom left in the UI): 2.2.5
Installation option (single install/HA): HA

Cluster information

Cluster type (Hosted/Infrastructure Provider/Custom/Imported):
Machine type (cloud/VM/metal) and specifications (CPU/memory):
Kubernetes version (use kubectl version):

(paste the output here)

Docker version (use docker version):

(paste the output here)

rancher/rancher:v2.3-head 18fc2d261

Created local Rancher server

Setup proxy vars, and NO_PROXY to exclude private network for vSphere

Connect via VPN to access vSphere
Created tunnel via ngrok & finish Rancher install
Create vSphere cluster successfully

No logs are shown in proxy

Create ec2 cluster and see connect logs in proxy


Support Cutter

IDA Pro is not the only reverse engineering platform in the world. Another popular one is Radare2, a highly-portable cross-platform reverse engineering framework and a toolkit without dependencies. It has support for analyzing binaries, disassembling code, debugging programs, attaching to remote GDB/LLDB, WinDbg servers, rich plugin system (see r2pm), and integration with various decompilers. For example, ghidra decompiler plugin - r2ghidra-dec. It is actively developed and can be easily integrated in various open source and commercial products. I believe, it will be highly beneficial to support these and provide a package for install from r2pm, see the package repository here: https://github.com/radareorg/radare2-pm

For documentation on writing plugins for radare2 see Scripting and Plugins Radare2 Book chapters.
Cutter is a crossplatform Qt/C++ GUI frontend to radare2:

For documentation on writing plugins for Cutter see the official tutorial and the curated list of various popular plugins.

analyse_fw_r2.py is a r2pipe-based script that allows you to do the same as analyse_fw_ida.py
A plugin for the cutter may appear in the future

Check out https://cutter.re/docs/plugins/tutorial-python.html for info on making cutter plugins

@yeggor why have you removed the radare2 support?

I removed the r2pipe based scripts because they don't work as expected with the latest radare2.
The r2pipe-based scripts are still available here: https://github.com/yeggor/UEFI_RETool/tree/88455f9be1a037d2d20a16cf3b8c9c991b13b2a0, but you probably want to fix them a bit before using.
Most likely, I will bring them up to date to build a dependency graph between UEFI drivers without using IDA Pro.

@yeggor Ideally this uefi thing should be packaged into r2pm, for visibility and maintaince, we run tests in radare2-extras to ensure things dont break.
Also, i didnt knew about the existence of it, or didnt tried to use, but overlooking the script i think the problem is related to some changes in the JSON of aflj (offset is now addr), not sure if there are more breaking changes. but it should be easy to spot the other errors. do you have some details about the reason why its not working?

The problem is more likely in the analyzing engine I wrote. I was doing a statistical calculation of the UEFI protocols and services found and found that the script began to find less.
If this script is interesting to the community, I will be happy to update it.
I just verified that the script can still be used (r2 commit: 324a26f5c6290bcfd6d214b1665230d4d542c197).

@trufae @krsh some features from uefi_retool are implemented in uefi_r2 (esil based UEFI analyzer):

search for EFI protocols
search for EFI GUIDs
search for Boot Services
search for Runtime Services

further work is planned on this project


fix: Support Pydantic v2 via shims.

Weheat is a dependency from HA, so this PR tries to make it compatible with multiple versions of Pydantic to help with the upgrade described in https://github.com/home-assistant/core/issues/99218.
I'm aware that the files are auto-generated, but the OpenAPI generator seems to support either pydantic v1 or v2, not the backwards compatible version. I'd suggest first generating a release with the backwards compatible version, the doing an upgrade to the version two at some point in the future.
There are no tests on the repo, so I did basic import testing with a dockerfile:
FROM python:3.12

WORKDIR /app
COPY . /app
RUN pip install .
RUN python -c "from weheat import *"

I will merge this and release it soon. I'm looking into making some basic test cases to avoid regression on this topic.

Thanks for the fast response @jesperraemaekers , I thought about adding basic test cases but I thought it would be too much out of scope, since the files are auto generated.


Signature is not deterministic?

Hi, when I use signWithK1 function to sign the same hash value with same private key, the signatures I get are different each time. And none of them is the same as I get from other language(like eos-go). When I check the code in eos-go, it seems use compact signature there which make sure the signature is deterministic. How can I get the same result in this library. Thank you very much.

I encountered the same problem, and the signature was not verified in Dapp

To do this you would have to provide a signature method that removed the random generator input to the message signing.  The library currently does not support this.  If you look at the current implementation, you could modify it to do this, but it is not recommended.


Pumpkin mcu service

PR Overview:

Installation package for Pumpkin MCU Kubos Service

Documentation:

No documentation changes are needed.

Integration Tests:

@CatGarab is there a test that needs to be included?

 Pumpkin MCU Service runs on boot

@jacoffey3 I added the Makefile, in case you wanted to look at it. Don't worry about the PR template (docs + integration tests)


Update virtio-win signed drivers

As one of the users experiencing issues with virtio-win 217 drivers, I see there have been commits to address reported issues that I've also experienced, like not installing correctly & the numerous virtiofs patches to issues since 217 was published.  I'd like to install these updates, test, and provide feedback on these known bugs.
217 is dated 5-31-22 (six weeks behind) and, like probably hundreds of other users, I've been patiently waiting for a new build to be published.  Unfortunately I don't have the time / resources to build & sign the drivers myself. Self-builds also unnecessarily open the door to more QA confusion by introducing additional variables into the discussion.  "What commit did you build??", etc.
Can the project team please produce an updated signed build some time soon?  Can someone give us an ETA on when we might expect it?
I'd suggest someone look into just automatically publishing builds out of your CI server when they pass validation (with appropriate warnings of course) so testers can provide better feedback and stop whining about stale builds :)
Along with this I'd include validation test output somewhere so testers can more easily see what's tested, how it's tested, and that it passes.

I took a stab at this in my dev branch to just have GitHub actions take care of this. That way you can have "a build" from a given commit in both pre and post commit workflows. That build may or may not have passed every validation test under the sun, but at least it would be "there".
That and having some of the CI actions in GitHub actions makes things quite a bit more transparent rather than having it locked behind a private CI exclusively
Any thoughts? Happy to push a PR for this to at least have basic build matrix support, which could then publish an artifact

@fubar-1
Thank you for your feedback and suggestions. At the moment we are working on publishing a new "stable" based on build 221. We will try to push the new "latest" one right after that.
Best,
Vadim.

@JonKohler We already copy some of the build artefacts for each PR. For example, you can see pvpanic driver package and the test results: https://www.dropbox.com/sh/nfywarl6wr40y6i/AACajqMTbhR2eJIBmM890TVza?dl=0&lst=
We can change CI to publish all the drivers, but they will not be signed at all and we need to think about build versioning in order not to create confusion.
We don't want to use publicly available CIs as there is a restriction on free tiers for the time that CI can run. And we want to test SDV as part of the CI build. Currently for all the platform + SDV for all the drivers runs for ±two hours.

Hi @fubar-1
I'm really sorry for delay.
I will try pushing 221 on the weekend.
The team is currently working on new features and internal releases,
this is the main reason why you don't see  many commits recently.
Cheers,
Vadim.

We can  continue maintaining the upstream drives at https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/upstream-virtio/
But it will take some time to make adjustments in our build and attestation signing process.
@YanVugenfirer Yan, what do you think?

We can continue maintaining the upstream drives at https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/upstream-virtio/
But it will take some time to make adjustments in our build and attestation signing process.
@YanVugenfirer Yan, what do you think?

Yes, it looks like a good idea. There is a need for upstream builds.
Another thing is that we might need to mark somehow stable and integration builds.

A new upstream build is available at
https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/upstream-virtio/virtio-win-08032022.zip
The build was done from the upstream snapshot on Aug 3rd, 2022. All Win10 binaries are attestation signed
by Microsoft.
Best,
Vadim.


[Help] The definition of predefined_val_ts?

Hi, thank you for the excellent work. After reading the paper and the code, I am still confused about the definition of predefined_val_ts. Could you give suggestions for applying to other scenes in DFC2019?
https://github.com/centreborelli/satnerf/blob/de7bd6ec1cbd6f3895949350d6ca98ba56e87bdd/eval_satnerf.py#L186-L209

Hello! Thank you for your comment.
Sat-NeRF uses the index of each input image in the training set to learn a transient embedding.
Validation images are not part of the training set, so there are different ways to handle them: e.g., do not use any transient embedding, or assign them a random index or a predefined index.
With predefined_val_ts, I simply fixed the selected image indexes for the validation views, but you can use any of the choices I listed, there is no correct way to do it.
Best,
Roger

Dear Roger,
Thank you for your reply.  Does it mean that the value of predefined_val_ts has no effect on the validation performance?
Best,
Dapeng

Exactly, predefined_val_ts does not alter the output rgb values or the output geometry. PSNR and elevation MAE will not be affected.
Best,
Roger

Well, that is cool. Thank you.

Hi @rogermm14 sorry for asking this again. Does it mean for any other AOI, we can fix the predefined_val_ts to any number, and the validation metrics wouldn't be affected?


Add concurrency to GitHub Actions.

Very delayed follow-up to https://github.com/vector-im/element-ios/pull/5101, this PR adds concurrency to the SDK actions.
Closes https://github.com/vector-im/element-ios/issues/5039.

Codecov Report

Merging #1505 (3486551) into develop (a14a6be) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop    #1505   +/-   ##
========================================
  Coverage    11.66%   11.67%
========================================
  Files          507      507
  Lines        83158    83158
  Branches     35522    35522
========================================
+ Hits          9704     9707    +3
+ Misses       73096    73093    -3
  Partials       358      358

Impacted Files
Coverage Δ

MatrixSDK/Utils/MXHTTPClient.m
30.39% <0.00%> (+0.50%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update a14a6be...3486551. Read the comment docs.


Get request with dto as input

Hi,
Is there a way to generate a get request with a dto as input?
This is my api:
        [HttpGet]
        [Route("api/sleeve-bearing-report")]
        [ProducesResponseType(typeof(FileContentResult), 200)]
        public async Task<IActionResult> SleeveBearingReport([FromQuery]SleeveBearingInputParametersDto inputParameters)
        {
          //code here
        }

This is the generated getrequest:
sleeveBearingReport(diameter: number, minToleranceDSmall: number, maxToleranceDSmall: number, housing: number, minToleranceDBig: number, isRpm: boolean, maxToleranceDBig: number, bearingLength: number, load: number, speed: number, maxTemp: number, rot: number, splitBushing: boolean, retainedBearing: boolean, unlubricatedOrOnlyLubricatedAtAssembly: boolean, wetSurroundings: boolean, unitOfMeausureId: number, productId: number): Promise<FileResponse>{}

Is it possible to generate the following request?
sleeveBearingReport(inputParameters: sleeveBearingInputParametersDto ): Promise<FileResponse>{}

At the moment this is not possible because in Swagger you cannot describe this - you only have multiple query parameters... For that we need a custom property to tell the generator to merge the parameters into a request object/class.

@RSuter  Hopefully this will add support and the order of the fields will change as DTO fields increase or decrease

At the moment this is not possible because in Swagger you cannot describe this - you only have multiple query parameters... For that we need a custom property to tell the generator to merge the parameters into a request object/class.

any progress? would be really nice! right now the client code generates breaking changes whenever we add new query parameters in the backend :(


php artisan infyom:api $MODEL_NAME --paginate=10    invalid

No difference is found with no paginate

limit:Limit the number of returned results
search:Global search on all searchable fields
offset:Offset from where results should be returned
sortedBy:Sort result by given field
How do I use this

laravel 5.5

are you replacing $MODEL_NAME with the actual name of the model you want to generate?

@zhaomengqiao can you post the exact command that you are trying to run?

it's adminlte known issue  https://github.com/InfyOmLabs/adminlte-templates/issues/2

completed and merged in templates repo.


Have a cleaned up V1 for TableQG

jaydesen created the following on Apr 21:
Some hardcodings need to be removed. Make it ready for V1.
Original issue: https://github.ibm.com/ai-foundation/PrimeQA/issues/147

duplicate #46


ME conduits disconnecting from AE cables

I noticed this happens whenever our server gets restarted, and each time chunks are unloaded (logging off etc).
ME conduits disconnect themselves from AE cables, cutting off portions of the network. This also happens visually - conduit connector disappears and also AE cable changes appearance (looses the "branch", as if nothing was connected).
This can be fixed by replacing the AE cable (conduit connector doesn't reappear, but everything works), but not by replacing the ME conduit (doing so just breaks the AE cable it's connecting to).
Using AE2 rv2 beta 7  and EiO <IP_ADDRESS>9
Server is using Cauldron

You are using an old EnderIO version - all known ME conduit issues have been fixed in EnderIO 2.2.8 build 349

I agree with @MatthiasMann, he fixed all ME Conduit bugs, you can download the version: ci.tterrag.com/job/EnderIO/lastSuccessfulBuild/

Thank you, will update then.

Okay, if this is fixed, then please close this issue.


Made changes to support pyspark 3.0.x and python 3.8 runtime

PySpark 2.4.x is getting deprecated along with python 3.7 so changed notebooks to use Pyspark 3.0.3 and Python 3.8
Tracker --> https://github.ibm.com/aiopenscale/tracker/issues/22911

@arsuryan - Can you please review?


get signal killed every time

I have no idea what's the problem.
% zeus start                                                                                (git)-[master]
Starting Zeus server v0.15.4
[ready] [crashed] [running] [connecting] [waiting]
boot
└── default_bundle
    ├── development_environment
    │   └── prerake
    └── test_environment
        └── test_helper

Available Commands: [waiting] [crashed] [ready]
zeus rake [run to see backtrace]
zeus runner (alias: r) [run to see backtrace]
zeus console (alias: c) [run to see backtrace]
zeus server (alias: s) [run to see backtrace]
zeus generate (alias: g) [run to see backtrace]
zeus destroy (alias: d) [run to see backtrace]
zeus dbconsole [run to see backtrace]
zeus test (alias: rspec, testrb) [run to see backtrace]
signal: killed

Does your app run without zeus?

sorry, it seems the zeus version in my Gemfile and the rbenv's default  zeus version are mismatch. pleas
e help me close this issue thanks

I'm actually getting this one now. I simply gem installed zeus and everything worked fine. but everytime I run zeus start, it gets the signal: killed error. This is the first line of the error:

I just had this issue, and it was because I had to install pending gems in my bundle. Make sure you can run rails s successfully first.

Hi, I'm actually getting the same error but I don't have any problem running my app in local.
Actually some of my coworkers use zeus on the same app, but for some reason I get signal killed at all time.
Please let me know if you need any information.
Thank you!


EXEC SQL sections are removed in the parsed program

I tried printing the text of a compilation unit retrieved using getCtx().getText(), and the EXEC SQL sections are completely ignored. Is this a missing feature of the parser? Or is there another way to retrieve the embedded SQL statements?
Thank you

Correct, the current implementation of the COBOL preprocessor deletes EXEC SQL statements from the output stream (cf. CobolPreprocessorImpl.java).
However I could commit an implementation, which conserves EXEC SQL statements by converting them to comment lines. E.g. EXEC SQL chardata END-EXEC would be converted to something like >*EXEC SQL chardata END-EXEC in the COBOL preprocessor output stream, so that the COBOL parser does not try to parse SQL statements as COBOL statements.
Can you please provide an exemplary COBOL snippet including the EXEC SQL statement you are trying to parse? That would help me to write a unit test.

Thank you, but aren't comments also removed in the parsed code?
Anyway, I tried an example from tutorialspoint:
IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.
DATA DIVISION.
WORKING-STORAGE SECTION.
EXEC SQL
INCLUDE SQLCA
END-EXEC.
EXEC SQL
INCLUDE STUDENT
END-EXEC.
EXEC SQL BEGIN DECLARE SECTION
END-EXEC.
01 WS-STUDENT-REC.
05 WS-STUDENT-ID PIC 9(4).
05 WS-STUDENT-NAME PIC X(25).
05 WS-STUDENT-ADDRESS X(50).
EXEC SQL END DECLARE SECTION
END-EXEC.
PROCEDURE DIVISION.
EXEC SQL
SELECT STUDENT-ID, STUDENT-NAME, STUDENT-ADDRESS
INTO :WS-STUDENT-ID, :WS-STUDENT-NAME, WS-STUDENT-ADDRESS FROM STUDENT
WHERE STUDENT-ID=1004
END-EXEC.
IF SQLCODE=0
DISPLAY WS-STUDENT-RECORD
ELSE DISPLAY 'Error'
END-IF.
STOP RUN.

The preprocessor now adds EXEC SQL as comments to the output stream (cf. CobolParserPreprocessorListenerImpl.java and ExecSqlMultiline.cbl.preprocessed).
Currently, those and all other comments are skipped by the COBOL parser (cf. Cobol85.g4). I will modify the grammar and parser this week, so that comments are processed on a hidden channel and can be accessed via the abstract semantic graph (ASG).

Ok, now it should work.
The preprocessor now extracts EXEC SQL data description entries from data divisions as well as EXEC SQL statements, EXEC SQLIMS statements and EXEC CICS statements from procedure divisions. In the preprocessor output stream they are marked by the line tags >*EXECSQL, >*EXECSQLIMSand >*EXECCICS (e.g. cf. ExecSql.cbl.preprocessed).
In the ASG the contents of those EXEC statements are provided as texts (cf. ExecSqlStatementMultilineTest.java). The linked unit tests shows how to access those texts, giving an answer to your original question.
Additional parsers would be required for parsing these textual EXEC SQL and EXEC CICS statements, e.g. for extracting table names. However, imo that is a different issue and I am not sure, if that feature is required, currently.

Thank you very much, it works perfectly.
I don't think your parser should parse SQL code, there already exist SQL parsers for Java.


Does 'providers' support customized LLM Model service?

Hi team,
I'm trying to use promptfoo to test my own LLM Model for prompts, but I failed to find the example/cases or userguide how to setup promptfooconfig.yaml. For example, my LLM Model service provide api like https://${host}:${port}/${api}, is it possible?
Thanks so much!

For starters, what is your LLM service API, is it custom?  Does it match one of these providers: https://promptfoo.dev/docs/providers/?
https://promptfoo.dev/docs/providers/custom-api details how to make a custom provider.
From there, check promptfoo init to create the initial config: https://promptfoo.dev/docs/getting-started

Thanks @jamesbraza!  @ooooona the Custom APIs documentation is the way to go - feel free to follow up if anything is unclear.

Thanks @jamesbraza and @typpo, I want to test prompt with my own finetune Code-Llama Model, which I wrapped with my python code, providing v2 inference api like: _http://localhost:8080/v2/models/{model_name}/infer_.
It seems that (current providers)[https://promptfoo.dev/docs/providers/] do not match my case, and I have to use custom-api? And I need to implement call my v2 inference api, like _http://localhost:8080/v2/models/{model_name}/infer_, in callApi: (prompt: string, context: { vars: Record<string, any> }) => Promise<ProviderResult>?

That is correct.  You could either use a custom API module, or if you're willing to conform to a predefined format you can use a webhook provider, which sends an HTTP POST request {prompt: string} and expects {output: string} in return from your server.


Idle notifier fix combat notifications, reset notifications on interaction

To make the notifier more precise make combat idle notifier use
interacting changed.
In order to make animation idle notifier work with timer resetting, make
it remember last animation.
When someone logs out, reset all timers to not have sent the
notifications after they are no longer relevant.
When player clicks in-game, he obviously changed his activity, so reset
all idle timers to default non-valid state.

Closes #1614
Closes #5013
Signed-off-by: Tomas Slusny<EMAIL_ADDRESS>

I believe this fixes #1614.

Actually changed the mouseIdleTicks counting back to mouseManager listener because it was not working correctly (apparently they were correct only when window was focused/not changing focus).

Any update on this @Adam- ?

Reworked the PR to keep using AnimationChanged and moved the combat idle checks to interacting changed

Added tests


frontend npm install gives warnings

Bug Description
Hi,
Slowly boarding into this project... while installing dependencies I got information regarding '2 high severity vulnerabilities'
I wonder if its something that should be taken care of?
(venv) PS C:\Users\arek\Desktop\LLM\langflow\src\frontend> npm install
npm warn deprecated<EMAIL_ADDRESS>This module is not supported, and leaks memory. Do not use it. Check out lru-cache if you want a good and tested way to coalesce async requests by a key value, which is much more comprehensive and powerful.
(...)
added 1159 packages, and audited 1161 packages in 48s

294 packages are looking for funding
  run `npm fund` for details

2 high severity vulnerabilities

Reproduction
\src\frontend> npm install
Expected behavior
no warnings
Who can help?
No response
Operating System
Windows 11 pro
Langflow Version
1.0.13
Python Version
None
Screenshot
No response
Flow File
No response

Hi @severfire,
I don't believe this warning is related to langflow core; it seems to be associated with shadcn-ui. We will monitor their repository, and once the issue is resolved, we will update it as soon as possible.
track the issue here


back to top when clicking on paper accordion element

The anchor (href='#') inside paper-accordion element causes this bug.
This also is a bad practice for SEO. The element is clickable so it should get a pointer once you hover over it. However, it isn't a link (read linking to other location/source).

Hi @ichigo77 , what do you think?


[SPARK-18710][ML] Add offset in GLM

What changes were proposed in this pull request?
Add support for offset in GLM. This is useful for at least two reasons:

Account for exposure: e.g., when modeling the number of accidents, we may need to use miles driven as an offset to access factors on frequency.
Test incremental effects of new variables: we can use predictions from the existing model as offset and run a much smaller model on only new variables. This avoids re-estimating the large model with all variables (old + new) and can be very important for efficient large-scaled analysis.

How was this patch tested?
New test.
@yanboliang @srowen @felixcheung @sethah

Test build #71974 has started for PR 16699 at commit a1f5695.

Test build #71974 has finished for PR 16699 at commit a1f5695.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71974/
Test FAILed.

jenkins, retest this please

Test build #71975 has started for PR 16699 at commit d071b95.

Test build #71975 has finished for PR 16699 at commit d071b95.

This patch fails MiMa tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71975/
Test FAILed.

you should not modify sharedParams directly.
"// DO NOT MODIFY THIS FILE! It was generated by SharedParamsCodeGen."
And if there is no other algorithms inheriting hasoffset, I suggest that do not create this

can you please:
1.) add [MLLIB] to the title
2.) fix the MiMa tests
The MiMa tests error seems to be related to the interface change, probably you need to add an exclude to the file "project/MimaExcludes.scala":
[error]  * abstract method getOffsetCol()java.lang.String in trait org.apache.spark.ml.param.shared.HasOffsetCol is inherited by class GeneralizedLinearRegressionBase in current version.
[error]    filter with: ProblemFilters.excludeInheritedNewAbstractMethodProblem
[error]  * abstract method offsetCol()org.apache.spark.ml.param.Param in trait org.apache.spark.ml.param.shared.HasOffsetCol is inherited by class GeneralizedLinearRegressionBase in current version.
[error]    filter with: ProblemFilters.excludeInheritedNewAbstractMethodProblem
[error]  * abstract synthetic method org$apache$spark$ml$param$shared$HasOffsetCol$setter$offsetCol_=(org.apache.spark.ml.param.Param)Unit in trait org.apache.spark.ml.param.shared.HasOffsetCol is inherited by class GeneralizedLinearRegressionBase in current version.
[error]    filter with: ProblemFilters.exclude[InheritedNewAbstractMethodProblem]("

Test build #71994 has started for PR 16699 at commit d2afcb0.

Test build #71994 has finished for PR 16699 at commit d2afcb0.

This patch fails Scala style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71994/
Test FAILed.

Merged build finished. Test FAILed.

Test build #71995 has started for PR 16699 at commit 9eca1a6.

Test build #71995 has finished for PR 16699 at commit 9eca1a6.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/71995/
Test PASSed.

Merged build finished. Test PASSed.

@zhengruifeng @imatiach-msft
Thanks much for pointing out the issue due to the hasOffset trait. This is what caused the test to fail. I have moved it to the GLRBase class. Things are working now 👍

@actuaryzhang  added a couple comments, please take a look, thanks!

Test build #72003 has started for PR 16699 at commit d44974c.

@imatiach-msft Thanks so much for your detailed review. Incredibly helpful. I've addressed all your comments in the new commit. Major changes are highlighted below:

Create OffsetInstance in Instance.scala and remove dependency of IWLS on regression.
Add param checking for offset.

Let me know if there is anything else that needs improvement.

Test build #72003 has finished for PR 16699 at commit d44974c.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72003/
Test PASSed.

Merged build finished. Test PASSed.

This looks really good.  I made a couple more comments.  I'm not sure about the class hierarchy of Instance/OffsetInstance.  Maybe committers can take a look/comment on this PR? @jkbradley @srowen @yanboliang Thanks!

@imatiach-msft Many thanks again for the review. I have incorporated some of your suggestions:

Create initialization of instance directly if it is Gaussian(identity) to avoid expensive conversion from OffsetInstance.
Fix spacing issues in test
Change off to offset to be more informative.

Feedback on the other suggestions:

move import to beginning, use ~==:  I did not change this for consistency with other tests in GLM. It's probably better to update all tests in a separate PR if this is preferred.
Inheriting Instance: I'm not sure about this and leaning toward not touching Instance since it is used a lot elsewhere. Let's see what others suggest.
Question: if OffsetInstance is a subclass of Instance, will a method with a signature RDD[Instance] also work with RDD[OffsetInstance]?
Would you please elaborate on the following comment:

if family = Gaussian and link = Identity we shouldn't be passing any offset.

This is the linear regression case and it still allows offset. The offset will be subtracted from the response as can be seen from the initialization in WLS Instance(label - offset, weight, features).

Test build #72040 has started for PR 16699 at commit e183c08.

Test build #72040 has finished for PR 16699 at commit e183c08.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72040/
Test PASSed.

Test build #72067 has started for PR 16699 at commit 58f93af.

Merged build finished. Test FAILed.

Test build #72068 has started for PR 16699 at commit da4174a.

Test build #72068 has finished for PR 16699 at commit da4174a.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72068/
Test PASSed.

@actuaryzhang thank you for updating the PR - I agree with your first two points.  With regards to the method of RDD[Instance] also working with RDD[OffsetInstance], I think it should work, as long as you define the method such that it takes in the parameter myAwesomeMethod(myRdd: RDD[_ <: Instance])
I think you need that "<:" in the method definition to allow the method to take in anything that inherits from Instance.
With regards to "Would you please elaborate on the following comment..." it looks like I made a mistake, I did not notice that you were subtracting the offset from the response when creating the Instance object in the previous commit, in which case you can ignore that comment.
The PR looks good to me, maybe the committers can comment on it?

@zhengruifeng Thanks for the suggestions. Added casting and instrumentation.
@imatiach-msft Thanks for the clarification! It is probably worth  another PR to clean up all tests in GLM.
Let me know if there is any additional comments!

Test build #72095 has started for PR 16699 at commit 52bc32b.

Test build #72095 has finished for PR 16699 at commit 52bc32b.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72095/
Test PASSed.

LGTM, thanks!

Test build #72162 has started for PR 16699 at commit 59e10f7.

@imatiach-msft Thanks much for your review. Renamed off.

@yanboliang @zhengruifeng @srowen
Could you guys take a look and let me know if there is any changes needed? Thanks much!

Test build #72162 has finished for PR 16699 at commit 59e10f7.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72162/
Test PASSed.

Test build #72672 has started for PR 16699 at commit fc64d32.

@sethah @imatiach-msft
Please review the new commit. Main changes:

Fix issue in null deviance calculation in the presence of offset. Except for special cases (Gaussian with identity link), we need to re-estimate a intercept only model. The discussions in #16740 are indeed very helpful here.
Updated the test for weighted GLM to include offset as well in order to test the addition of offset thoroughly.

Please let me know if there is any issue. Thanks!

Test build #72672 has finished for PR 16699 at commit fc64d32.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72672/
Test PASSed.

Merged build finished. Test PASSed.

whew, this was a lot of work :)

@sethah Yes, that is lots of work. However, the only critical change (since the last commit) is on the calculation of the null deviance. The other changes are mainly because of updating existing tests as you suggested (which makes sense and indeed helps to capture a few issues). This requires adding new offsets and updating all inference results... I think it makes sense to just focus on the null deviance part.

@actuaryzhang This is looking pretty good overall. Regarding the prediction logic, R glm does not allow you to predict with offsets, correct? I notice that statsmodels in Python does allow it. For, e.g. Poisson regression, predicting with offsets is a slight convenience, I guess. The point of training with offsets in this case is to effectively model rates instead of counts, and so when you predict you'd be interested in the rates and predicting without offsets, that's exactly what you get. If you want to apply those rates to a specific exposure time, then you just take your predictions and multiply them with exposure * y_hat where y_hat is the predicted rate. I think I have that correct. My question then, is do we need to support prediction with offset? It makes things clunkier in the code, so there is some reason not to. Since you presumably have more experience with using GLMs, I'd like to hear your thoughts.

@sethah Thanks much for your review.
Regarding prediction, both R and my implementation here allow prediction with offsets. If the users want to get the predicted rates (instead of counts), then they can specify the offset to be lit(0.0) in the new data set when making predictions.
The following shows that R prediction includes offsets:
A <- matrix(c(0, 1, 2, 3, 5, 7, 11, 13), 4, 2)
b <- c(1, 0, 0.2, 2)
off <- c(2, 3, 5, 4)
df <- as.data.frame(cbind(A, b))
model <- glm(formula = "b ~ .", family = "poisson", data = df,
             offset = off)
p1 <- predict(model, df, type = "response")
p2 <- as.numeric(exp(cbind(1, A) %*% coef(model) + off))
sum(p1 - p2)

This part of the code in the GeneralizedLinearRegression file shows that we do need offset when making the prediction.
  protected def predict(features: Vector, offset: Double): Double = {
    val eta = predictLink(features, offset)
    familyAndLink.fitted(eta)
  }

And this is the test for the prediction:
val familyLink = FamilyAndLink(trainer)
        model.transform(dataset).select("features", "offset", "prediction", "linkPrediction")
          .collect().foreach {
          case Row(features: DenseVector, offset: Double, prediction1: Double,
          linkPrediction1: Double) =>
            val eta = BLAS.dot(features, model.coefficients) + model.intercept + offset
            val prediction2 = familyLink.fitted(eta)
            val linkPrediction2 = eta
            assert(prediction1 ~= prediction2 relTol 1E-5, "Prediction mismatch: GLM with " +
              s"family = $family, and fitIntercept = $fitIntercept.")
            assert(linkPrediction1 ~= linkPrediction2 relTol 1E-5, "Link Prediction mismatch: " +
              s"GLM with family = $family, and fitIntercept = $fitIntercept.")
        }

Does this make sense?

I'm finding R's behavior for prediction with offsets to be a bit strange. Yes, R does use the original offsets supplied during training to do prediction, but what if I want to make predictions on a completely new dataset, with completely different offsets? R still tries to use the original ones, which doesn't make sense to me. Then, maybe predicting on "new" data when you've trained using offsets doesn't make sense? Regardless, here we are allowing users to pass in a new dataframe to make predictions on, where they can supply an offset column for the new "test" data - offsets that are not the same as the training offsets. This is a feature not provided by R, it seems.

@sethah The predict method can work with new data in R. See below. Shall we focus on the current implementation, instead of discussing the details of the R behavior? :)
Let me know if there is any suggestion on the last commit. Thanks.
df <- data.frame(y = rnorm(10), x = rnorm(10), off = rnorm(10))
model <- glm(y ~ x, data = df, offset = off)
df2 <- data.frame(y = rnorm(20), x = rnorm(20), off = rnorm(20))
p1 <- predict(model, df2, type = "response")
p2 <- as.numeric(model.matrix(y~x, df2) %*% coef(model) + df2$off)
sum(p1 - p2)

Ah, thank you very much for that clarification, I don't have much experience using R. I tried this out earlier, and it seems you have to use the same offset column name as you show above. I apologize for not making my intentions clear earlier - I wanted to discuss the R behavior to make sure we were doing the right thing here.
We actually discuss R's behavior a lot and even use it as a baseline for our unit tests, as you know. I think being able to compare new Spark PRs to other packages enables reviewers who may not be experts in a particular area to still contribute. More importantly, since it takes quite a lot of time and effort to do code review I like to make sure we have decided on the high-level approach before spending time on implementation details, which I find saves time in the long run. JMO, hope it makes sense :)

Merged build triggered.

Merged build started.

Test build #72857 has started for PR 16699 at commit 90d68a6.

Merged build triggered.

Merged build started.

Test build #72858 has started for PR 16699 at commit e95c25b.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72857/
Test FAILed.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72858/
Test FAILed.

Merged build finished. Test FAILed.

@sethah Thanks much for your review. I've made a new commit that addressed all your comments. Please see my inline comments.  Let me know if there is any other suggestions. Thanks.

Merged build triggered.

Merged build started.

Test build #72863 has started for PR 16699 at commit 4b336be.

Test build #72863 has finished for PR 16699 at commit 4b336be.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/72863/
Test PASSed.

@sethah Is there anything else you would recommend for this PR? Thanks.

@yanboliang @sethah
Any suggestion on moving this PR forward? Appreciate your comments and reviews.

@actuaryzhang Sorry for late response. I will review this PR after finishing 2.2 release, thanks for your patience.

@yanboliang Thanks for the update.
This is a big change, although lots of then are on the test side.
Let me know if there is anything I can do to help make the review easier.

@yanboliang Thanks much for the review. The new commit includes everything you suggested except implementing WeightLeastSquares interface for OffsetInstance. Please see my incline comments above.

Test build #78727 has started for PR 16699 at commit 1e47a11.

Test build #78727 has finished for PR 16699 at commit 1e47a11.

This patch fails PySpark pip packaging tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78727/
Test FAILed.

jenkins, retest this please

Test build #78729 has started for PR 16699 at commit 1e47a11.

Test build #78729 has finished for PR 16699 at commit 1e47a11.

This patch fails PySpark pip packaging tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78729/
Test FAILed.

Not sure what this error msg means, but it seems unrelated to this PR.

this is a known issue in test runs currently - it's mentioned in<EMAIL_ADDRESS>just so you know.

Got it. I should pay more attention to that mailing list from now on :)

Jenkins, test this please.

Test build #78812 has started for PR 16699 at commit 1e47a11.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78812/
Test PASSed.

Made a new commit that fixes the issues you pointed out.

Test build #78920 has started for PR 16699 at commit db0ac93.

Test build #78920 has finished for PR 16699 at commit db0ac93.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/78920/
Test PASSed.

Merged into master. Thanks for contribution and all reviews! This great feature will benefit lots of users.
@actuaryzhang Could you send follow-up PRs to address the two inline comments? Thanks.

@yanboliang @actuaryzhang this PR breaks the scala-2.10 build:
[error] /home/jenkins/workspace/spark-master-compile-maven-scala-2.10/mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala:1117: value absTol is not a member of Int
[error]     assert(summary.dispersion ~== dispersionR absTol 1E-3)
[error]                                               ^
[error] /home/jenkins/workspace/spark-master-compile-maven-scala-2.10/mllib/src/test/scala/org/apache/spark/ml/regression/GeneralizedLinearRegressionSuite.scala:1232: value absTol is not a member of Int

See: https://amplab.cs.berkeley.edu/jenkins/view/Spark QA Compile/job/spark-master-compile-maven-scala-2.10/4721/

@hvanhovell I will send a quick fix soon, thanks for your kindly remind.

#18489 fixed the build failure. Thanks.


touched/untouched not working with ng build --prod

Versions
ng --version

Angular CLI: 1.6.2
Node: 9.3.0
OS: win32 x64
Angular: 5.1.2
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cli: 1.6.2
@angular-devkit/build-optimizer: 0.0.36
@angular-devkit/core: 0.0.22
@angular-devkit/schematics: 0.0.42
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.9.2
@schematics/angular: 0.1.11
@schematics/schematics: 0.0.11
typescript: 2.4.2
webpack: 3.10.0

Repro steps

Step 1
ng new testProject
Step 2
cd testProject
Step 3
npm install
Step 4
add code in /src/app/app.component.html:

Angular blog

      field
      <input [(ngModel)]="model.field" #field="ngModel"/>

untouched - {{field.untouched}}

pristine - {{field.pristine}}

Step 5
add code in /src/app/app.component.ts:

export class AppComponent {
  title = 'app';
  // my code begin
  model = {
    field: ''
  };
  // my code end
}

Step 6
add code in /src/app/app.module.ts:

import { FormsModule, ReactiveFormsModule } from '@angular/forms';
...
  imports: [
    BrowserModule,
    FormsModule,
    ReactiveFormsModule
  ],

Step 7
npm run build

Observed behavior

Desired behavior
as after run ng serve:

In fact it seems that more things are not working here. I comment another issue in Material 2 repo, but I think it's related. Below is the copy of what I posted in Material Repo.

(...) I could reproduce it in a test app, available in heroku. In the images below you can notice that not just the label stops working, but also the interpolation (see the gifs, just beneath the inputs). It doesn't seems to be related just to material... maybe @angular/cli's build-optimizer flag has something to do with it. I noticed the problem using Chrome and Firefox (Windows 10)

without --prod

with --prod

If it helps, the repo for the test app above: https://github.com/julianobrasil/material2-test
Some time after you push to master (it's not protected - anyone can push), the app can be viewed from https://angular-julianobrasil.herokuapp.com/ (it's a free heroku account so it goes to sleep after 30 min idle and takes some time to wake up after someone tries do access it for the first time)

I have slimier problem.
If publish site with 'ng build --prod', (keydown) and (click) of a input control will not fire.
If use ' ng build --aot --build-optimizer true --output-hashing all --extract-css true --name
d-chunks false --sourcemaps false --environment prod', the events will be fired normally.
cli version: 1.6.2
angular version: 5.1.2

Likely to be caused by uglify-es's latest update.

@cuiliang, I'll try that. But the funny part is that if I compile the project locally (in my Windows 10 machine), it seems to work just fine. When I deploy to heroku (compile remotely), the problem comes up.

@cuiliang, it worked

This @cuiliang's suggestion (https://github.com/angular/angular-cli/issues/8997#issuecomment-353957698) worked for me:
ng build --aot --build-optimizer true --output-hashing all --extract-css true --name
         d-chunks false --sourcemaps false --environment prod

new UglifyJSPlugin({
    sourceMap: buildOptions.sourcemaps,
    uglifyOptions: {
        ecma: wco.supportES2015 ? 6 : 5,
        warnings: buildOptions.verbose,
        ie8: false,
        // mangle: {
        //     safari10: true,
        // },
        mangle: false,
        compress: uglifyCompressOptions,
        output: {
            ascii_only: true,
            comments: false,
            webkit: true,
        },
    }
}),

Confirmed it can be solved by disable the mangle option in uglifyJS plugin.

@julianobrasil Thank you for putting up a repro, it helps a lot. I'm trying to see what the issue with it but I don't think I can reproduce... This is what I did:
git clone https://github.com/julianobrasil/material2-test
cd material2-test
npm i
npm run ng -- serve --prod

Then I opened http://localhost:4200/ and interacted with the boxes:

The input box content seems to be reflected outside of it.
@julianobrasil mentioned it doesn't happen locally for him (https://github.com/angular/angular-cli/issues/8997#issuecomment-353959248) so maybe that's why it doesn't happen locally for me either.
Other users mentioned disabling Uglify mangle helped. Maybe it is related to the Uglify version. We use uglifyjs-webpack-plugin which internally uses uglify-es. You can check the versions with npm ls uglify-es:
$ npm ls uglify-es
<EMAIL_ADDRESS>D:\sandbox\material2-test
`--<EMAIL_ADDRESS>  `--<EMAIL_ADDRESS>    `--<EMAIL_ADDRESS>
For users that are experiencing this problem, what version of uglify-es is displayed on the machine where the bundle is built?

@filipesilva<EMAIL_ADDRESS>will cause the error.

@filipesilva, I did a lot of things after my comment above saying it worked fine over here... now the problem is happening locally too. I don't know whether the uglify was 3.2.1 at that time, but now:

Bisected to 3.2.2(OK) to 3.3.0(Not OK)

Just to confirm @trotyl's info, in other projects that are still working right:

I'm investigating what changed in the uglify output between those two versions.
Meanwhile for those affected you can force install<EMAIL_ADDRESS>by running this command:
npm i<EMAIL_ADDRESS>--save-exact && rm -rf package-lock.json node_modules && npm i && npm ls uglify-es

At the end it should output the following:
+--<EMAIL_ADDRESS>| `--<EMAIL_ADDRESS>|   `--<EMAIL_ADDRESS>deduped
`--<EMAIL_ADDRESS>
You can see the right version (3.2.2) version is listed under uglifyjs-webpack-plugin. This is a temporary workaround.

I've been looking into this and saving output in https://github.com/filipesilva/material2-test/tree/uglifyjs-debug.
I haven't found what changed exactly yet. You can see the different in output for the main bundle in https://github.com/filipesilva/material2-test/commit/cf6a59e3961be5c4b6a0f9671690200cd19a762d?w=1 (click Load diff for<EMAIL_ADDRESS>on the left and<EMAIL_ADDRESS>on the right).
I can say that a fair bit has changed though. I can also see that the main bundle is 12k bigger with 3.3.2.

@kzc do you perhaps have an idea of what might have changed, or what I can do to narrow down my search of what's broken?

@filipesilva our project is also affected. When we deploy, a flag --prod is used. I've followed this direction and it helped: https://github.com/angular/angular-cli/issues/8997#issuecomment-353957698
Replacing --prod with list of options eliminates one of the steps that invokes UglifyJS (https://github.com/angular/angular-cli/wiki/build).

@filipesilva Please take a look at the uglify-es commit log for recent changes and bug fixes:
https://github.com/mishoo/UglifyJS2/commits/harmony
All I can suggest is to update to the most recent version and if it still fails then file an uglify issue with a reproducible JS input.

Can confirm this as well. I was just about to publish my Angular chrome extension and when running --prod the styles are all messed up. As @mlurker and @cuiliang mentioned, running the commands fixes the problem.

@mlurker I suggest you use my workaround in https://github.com/angular/angular-cli/issues/8997#issuecomment-353970841 instead of replacing the build command, as doing the latter will leave you with much bigger bundles.
@kzc I'm going through the most recent commits (https://github.com/mishoo/UglifyJS2/pull/2656, https://github.com/mishoo/UglifyJS2/pull/2658 and https://github.com/mishoo/UglifyJS2/pull/2659) and applying them manually but it doesn't seem to make a difference. Also tried https://github.com/mishoo/UglifyJS2/pull/2661.
Trying to get in touch with the core team to figure out which bits of core code could originate the current behaviour to narrow it down and get you a good issue repro.

For what its worth very similar problems with other non material components as well (ngx-bootstrap if it matters). They are non functional, silently not reacting to user clicks. Curiously, the same components used elsewhere on the same page (in other lazy loaded components) continue to work.
Running with ng serve --aot --build-optimizer true --output-hashing all --extract-css true --named-chunks false --sourcemaps false --environment prod as suggested above, everything works.

https://github.com/angular/angular-cli/issues/8992 has the same root cause as this issue.

How resolve problem
[alekcei@localhost BID]$ npm ls uglify-es
<EMAIL_ADDRESS>/home/alekcei/workspase/BID
└─┬<EMAIL_ADDRESS>  └─┬<EMAIL_ADDRESS>    └──<EMAIL_ADDRESS>

    _                      _                 ____ _     ___
   / \   _ __   __ _ _   _| | __ _ _ __     / ___| |   |_ _|
  / △ \ | '_ \ / _` | | | | |/ _` | '__|   | |   | |    | |
 / ___ \| | | | (_| | |_| | | (_| | |      | |___| |___ | |
/_/   \_\_| |_|\__, |\__,_|_|\__,_|_|       \____|_____|___|
               |___/

Angular CLI: 1.6.2
Node: 8.1.2
OS: linux x64
Angular: 5.1.2
... animations, common, compiler, compiler-cli, core, forms
... http, language-service, platform-browser
... platform-browser-dynamic, router

@angular/cli: 1.6.2
@angular-devkit/build-optimizer: 0.0.36
@angular-devkit/core: 0.0.22
@angular-devkit/schematics: 0.0.42
@ngtools/json-schema: 1.1.0
@ngtools/webpack: 1.9.2
@schematics/angular: 0.1.11
@schematics/schematics: 0.0.11
typescript: 2.4.2
webpack: 3.10.0

But not working
The problem is similar in the commentary
https://github.com/angular/angular-cli/issues/8997#issuecomment-353950777

@Alekcei, take a look at https://github.com/angular/angular-cli/issues/8997#issuecomment-353970841

Sorry.
npm i<EMAIL_ADDRESS>--save-exact
[alekcei@localhost BID]$ npm ls uglify-es
<EMAIL_ADDRESS>/home/alekcei/workspase/BID
├─┬<EMAIL_ADDRESS>│ └─┬<EMAIL_ADDRESS>│   └──<EMAIL_ADDRESS>deduped
└──<EMAIL_ADDRESS>
Yes, it's working now
большое спасибо

But not working  --no-build-optimizer
ERROR Error: No NgModule metadata found for 'function (){}'.
    at t.TMtu.t.resolve (2.b2a6116….chunk.js:1)
    at t.TMtu.t.getNgModuleMetadata (2.b2a6116….chunk.js:1)
    at e.TMtu.t._loadModules (2.b2a6116….chunk.js:1)
    at e.TMtu.t._compileModuleAndAllComponents (2.b2a6116….chunk.js:1)
    at e.TMtu.t.compileModuleAndAllComponentsAsync (2.b2a6116….chunk.js:1)
    at t.TMtu.t.compileModuleAndAllComponentsAsync (2.b2a6116….chunk.js:1)
    at Function.TMtu.t.createDinamicPetitionOperationComponent (2.b2a6116….chunk.js:1)
    at t.TMtu.t.createOperationsPetitionContent (2.b2a6116….chunk.js:1)
    at e._next (2.b2a6116….chunk.js:1)
    at e.E9/g.e.__tryOrUnsub (vendor.07efdf9….bundle.js:1)
(anonymous) @ vendor.07efdf9….bundle.js:1
LMZF.t.handleError @ vendor.07efdf9….bundle.js:1
next @ vendor.07efdf9….bundle.js:1
e.object.i @ vendor.07efdf9….bundle.js:1
E9/g.e.__tryOrUnsub @ vendor.07efdf9….bundle.js:1
E9/g.e.next @ vendor.07efdf9….bundle.js:1
E9/g.e._next @ vendor.07efdf9….bundle.js:1
E9/g.e.next @ vendor.07efdf9….bundle.js:1
TO51.e.next @ vendor.07efdf9….bundle.js:1
LMZF.e.emit @ vendor.07efdf9….bundle.js:1
(anonymous) @ vendor.07efdf9….bundle.js:1
t.invoke @ polyfills.947cc91….bundle.js:1
r.run @ polyfills.947cc91….bundle.js:1
LMZF.t.runOutsideAngular @ vendor.07efdf9….bundle.js:1
onHandleError @ vendor.07efdf9….bundle.js:1
t.handleError @ polyfills.947cc91….bundle.js:1
r.runTask @ polyfills.947cc91….bundle.js:1
e.invokeTask @ polyfills.947cc91….bundle.js:1
v @ polyfills.947cc91….bundle.js:1
d @ polyfills.947cc91….bundle.js:1

again missing MetaData

Got a weird behavior: focus event on input fired four times on page load (no autofocus, ofc), both ngSubmit and submit events stopped working (page loaded on submit) and many other html event issues. Fixed by removing --prod.

I have similar issue - events from a nested component which are emmitted from ngFor are not handled properly by parent component.
Example:
@Component({
  selector: 'cdg-list',
  template: `

                        Add
                        Delete
                        Edit

            `,
})
export class ListComponent {
  @Input() items: any[] = [];

  @Output() addItem = new EventEmitter();
  @Output() deleteItem = new EventEmitter();
  @Output() editItem = new EventEmitter();
}

@Component({
  selector:        'app',
  template:        `
                       <cdg-list [items]="['foobar', 'foo', 'test', 'bar']"
                                 (addItem)="onAdd()"
                                 (editItem)="onEdit()"
                                 (deleteItem)="onDelete()">
                       </cdg-list>
                   `,
  changeDetection: ChangeDetectionStrategy.OnPush
})
export class AppComponent {
  onAdd() { alert('ADD'); }
  onDelete() { alert('DELETE'); }
  onEdit() { alert('EDIT'); }
}

In app component, onEdit() handler will always be called, no matter which event gets triggered from 'dumb' component(ListComponent).
This only happens when building with --prod switch and if I set uglifier mangle=false  then the issue is gone.
Not sure if this is the same issue, but the source of this issue is definately the same.
One thing to note is that I'm having this issue with angular/cli 1.5+ (tested it with 1.5.0, up to 1.6.3) and angular 5+ (5.0.0, up to 5.2.0-beta)
Example

@Ledzz, @dojchek : are you seeing the bug after aplying @filipesilva's suggested workaround: https://github.com/angular/angular-cli/issues/8997#issuecomment-353970841?

No, the bug is gone with that walk-around (can also be done just by setting mangle=false as suggested in above).

Why angular cli not fixed stable versions of dependency packages in package.json? I think, that this strategy saved all angular builds in the world in this case)

@MaxGorshkov uglify-es is the sub-dependency of uglify-js-pllugin, fixing uglify-js-pllugin version will not help with that.

@julianobrasil didn't try this, I've used this method

Solution #8997 from @filipesilva worked for me.

@MaxGorshkov Angular CLI does not have a direct dependency in uglify-es, like @trotyl said, and so we cannot pin it. Even if we did<EMAIL_ADDRESS>is not a prerelease version or anything, it is an official stable version.
Pinning (using a specific version of a dependency) does not work because you cannot pin subdependencies. In the NPM ecosystem the way you ensure reproducible installs is not via pinning dependencies but rather by using lockfiles, either package-lock.json or yarn.lock.

@filipesilva => #9006

Thanks @d3viant0ne, will try to get a release of Angular CLI up. @danbucholtz you might want to pin as well on Ionic's side.

Keeping issue opened until the workaround is released.

The bug causing these problems is reported with repros at https://github.com/mishoo/UglifyJS2/issues/2663.

<EMAIL_ADDRESS>is now released, which should set the minimum version of uglifyjs-webpack-plugin to use a pinned<EMAIL_ADDRESS>Big thanks to @d3viant0ne for updating and releasing it 👍
Meanwhile https://github.com/mishoo/UglifyJS2/issues/2663 has also been fixed and should be part of the next release of uglify-es.

nice work . i can update now.


As staff I can view adopter applications for a given pet via the /pet/:id Applications tab

Currently, all applications are viewable at /adoption_application_reviews
We want to also see the applications for a specific pet on that pet's show page under the Applications tab
Acceptance criteria

a pet's adopter applications are shown on /pet/:id Applications tab
follows implementation to render tab partials used for checklists (PR still in progress)

I'd like to take a stab at this.

2 quick questions...
It looks like as of now there's no application partial, so my thought is to move the current application rendering that appears in adoption_application_reviews/index.html.erb into a partial that can be shared by both that view and the applications tab partial for the pet show view. Is it okay to go ahead and do that and then update the adoption_application_reviews/index.html.erb accordingly?
Second, it looks like the controller action for the tabs is going to be taken care of in pull #265 and that this line in the pets show view will render the partial for the selected tab:

Does this mean I should add the tab partial for the pet's applications in the organizations/tasks directory?

As of now, I've created the shared partial. It leaves the layout of adoption applications reviews index unchanged. For the applications tab on the show page, I could see the partial rendered in a card like:

Or without the header:

Either way, that's what it would look like if dropped into the tasks/applications.html.erb file of pull #265. Since that pull hasn't been merged yet, I could put the card in its own partial file? Or hold off until that pull has been merged? Let me know.

Great! I think there's nothing to stop you carring on here. If yours is merged first, then @ErinClaudio can just accept the incoming on a conflict to overwrite his current file.
If you want to continue working on this in this PR, you could fix up the _applications partial and render each application in a table like we use for the staff index or pets index table. So we would have an Applicant column, Status Column and links or buttons to view adopter profile, edit application or create an adoption. Does that make sense?

Yes, makes sense. I'll start working on that.

Ok so let me know what you think of this. This would be how the partial would render on the adoption applications reviews index page:

And on the pet show applications tab, it would be rendered like:

I kept the conditional rendering of the adopter profile and create adoption links from how it was before. But changed the status to a badge to be consistent with the pet index table, replacing the bolded withdrawn status with a different background color. Obviously doesn't have to be yellow. Could also do different colors for the different statuses.

@MooseCowBear Great! I think the table looks cleaner. I am open to ideas (I am no designer), but have some ideas:
Column Order

related things are better next to each other, so let's put the Adopter Profile column second, next to Applicant column. I think we could just rename the Adopter Profile column to just Profile. Then have the Edit Application column third, renamed to just Edit, followed by Status and finally Confirm Adoption (renamed from Create Adoption).

Status column

I think we can get rid of 'status:' text in each row and just show the status value
Some ideas on colours: we can either make them all the same colour for now and revisit colours later when we have UX support, or if you feel like making some colours depending on status, can do something like:
Withdrawn could be black text on light grey background
Successful Applicant and Adoption Made could be black text on pale green background
Adoption Pending can be black text on pale yellow background
Under review can be the (current) white text on blue background
Awaiting review can be white text on darker purple background

The rest of the columns

What do you think about using an icon in each instead of the current text or button? E.g.:
Profile column we use fe-file-text class icon
Edit column we use fe-edit class icon
Confirm Adoption we use something like fe-plus

All that sounds good. I can go ahead and make the changes in this PR.
One question: Right now, displaying the applicant's name creates an n+1 problem for the applications review index page. Do you want me to adjust the Pet.org_pets_with_apps method that the applications reviews controller index method uses to take care of this while I'm at it?

All that sounds good. I can go ahead and make the changes in this PR.
One question: Right now, displaying the applicant's name creates an n+1 problem for the applications review index page. Do you want me to adjust the Pet.org_pets_with_apps method that the applications reviews controller index method uses to take care of this while I'm at it?

Good catch. You can eager load the associated data with includes on the initial query.
Also, for future ref, it might be easier to create a draft PR and continue the discussion on there because then reviewers can see any code changes to date, and it's easier to review conversations on the PR (the issue is not the first place to look).


SSLv3 alert handshake failure

How to reproduce:

Run third_party/webpagereplay/replay.py --port 0 --ssl_port 0 --no-dns_forwarding --use_closest_match -r foo.wpr
Run /opt/google/chrome/chrome --no-default-browser-check --no-first-run --ignore-certificate-errors --host-resolver-rules="MAP * <IP_ADDRESS>,EXCLUDE localhost" --testing-fixed-http-port=HTTPS_PORT --testing-fixed-https-port=HTTP_PORT --no-proxy-server --user-data-dir=$(mktemp -d) in a separate terminal with HTTP_PORT and HTTPS_PORT taken from the output of the first command.
Go to https://i1.sndcdn.com/artworks-000139618215-ubm1i6-t500x500.jpg.

Looks like force using python2.7.9 is not a good solution since most machine have python 2.7.9. A proposed solution from @nickie is to package pyopenssl in third_party/ instead.

I can reproduce this issue on Mac OS X Version 10.11.5 (15F34), running Python 2.7.11 installed via Homebrew. I have checked, and the ssl library is using Homebrew's OpenSSL (version 1.0.2h as of this writing):
import ssl
print ssl.OPENSSL_VERSION

Thus, it does not appear to be an issue with an old version of OpenSSL.

Ok, I have more time to dig into this today. For the case of loading theverge.com, it looks like this is not an issue with  an old version of OpenSSL but the https request we issue to the server. In particular, the header may cause SSL failure.
This is the minimal reproducible:
`import httplib
connection = httplib.HTTPSConnection('tags.api.umbel.com')
headers = {
'accept-language': 'en-US,en;q=0.8',
'accept-encoding': 'gzip,deflate,br',
'connection': 'keep-alive',
'accept': 'image/webp,image/,/*;q=0.8',
'user-agent': 'Mozilla/5.0 (Macintosh;Intel Mac OS X 10_11_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2793.0 Safari/537.36',
'host': 'm3jnlukh12.execute-api.us-east-1.amazonaws.com',
'referer': 'http://www.theverge.com/2015/6/5/8727399/parrot-bebop-drone-review',
'cache-control': 'max-age=0'
}
RETURN {"message":"Forbidden"}
connection.request('GET', '/lrjhazrpqbgtnrij/w.js', headers=headers)
response = connection.getresponse()
print response.read()
RETURN ACTUAL JS CONTENT
connection.request('GET', '/lrjhazrpqbgtnrij/w.js', headers={})
response = connection.getresponse()
print response.read()`

Please ignore my comment above, after digging more, the root cause is because we translate the string host name to ip address in https://github.com/chromium/web-page-replay/blob/7142ee71f87f300bcd325b516c18d33912915952/httpclient.py#L298

# Use an IP address because WPR may override DNS settings.
 connection_ip = self._real_dns_lookup(connection_host)

To reproduce this bug:
import httplib
connection = httplib.HTTPSConnection('tags.api.umbel.com')
connection.connect()  # OK
connection = httplib.HTTPSConnection('<IP_ADDRESS>')  # The IP address of tags.api.umbel.com
connection.connect()  # ssl.SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590) got thrown
I believe fix here is to not translate the string host name to ip address.  WPR shouldn't override DNS setting on recording mode.

FYI: @pasko @gabadie

Another problem while fetching HTTPS:
import httplib
connection = httplib.HTTPSConnection('sdk.streamrail.com')
connection.connect()
connection.request('GET', '/player/sr.ads.js')
Running the above yields:
Traceback (most recent call last):
File "test.py", line 3, in
connection.connect()
File "/usr/lib/python2.7/httplib.py", line 1182, in connect
self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file)
File "/usr/lib/python2.7/ssl.py", line 487, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 243, in init
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 405, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:510: error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert handshake failure
On the other hand, once can navigate to https://sdk.streamrail.com/player/sr.ads.js with no problem on Chrome. (Though navigating to https://sdk.streamrail.com/ return a 403)

Might not be the root cause but support for SSLv3 is being disabled across the web and it looks like the default HTTPSConnection implementation doesn't use TLS.
Here is a gist that looks like it does what you need: https://gist.github.com/flandr/74be22d1c3d7c1dfefdd
I'll try quickly in your test above to see if it helps.

Argh, I can't reproduce the problem on my main dev machine (Windows).  Let me know if the gist doesn't help and I can try on my Mac at home tonight.

I just try it & still has the problem:
import httplib
import urllib2
import socket
import ssl

class TLS1Connection(httplib.HTTPSConnection):
    """Like HTTPSConnection but more specific"""
    def __init__(self, host, **kwargs):
        httplib.HTTPSConnection.__init__(self, host, **kwargs)

    def connect(self):
        """Overrides HTTPSConnection.connect to specify TLS version"""
        # Standard implementation from HTTPSConnection, which is not
        # designed for extension, unfortunately
        sock = socket.create_connection((self.host, self.port),
                self.timeout, self.source_address)
        if getattr(self, '_tunnel_host', None):
            self.sock = sock
            self._tunnel()

        # This is the only difference; default wrap_socket uses SSLv23
        self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file,
                ssl_version=ssl.PROTOCOL_TLSv1)

class TLS1Handler(urllib2.HTTPSHandler):
    """Like HTTPSHandler but more specific"""
    def __init__(self):
        urllib2.HTTPSHandler.__init__(self)

    def https_open(self, req):
        return self.do_open(TLS1Connection, req)

# Override default handler
urllib2.install_opener(urllib2.build_opener(TLS1Handler()))

connection = TLS1Connection('sdk.streamrail.com')
connection.connect()
connection.request('GET', '/player/sr.ads.js')
res = connection.getresponse()

print res.read()

Traceback (most recent call last):
File "test.py", line 39, in
connection.connect()
File "test.py", line 23, in connect
ssl_version=ssl.PROTOCOL_TLSv1)
File "/usr/lib/python2.7/ssl.py", line 487, in wrap_socket
ciphers=ciphers)
File "/usr/lib/python2.7/ssl.py", line 243, in init
self.do_handshake()
File "/usr/lib/python2.7/ssl.py", line 405, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [Errno 1] _ssl.c:510: error:14094410:SSL routines:SSL3_READ_BYTES:sslv3 alert handshake failure
Also it's worth noting that we don't have this problem with http.client lib in python3.

I'll fire up a VM and see if I can reproduce it.
I wouldn't be surprised if it was still an issue with the TLS config and TLSv1 may not be good enough.  TLS v 1.0 is also turned off in a lot of places though not everywhere might support 1.2.
In this case cloudfront appears to support TLS 1.0 -> 1.2 so I'm surprised it didn't work but maybe something else is causing the failure: https://www.ssllabs.com/ssltest/analyze.html?d=sdk.streamrail.com&s=<IP_ADDRESS>&latest
A tcpdump may help identify what option is causing the failure.

Sorry, did you really mean python 2.7.6?  As far as I can tell, SNI and a lot of other things are broken pre-2.7.9 so you're going to want that as a minimum.

Thanks, I will try python2.7.9 to see if it fixes the problem. The challenges is most of the bots on chromium still use python2.7.6, hence I had to revert https://github.com/chromium/web-page-replay/commit/03de0c5863e60943145df2b7bf93ee5bcb534048
If switching python to 2.7.9 is the fix, probably I will enforce python2.7.9 only for recording mode.

Maybe I'm missing something but on Ubuntu 16.0.4 with Python 2.7.10 this works fine:
import httplib
connection = httplib.HTTPSConnection('sdk.streamrail.com')
connection.connect()
request = connection.request('GET', '/player/sr.ads.js')
response = connection.getresponse()
print response.read()

Python < 2.7.9 isn't going to work with any server that needs SNI and it looks like protocol negotiation is working fine otherwise.

Thanks Patrick, I confirm that updating the python version to 2.7.9+ is the solution to fetch these https URL. I will make a change to throw exception when people try recording WPR with python version < 2.7.9

I verified that we can now load https://i1.sndcdn.com/artworks-000139618215-ubm1i6-t500x500.jpg on webpagereplay recording.


removed p2p and trinity dependencies from py-evm(eth)

What was wrong?
Installing py-evm installed p2p and trinity
How was it fixed?
modified install_requires to only include eth dependencies.
https://github.com/ethereum/py-evm/pull/1188#issuecomment-413697522
Cute Animal Picture

Gonna wait till Monday/Tuesday for merge as I don't want to accidentally cause a bunch of un-intended breakage due to something that we didn't consider.  External libraries that depend on py-evm are eth-tester and web3.py.  There are probably a few 3rd party things as well that we're un-aware of (like how @arachnid used py-evm for the solidity golf competition).

I don't want to accidentally cause a bunch of un-intended breakage due to something that we didn't consider.

Yes please test it thoroughly. I have only relied on unit tests to tell me what I've done is sane.
I don't know much about this project to test it manually.

External libraries that depend on py-evm are eth-tester and web3.py

Coming soon eth-tester-rpc which is basically a rewrite of @pipermerriam's eth-testrpc for eth-tester. It uses py-evm internally.


Simulate an SPO setup

Each SPO should have:

one block producer node;
connected only to two relays.

We should verify that even if the relays, for some time cannot connect to the
block producer, they should re-connect to the produer (which shouldn't take
longer than some specified interval).

Hi @coot do we need to add this to ouroborus-network project?


'none' is not a valid value for min-height

https://developer.mozilla.org/en-US/docs/Web/CSS/min-height
Here's an example of a site that has a global textarea style that sets min-height and ignores the invalid inlined min-height style.

Thanks


[ko] Enhance docs/tasks/tools/install-kubectl-windows.md

This is a Feature Request

What would you like to be added

Apply review #29167.
Enhance content/ko/docs/tasks/tools/install-kubectl-windows.md
Why is this needed
Change line format and some words to common.
Comments

https://kubernetes.io/ko/docs/tasks/tools/install-kubectl-windows/
/language ko

/triage accepted


Questions about commands and installation

Hi
I'm having trouble deploying the service locally (self-hosted) and some doubts due to my lack of docker experience. My first question is about docker-compose.yml in the db part that I configure eg MARIADB_USER with my user from my local machine?

My second question regarding commands related to docker-compose, the only command to use is docker-compose up -e SOCIAL_AUTH_GITHUB_KEY=xxx -e SOCIAL_AUTH_GITHUB_SECRET=yyy.
Because if the only command used doesn't generate a result.

Where am I making a mistake?
Thanks in advance and forgive my english

I think I've actually got the command wrong - can you try adding the *GITHUB* vars to an .env file in the current folder instead? https://docs.docker.com/compose/env-file/