ERR_EMPTY_RESPONSE

I get the following error message when I follow the instructions in the installation guide and get the docker images up and running and visit the webpage (localhost:8080):

What could be the issue causing this error to appear? And how can I investigate what is happening under the hood that results in this error?
Thanks in advance.

@MuadDev ,
Please run docker logs cvat in your console and send us the output.

Thanks for your response, what follows is the output of docker logs cvat
2019-07-03 08:05:34,456 INFO RPC interface 'supervisor' initialized
2019-07-03 08:05:34,456 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2019-07-03 08:05:34,456 INFO supervisord started with pid 1
2019-07-03 08:05:35,459 INFO spawned: 'ssh-agent' with pid 8
2019-07-03 08:05:35,462 INFO spawned: 'git_status_updater' with pid 9
2019-07-03 08:05:35,465 INFO spawned: 'rqworker_low' with pid 10
2019-07-03 08:05:35,468 INFO spawned: 'runserver' with pid 11
2019-07-03 08:05:35,470 INFO spawned: 'rqworker_default_0' with pid 12
2019-07-03 08:05:35,472 INFO spawned: 'rqworker_default_1' with pid 14
2019-07-03 08:05:35,478 DEBG 'ssh-agent' stdout output:
SSH_AUTH_SOCK=/tmp/ssh-agent.sock; export SSH_AUTH_SOCK;
echo Agent pid 15;

2019-07-03 08:05:35,478 DEBG 'ssh-agent' stderr output:
debug2: fd 3 setting O_NONBLOCK

2019-07-03 08:05:35,480 DEBG 'git_status_updater' stderr output:
wait-for-it.sh: waiting for redis:6379 without a timeout

2019-07-03 08:05:35,481 DEBG 'runserver' stderr output:
wait-for-it.sh: waiting for db:5432 without a timeout

2019-07-03 08:05:35,482 DEBG 'rqworker_low' stderr output:
wait-for-it.sh: waiting for redis:6379 without a timeout

2019-07-03 08:05:35,483 DEBG 'rqworker_default_1' stderr output:
wait-for-it.sh: waiting for cvat_redis:6379 without a timeout

2019-07-03 08:05:35,484 DEBG 'runserver' stderr output:
wait-for-it.sh: db:5432 is available after 0 seconds

2019-07-03 08:05:35,484 DEBG 'git_status_updater' stderr output:
wait-for-it.sh: redis:6379 is available after 0 seconds

2019-07-03 08:05:35,485 DEBG 'rqworker_low' stderr output:
wait-for-it.sh: redis:6379 is available after 0 seconds

2019-07-03 08:05:35,485 DEBG 'runserver' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:05:35,485 DEBG 'git_status_updater' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:05:35,486 DEBG 'rqworker_default_1' stderr output:
wait-for-it.sh: cvat_redis:6379 is available after 0 seconds

2019-07-03 08:05:35,486 DEBG 'rqworker_low' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:05:35,487 DEBG 'rqworker_default_1' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:05:35,489 DEBG 'rqworker_default_0' stderr output:
wait-for-it.sh: waiting for cvat_redis:6379 without a timeout

2019-07-03 08:05:35,492 DEBG 'rqworker_default_0' stderr output:
wait-for-it.sh: cvat_redis:6379 is available after 0 seconds

2019-07-03 08:05:35,493 DEBG 'rqworker_default_0' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:05:35,563 DEBG 'ssh-agent' stderr output:
debug2: fd 4 setting O_NONBLOCK

2019-07-03 08:05:36,564 INFO success: ssh-agent entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:05:36,564 INFO success: git_status_updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:05:36,565 INFO success: rqworker_low entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:05:36,565 INFO success: runserver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:05:36,565 INFO success: rqworker_default_0 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:05:36,565 INFO success: rqworker_default_1 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:08:06,514 WARN received SIGTERM indicating exit request
2019-07-03 08:08:06,514 DEBG killing rqworker_default_1 (pid 14) with signal SIGTERM
2019-07-03 08:08:06,514 DEBG killing rqworker_default_0 (pid 12) with signal SIGTERM
2019-07-03 08:08:06,514 INFO waiting for git_status_updater, rqworker_low, ssh-agent, runserver, rqworker_default_0, rqworker_default_1 to die
2019-07-03 08:08:06,515 DEBG fd 31 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>50712 for <Subprocess at<PHONE_NUMBER>36184 with name rqworker_default_1 in state STOPPING> (stdout)>
2019-07-03 08:08:06,515 DEBG fd 35 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>51000 for <Subprocess at<PHONE_NUMBER>36184 with name rqworker_default_1 in state STOPPING> (stderr)>
2019-07-03 08:08:06,515 INFO stopped: rqworker_default_1 (terminated by SIGTERM)
2019-07-03 08:08:06,515 DEBG received SIGCLD indicating a child quit
2019-07-03 08:08:06,515 DEBG fd 26 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>50136 for <Subprocess at<PHONE_NUMBER>36112 with name rqworker_default_0 in state STOPPING> (stdout)>
2019-07-03 08:08:06,515 DEBG fd 30 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>50424 for <Subprocess at<PHONE_NUMBER>36112 with name rqworker_default_0 in state STOPPING> (stderr)>
2019-07-03 08:08:06,515 INFO stopped: rqworker_default_0 (terminated by SIGTERM)
2019-07-03 08:08:06,515 DEBG received SIGCLD indicating a child quit
2019-07-03 08:08:06,515 DEBG killing runserver (pid 11) with signal SIGTERM
2019-07-03 08:08:09,520 INFO waiting for git_status_updater, rqworker_low, ssh-agent, runserver to die
2019-07-03 08:08:12,524 INFO waiting for git_status_updater, rqworker_low, ssh-agent, runserver to die
2019-07-03 08:08:15,528 INFO waiting for git_status_updater, rqworker_low, ssh-agent, runserver to die
2019-07-03 08:08:16,529 WARN killing 'runserver' (11) with SIGKILL
2019-07-03 08:08:16,530 DEBG killing runserver (pid 11) with signal SIGKILL
2019-07-03 08:08:16,530 DEBG fd 25 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>49848 for <Subprocess at<PHONE_NUMBER>71704 with name runserver in state STOPPING> (stderr)>
2019-07-03 08:08:16,530 DEBG fd 21 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>49560 for <Subprocess at<PHONE_NUMBER>71704 with name runserver in state STOPPING> (stdout)>
2019-07-03 08:08:16,531 INFO stopped: runserver (terminated by SIGKILL)
2019-07-03 08:08:16,531 DEBG received SIGCLD indicating a child quit
2019-07-03 08:08:16,531 DEBG killing rqworker_low (pid 10) with signal SIGTERM
2019-07-03 08:08:16,532 DEBG fd 20 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>48840 for <Subprocess at<PHONE_NUMBER>36328 with name rqworker_low in state STOPPING> (stderr)>
2019-07-03 08:08:16,532 DEBG fd 16 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>48624 for <Subprocess at<PHONE_NUMBER>36328 with name rqworker_low in state STOPPING> (stdout)>
2019-07-03 08:08:16,532 INFO stopped: rqworker_low (terminated by SIGTERM)
2019-07-03 08:08:16,533 DEBG received SIGCLD indicating a child quit
2019-07-03 08:08:16,533 DEBG killing git_status_updater (pid 9) with signal SIGTERM
2019-07-03 08:08:16,533 DEBG 'ssh-agent' stderr output:
debug1: XXX shrink: 3 < 4

2019-07-03 08:08:16,533 INFO reaped unknown pid 62
2019-07-03 08:08:16,533 DEBG received SIGCLD indicating a child quit
2019-07-03 08:08:16,534 DEBG fd 10 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>49344 for <Subprocess at<PHONE_NUMBER>35968 with name git_status_updater in state STOPPING> (stdout)>
2019-07-03 08:08:16,534 DEBG fd 15 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>48120 for <Subprocess at<PHONE_NUMBER>35968 with name git_status_updater in state STOPPING> (stderr)>
2019-07-03 08:08:16,534 INFO stopped: git_status_updater (terminated by SIGTERM)
2019-07-03 08:08:16,534 DEBG received SIGCLD indicating a child quit
2019-07-03 08:08:16,535 DEBG killing ssh-agent (pid 8) with signal SIGTERM
2019-07-03 08:08:16,535 DEBG fd 9 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>35480 for <Subprocess at<PHONE_NUMBER>35752 with name ssh-agent in state STOPPING> (stderr)>
2019-07-03 08:08:16,535 DEBG fd 7 closed, stopped monitoring <POutputDispatcher at<PHONE_NUMBER>35192 for <Subprocess at<PHONE_NUMBER>35752 with name ssh-agent in state STOPPING> (stdout)>
2019-07-03 08:08:16,535 INFO stopped: ssh-agent (terminated by SIGTERM)
2019-07-03 08:08:16,535 DEBG received SIGCLD indicating a child quit
2019-07-03 08:10:21,251 INFO RPC interface 'supervisor' initialized
2019-07-03 08:10:21,251 CRIT Server 'unix_http_server' running without any HTTP authentication checking
2019-07-03 08:10:21,251 INFO supervisord started with pid 1
2019-07-03 08:10:22,254 INFO spawned: 'ssh-agent' with pid 8
2019-07-03 08:10:22,257 INFO spawned: 'git_status_updater' with pid 9
2019-07-03 08:10:22,259 INFO spawned: 'rqworker_low' with pid 10
2019-07-03 08:10:22,262 INFO spawned: 'runserver' with pid 11
2019-07-03 08:10:22,265 INFO spawned: 'rqworker_default_0' with pid 13
2019-07-03 08:10:22,267 INFO spawned: 'rqworker_default_1' with pid 15
2019-07-03 08:10:22,269 DEBG 'ssh-agent' stdout output:
SSH_AUTH_SOCK=/tmp/ssh-agent.sock; export SSH_AUTH_SOCK;
echo Agent pid 16;

2019-07-03 08:10:22,269 DEBG 'ssh-agent' stderr output:
debug2: fd 3 setting O_NONBLOCK

2019-07-03 08:10:22,271 DEBG 'rqworker_low' stderr output:
wait-for-it.sh: waiting for redis:6379 without a timeout

2019-07-03 08:10:22,274 DEBG 'git_status_updater' stderr output:
wait-for-it.sh: waiting for redis:6379 without a timeout

2019-07-03 08:10:22,274 DEBG 'runserver' stderr output:
wait-for-it.sh: waiting for db:5432 without a timeout

2019-07-03 08:10:22,275 DEBG 'rqworker_low' stderr output:
wait-for-it.sh: redis:6379 is available after 0 seconds

2019-07-03 08:10:22,276 DEBG 'git_status_updater' stderr output:
wait-for-it.sh: redis:6379 is available after 0 seconds

2019-07-03 08:10:22,277 DEBG 'rqworker_low' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:10:22,278 DEBG 'git_status_updater' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:10:22,278 DEBG 'runserver' stderr output:
wait-for-it.sh: db:5432 is available after 0 seconds

2019-07-03 08:10:22,279 DEBG 'rqworker_default_1' stderr output:
wait-for-it.sh: waiting for cvat_redis:6379 without a timeout

2019-07-03 08:10:22,280 DEBG 'runserver' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:10:22,283 DEBG 'rqworker_default_1' stderr output:
wait-for-it.sh: cvat_redis:6379 is available after 0 seconds

2019-07-03 08:10:22,283 DEBG 'rqworker_default_0' stderr output:
wait-for-it.sh: waiting for cvat_redis:6379 without a timeout

2019-07-03 08:10:22,284 DEBG 'rqworker_default_1' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:10:22,286 DEBG 'rqworker_default_0' stderr output:
wait-for-it.sh: cvat_redis:6379 is available after 0 seconds

2019-07-03 08:10:22,288 DEBG 'rqworker_default_0' stderr output:
bash: cannot set terminal process group (-1): Inappropriate ioctl for device
bash: no job control in this shell

2019-07-03 08:10:22,352 DEBG 'ssh-agent' stderr output:
debug2: fd 4 setting O_NONBLOCK

2019-07-03 08:10:23,354 INFO success: ssh-agent entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:10:23,354 INFO success: git_status_updater entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:10:23,354 INFO success: rqworker_low entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:10:23,354 INFO success: runserver entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:10:23,354 INFO success: rqworker_default_0 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)
2019-07-03 08:10:23,354 INFO success: rqworker_default_1 entered RUNNING state, process has stayed up for > than 1 seconds (startsecs)

Furthermore, using "netstat -an|grep LISTEN" inside the docker I can see that there is nothing listening to port 8080:

Are there perhaps some log files I can check to see why the server fails to start?

@MuadDev , at the moment I have no idea what is going on in the case. Which command do you use to run CVAT? OS? Docker and docker-compose version?

Are you running develop branch for CVAT from github? Do you have any local modifications?

To answer your questions:

Command used to start CVAT: docker-compose up
OS:

Docker version:

Docker compose version:

Branch & changes:

Added nettools to investigate if a process was listening to a port:

Allowed all hosts:

The problem was as follows: The umask on our system was set to 077 by our admin instead of the default 022. This caused some trouble since now the server did not automatically start once the docker was built, hence the ERR_EMPTY_RESPONSE. The fault was ours.

@MuadDev , thank you for the investigation. It saves our time for new features.


Crash when rendering transparent bit

This crash occurs when breaking bits in a block with broken rendering (caused by transparent bits).
Now say that three times fast!
Crash log: https://pastebin.com/XSpvuG8P

And another crash log, this time from rendering the bit item: https://pastebin.com/ajcb3w9i


fix(httpParamSerializerJQLike): First array element always non-indexed

What kind of change does this PR introduce? (Bug fix, feature, docs update, ...)
Bug fix
What is the current behavior? (You can also link to an open issue here)
Serializing an array of objects with $httpParamSerializerJQLike is ambiguous with serializing an object with numeric keys.
params = { a: [{ b: 1 }, { c: 2 }] }
=> a[0][b]=1&a[1][c]=2

This could falsely be deserialized as: { a: { "0": { b: 1 }, "1": { c: 2 } } } (and it absolutely will).
What is the new behavior (if this is a feature change)?
params = { a: [{ b: 1 }, { c: 2 }] }
=> a[][b]=1&a[1][c]=2

This first element ensures a should be considered an array.
Does this PR introduce a breaking change?
Unfortunately this is not backward compatibly (would actually break rack before it pulls the mentioned request), but this is somewhat mandatory as you simply cannot properly serialize some scenarios (see the tests in rack's PR for more scenarios). Should this be a configuration option somewhere?
Please check if the PR fulfills these requirements

[x] The commit message follows our guidelines: https://github.com/angular/angular.js/blob/master/CONTRIBUTING.md#commit-message-format
[x] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

Other information:
This follows these PR's in jQuery and rack: jquery/jquery#3666, rack/rack#1165
This PR is pretty much copied from the jQuery PR, and it should probably only be accepted after jQuery accepts their PR (since this method is a clone of theirs).

We should definitely only consider this after the jQuery PR has been accepted. It is not actionable atm.

The jquery PR https://github.com/jquery/jquery/pull/3666 has been closed because it's stale. For the time being, this should be closed, too.


Checkpoint/Restore failure with internal IPv6 connections

There was a bug report in the CRIU bug tracker (https://github.com/checkpoint-restore/criu/issues/735) that checkpoint/restore did not work with docker because it lacks support for checkpointing restoring containers with established TCP connections. The failure happened during checkpointing.
I recommend to try out Podman and checkpointing works but restoring fails.
The reason for the failure is that the container, in this special case, uses internal IPv6 TCP connections. This TCP connections are bound to the link-local IPv6 address.
On restore Podman (CNI) currently creates a network namespace with the same IPv4 address the container had during checkpointing. IPv6 is ignored. This means a new IPv6 link-local IP address is used in the restored container and the restore of the established TCP connection fails.
The upcoming IPv6 support in Podman is probably a good starting point to add IPv6 addresses to the checkpoint so that they are restored correctly. As far as I understand it right now this will probably be not enough for the special case where the link-local IPv6 address is used by an application in the container. I still need to figure out the details.
I am assigning this to me. This is mainly as a reminder for me and that it is known and documented.

SGTM

Added the do-not-close label.

@adrianreber Any progress on this one?

Unfortunately no. Nothing from my side so far. Still planing to fix it.

@adrianreber ping

@adrianreber ping again

@adrianreber This issue is 1.5 years old, do you want to keep it open?

I will just close it. Nobody has been complaining about this in the last 1.5 years so it does not seem to be really important.


fluid+juicefs The display directory is read-only

Discussed in https://github.com/juicedata/juicefs/discussions/3474

Originally posted by xiashengjie April 13, 2023
Use the following configuration file
dataset.yaml
apiVersion: data.fluid.io/v1alpha1
kind: Dataset
metadata:
name: jfsdemo
spec:
mounts:
- name: minio
mountPoint: "juicefs:///demo"
options:
bucket: "http://<IP_ADDRESS>:9000/fluid"
storage: "minio"
encryptOptions:
- name: metaurl
valueFrom:
secretKeyRef:
name: jfs-secret
key: metaurl
- name: access-key
valueFrom:
secretKeyRef:
name: jfs-secret
key: access-key
- name: secret-key
valueFrom:
secretKeyRef:
name: jfs-secret
key: secret-key
runtime.yaml
apiVersion: data.fluid.io/v1alpha1
kind: JuiceFSRuntime
metadata:
name: jfsdemo
spec:
replicas: 1
tieredstore:
levels:
- mediumtype: MEM
path: /dev/shm
quota: 40Gi
low: "0.1"
log
2023/04/13 04:37:56.318179 juicefs[6] : Meta address: redis://<IP_ADDRESS>:6379/1 [interface.go:402]
2023/04/13 04:37:56.332463 juicefs[6] : Ping redis: 979.547µs [redis.go:2878]
2023/04/13 04:37:56.336622 juicefs[6] : Data use minio://<IP_ADDRESS>:9000/fluid/minio/ [format.go:435]
2023/04/13 04:37:56.375521 juicefs[6] : Volume is formatted as {
"Name": "minio",
"UUID": "48b0ef8c-5040-415e-b228-96e4e9a2c708",
"Storage": "minio",
"Bucket": "http://<IP_ADDRESS>:9000/fluid",
"AccessKey": "minioadmin",
"SecretKey": "removed",
"BlockSize": 4096,
"Compression": "none",
"KeyEncrypted": true,
"TrashDays": 1,
"MetaVersion": 1
} [format.go:472]
2023/04/13 04:37:56.648638 juicefs[22] : Meta address: redis://<IP_ADDRESS>:6379/1 [interface.go:402]
2023/04/13 04:37:56.655804 juicefs[22] : Ping redis: 855.124µs [redis.go:2878]
2023/04/13 04:37:56.657691 juicefs[22] : Meta redis://<IP_ADDRESS>:6379/1 is not available: lookup subdir demo: read-only file system [interface.go:409]
How can I solve this problem?

Hi @xiashengjie , dataset default accessModes is ReadOnlyMany. You can set accessMode to ReadWriteMany referring to https://github.com/fluid-cloudnative/fluid/blob/master/docs/zh/samples/data_accessmodes.md

How can I fetch juicefs metrics when using juicefsruntime in fluid? Any examples?
@zwwhdls


TypeScript fixes again

This is an updated branch for #8903.  I hate to lose the conversation from there but submitting these changes as a single commit allowed me to significantly reduce the number of files in the patchset -- a lot of them were only related to when.js Promises, and that's no longer relevant.  Hopefully this makes it easier to review and merge.

Thanks for the pull request @thw0rted!

:heavy_check_mark: Signed CLA found.
:grey_question: CHANGES.md was not updated.

If this change updates the public API in any way, please add a bullet point to CHANGES.md.

Reviewers, don't forget to make sure that:

[ ] Cesium Viewer works.
[ ] Works in 2D/CV.

I just noticed that #10265 was called out in the changelog -- turns out that two of the "fixes" I made to the return type info should have been correct and it was the code that was wrong.  It might be worth looking at the function bodies for the other return-type changes I have in this PR to double check that there aren't other similar errors waiting to bite somebody.

Thanks @thw0rted! After the last commit, the build-ts task is failing in CI. Could you please take a look?

Thanks @thw0rted! Latest changes look good. Can you back out the Documentation/ changes since they're no longer needed? Then this is ready to be merged. :+1:

Re the Documentation changes (and the corresponding JSDoc template update), does this mean that the team isn't willing to allow for any function overloading?  I think I had this conversation with Matt on the previous PR, several years ago, but at the time I hadn't figured out a way to avoid having multiple entries in the actual docs -- which is of course pretty confusing.
5+ overloads in one file looks bad, but the previous case was just one, I think so that exportKML gave the correct return type based on the options with which it was called.  That would be a lot less verbose/confusing with the new system.  I guess in the worst case I can always dig up the old commit if we need it again, but it'd be nice to leave it in place as an option.  Your call?

I would say hold off on adding the guide and the tooling until we have a verified use case in the code. It may be the path we end up going down eventually, and we'll have the record of it in this PR. But until we have a use case in the code itself, maintaining the documentation and tolling is questionable.

Awesome, thanks @thw0rted! Thanks for your patience throughout the process.


Add Rector PHP to ease code improvement

Coverage Report

Totals

Statements:
96.46% ( 4002 / 4149 )

Methods:
93.8% ( 711 / 758 )

Lines:
97.05% ( 3291 / 3391 )

Coverage Report

Totals

Statements:
96.54% ( 4068 / 4214 )

Methods:
93.77% ( 708 / 755 )

Lines:
97.14% ( 3360 / 3459 )


cocotb fails trying to import a module called png

You'll need pip install pypng (is there a place to document that requirement?) alternatively, i could take it out or make it conditional or something


[Event Request] Database::"Assemble-to-Order Link" - Function NeedsSynchronization - OnBeforeNeedsSynchronization

We like to request the following Event:
Object: Database::"Assemble-to-Order Link"
Function: NeedsSynchronization
Publisher Event: OnBeforeNeedsSynchronization
Reason: To implement custom business logic
Where is the event placed:
local procedure NeedsSynchronization(SalesLine: Record "Sales Line"): Boolean
var
IsHandled: Boolean;
IsNeedSynchronization: Boolean;
begin
GetAsmHeader;
AsmHeader.CalcFields("Reserved Qty. (Base)");
OnBeforeNeedsSynchronization(AsmHeader, SalesLine, IsNeedSynchronization, IsHandled);
if IsHandled then
exit(IsNeedSynchronization);
Event Code:
[IntegrationEvent(false, false)]
local procedure OnBeforeNeedsSynchronization(AssemblyHeader: Record "Assembly Header"; SalesLine: Record "Sales Line"; var IsNeedSynchronization: Boolean; var IsHandled: Boolean)
begin
end;

Thanks for reporting this. We agree, and we’ll publish a fix asap, either in an update for the current version or in the next major release. We will update this issue with information about availability.


check_subscription_before_vccn

this stops acceptance_store queries if not required.

In summary, the modification stops acceptance_store queries in cases where they are not required, aiming to enhance the overall efficiency and performance of the system by optimizing resource usage. This aligns with a common software development principle of minimizing unnecessary operations for improved speed and resource utilization.


Feature Request

The time based permissions you added are incredibly helpful for the image I have for my server however is it possible to add a similar system for the amount of times a permissions-based command is allowed to be used. For example, allowing players to use /randomtp say only 5 times per day, This would readily apply itself to various needs of prison, games or faction based servers in addition to making donor perks easier to control.

That is not the job of the permissions plugin. This should either be managed by the plugin the command stems from or a specialized plugin.
Also it’s not possible to reliably implement this. As a command permission can be checked without the command being executed!


Positioning pusher with left and top instead of translate

When translate is used in the pusher, position: fixed is not working for the child elements inside of the pusher. Instead of that, using left: @width and top: @height fixes this problem.

Translate3d is necessary for GPU acceleration, top/left are not gpu accelerated.


[web]: Routes have /#/ in them

I'm defining named routes and there's always this /#/ addition, for example:
localhost:8080/#/ and this is pretty bad in my opinion, is there an option of removing it?

I think it's technically possible, but there are reasons why it is the way it is.
Everything after the # doesn't reload the page when updating it so I think it's a good way to maintain state. Updating the history would also be possible.

I think it's technically possible, but there are reasons why it is the way it is.
Everything after the # doesn't reload the page when updating it so I think it's a good way to maintain state. Updating the history would also be possible.

Theres the History API for that. It changes the url but doesn't do a page reload.

That was what I meant

We currently provided only the hash strategy. We should provide support for push/pop state and other strategies, including user-defined strategies. I'll mark this as a new feature.

/cc @mdebbar (perhaps we should expose LocationStrategy to developers)

A HTML5 History strategy would be awesome! ❤
However it does require a little more work for users to setup on static hosting providers and some doesn't support it.
It would also require some extra code in the local web server, so it tries to resolve it as an asset first and if it  is not an asset it let's the strategy handle it.


Nodes outside view port when physics layout enabled

It appears that when using the recently merged physics layout option @jbrekelmans that nodes can end up outside the view port of the canvas.
Using vizceral ver 4.2.1, vizceral-react ver 4.2.0, and vizceral-example commit 0a17500211f12130cffb4b88145ad503ca6169bc (Note: vizceral-example needs a version tick in package.json), and the sample_data.json that is in that same vizceral-example commit.
It seems that the physics based layout is "skipping" a updatePosition call or needs something like the recalculateOrbitSize() call that was added in 386ee4edfa145cb964d47704c24d29213f5bd96f in globalTrafficGraph.js, but has moved to the ringCenterLayout in the latest commit.
Thanks for all the fixes and improvements @jbrekelmans and @jrsquared.  My own work has seen immediate improved performance thanks to your efforts.

First, this issue #44 and issue #77 are one and the same.
Second, I think finally got a line on what the source of the issue is.  It isn't the size of the canvas, it is the field of view of the camera.  Since it is an orthographic camera it doesn't actually have a field-of-view, but it does have a viewing frustum (see THREE.js's docs on OrthographicCamera).
At any rate, I was digging around in vizceral-example trying to figure out why if the details panel was open, that I could drag a node to position that is not visible when the panel closed.  I noticed that the camera's top and bottom frustum values changed when the side panel opened.  This is to preserve the aspect ratio... blah, blah, blah.
If we put a check in somewhere (?? not sure where it should live ??) that checks the positions of nodes against the camera's frustum dimensions we could find and push back into view nodes that have "fallen" off the edge of the frustum.
This leads to another thing I'm not sure how to address in that if we just push things that are beyond the edge of the frustum back into view that could cause things to be overlapping in the layout... so how do would/do you make it check the layout and/or re-layout?
Is calling an update going to work?  Would setting classInvalidated work?  I'm really over my head at this level.  I've been trying to add a new render to create background grid and trying to integrate it with the "update" cycle isn't going well.  I can only image this is even trickier.
Thought?  Suggestions?

@jbrekelmans wrote all the re-layout stuff and it's not something we use interally, so I'm pretty over my head with that part of the codebase as well. Since he hasn't been active or responding to this, I'm going to close this for now since it's not something we will be actively looking at. Happy to review any PRs related to this, or to reopen it if @jbrekelmans can address it.


Add a button to change all colours at once

I agree. When I want just one color, but have to set each of the 8, it's a slow process. Maybe just a button to sync it to the first color. Like

Pick first color
Press button
All colors synced to 1st one

Unfortunately, I'm unable to work on MSIRGB until the summer, but I'll look into this when I can.

Ok! Thanks!
On Tue, Mar 3, 2020 at 12:51 PM Pedro Fanha<EMAIL_ADDRESS>wrote:

Unfortunately, I'm unable to work on MSIRGB until the summer, but I'll
look into this when I can.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/ixjf/MSIRGB/issues/81?email_source=notifications&email_token=AMBOVTOWKQEEDSK6O4Y3JT3RFU7RFA5CNFSM4KQKULV2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOENUPKIQ#issuecomment-594081058,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AMBOVTPMTDGBDKOF5GDB3I3RFU7RFANCNFSM4KQKULVQ
.

--
Best Regards,
PicoPlanet Developing

Added in commit 0c709d2.


Total eligible amount is not showing for NOA (part-time and full-time)

Acceptance Criteria

[x] For the NOA , Eligible for the amount is not showing correctly,
[x] Check all NOA Values to identify the source of truth and ensure they are coming from the database instead of camunda variables
[x] Everything that they are eligible for prior to ecert generation
[x] update test cases

Private Zenhub Image

From #2222 we found that in assessment_data we map childcareCost from calculatedDataChildCareCost. Looks like we must use calculatedDataTotalChildCareCost  instead.
Private Zenhub Image

Question to be answered in #2331

Requested Hugh to create a placeholder ticket #2381 for the reviewing the assessment and awards for both the Full time and Part time applications

https://teams.microsoft.com/l/message/19:meeting_MWY2MWExZjYtODMyZi00YmIxLWI1ZWMtOTFjOWY0M2M5ZTMz@thread.v2/1696957400063?context={"contextType"%3A"chat"}


chore: update the yarn.lock file

This should resolve the outstanding security alerts - although I don't think any of the security alerts actually affect this repo anyway.

:tada: This PR is included in version 3.3.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Upload fails with datetime64[ns]

pd-to-mssql recently started failing on me and giving me the following error:
There are more columns in the INSERT statement than values specified in the VALUES clause.
When I checked the debugging script I saw that the date values were missing from the insert statement. After some testing, I found that if I convert dates to strings first, the upload is successful.
I believe the issue comes down to this if statement, which never considers the type datetime64[ns]:
https://github.com/veyron8800/pd_to_mssql/blob/0b0f03103ceaadee6fc5576b2bb5afc5eb47e559/pd_to_mssql/pd_to_mssql.py#L62

Hello @galexioza
The type names listed on that if statement are for the tsql data types, not the python ones. I'm confused by the fact that your solution works, as all columns are stringified before the upload, so converting it explicitly before calling to_sql shouldn't change anything. Could you verify that the data type of the column in the table you are uploading to is datetime2. If it is the old datetime then that could be causing this issue, as the package does not support it.
If that is not the issue, then I would need some more info to figure out the problem. If you could share code and example data, that would be great. If the data and code in question are sensitive in nature, then a minimum example with fake data would be much appreciated.
Thanks

Since pd-to-mssql can't create tables, I am using pandas to_sql method first to create the table with one row, then truncating it, then uploading the rest with pd-to-mssql.
Converting to a string works because it sets the column on the server to varchar. So that isn't really what I want.
The datatype is 'datetime', which must be the default set by pandas. Any idea if there is a way to specify this type in pandas? If not, then I will just create the table first with a text SQL statement.

Manually creating a table with datetime2 instead of relying on pandas worked, so I'll close the issue as it's my problem to work around.
Thanks for your help and surprisingly quick response.

Hello @galexioza
The documentation for Pandas' to_sql function suggests that it is possible for you to specify the datatypes of the columns in the resulting sql table through the dtype parameter. However, when I use this package, I always just create the table using a tsql statement like you did with your workaround.
Thank you for using the package!


fix: Fix the BUG of incorrectly clearing whitespace characters in Mak…

…efile
Description

Related Issue

Type of Change

[ ] 📚 Examples / docs / tutorials / dependencies update
[ ] 🔧 Bug fix (non-breaking change which fixes an issue)
[ ] 🥂 Improvement (non-breaking change which improves an existing feature)
[ ] 🚀 New feature (non-breaking change which adds functionality)
[ ] 💥 Breaking change (fix or feature that would cause existing functionality to change)
[ ] 🔐 Security fix

Checklist

[ ] I've read the CODE_OF_CONDUCT.md document.
[ ] I've read the CONTRIBUTING.md guide.
[ ] I've updated the code style using make codestyle.
[ ] I've written tests for all new methods and classes that I created.
[ ] I've written the docstring in Google format for all the methods and classes that I used.

Thanks!


First step towards a zero-rotating marker/popup/tooltip pane

Note: Work in Progress
In reference to the comment at https://github.com/fnicollet/Leaflet/issues/8#issuecomment-249875366, that the markers, popups and tooltips should not be rotated back and forth, but instead be rotated zero times, I started this branch to try to accomplish that.
I split the map panes in two groups, leaving the tilePane and the overlayPane within the rotatePane, but creating a new norotatePane for the three other panes.
I adapted a bit of the positioning logic, but I am now stuck.
What works:
Panning, zooming, initial rotating
What does not work:
Rotating, then panning, then rotating (or zooming). The origin of the second rotate/zooming action is wrong for the elements in the no-rotate pane. I don't know how to fix it currently and am hoping for the input of experienced Leaflet devs here ;-) I'm guessing I need to reset the origin somewhere, somehow...?
You can try out the changes and problems at
https://rawgit.com/sisou/Leaflet/zero-rotate/debug/rotate/rotate.html
https://rawgit.com/sisou/Leaflet/zero-rotate/debug/rotate/rotate-and-drag.html

Hello @sisou and thanks for that PR!
Isolating the markers / popups to a non-rotating pan is a good idea, instead of counter-rotating the objects.
The fact that it is off when rotating then paning then rotating probably comes from the mapPivot calculation, which will need to take into account the pane container offset or something. I lost a few nights over issues like this and I am not sure how to fix it properly, but there is a way
i'll try your branch when I have a little time!
Fabien

Progress
I managed to make the markers and divOverlays behave nicely.
Next up: zooming positioning of divOverlays after a rotated panning... -.-

Progress
Animated zooming of popups and tooltips (I hope, did not test tooltips) works.
Next up:

Ups, wrong button :D

Completed
Hi Fabien,
I think I finished this! I tested everything (also with rotation off) and it works great. Please have a look through and see if you find any problems =)

This PR has been superseeded by #19.


Hybrid (SPA and non SPA) page communications

Before my application was written in jQuery and wasn't SPA. I have rewritten some of the pages to AngularJS and made them SPA.
Now I need to have proper comunication beetwen these apps. Because when I go from Angular page to Angular page, I don't want to reload the page, but when I go to jQuery page, I do want to reload the page so server could send me other source files.
So, when the link goes to the Angular page (from Angular page), it's simple:

And ui-router catches "store/categories", changes the state and it doesn't reload the page.
However if the link goes to jQuery page (from Angular page), I can't just specify , because ui-router will catch it and the page won't reload.
Then I decided to use window.location.href to force page reload:

$urlRouterProvider.otherwise(function($injector, $location){
  window.location.href= window.location.origin + $location.url();
});

And it worked! When you now click on the link it will reload the page with the correct URL.
However if you go to a page using window.location.href and then click the browser back button, the URL will change but the page won't reload.

Use target="_self" instead of window.location.href


ovirt-web-ui 1.6.1-1

dropping compatibility with 4.3, adding brand new New VM dialog, moving to PF4...doesn't it deserve a 1.7 at least?

dropping compatibility with 4.3, adding brand new New VM dialog, moving to PF4...doesn't it deserve a 1.7 at least?

@michalskrivanek Based on Zanata project version #1162, it should stay as 1.6. We can change that on next build

ci build please


jsd loss

Hi,
many thanks for making your code open source!
I'm currently experimenting with it and have integrated it into the timm training pipeline.
I saw you have a flag for the jsd loss. Did you use this for the results in the paper? I assume no, because it wasn't mentioned and it would make training quite a bit slower.
But since the option is there, did you try it? Did it help?

Hi,
Thank you for your interest in our work!
Regarding your question about the JSD loss, yes, we did use it for the results reported in our paper. Although it was not explicitly highlighted, the use of JSD loss was instrumental in achieving the enhanced performance metrics we presented. We found that JSD loss significantly improves accuracy and other safety metrics, despite indeed prolonging the training time. We acknowledge this oversight and plan to update the paper to include a mention of JSD loss at an appropriate place.
We included the JSD loss flag in our codebase to offer users the option to explore its benefits, acknowledging that the extended training time might be a worthy trade-off for the gains in model performance and robustness.
Best wishes,
Zhenglin

Thanks Zhenglin!
I can confirm I also get better results when using JSD loss. I am a bit surprised that PixMix, which was follow-up work on AugMix doesn't use it.

Hi,
Thank you for your reply!
I agree with your points. In fact, it's mentioned in PixMix that " the Jensen-Shannon Divergence consistency loss, which requires at least thrice the memory per batch."  I think this may be the reason why authors don't use it.

Besides, in my view, perhaps the reason is that the method of generating images in PixMix is simpler, as it involves merging the same image with a fractal image or an enhanced version of the original image within a single pipeline. This method does not require the use of JSD loss to maintain semantic consistency in the generated images. I also suspect that using JSD loss might not yield better results for PixMix, whereas the methods used in AugMix and IPmix are more complex. IPmix, in particular, involves merging images across multiple levels and uses more complex mixing methods (e.g., random mixing), hence the need for JSD loss to maintain consistency in predictions.
If you are interested in this topic, you might consider trying out JSD loss with PixMix to observe its effects.
If you have more questions, feel free to contact me.
Best wishes,
Zhenglin


Can't update to 0.70.

Microsoft PowerToys version
0.70
Installation method
GitHub
Running as admin
Yes
Area(s) with issue?
Installer
Steps to reproduce
I tried to install.
[4298:43E4][2023-05-25T10:11:04]i001: Burn v<IP_ADDRESS>26, Windows v10.0 (Build 22621: Service Pack 0), path: C:\WINDOWS\Temp{354F84A1-0D93-420D-B0C7-B78DA3B2C12C}.cr\PowerToysUserSetup-0.70.0-x64.exe
[4298:43E4][2023-05-25T10:11:04]i000: Initializing string variable 'InstallFolder' to value '[LocalAppDataFolder]PowerToys'
[4298:43E4][2023-05-25T10:11:04]i000: Initializing string variable 'MsiLogFolder' to value '[LocalAppDataFolder]\Microsoft\PowerToys'
[4298:43E4][2023-05-25T10:11:04]i000: Initializing version variable 'MinimumVersion' to value '<IP_ADDRESS>'
[4298:43E4][2023-05-25T10:11:04]i000: Initializing version variable 'TargetPowerToysVersion' to value '0.70.0'
[4298:43E4][2023-05-25T10:11:04]i000: Initializing version variable 'DetectedPowerToysVersion' to value '<IP_ADDRESS>'
[4298:43E4][2023-05-25T10:11:04]i000: Initializing version variable 'DetectedPowerToysUserVersion' to value '<IP_ADDRESS>'
[4298:43E4][2023-05-25T10:11:04]i000: Initializing version variable 'DetectedWindowsBuildNumber' to value '0'
[4298:43E4][2023-05-25T10:11:04]i009: Command Line: '-burn.clean.room=G:\download\PowerToysUserSetup-0.70.0-x64.exe -burn.filehandle.attached=604 -burn.filehandle.self=760'
[4298:43E4][2023-05-25T10:11:04]i000: Setting string variable 'WixBundleOriginalSource' to value 'G:\download\PowerToysUserSetup-0.70.0-x64.exe'
[4298:43E4][2023-05-25T10:11:04]i000: Setting string variable 'WixBundleOriginalSourceFolder' to value 'G:\download'
[4298:43E4][2023-05-25T10:11:04]i000: Setting string variable 'WixBundleLog' to value 'C:\temp\powertoys-bootstrapper-msi-0.70.0_20230525101104.log'
[4298:43E4][2023-05-25T10:11:04]i000: Setting string variable 'WixBundleName' to value 'PowerToys (Preview) x64'
[4298:43E4][2023-05-25T10:11:04]i000: Setting string variable 'WixBundleManufacturer' to value 'Microsoft Corporation'
[4298:2A30][2023-05-25T10:11:04]i000: Setting numeric variable 'WixStdBALanguageId' to value 1033
[4298:2A30][2023-05-25T10:11:04]i000: Setting version variable 'WixBundleFileVersion' to value '<IP_ADDRESS>'
[4298:43E4][2023-05-25T10:11:05]i100: Detect begin, 3 packages
[4298:43E4][2023-05-25T10:11:05]i000: Product or related product not found: {D8B559DB-4C98-487A-A33F-50A8EEE42726}
[4298:43E4][2023-05-25T10:11:05]i000: Setting version variable 'DetectedPowerToysUserVersion' to value '<IP_ADDRESS>'
[4298:43E4][2023-05-25T10:11:05]i000: Setting version variable 'DetectedPowerToysVersion' to value '<IP_ADDRESS>'
[4298:43E4][2023-05-25T10:11:05]i000: Setting string variable 'DetectedWindowsBuildNumber' to value '22621'
[4298:43E4][2023-05-25T10:11:05]i000: Registry key not found. Key = 'Software\Microsoft\EdgeUpdate\Clients{F3017226-FE2A-4295-8BDF-00C3A9A7E4C5}'
[4298:43E4][2023-05-25T10:11:05]i000: Setting numeric variable 'HasWebView2PerUser' to value 0
[4298:43E4][2023-05-25T10:11:05]i000: Setting numeric variable 'HasWebView2PerMachine' to value 1
[4298:43E4][2023-05-25T10:11:05]i102: Detected related bundle: {f23d8111-a4c9-401f-9456-171fd17aac28}, type: Upgrade, scope: PerMachine, version: <IP_ADDRESS>, operation: MajorUpgrade
[4298:43E4][2023-05-25T10:11:05]i052: Condition 'HasWebView2PerMachine OR HasWebView2PerUser' evaluates to true.
[4298:43E4][2023-05-25T10:11:05]i101: Detected package: TerminatePowerToys, state: Absent, cached: None
[4298:43E4][2023-05-25T10:11:05]i101: Detected package: WebView2, state: Present, cached: None
[4298:43E4][2023-05-25T10:11:05]i101: Detected package: PowerToysUserSetup_0.70.0_x64.msi, state: Absent, cached: None
[4298:43E4][2023-05-25T10:11:05]i052: Condition 'MinimumVersion >= DetectedPowerToysVersion' evaluates to false.
[4298:43E4][2023-05-25T10:11:05]e000: PowerToys is already installed on this system for all users. We recommend first uninstalling that version before installing this one.
[4298:43E4][2023-05-25T10:11:05]e000: Error 0x81f40001: Bundle condition evaluated to false: MinimumVersion >= DetectedPowerToysVersion
[4298:43E4][2023-05-25T10:11:05]i199: Detect complete, result: 0x0
✔ Expected Behavior
No response
❌ Actual Behavior
No response
Other Software
No response

It seems like you're trying to install a user installation of PowerToys on a machine that already has a machine installation.

PowerToys is already installed on this system for all users. We recommend first uninstalling that version before installing this one.

Please uninstall machine-wide PowerToys first or try to install 0.70.0 with the machine installer instead of the user installer.
Does this help? /needinfo

I downloaded good installer and update was fine. Thank you very much


TimePickerPreference style error

@Gericop TimePickerPreference button style error.

Style:
    <style name="AppTheme" parent="Theme.MaterialComponents.Light.NoActionBar">
        <item name="colorPrimary">@color/colorPrimary</item>
        <item name="colorPrimaryDark">@color/colorPrimaryDark</item>
        <item name="colorSecondary">@color/colorSecondary</item>
    </style>

@Gericop Use AppCompatButton Instead of Button will fixed Timepicker Button Style Problems. But Datapicker also has this problem, please check.

This will be fixed by the date picker lib v1.0.7-alpha3 (actually alpha1 fixes it but this problem is non-existent in the support lib v28.0.0-alpha1).

Thanks for fixing, Sync “preference-v7-extras” please.

v<IP_ADDRESS>-alpha3 is not released yet.

@Gericop "preference-v7-extras" version updated to "v<IP_ADDRESS>-alpha3" will sync date picker lib v1.0.7-alpha3？

Yes.


Example using AoT compilation with ngc

Not sure if it makes sense to have two different build configurations and bootstrap.ts files, but it would be nice to have an example of the setup for the new Ahead-of-Time compilation in RC5.

https://github.com/AngularClass/angular2-seed/issues/11


can you change game name or tag using api?

There is an api to do it but it's on the website so using it likely requires cloudflare bypasses. If you want to look more into it, I'd recommend changing your name on an alt account or something and recording the form submission data using the browser network tools.
If I remember correctly, it requires a csrf token obtained from an authenticated request to the account settings page.


How to do RAG + Conversational Memory?

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code
from langchain_core.runnables import RunnablePassthrough
def format_docs(docs):
return "\n\n".join(doc.page_content for doc in docs)
rag_chain = (
{"context": retriever | format_docs, "input": RunnablePassthrough()}
| custom_rag_prompt
| llm
| output_parser
)
response = rag_chain.invoke("How LCNC is disrupting the development market ? Is it for real ? ")
This has a custom prompt template , vector store and rag, but I need to add a conversation buffer to it, is that possible?
Error Message and Stack Trace (if applicable)
No response
Description
I am currently working in RAG + Vectorstore + Langchain . In this method I need to add conversational memory, which will help me to answer with the context of the previous response. Is there any method to do it?
System Info
Linux
Langchain

I am working on the RAG+SQL+Langchain combination.
Its very similar to what you are doing here.
This is my SQL generation prompt:
    template = """
    You are a helpful and clever data analyst at a company. You are interacting with a user who is asking you questions about the company's database.
    Based on the table schema below, write a SQL query that would answer the user's question. Take the conversation history into account.

    <SCHEMA>{schema}</SCHEMA>

    Conversation History: {chat_history}

    Write only the SQL query and nothing else. Do not wrap the SQL query in any other text, not even backticks.

    For example:
    Question: which 3 artists have the most tracks?
     You are interacting with a user who is asking you questions about the company's database.
    Based on the table schema below, question, sql query, and sql response, write a natural language response.strictly use the underlying database for the answers.If answer is not from the database..then say i dont know the answer. If you don't know the answer, just say that you don't know, don't try to make up an answer.dont mention 'sql' in the answer.

    <SCHEMA>{schema}</SCHEMA>

    Conversation History: {chat_history}
    SQL Query: <SQL>{query}</SQL>
    User question: {question}
    SQL Response: {response}"""

Prompt generation and chain creation
   prompt = ChatPromptTemplate.from_template(template)

   chain = (
    RunnablePassthrough.assign(query=sql_chain).assign(
        schema=lambda _: db.get_table_info(),
        response=lambda vars: print(f"\n\nExecuting SQL query: {vars['query']}") or db.run(vars["query"]) if
  is_runnable_query(vars["query"],db) else "Sorry, i cant get this data right now",
    )
    | prompt
    | llm
    | StrOutputParser()
)

Last step:
return chain.invoke({
    "question": user_query,
    "chat_history": chat_history,
})

Can you please share some more info about your application?
Give me more info like what vectorstore you are using (chroma or FAISS or pinecone ?)  So that i can go into specific details

Thanks swapnil for your reply
I am using qdrant
`
from langchain_community.document_loaders.csv_loader import CSVLoader
from langchain.text_splitter import RecursiveCharacterTextSplitter
from langchain_openai import  OpenAIEmbeddings
from langchain_community.vectorstores import Qdrant
from langchain import hub
from langchain_openai import ChatOpenAI
import qdrant_client
loader = CSVLoader("/content/Data/fivehundreddatafortesting.csv",encoding="latin-1")
docs = loader.load()
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=1000,
chunk_overlap=200,
)
splits = text_splitter.split_documents(docs)
url = "Your qdrant url"
#api_key = "your api key"
vectorstore = Qdrant.from_documents(
documents=docs,
embedding=OpenAIEmbeddings(),
url=url,
prefer_grpc=True,
api_key=api_key,
collection_name="final_chunk",
)
llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)
    # Initialize QdrantClient

client = qdrant_client.QdrantClient(
"Your qdrant url,
api_key="your qdrant api key",
)
    # Initialize Qdrant Vector Store

vectorstore = Qdrant(
client=client,
collection_name="equal_chunk",
embeddings=OpenAIEmbeddings(),
)
    # Initialize retriever

retriever = vectorstore.as_retriever()
`
In the above code,  i have initialized an existing collection , but also i have attached the code to create a collection

from langchain_core.prompts import (
    ChatPromptTemplate,
    FewShotChatMessagePromptTemplate
)

examples = [
   #Your examples for few shot prompt goes here
]

example_prompt = ChatPromptTemplate.from_messages(
    [
        ("human", "{input}"),
        ("ai", "{output}"),
    ]
)

few_shot_prompt = FewShotChatMessagePromptTemplate(
    example_prompt=example_prompt,
    examples=examples,
)

I have added few shot prompting technique, i have a doubt as this is working or not , please provide  insights on this too.

from langchain.output_parsers import CommaSeparatedListOutputParser
from langchain_openai import ChatOpenAI

custom_rag_prompt = ChatPromptTemplate.from_messages(
            [
                ("system", """
          Your system prompt here
    """),
                few_shot_prompt,
                ("human", "{input}"),
            ]
        )

print(custom_rag_prompt)

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

output_parser = CommaSeparatedListOutputParser()

from langchain_core.runnables import RunnablePassthrough
def format_docs(docs):
    return "\n\n".join(doc.page_content for doc in docs)

rag_chain = (
    {"context": retriever | format_docs, "input": RunnablePassthrough()}
    | custom_rag_prompt
    | llm
    | output_parser
)

response = rag_chain.invoke("Question here")

For this code , I need to make this particular thing as conversational can you please provide insights on this


Generated SDK files for LTR backup

Hi @hchungmsft, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes. I promise there's no faxing. https://cla.azure.com.

TTYL, AZPRBOT;


fix(smt2): except_t & test on windows

Hello all.
I updated files so they correctly use newly updated except_t.
I checked that it runs well by running *.lean files in test directory. (To test them on Windows (msys), I updated parse_smt2_result so it ignores trailing \r characters as well.)

Thanks.


Move controllers paierd using #634 but after shut down of pc not connecting.

The controllers are visible in settings in devices tab, but when I press the ps button won't connect. I have changed the psmoveserviceadmin.exe file to the original after pairing using the one from #634. Please help asap.

I have linked both .log files in #677 issue.


Add support to filter instances by price range, and expose the prices/spot prices in output

It would be great if the support is added to filter out instances based on live price range. Also, to show the price of the instances (both spot and on-demand) in the output.

I think price column in output could be very handy. We can get this info from pricing api from aws
Cause, even with same specs some instances can be twice more expensive

This has been merged and can be built from source for testing.  I'm going to hold off on doing an official release because we'll have to increment the major version to v3 because of some lib changes that were required to support the pricing retrieval changes.

Thanks a lot! @bwagner5

Would love to hear feedback here!

A few things. First the nit picks:
1. In --help text, mention that accepted price is in USD.
Unexpected behaviour:
1. Can't get On-Demand pricing for us-west-1, us-west-2, us-east-2 upon multiple retries. Interestingly, us-west-1 works fine all the time. My credentials have admin privileges, so it is not an IAM issue.
Krishna-Birla:amazon-ec2-instance-selector krishna.birla$ ./build/ec2-instance-selector --region us-east-2 --max-results 5 -o table-wide --price-per-hour-max 0.2
Could not retrieve hourly price for instance type r4.16xlarge
Could not retrieve hourly price for instance type m5a.16xlarge
Could not retrieve hourly price for instance type t2.2xlarge
Could not retrieve hourly price for instance type m5d.8xlarge
Could not retrieve hourly price for instance type m6g.metal
Could not retrieve hourly price for instance type m6gd.medium
Could not retrieve hourly price for instance type r5dn.8xlarge
Could not retrieve hourly price for instance type c6gn.4xlarge
...

Krishna-Birla:amazon-ec2-instance-selector krishna.birla$ ./build/ec2-instance-selector --region us-west-1 --max-results 5 -o table-wide --price-per-hour-max 0.2
Could not retrieve hourly price for instance type c5a.xlarge
Could not retrieve hourly price for instance type g4dn.12xlarge
Could not retrieve hourly price for instance type r5ad.12xlarge
Could not retrieve hourly price for instance type m5.xlarge
Could not retrieve hourly price for instance type c4.large
^C

Krishna-Birla:amazon-ec2-instance-selector krishna.birla$ ./build/ec2-instance-selector --region us-west-2 --max-results 5 -o table-wide --price-per-hour-max 0.2
Could not retrieve hourly price for instance type c6g.xlarge
Could not retrieve hourly price for instance type m5dn.12xlarge
Could not retrieve hourly price for instance type r6g.medium
Could not retrieve hourly price for instance type r6g.8xlarge
Could not retrieve hourly price for instance type c4.4xlarge
Could not retrieve hourly price for instance type c6gn.xlarge
Could not retrieve hourly price for instance type i3en.24xlarge
Could not retrieve hourly price for instance type c6g.12xlarge
Could not retrieve hourly price for instance type m5d.16xlarge
Could not retrieve hourly price for instance type c3.8xlarge
Could not retrieve hourly price for instance type r5n.12xlarge
Could not retrieve hourly price for instance type inf1.xlarge
Could not retrieve hourly price for instance type c5n.large
Could not retrieve hourly price for instance type m5a.16xlarge
...

Krishna-Birla:amazon-ec2-instance-selector krishna.birla$ ./build/ec2-instance-selector --region us-east-1 --max-results 5 -o table-wide --price-per-hour-max 0.2
Instance Type  VCPUs   Mem (GiB)  Hypervisor  Current Gen  Hibernation Support  CPU Arch      Network Performance  ENIs    GPUs    GPU Mem (GiB)  GPU Info  On-Demand Price/Hr
-------------  -----   ---------  ----------  -----------  -------------------  --------      -------------------  ----    ----    -------------  --------  ------------------
a1.large       2       4          nitro       false        false                arm64         Up to 10 Gigabit     3       0       0                        $0.051
a1.medium      1       2          nitro       false        false                arm64         Up to 10 Gigabit     2       0       0                        $0.0255
a1.xlarge      4       8          nitro       false        false                arm64         Up to 10 Gigabit     4       0       0                        $0.102
c1.medium      2       1.69922    xen         false        false                i386, x86_64  Moderate             2       0       0                        $0.13
c3.large       2       3.75       xen         false        true                 i386, x86_64  Moderate             3       0       0                        $0.105
NOTE: 81 entries were truncated, increase --max-results to see more

Suggestions
1. Display On-Demand/Spot pricing for all table-wide outputs
In current code, we only hydrate the cache if --price-per-hour filter is set. Can we change this behaviour, and use --usage-class instead to hydrate the respective cache. I know this comes at the expense of startup time for hydrating the cache, but there is a strong use case here:
For any AWS user, generally price is the "deciding" factor but not the "limiting" factor. The most common use case IMO is: I have a list of EC2 instances as per my vCPU, memory, ENI, I/O and other hardware requirements; now I have a list of instances I can use and I want to choose the cheapest one. For that I need the prices listed.
The price filter is for people who already have some idea about how AWS prices their instances.

Thanks so much for taking the time to try it out and write this feedback!
I'll investigate the regions not working, thanks for bringing that up.
I'm fine with doing price retrieval when --usage-class is specified too. Just to clarify, you mean retrieve pricing on usage-class and price-per-hour filter right (not just on usage-class) ? Feel free to raise a PR for that!

I'm fine with doing price retrieval when --usage-class is specified too. Just to clarify, you mean retrieve pricing on usage-class and price-per-hour filter right (not just on usage-class) ? Feel free to raise a PR for that!

I am suggesting to retrieve price in all cases of table wide output. (On-demand price for usageClass == nil || usageClass == on-demand and Spot price for usageClass == spot)
Current code (only hydrate if price per hour is given):
if pricePerHour != nil {
	if usageClass == nil || usageClass == on-demand {
		// Hydrate on demand cache
	} else {
		// Hydrate spot cache
	}
}

My suggestion (always hydrate):
if usageClass == nil || usageClass == on-demand {
	// Hydrate on demand cache
} else {
	// Hydrate spot cache
}

Just like we list hypervisor, current gen, gpu, ENI, etc. we should always list price. Is that okay?

I see, I'm not opposed to figuring out a way that we could enable it all the time for table-wide output, but as it stands right now, hydrating the cache will still not expose it on the table. Currently the outputs know nothing about how the filtering works. Hydrating the cache will speed up price retrieval when calling the selector, but the selector will only retrieve pricing data if a pricing filter is present. So basically the change you're suggesting would not be sufficient to show the price on the table-wide view and if we make a further change in selector to fetch pricing data on every filter call, then it would slow down for all filters not just ones that are displayed w/ table-wide.
https://github.com/aws/amazon-ec2-instance-selector/blob/main/pkg/selector/selector.go#L189-L207
We may be able to refactor a bit to check if the cache is already hydrated and if it is, then go ahead and add pricing data... It feels little dirty to do that since we're inferring information about which output we're using to do further filter work (even if it's just attaching the data to the instanceType struct).

but as it stands right now, hydrating the cache will still not expose it on the table.

Yes, ofc I meant not just this if condition but other relevant changes overall to make it possible. If you are okay with the startup time of hydration, I can add that in table-wide, make whatever change is needed. Should I start development then?

So basically the change you're suggesting would not be sufficient to show the price on the table-wide view and if we make a further change in selector to fetch pricing data on every filter call, then it would slow down for all filters not just ones that are displayed w/ table-wide.

I am sure we can prevent this, so that if a user does not want table wide output, we do not hydrate. I can try.

Sure! Would love for you to help on this! I agree, I think it's solvable.  Looking forward to seeing what you come up with :)

I haven't had time to investigate the other regions not pulling pricing data but I will check on that probably tomorrow.

@bwagner5 I figured out why other regions are not working. https://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/using-pelong.html#pe-endpoint
I could not find a good alternate, may have to dig deeper.
Except this region issue, I have given a PR for the other things we discussed.

This has been released for a while just forgot to close :D


can't find some entry in polkadotjs portal

Just follow the workshop step by step, deploy a two node (polkadot's bkchr-cumulus-branch branch) local testnet with Alice & Bob, build the polkadotjs (master branch & most updated code) locally and start the frontend server, visit this server by:
http://localhost:3000/#/?rpc=ws://IP_ADDRESS_OF_THE_SERVER:9944

And I found there are some entry missing in the protal, such like: chain states & extrinsics. Is there any version compatible issue between the node backend & frontend?

Go back to the settings tab, and ensure you've selected "Fully featured" for interface operation mode.
 Click save, and (if necessary) refresh the page.
I'm 99% sure that's your issue, but if you still don't get the tabs to show up, please try the endpoint in that screenshot.
If you continue to have problems, please re-open. I'm happy to help.


Support dm-multipath, update storage msg

This is for https://github.com/rackerlabs/getData/issues/5. It also moves the msg covering storage above the vgcfgbackup so if that is slow/fails its clear from the console what is happening. Finally, the san_packages output is removed since its very specific to a single set of components.

All looks good.


File Explorer window size and position gets reverted after using FancyZones

Microsoft PowerToys version
"0.49.1"
Running as admin

[X] Yes

Area(s) with issue?
FancyZones
Steps to reproduce

Open File Explorer window
Move File Explorer window to a FancyZone
Close File Explorer window
Open File Explorer window again (window size and position is now reverted to before FancyZone was used)

✔ Expected Behavior
File Explorer window position and size to be remembered when opening new window after using FancyZone.
❌ Actual Behavior
File Explorer window position and size gets reverted back to before FancyZone was used.
Other Software
No response

Duplicate of #12211

Thanks for reporting this. This issue is a duplicate.
I'm linking this issue to #12211 to centralize the discussion thread for tracking.
Thanks for your help to make PowerToys a better piece of software!


Determine containerised database migrations approach

https://pythonspeed.com/articles/schema-migrations-server-startup/
https://engineering.instawork.com/elegant-database-migrations-on-ecs-74f3487da99f
Database migrations are the primary use case for this but it may capture other things, such as configuration changes, so it should be thought of more as dependency changes between versions than just database migrations.

All dependency changes are backwards compatible by one version, database changes are additive -- deprecated database configuration is cleaned up in later deploys
Application image has an alternative "upgrade" entrypoint which is triggered explicitly by passing in a flag at runtime -- by default the entrypoint is nginx and the upgrade (migrate + any other tasks) can only happen explicitly
Container healthcheck determines if it is compatible with all dependencies

e.g, database: if the application version (baked into the image) is greater than the database version (stored in the database) then the healthcheck fails
The reason to check for greater than rather than matching is because we need the old application versions to be able to continue to start, even if the database has been upgraded, because it's possible the new application versions won't start and the old will need to continue to run

Deployment process is responsible for triggering a single instance of upgrade and pushing a new version of the application to each running instance: healthchecks will fail for the new versions of the application until the migration is finished

optionally, in a strict environment (where any failing healthcheck during a deployment is an alarm) the upgrade task could be waited on before triggering the new application version deployment -- this would create a dependency between deployment command order but that shouldn't be an issue

A dependency has a version: an application has a list of dependencies and the minimum versions which must be met for the healthcheck to pass.
A Database has a Schema Version.

This is now supported with the addition of Conductor which provides a Schema Version check. Last outstanding step is to document the implementation.


Very slow for inputs like "a" * 100000

I've noticed that Tiktoken is really slow for strings of repeated characters like "a" * 100_000. Interestingly, when you add spaces, like "a " * 50_000, the performance is orders of magnitude better:

Is this a bug or a fundamental property of BPE?
My Tiktoken version is 0.5.0.

It's a little bit of both. The first step we do in tokenising is regex splitting, where we split on things like whitespace. The ensuing chunks are then fed to BPE. When tokenising natural language, these chunks usually end up very small.
BPE is superlinear in chunk length, so what you're seeing is approximately expected. But the implementation in tiktoken is optimised for short chunk lengths where constant factors dominate, so time complexity as chunk length scales is worse than what it could be. Thus far, this hasn't mattered in practice, so I haven't bothered branching the logic. Do you have a use case, or are you just curious?

Thanks! The use case is that we're building an API that accepts text produced/consumed by GPT and we limit the length of the input to a certain number of tokens. Checking if the text is within the limit requires tokenizing it, and if tokenizing certain adversarial long strings takes a long time, this would be a way to DoS the server.

Isn't this also an issue that OpenAI itself has to deal with?

Thanks, that's a reasonable use case. Every token is at least one byte, so for now I'd recommend running a quick length check against your input first. At typical maximum context lengths, this should provide a reasonable guarantee.
I work on the research side of things, but OpenAI is probably pretty happy for you to pay us for these tokens since CPU time is basically free. We have other mitigations against denial of service as well.

Every token is at least one byte, so for now I'd recommend running a quick length check against your input first.

But I guess the maximum token length is quite long, which is what I'd need to make statements like "for inputs longer than X bytes, they definitely have more than Y tokens so I can reject them"
The way we worked around this this in the meanwhile is that for extremely long prompts we require a certain number of spaces + other mitigations against DoS as you mention.

If you want a more precise (but not perfectly exact) way to check, you can do something like this, which parses progressively larger portions of the input. This is much more precise than bounding the length of the input, and in practice tends to be reasonably fast (tiktoken parsing of small strings is insanely fast, so if your input is only a few thousand code points then the code below should take low single-digit milliseconds).
@dataclass
class TokenCheck:
    verification_passed: bool
    num_tokens: int

def get_number_tokens(message: str, limit: int, processing_size=500) -> TokenCheck:
    for end_index in range(0, len(message), processing_size):
        if len(encoding.encode(message[:end_index], disallowed_special=())) > limit:
            return TokenCheck(verification_passed=False, num_tokens=limit * 2)
    num_tokens = len(encoding.encode(message, disallowed_special=()))
    if num_tokens > limit:
        return TokenCheck(verification_passed=False, num_tokens=limit * 2)
    else:
        return TokenCheck(verification_passed=True, num_tokens=num_tokens)

For what it's worth, I just ran into a bug with tiktoken taking crazy amount of time which broke some code I was working on trying to do data analysis. Here's a plot I generated of runtime across token length:

The ultimate problem was I was trying to encode a dataset of output generated by GPT-J with tiktoken, and the model just fell into one of its loops where it just wrote out a bunch of whitespace over and over. And this caused nearly 100,000 whitespace tokens to be tokenized back-to-back which just stalled my tiktoken encoding forever.
I think it would probably be a good idea to implement a fix for this. While tiktoken was probably designed for encoding GPT-4 output, it's not the only thing people use it for. And having superlinear runtimes can cause other people a lot of pain in ways they wouldn't expect.

@qxcv Thanks! Looks like a good approximate workaround. Btw somebody sent Tensor Trust to me a few weeks ago and I really like the idea, although I haven't had time to play around with it yet. (We made https://gandalf.lakera.ai/, so we're working in the same space 🙂)

This can be solved by a different rank merge algorithm - I've fixed it in https://github.com/knuddelsgmbh/jtokkit (new version not released yet), might also contribute it back to tiktoken if it's an important usecase.
The blue is the current array based algo (doing linear minimum search) and the red is the new AVL tree based:

I pushed a PR here as well to tackle this exact scenario, see: https://github.com/openai/tiktoken/pull/239


submission deadline moved to Sept 20.

WG: no one objects so far.

we already announced on the mailing list.


Revert "Add aria-selected value to ActionList.Item."

This reverts commit 242c634.
There was some disagreement about how ActionList items should announce if they're selected or not. There's a functional difference between an item being "selected" (the user is currently focused on it) and "checked" (the option was previously chosen and is active). We want to communicate the former, but the aria-selected attribute is the incorrect way to do this.
This should not impact anything major in dotcom, as both changes were internal to the component, and did not require any changes by consumers.

@joshblack could you please give this a review when you have a moment and, if all looks good to you and tests pass, merge on behalf of @radglob? Thank you 🙏🏻


Step reports format for scenario outlines with examples

Compared with other cucumber frameworks the reporting is slightly different:
It would be more intuitive if the report should report the actual used values in the test, this is the case e.g. for all Scenario outlines with examples.
Taking the report from examples/2_scenario_outline.feature
Scenario Outline: First Scenario Outline  examples/features/2_scenario_outline.feature:5
[   PASSED    ] Given An empty box  examples/features/2_scenario_outline.feature:6
[   PASSED    ] When I place <count> x <item> in it  examples/features/2_scenario_outline.feature:7
[   PASSED    ] Then The box contains <count> item(s)  examples/features/2_scenario_outline.feature:8
  With Examples:
  | count | item    |
  | 1     | apple   |

The output would be more intuitive with
Scenario Outline: First Scenario Outline  examples/features/2_scenario_outline.feature:5
[   PASSED    ] Given An empty box  examples/features/2_scenario_outline.feature:6
[   PASSED    ] When I place 1 x "apple" in it  examples/features/2_scenario_outline.feature:7
[   PASSED    ] Then The box contains 1 item(s)  examples/features/2_scenario_outline.feature:8

done ✅ https://github.com/ThoSe1990/cwt-cucumber/pull/42


removed <certainty>

For some reason in 3573 line1, 0722 line 2 and 0436 line 1 the <orig> element has simply ben removed instead, but this can be manually tweaked in due course, if it's even worth worrying about right now.

For some reason in 3573 line1, 0722 line 2 and 0436 line 1 the <orig> element has simply ben removed instead, but this can be manually tweaked in due course, if it's even worth worrying about right now.

Ah, I did that because these look like parts of names and will eventually be tagged as such, so they're not totally uninterpretable.


Filter edits that are obviously not damaging for damaging_and_goodfaith campaigns.

Isn't this done by meta:Research talk:Revision scoring as a service/Work log/2015-05-16#Saturday, May 16, 2015?

Yup!  :)


lake: Parallelize tsdir fetch

The walk of lake stored in s3 is slowed greated in that tsDirs are fetched
sequentially. The main thread sits by idly while the metadata for all chunks in
a tsdir are retrieved over the network. Speed this process up by running the
tsdir fetches each in a separate goroutine.
Because it is important that results be returned in order, save the retrieved
tsdirs in a min/max heap, and release results from the stream when the next
min/max is received.
Closes #2017

Some rough performance metrics:
On a 8 cpu aws box running on corelight dataset with threshold set to 1MiB (40 tsdirs w/ ~5 chunks per day)

master: ~5 secs
parallelize-fetch-tsdirs: ~2.5 secs

@alfred-landrum I incorporated your changes with some minor adjustments.


error scheduling: no minions available to schedule pods

Hi,
I have setup a 4 nodes (1 master, 3 slaves) kubernetes over coreos, but when I try to create a pod it does not get deployed
Kubernetes v0.14.1
Coreos 633.1.0
kubmaster ~ # kubectl get nodes
NAME      LABELS    STATUS
kubmaster guestbook # fleetctl list-machines
MACHINE         IP              METADATA
1eec218a...     <IP_ADDRESS>    role=node
5b6c8966...     <IP_ADDRESS>    role=master
86d46f7a...     <IP_ADDRESS>    role=node
a20f07f4...     <IP_ADDRESS>    role=node
kubmaster guestbook # kubectl get pods
POD                  IP        CONTAINER(S)   IMAGE(S)           HOST           LABELS              STATUS    CREATED
redis-master-ioeb9             master         dockerfile/redis      name=redis-master   Pending   49 minutes
kubmaster guestbook # kubectl describe pod redis-master-ioeb9
Name:                           redis-master-ioeb9
Image(s):                       dockerfile/redis
Host:                           /
Labels:                         name=redis-master
Status:                         Pending
Replication Controllers:        redis-master (1/1 replicas created)
Events:
FirstSeen                             LastSeen                        Count   From            SubobjectPath   Reason                  Message
Thu, 09 Apr 2015 06:56:42 +0000       Thu, 09 Apr 2015 07:02:45 +0000 17      {scheduler }                    failedScheduling        Error scheduling: no minions available to schedule pods
Thu, 09 Apr 2015 07:03:36 +0000       Thu, 09 Apr 2015 07:16:45 +0000 42      {scheduler }                    failedScheduling        Error scheduling: no minions available to schedule pods
Thu, 09 Apr 2015 07:17:22 +0000       Thu, 09 Apr 2015 07:46:18 +0000 89      {scheduler }                    failedScheduling        Error scheduling: no minions available to schedule pods
kubmaster guestbook # journalctl -u kube-register
[...]
Apr 09 10:40:47 kubmaster kube-register[770]: error registering: <IP_ADDRESS>
Apr 09 10:40:47 kubmaster kube-register[770]: Response: &http.Response{Status:"422 status code 422", StatusCode:422, Proto:"HTTP/1.1", ProtoMajor:1, ProtoMinor:1, Heade
Apr 09 10:40:47 kubmaster kube-register[770]: Response Body:
Apr 09 10:40:47 kubmaster kube-register[770]: {
Apr 09 10:40:47 kubmaster kube-register[770]: "kind": "Status",
Apr 09 10:40:47 kubmaster kube-register[770]: "creationTimestamp": null,
Apr 09 10:40:47 kubmaster kube-register[770]: "apiVersion": "v1beta1",
Apr 09 10:40:47 kubmaster kube-register[770]: "status": "Failure",
Apr 09 10:40:47 kubmaster kube-register[770]: "message": "Node "<IP_ADDRESS>" is invalid: [status.Capacity: required value, spec.ExternalID: required value]",
Apr 09 10:40:47 kubmaster kube-register[770]: "reason": "Invalid",
Apr 09 10:40:47 kubmaster kube-register[770]: "details": {
Apr 09 10:40:47 kubmaster kube-register[770]: "id": "<IP_ADDRESS>",
Apr 09 10:40:47 kubmaster kube-register[770]: "kind": "Node",
Apr 09 10:40:47 kubmaster kube-register[770]: "causes": [
Apr 09 10:40:47 kubmaster kube-register[770]: {
Apr 09 10:40:47 kubmaster kube-register[770]: "reason": "FieldValueRequired",
Apr 09 10:40:47 kubmaster kube-register[770]: "message": "status.Capacity: required value",
Apr 09 10:40:47 kubmaster kube-register[770]: "field": "status.Capacity"
Apr 09 10:40:47 kubmaster kube-register[770]: },
Apr 09 10:40:47 kubmaster kube-register[770]: {
Apr 09 10:40:47 kubmaster kube-register[770]: "reason": "FieldValueRequired",
Apr 09 10:40:47 kubmaster kube-register[770]: "message": "spec.ExternalID: required value",
Apr 09 10:40:47 kubmaster kube-register[770]: "field": "spec.ExternalID"
Apr 09 10:40:47 kubmaster kube-register[770]: }
Apr 09 10:40:47 kubmaster kube-register[770]: ]
Apr 09 10:40:47 kubmaster kube-register[770]: },
[...]
Could you help me?
thanks,

I've found that the kube-kubelet service on slaves are complaining:
Apr 09 09:33:51 kubslave1 kubelet[772]: E0409 09:33:51.103942     772 kubelet.go:1729] error updating node status, will retry: error getting node "kubslave1": minion "kubslave1" not found
Apr 09 09:33:51 kubslave1 kubelet[772]: E0409 09:33:51.106384     772 kubelet.go:1729] error updating node status, will retry: error getting node "kubslave1": minion "kubslave1" not found
Apr 09 09:33:51 kubslave1 kubelet[772]: E0409 09:33:51.106413     772 kubelet.go:557] Unable to update node status: Update node status exceeds retry count
any ideas?

I also have found that:
kubslave1 ~ # curl http://<IP_ADDRESS>:10250/healthz
[+]ping ok
[+]docker ok
[-]hostname failed: Kubelet hostname "kubslave3" does not match the hostname expected by the master "<IP_ADDRESS>"
healthz check failed

forget about it. I updated to 0.14.2 and its solveds. it seems
https://github.com/kelseyhightower/kube-register/issues/13


Expand l10n documentation

Refer to http://discuss.flarum.org/d/1085/8

Adds a page for langpack installation instructions.
Renames the i18n page in the Extensions section.
Modifies the packaging page in same to match.
Adds a new section of l10n pages for translators.

Whoops ... this one seems to have been half-backed. Folder- and filename mismatches.
I'll try again.


Syntax Highlight for solidity

Add syntax highlight support for solidity.
https://github.com/ethereum/solidity
For testing purpose
https://gist.github.com/hackjutsu/676539163fbe1093062ca14006d1b3ba
Candidate package
https://www.npmjs.com/package/highlightjs-solidity

https://medium.com/@danielque/psa-how-to-fix-githubs-syntax-highlighting-for-solidity-4e9867c540b6

Thanks @meilinz !!


Evicted pod do not release controller ConfigMap lock

The controller pod becomes a leader by acquiring a lock on a configmap resource.
Due to a bug in my code which caused a memory leak, the pod was evicted after a few hours.
However, the evicted pod lock was not released and thus, the new pod could not become a leader.
The only way to fix that was to manually delete the evicted pod, which was locking the resource and delete the new pod which could not lock the resource and gave up after a few tries.
According to Kubernetes documentation, evicted pods locks are deleted as they evict.

HI @ilanuriel,
Note that when a pod is flagged as evict the API will try to remove it. Also, before removing the pod it will ensure that all pre-conditions to allowing it happens.
Shows that because of your code which caused a memory leak it was running out of resoources. I'd like to suggest you check the following documentation.

https://kubernetes.io/docs/concepts/workloads/pods/disruptions/
https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#eviction-policy

Please, let us know if the above information attends your expectations and if we can close this ticket or if has anything else that you are looking for to be done here.

Thanks Camila,
Due to my memory leak the POD was evicted. This part is OK. The leak was violating resource consumption so Kubernetes was right to evict the POD. However, the evicted POD was not removed and lock was not freed. Therefore the new POD replacing the evicted POD was unable to become the leader. This part in my opinion is a bug because if an evicted POD, which is a non working POD does not release resources, new PODs will fail to operate, so essentially eviction means service outage instead of replacing a sick POD with a new healthy one and resuming service. Unless I am missing something, I believe this is a bug. Do you agree?

Hi @ilanuriel,
In the following k8s documents you can have a  better idea over how k8s API works in these scenarios and how you can set up your environment to dealing better with this kind of disruption.
https://kubernetes.io/docs/concepts/workloads/pods/disruptions/
https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#eviction-policy
Also, could you please let us know why do you believe that it is a bug in the operator-framework?

Hi Camila,
Per documentation, evicted pod resources need to be released. The documentation suggests various methods to tune when this happens. But whenever this happens, the eviction should end with resources being released, which is what is not happening in this case.
The reason I believe this is an operator-framework bug is as follows:
The way the controller code in the framework works, is that a configmapis is created by the pod which wants to become the leader. Upon a successful configmap generation the pod becomes the leader, and it will stay the leader even if another pod instance tries to become one, because no second pod can generate the configmap as long as it is not deleted first.
However, when the leading pod is evicted, the one replacing it should be able to become a leader, which means that it should be able to generate a configmap. This will happen only if the evicted pod will delete the configmap when evicted, so the one replacing it can successfully generate the configmap again and become a leader. I found no code that is responsible to delete a configmap on eviction, therefore I conclude that there is a missing handler in the framework. However, I am not familiar with the framework code to a level that I can be sure my conclusion is correct, that is why I approached you.

Hi @ilanuriel,
Thank you for the information provided.
Note that as you described your scenario was:the evicted pod lock was not released and thus, the new pod could not become a leader. The only way to fix that was to manually delete the evicted pod, which was locking the resource and delete the new pod which could not lock the resource and gave up after a few tries.
Which means that it fails in the case of eviction of a pod due to the node being out-of-resources as described in the k8s. So, your cluster had no resources enough to allow the k8s API to perform the evited action and delete the pod as any other consequence action.
In order to try to avoid this scenario, according to k8s doc, you would be able to Configure Out Of Resource Handling.
The operator-framework use the client-go to call the API in order to request to do the actions, however, the indeed action to delete the evicted pod and etc is done by the API and not by it. So, I cannot see any relation with.
Please, let me know if the above information could clarify it and if you are ok in close to this issue.

Thank you Haseeb,
May I suggest a resolution?
leader.go in the framework deals with the owner. In line 85, if an owner is found and this is not a restart case of the same pod, it will wait for sometime and in the case I have raised, give up since the lock will never be released.
What we can do is check if the ownerexistingOwner.name belongs to a POD that is not in RUNNING or PENDING state. In such case, the lock will be regarded as an error lock and thus, we should allow the controller to proceed as a leader and even further,  delete the existing configmap and re-create it so its ownership is updated to be owned by the new pod. This way, the code operator will supervise locks are owned by living pods and not staying in ownership or invalid entities that for some reason where not released properly.

Since it blocks the CNV operators' development, I create a bug in BZ for this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1749620

What we can do is check if the ownerexistingOwner.name belongs to a POD that is not in RUNNING or PENDING state. In such case, the lock will be regarded as an error lock and thus, we should allow the controller to proceed as a leader and even further, delete the existing configmap and re-create it so its ownership is updated to be owned by the new pod.

@ilanuriel Unfortunately that approach would cause the leader-for-life election guarantees to fail in the scenario that the leader pod is evicted due to it being on an unresponsive or unreachable node. In that case, the pod may actually still be running so it would be invalid to delete its lock.
However, controller-runtime supports leader election with the leader-with-lease approach that may handle this particular issue correctly. See our leader election documentation for more details.
There are tradeoffs with either approach, so you should consider carefully which makes the most sense for your use case.

Thinking about this more, it seems there are two ways a pod can be evicted:

The pod is on a node that becomes unreachable.
The pod is killed by kubelet due to resource usage.

For 1) there are two scenarios:
1a. The pod is able to maintain a connection to the API server.
1b. The pod loses the connection to the API server.
To cover 1a., we could have the leader node monitor itself, checking to see if it ever gets evicted. If so, it could stop all of its controllers, delete the lock, and exit. This would allow an operator replica on a non-partitioned node to gain the lock and become the leader.
For 1b., if the leader is no longer able to communicate with the API server, it would no longer be able to monitor itself or delete the lock. In this case, it seems like there is still the possibility of a deadlock.
To cover 2, we could look into catching the signal from kubelet that terminates the container, which would allow us to stop all controllers, delete the lock, and exit cleanly.
Note that for 1a and 2. It is important that all of the controllers are stopped before the lock is deleted to guarantee that another operator replica is not able to run the same controllers simultaneously.
@shawn-hurley @hasbro17 @estroz @camilamacedo86 Anything I'm getting wrong or any different perspectives or ideas for other approaches? Any ideas for the 1b case?

@joelanford won't the deadlock only occur in case 1b for pod-eviction-timeout amount of time, since the pod will be deleted after that amount of unresponsiveness?

@estroz Not necessarily. If the node is unreachable the apiserver can't tell the kubelet to delete the pods after pod-eviction-timeout.
https://kubernetes.io/docs/concepts/architecture/nodes/#condition

In some cases when the node is unreachable, the apiserver is unable to communicate with the kubelet on the node. The decision to delete the pods cannot be communicated to the kubelet until communication with the apiserver is re-established. In the meantime, the pods that are scheduled for deletion may continue to run on the partitioned node.

In versions of Kubernetes prior to 1.5, the node controller would force delete these unreachable pods from the apiserver. However, in 1.5 and higher, the node controller does not force delete pods until it is confirmed that they have stopped running in the cluster. You can see the pods that might be running on an unreachable node as being in the Terminating or Unknown state.

So deadlock forever is possible on partitioned/unreachable nodes. I don't think we should try to handle 1b. 1a is just a best case scenario for 1 so I don't know if it's worth optimizing for.
@joelanford I need to read more on what are the sequence of events when a pod is evicted but from the termination lifecycle of a pod it doesn't seem that we need to handle the case of the pod being unresponsive to SIGTERM from the kubelet.
https://kubernetes.io/docs/concepts/workloads/pods/pod/#termination-of-pods

User sends command to delete Pod, with default grace period (30s)
The Pod in the API server is updated with the time beyond which the Pod is considered “dead” along with the grace period.
Pod shows up as “Terminating” when listed in client commands
(simultaneous with 3) When the Kubelet sees that a Pod has been marked as terminating because the time in 2 has been set, it begins the Pod shutdown process.
...

In the case of no network partitions, once the graceperiod expires the pod will be killed via SIGKILL and will be removed from the API server.
So for 2 we shouldn't be trying to catch SIGTERM/SIGKILL signals from the kubelet in the operator pod. At that point the pod will be successfully deleted anyway.
At best the pod can monitor its own terminationGracePeriodSeconds field in the spec to realize it's being marked for eviction and then shutdown on its own.
Going back to @ilanuriel's  original issue:

However, the evicted POD was not removed
What I want to know is what is the graceperiod for that eviction policy and how long does the pod stick around for after being evicted.
https://kubernetes.io/docs/tasks/administer-cluster/out-of-resource/#soft-eviction-thresholds

Okay so evicted pods will get "evicted" but not killed/deleted(i.e deletion timestamp is not set) from the API it seems. This is true for pods managed by Deployments but not for Daemonsets or other controllers. And it seems intentional.
See https://github.com/kubernetes/kubernetes/issues/54525#issuecomment-339847242 https://github.com/kubernetes/kubernetes/issues/54525#issuecomment-340033184 and https://github.com/kubernetes/kubernetes/issues/54525#issuecomment-340035375
I guess for the low resource eviction case if we know the pod is going to stick around with status.phase=Failed then the leader pod can give up the lock and exit itself.
@ilanuriel I don't know if you can confirm this anymore for your evicted pods to see if they were marked with status.phase=Failed?

I think it was marked with failed

I would like to emphasize that even if there are cases where the framework won't fix, please be advised that the newly created POD could not become the leader and after sometime stopped trying and remained in a RUNNING state, where it is essentially was  doing nothing. Only by deleting BOTH the evicted POD AND the new POD was I able to get things OK again, as the new POD stopped trying to become the leader, so deleting ONLY the evicted POD was NOT enough. From a DevOps perspective, the new POD looked perfect: it was running and no report that this operator will be doing nothing was generated.
From a system management perspective, this is for sure a problem because we get nothing to suggest that the operator is not working and we have to do something about it. A new POD is in a RUNNING state, everything looks OK, the system administrator relaxes with a cup of coffee that all is good, while in practice the new POD operator will NEVER work.
To make things even worse, even if deleting evicted POD, the new POD has to be deleted as well to make things work again.
So to summarize, even if there are situations where the framework cannot recover evicted PODs issues, at the very least, it cannot generate new PODs that cannot work and provide the illusion that all is great while they do not work. A POD that cannot become a leader and function, should switch into a Kubernetes state that allows admins to understand the system needs care.
Also, while every admin knows that evicted POD should be deleted eventually, they probably have no idea that the new operator POD should be deleted as well to recover, so at the current situation, they will delete the evicted POD, thinking that the new POD is OK since it is in a RUNNING state, while that POD gave up trying to become the leader a long time ago and remain in a "doing nothing" RUNNING state.

This looks like a dup of this issue that was reported back in April. See: https://github.com/operator-framework/operator-sdk/issues/1305

I agree that the simplest way to address this is for the non-leader to check the status of the leader pod and delete its lock if that pod's phase is Failed.

Only by deleting BOTH the evicted POD AND the new POD was I able to get things OK again, as the new POD stopped trying to become the leader

That sounds like a separate bug. Do you have any additional findings or info on why the new Pod stopped trying to become the leader?

Does this issue solved?

@avarf the patch isn't merged yet, but if you would like to help test it, the the PR is 2210.

BZ clone https://bugzilla.redhat.com/show_bug.cgi?id=1749620

Demo of bugfix: https://www.youtube.com/watch?v=qTzdAYWPdCQ


How to set the input when compiling model for non-image input?

Hi, I have trained a model whose input is a set of 3D points with a shape Nx3, N is not a fixed number. In this case, how to set the input during compiling my model?
For image, the input shape is like this:
inputs = [torch.randn((1, 3, 224, 224)).to("cuda").half()]

What if for my case? Thank you!
### Tasks

Hello - for models with tensors that have Dynamic dimensions, you can specify the minimum, optimal, and maximum shapes via the torch_tensorrt.Input API. For instance, when compiling via torch_tensorrt.compile, you could specify:
torch_tensorrt.compile(model,
                       inputs=[
                           torch_tensorrt.Input(
                               min_shape=(1, 3, 224, 224),
                               opt_shape=(3, 3, 224, 224),
                               max_shape=(10, 3, 224, 224))]
                      ...


Diffs from plugins cause Vscode to freeze and do other weird things.

Describe the bug
Using a plugin that sends diffs to the language server.
https://github.com/JanSharp/FactorioSumnekoLuaPlugin
This one is hard to reproduce but can be randomly and consistently reproduced on my end. Usually happens when moving whole lines of code (i.e. adding multiple lines inside a function)
Some things that have happened:
Syntax coloring below the current point gets all random
Vscode freezes and has to be closed and reopened
Entering characters starts typing in reverse
Using 3.1.0 and Vscode on Linux/PopOs/Ubuntu
I will add logs to here when I notice it happen along with what I noticed at the time.

Vscode freezes and has to be closed and reopened
Entering characters starts typing in reverse

This is the function of the client. Theoretically, the language server cannot affect this function.

My best guess is the data is getting corrupted because current diffs are being processed before past diffs are finished.
Well it's entirely possible it is vscodes fault it does only happen with this provider and a plug-in providing  diffs

Did you applied this patch: 22f29e50a340c6f8ab8bcafe983d2af07d123429 ?
Or try to disable format on paste or format on save

Have not noticed any more freezes/crashes in the latest few versions. I will close this for now. Thank you