Trying to get this to work with Spotpear LCD

This is my lcd
http://www.spotpear.com/index.php/index/study/detail/id/77.html
Type of Raspberry Pi
Raspberry Pi 3A
Linux Kernel version
Linux raspberrypi 5.4.79-v7+ #1373 SMP Mon Nov 23 13:22:33 GMT 2020 armv7l GNU/Linux
Expected behaviour
I expect a 3d box
Actual behaviour
Trying to get the display to work with the correct driver
If I try and run 3d_box.py I get the following error message
3d_box.py: error: I2C device not found on address: 0x3C
This is no surprise as
i2cdetect 1
Does not report anything, I get
0  1  2  3  4  5  6  7  8  9  a  b  c  d  e  f
00:          -- -- -- -- -- -- -- -- -- -- -- -- --
10: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
20: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
30: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
40: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
50: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
60: -- -- -- -- -- -- -- -- -- -- -- -- -- -- -- --
Does anyone have any pointers please?

first quick note - your LCD is not I2C, is SPI, so you need to let the driver know how to correctly reach your display - through the correct SPI port, using the right SPI settings and right GPIO pins. You should be able to get all of these parameters from the example code they're providing.

first quick note - your LCD is not I2C, is SPI, so you need to let the driver know how to correctly reach your display - through the correct SPI port, using the right SPI settings and right GPIO pins. You should be able to get all of these parameters from the example code they're providing.

Unfortunately, there is only one c program that you can download (to create fbcp) and I did not find it very helpful.
I did some digging around in /sys with
find /sys/bus | grep spi
and got this back
/sys/bus/platform/devices/3f204000.spi
/sys/bus/platform/drivers/spi-bcm2835
/sys/bus/platform/drivers/spi-bcm2835/bind
/sys/bus/platform/drivers/spi-bcm2835/unbind
/sys/bus/platform/drivers/spi-bcm2835/module
/sys/bus/platform/drivers/spi-bcm2835/uevent
/sys/bus/platform/drivers/spi-bcm2835/3f204000.spi
/sys/bus/spi
/sys/bus/spi/drivers_probe
/sys/bus/spi/devices
/sys/bus/spi/devices/spi0.0
/sys/bus/spi/devices/spi0.1
/sys/bus/spi/uevent
/sys/bus/spi/drivers
/sys/bus/spi/drivers/stmpe-spi
/sys/bus/spi/drivers/stmpe-spi/bind
/sys/bus/spi/drivers/stmpe-spi/unbind
/sys/bus/spi/drivers/stmpe-spi/uevent
/sys/bus/spi/drivers/fb_ili9340
/sys/bus/spi/drivers/fb_ili9340/spi0.0
/sys/bus/spi/drivers/fb_ili9340/bind
/sys/bus/spi/drivers/fb_ili9340/unbind
/sys/bus/spi/drivers/fb_ili9340/module
/sys/bus/spi/drivers/fb_ili9340/uevent
/sys/bus/spi/drivers/ads7846
/sys/bus/spi/drivers/ads7846/bind
/sys/bus/spi/drivers/ads7846/unbind
/sys/bus/spi/drivers/ads7846/spi0.1
/sys/bus/spi/drivers/ads7846/module
/sys/bus/spi/drivers/ads7846/uevent
/sys/bus/spi/drivers_autoprobe
So it is using the fb_ili9340 driver, that source code is here
https://github.com/adafruit/adafruit-rpi-fbtft/blob/master/fb_ili9340.c
Can I extract the correct information from there?

Unfortunately, there is only one c program that you can download (to create fbcp) and I did not find it very helpful.
I did some digging around in /sys with
find /sys/bus | grep spi
and got this back
/sys/bus/platform/devices/3f204000.spi
/sys/bus/platform/drivers/spi-bcm2835
/sys/bus/platform/drivers/spi-bcm2835/bind
/sys/bus/platform/drivers/spi-bcm2835/unbind
/sys/bus/platform/drivers/spi-bcm2835/module
/sys/bus/platform/drivers/spi-bcm2835/uevent
/sys/bus/platform/drivers/spi-bcm2835/3f204000.spi
/sys/bus/spi
/sys/bus/spi/drivers_probe
/sys/bus/spi/devices
/sys/bus/spi/devices/spi0.0
/sys/bus/spi/devices/spi0.1
/sys/bus/spi/uevent
/sys/bus/spi/drivers
/sys/bus/spi/drivers/stmpe-spi
/sys/bus/spi/drivers/stmpe-spi/bind
/sys/bus/spi/drivers/stmpe-spi/unbind
/sys/bus/spi/drivers/stmpe-spi/uevent
/sys/bus/spi/drivers/fb_ili9340
/sys/bus/spi/drivers/fb_ili9340/spi0.0
/sys/bus/spi/drivers/fb_ili9340/bind
/sys/bus/spi/drivers/fb_ili9340/unbind
/sys/bus/spi/drivers/fb_ili9340/module
/sys/bus/spi/drivers/fb_ili9340/uevent
/sys/bus/spi/drivers/ads7846
/sys/bus/spi/drivers/ads7846/bind
/sys/bus/spi/drivers/ads7846/unbind
/sys/bus/spi/drivers/ads7846/spi0.1
/sys/bus/spi/drivers/ads7846/module
/sys/bus/spi/drivers/ads7846/uevent
/sys/bus/spi/drivers_autoprobe
So it is using the fb_ili9340 driver, that source code is here
https://github.com/adafruit/adafruit-rpi-fbtft/blob/master/fb_ili9340.c
Can I extract the correct information from there?

There's still some information in the linked repo, specifically, the script that you're advised to use. The script uses the waveshare32b overlay from their repository, which is compiled. It's scummy from their side not to publish sources for the overlay, however, overlays are easy to decompile into a state where you can see the pins used:
dtc -I dtb -O dts -o waveshare35b-overlay.dts waveshare35b-overlay.dtb

/dts-v1/;

/ {
	compatible = "brcm,bcm2835\0brcm,bcm2708\0brcm,bcm2709";

	[...]

	fragment@1 {
		target = <0xffffffff>;

		__overlay__ {

			waveshare35b_pins {
				brcm,pins = <0x11 0x19 0x18>;
				brcm,function = <0x00 0x00 0x00>;
				phandle = <0x01>;
			};
		};
	};

	fragment@2 {
		target = <0xffffffff>;

		__overlay__ {
			#address-cells = <0x01>;
			#size-cells = <0x00>;

			waveshare35b@0 {
				compatible = "ilitek,ili9486";
				reg = <0x00>;
				pinctrl-names = "default";
				pinctrl-0 = <0x01>;
				spi-max-frequency = <0xe4e1c0>;
				txbuflen = <0x8000>;
				rotate = <0x5a>;
				bgr = <0x00>;
				fps = <0x1e>;
				buswidth = <0x08>;
				regwidth = <0x10>;
				reset-gpios = <0xffffffff 0x19 0x01>;
				dc-gpios = <0xffffffff 0x18 0x00>;
				debug = <0x00>;
				init = <0x10000b0 0x00 0x1000011 0x20000ff 0x100003a 0x55 0x1000036 0x28 0x1000021 0x10000c0 0x09 0x09 0x10000c1 0x41 0x00 0x10000c5 0x00 0x36 0x10000e0 0x00 0x2c 0x2c 0x0b 0x0c 0x04 0x4c 0x64 0x36 0x03 0x0e 0x01 0x10 0x01 0x00 0x10000e1 0x0f 0x37 0x37 0x0c 0x0f 0x05 0x50 0x32 0x36 0x04 0x0b 0x00 0x19 0x14 0x0f 0x1000011 0x20000ff 0x1000029>;
				phandle = <0x02>;
			};

			waveshare35b-ts@1 {
				compatible = "ti,ads7846";
				reg = <0x01>;
				spi-max-frequency = <0x1e8480>;
				interrupts = <0x11 0x02>;
				interrupt-parent = <0xffffffff>;
				pendown-gpio = <0xffffffff 0x11 0x01>;
				ti,x-plate-ohms = [00 3c];
				ti,pressure-max = [00 ff];
				phandle = <0x03>;
			};
		};
	};

	__overrides__ {
		speed = <0x02 0x7370692d 0x6d61782d 0x66726571 0x75656e63 0x793a3000>;
		txbuflen = [00 00 00 02 74 78 62 75 66 6c 65 6e 3a 30 00];
		rotate = [00 00 00 02 72 6f 74 61 74 65 3a 30 00];
		fps = [00 00 00 02 66 70 73 3a 30 00];
		bgr = [00 00 00 02 62 67 72 3a 30 00];
		debug = <0x02 0x64656275 0x673a3000>;
		swapxy = <0x03 0x74692c73 0x7761702d 0x78793f00>;
	};

	[...]
};

Full decompiled overlay sources:
waveshare35b-overlay.zip
The pins used for RST/DC should be easy to understand. Let me think for a second where we can go from here...

There's still some information in the linked repo, specifically, the script that you're advised to use. The script uses the waveshare32b overlay from their repository, which is compiled. It's scummy from their side not to publish sources for the overlay, however, overlays are easy to decompile into a state where you can see the pins used:
dtc -I dtb -O dts -o waveshare35b-overlay.dts waveshare35b-overlay.dtb

/dts-v1/;

/ {
	compatible = "brcm,bcm2835\0brcm,bcm2708\0brcm,bcm2709";

	[...]

	fragment@1 {
		target = <0xffffffff>;

		__overlay__ {

			waveshare35b_pins {
				brcm,pins = <0x11 0x19 0x18>;
				brcm,function = <0x00 0x00 0x00>;
				phandle = <0x01>;
			};
		};
	};

	fragment@2 {
		target = <0xffffffff>;

		__overlay__ {
			#address-cells = <0x01>;
			#size-cells = <0x00>;

			waveshare35b@0 {
				compatible = "ilitek,ili9486";
				reg = <0x00>;
				pinctrl-names = "default";
				pinctrl-0 = <0x01>;
				spi-max-frequency = <0xe4e1c0>;
				txbuflen = <0x8000>;
				rotate = <0x5a>;
				bgr = <0x00>;
				fps = <0x1e>;
				buswidth = <0x08>;
				regwidth = <0x10>;
				reset-gpios = <0xffffffff 0x19 0x01>;
				dc-gpios = <0xffffffff 0x18 0x00>;
				debug = <0x00>;
				init = <0x10000b0 0x00 0x1000011 0x20000ff 0x100003a 0x55 0x1000036 0x28 0x1000021 0x10000c0 0x09 0x09 0x10000c1 0x41 0x00 0x10000c5 0x00 0x36 0x10000e0 0x00 0x2c 0x2c 0x0b 0x0c 0x04 0x4c 0x64 0x36 0x03 0x0e 0x01 0x10 0x01 0x00 0x10000e1 0x0f 0x37 0x37 0x0c 0x0f 0x05 0x50 0x32 0x36 0x04 0x0b 0x00 0x19 0x14 0x0f 0x1000011 0x20000ff 0x1000029>;
				phandle = <0x02>;
			};

			waveshare35b-ts@1 {
				compatible = "ti,ads7846";
				reg = <0x01>;
				spi-max-frequency = <0x1e8480>;
				interrupts = <0x11 0x02>;
				interrupt-parent = <0xffffffff>;
				pendown-gpio = <0xffffffff 0x11 0x01>;
				ti,x-plate-ohms = [00 3c];
				ti,pressure-max = [00 ff];
				phandle = <0x03>;
			};
		};
	};

	__overrides__ {
		speed = <0x02 0x7370692d 0x6d61782d 0x66726571 0x75656e63 0x793a3000>;
		txbuflen = [00 00 00 02 74 78 62 75 66 6c 65 6e 3a 30 00];
		rotate = [00 00 00 02 72 6f 74 61 74 65 3a 30 00];
		fps = [00 00 00 02 66 70 73 3a 30 00];
		bgr = [00 00 00 02 62 67 72 3a 30 00];
		debug = <0x02 0x64656275 0x673a3000>;
		swapxy = <0x03 0x74692c73 0x7761702d 0x78793f00>;
	};

	[...]
};

Full decompiled overlay sources:
waveshare35b-overlay.zip
The pins used for RST/DC should be easy to understand. Let me think for a second where we can go from here...

For a start, the RST pin should be 25 and DC pin should be 24.

For a start, the RST pin should be 25 and DC pin should be 24.

lol, Spotpear seems to be a shitty company when it comes to open-sourcing even the most primitive stuff, not to mention even schematics. Would not be surprised if there's no original work put from their side, their products seem to be similar to Waveshare products, at least they're reusing the code and the overlays...
Wait, are they by any chance the same company? =D
Check this out, this might just help you a lot: https://github.com/rm-hull/luma.lcd/pull/136

lol, Spotpear seems to be a shitty company when it comes to open-sourcing even the most primitive stuff, not to mention even schematics. Would not be surprised if there's no original work put from their side, their products seem to be similar to Waveshare products, at least they're reusing the code and the overlays...
Wait, are they by any chance the same company? =D
Check this out, this might just help you a lot: https://github.com/rm-hull/luma.lcd/pull/136

Thanks for all of this, much appreciated.
You should see Spotpear's attempt at scripts, no formatting whatsoever, no subtle stuff like injecting parameters into configuration files. Oh no, just overwrite the files instead. My favourite is overwriting /boot/config.txt.
Anyway, I am still trying to get a successful test working, I am running this
python3 3d_box.py -i spi  --gpio-reset 25 --gpio-data-command 24 -d ili9486 --spi-device 1
I am getting this error
/usr/local/lib/python3.7/dist-packages/luma/core/interface/serial.py:200: RuntimeWarning: This channel is already in use, continuing anyway.  Use GPIO.setwarnings(False) to disable warnings.
fbcp is not running.
I tried to run test_ili9486.py and it did not return anything.
Any clues please?

Thanks for all of this, much appreciated.
You should see Spotpear's attempt at scripts, no formatting whatsoever, no subtle stuff like injecting parameters into configuration files. Oh no, just overwrite the files instead. My favourite is overwriting /boot/config.txt.
Anyway, I am still trying to get a successful test working, I am running this
python3 3d_box.py -i spi  --gpio-reset 25 --gpio-data-command 24 -d ili9486 --spi-device 1
I am getting this error
/usr/local/lib/python3.7/dist-packages/luma/core/interface/serial.py:200: RuntimeWarning: This channel is already in use, continuing anyway.  Use GPIO.setwarnings(False) to disable warnings.
fbcp is not running.
I tried to run test_ili9486.py and it did not return anything.
Any clues please?

do you have the overlay loaded? make sure it's not loaded.
This warning is likely expected and normal - the display should work nevertheless. What actually happens instead? You're not describing the result you're getting.

do you have the overlay loaded? make sure it's not loaded.
This warning is likely expected and normal - the display should work nevertheless. What actually happens instead? You're not describing the result you're getting.

hmm, sorry, I'm on the go. What does --spi-device mean? I'd imagine the display is connected to SPI0.0 and touch is connected to SPI0.1 - that's been my experience with the display I have, but it might not be the case with whatever breakout you're trying out. Or is it something other than SPI port/SPI CS number?

hmm, sorry, I'm on the go. What does --spi-device mean? I'd imagine the display is connected to SPI0.0 and touch is connected to SPI0.1 - that's been my experience with the display I have, but it might not be the case with whatever breakout you're trying out. Or is it something other than SPI port/SPI CS number?

also, it's no wonder test_ili9486.py returns nothing - it actually isn't written to operate on a real display, it's just a luma test file that helps luma developers check whether the code is still working.

also, it's no wonder test_ili9486.py returns nothing - it actually isn't written to operate on a real display, it's just a luma test file that helps luma developers check whether the code is still working.

OK. I have removed dtoverlay=waveshare32b:rotate=270 from /boot/config.txt
The following devices are created
/dev/spidev0.0
/dev/spidev0.1

I have rerun with
python3 3d_box.py -i spi  --gpio-reset 25 --gpio-data-command 24 -d ili9486
I get
3d_box.py: error: Unsupported display mode: 128 x 64
So I am getting closer!

OK. I have removed dtoverlay=waveshare32b:rotate=270 from /boot/config.txt
The following devices are created
/dev/spidev0.0
/dev/spidev0.1

I have rerun with
python3 3d_box.py -i spi  --gpio-reset 25 --gpio-data-command 24 -d ili9486
I get
3d_box.py: error: Unsupported display mode: 128 x 64
So I am getting closer!

seems like you need to give it height and width parameters =)

seems like you need to give it height and width parameters =)

I tried
--width 320 --height 240
Got
3d_box.py: error: Unsupported display mode: 320 x 240
All I have is a white screen....it must be something simple

I tried
--width 320 --height 240
Got
3d_box.py: error: Unsupported display mode: 320 x 240
All I have is a white screen....it must be something simple

searching "ili9486" in this repo on github brought me to this page, which says that only 480x320 displays are supported at the moment. Are you sure your display is 320x240?

searching "ili9486" in this repo on github brought me to this page, which says that only 480x320 displays are supported at the moment. Are you sure your display is 320x240?

Unfortunately it is 320 x 240

Unfortunately it is 320 x 240

If I make a change in that code to take out that check (in device.py) do I have to recompile it?

If I make a change in that code to take out that check (in device.py) do I have to recompile it?

Good thinking! Answering your question directly - no, but you'll have to make sure that your changes actually get used by your scripts, i.e. git-clone-ing this code locally, making your changes then using something like python setup.py develop to install the locally-modified library - or just edit the library files in the place they're currently stored after you install using pip.
Answering the bigger question - the check is there for a reason. First of all, it's a sanity check of whether a known-valid width&height pair is passed, but also, it's a check to see whether the library has actually been tested with the display in question. The library code adapts to different widths and heights when it comes to i.e. graphics primitives and so on, but there's one more thing you might want to keep in mind - some initialization commands will be different depending on the display size (width&height in pixels), as displays of different size, even if they use the same controller for driving the panel's pixels, will wire the panel differently and might have a lot of tiny but significant bits&pieces configured in a different way, to the point of no image appearing on the display.
What can you do about this if simply removing the width&height check will not work? The major thing is checking initialization commands against ones from a library that's known-good and works for your display, then using them. Order of the commands will be different and that's okay. After you do this, the display should work. Next step would be to contribute your changes to this repository so that everyone interested would be able to use them, and as such, make the world a bit of a brighter place =)
What you can also do is use the overlay (that seems to actually work for you), not enable fbcp and instead use the luma's linux-framebuffer support to draw to the /dev/fb1 device created by the display driver. If you don't want to play with adjusting init commands, this would probably be the easiest solution for you.

Good thinking! Answering your question directly - no, but you'll have to make sure that your changes actually get used by your scripts, i.e. git-clone-ing this code locally, making your changes then using something like python setup.py develop to install the locally-modified library - or just edit the library files in the place they're currently stored after you install using pip.
Answering the bigger question - the check is there for a reason. First of all, it's a sanity check of whether a known-valid width&height pair is passed, but also, it's a check to see whether the library has actually been tested with the display in question. The library code adapts to different widths and heights when it comes to i.e. graphics primitives and so on, but there's one more thing you might want to keep in mind - some initialization commands will be different depending on the display size (width&height in pixels), as displays of different size, even if they use the same controller for driving the panel's pixels, will wire the panel differently and might have a lot of tiny but significant bits&pieces configured in a different way, to the point of no image appearing on the display.
What can you do about this if simply removing the width&height check will not work? The major thing is checking initialization commands against ones from a library that's known-good and works for your display, then using them. Order of the commands will be different and that's okay. After you do this, the display should work. Next step would be to contribute your changes to this repository so that everyone interested would be able to use them, and as such, make the world a bit of a brighter place =)
What you can also do is use the overlay (that seems to actually work for you), not enable fbcp and instead use the luma's linux-framebuffer support to draw to the /dev/fb1 device created by the display driver. If you don't want to play with adjusting init commands, this would probably be the easiest solution for you.

I figured out how to remove luma.lcd and make changes to a locally cloned copy (in /home/pi) and install that.
Just sharing here if anyone is curious
First part solution : sudo pip3 uninstall luma.lcd
Second part solution : sudo pip3 install -e /home/pi/luma.lcd
I just want to remove the check to see if anything is displayed, I am still looking at a white screen even though this command now runs.
python3 luma.examples/examples/3d_box.py --interface spi -d ili9486 --width 320 --height 240
If I put dtoverlay=waveshare32b:rotate=270 back into /boot/config.txt I get a black screen.
If I try to run
python3 luma.examples/examples/3d_box.py --interface spi --gpio-reset 25 --gpio-data-command 24 -d ili9486 --width 320 --height 240 --framebuffer-device /dev/fb1
I get
/usr/local/lib/python3.7/dist-packages/luma/core/interface/serial.py:200: RuntimeWarning: This channel is already in use, continuing anyway.  Use GPIO.setwarnings(False) to disable warnings.
  self._gpio.setup(pin, self._gpio.OUT)

I will share my efforts when I get this working :) I also understand why sanity checks are put into code, I will put them back once I get something displayed.

I figured out how to remove luma.lcd and make changes to a locally cloned copy (in /home/pi) and install that.
Just sharing here if anyone is curious
First part solution : sudo pip3 uninstall luma.lcd
Second part solution : sudo pip3 install -e /home/pi/luma.lcd
I just want to remove the check to see if anything is displayed, I am still looking at a white screen even though this command now runs.
python3 luma.examples/examples/3d_box.py --interface spi -d ili9486 --width 320 --height 240
If I put dtoverlay=waveshare32b:rotate=270 back into /boot/config.txt I get a black screen.
If I try to run
python3 luma.examples/examples/3d_box.py --interface spi --gpio-reset 25 --gpio-data-command 24 -d ili9486 --width 320 --height 240 --framebuffer-device /dev/fb1
I get
/usr/local/lib/python3.7/dist-packages/luma/core/interface/serial.py:200: RuntimeWarning: This channel is already in use, continuing anyway.  Use GPIO.setwarnings(False) to disable warnings.
  self._gpio.setup(pin, self._gpio.OUT)

I will share my efforts when I get this working :) I also understand why sanity checks are put into code, I will put them back once I get something displayed.

Finally got it running
python3 luma.examples/examples/3d_box.py --interface spi --gpio-reset 25 --gpio-data-command 24 -d linux_framebuffer --width 320 --height 240 --framebuffer-device /dev/fb1
No code changes required, just pass the correct parameters!

Finally got it running
python3 luma.examples/examples/3d_box.py --interface spi --gpio-reset 25 --gpio-data-command 24 -d linux_framebuffer --width 320 --height 240 --framebuffer-device /dev/fb1
No code changes required, just pass the correct parameters!

happy to hear that! It's not even clear to me that you need the SPI and GPIO parameters, I'd bet they're just unused, try removing them and my hunch is that it will work just as well.

happy to hear that! It's not even clear to me that you need the SPI and GPIO parameters, I'd bet they're just unused, try removing them and my hunch is that it will work just as well.

happy to hear that! It's not even clear to me that you need the SPI and GPIO parameters, I'd bet they're just unused, try removing them and my hunch is that it will work just as well.

happy to hear that! It's not even clear to me that you need the SPI and GPIO parameters, I'd bet they're just unused, try removing them and my hunch is that it will work just as well.

Just to confirm, this worked
python3 3d_box.py --display linux_framebuffer --width 320 --height 240 --framebuffer-device /dev/fb1

Just to confirm, this worked
python3 3d_box.py --display linux_framebuffer --width 320 --height 240 --framebuffer-device /dev/fb1


Ad/init cmd

The action that's created hasn't been tested and there are a few outstanding questions/comments on this document that should probably be resolved before this gets merged.

@anuraagnalluri the previous init command is now build, so I think this will change the behavior if they're not pinning their version.

@anuraagnalluri the previous init command is now build, so I think this will change the behavior if they're not pinning their version.

Right, this is what I want to make sure of. If they're pinning their version, we could be safe. Just want to make sure we're not releasing something in to their workflow with no prior warning as this won't be backwards compatible.
Will need a version increment + specific callout in our changelog, but haven't really looked in to how our release process works nor how they are consuming.


SNS topic support for the SQS transport

Breaking this out for now.

Wrong issue, this is still open.

@jeremydmiller - I would just like to find out about the nearness to delivery for this topic.
As you know from one of my other issues, I am currently using RabbitMq. Our systems, however, call for AWS platforms and thus, AWS SNS.
Since they rely on event publishing, which is currently not (fully) supported with the SQS-only implementation, and rabbit is so expensive to run in AWS, I am eagerly awating this feature! :) It would so amazing to see this come to a release soon.
Thanks.
Kurt


WeaviateIndex doesn't build when using NotionPageReader documents

  documents = NotionPageReader(integration_token=notion_token).load_data(page_ids=page_ids)

  index = GPTWeaviateIndex(
    documents,
    weaviate_client=weaviate_client
  )

This could also apply to other document types.
The issue is twofold: 1) doc_id is missing in the documents, 2) extra_info is there but is a map so weaviate can't store it
I worked around the issue by manually setting doc_id and extra_info for the nodes (using copy pasted library code)
  index_struct = index.index_struct_cls(class_prefix=index.class_prefix)

  # Fix for doc_id issue
  for doc in documents:
    doc.embedding = OpenAIEmbedding(model = EMBEDDING_MODEL).get_text_embedding(doc.get_text())

    # Manually do this
    nodes = index._get_nodes_from_document(doc, index._text_splitter)
    for n in nodes:
      n.doc_id = doc.extra_info["page_id"]
      n.extra_info = ""
      WeaviateNode.from_gpt_index(weaviate_client, n, index_struct.get_class_prefix())

Oh interesting. Thanks for surfacing, will take a look later today!

I figured out the issue - basically when adding the extra_info field to Nodes, it broke our Weaviate serializer (since Weaviate can't serialize an arbitrary dict). I'll json-serialize extra_info as a string for now and deserialize it.

Awesome thank you! Amazingly fast response.
You may also encounter issues with doc_id being null. When I set node.extra_info = "" I still got errors about the doc_id.

Ah could you post stack trace of doc_id?

No stack trace as the error is swallowed, but it does log this, which I believe is being thrown by Weaviate at the WeaviateNode.from_gpt_index call.
{'error': [{'message': "invalid string property 'doc_id' on class 'Gpt_Index_5469264052329293341_Node': not a string, but <nil>"}]}

This should repro with this:
  documents = NotionPageReader(integration_token=notion_token).load_data(page_ids=page_ids)

  index = GPTWeaviateIndex(
    documents,
    weaviate_client=weaviate_client
  )

Possibly specific to Notion docs as they don't set the doc_id. However, I noticed that even if you set the doc_id in the Document, it's still explicitly not set at the Node level in _get_nodes_from_document:
(https://github.com/jerryjliu/gpt_index/blob/593e95ff1b329275ad78079608b742df9fa95b3c/gpt_index/indices/base.py#L209)
            node = Node(
                text=text_chunk,
                index=start_idx + i,
                ref_doc_id=document.get_doc_id(),
                embedding=document.embedding,
                extra_info=document.extra_info if self._include_extra_info else None,
                node_info=index_pos_info,
            )


Reduce size of distribution archive

@sezero said:
Here is some crude draft that reduces tarball size from 84.6 MB to 21.5 MB.
Not build-tested after a make dist.

Fixed by https://github.com/libsdl-org/SDL_image/commit/d67e9ed513f6061e5791c21a7d22f1f074d587fa


Add check to ensure TSA algo type exists

Pull Request Description
What issue does this change request address? (Use "#" before the issue to link it, i.e., #42.)
Refs #1443
What are the significant changes in functionality due to this change request?
We want to make sure that a TSA algorithm is available and specified for any given SythethicHistory ROM. This adds a check to make sure a TSA algo is found in the input specification and produces IOError if not found.

For Change Control Board: Change Request Review
The following review must be completed by an authorized member of the Change Control Board.

[x] 1. Review all computer code.
[x] 2. If any changes occur to the input syntax, there must be an accompanying change to the user manual and xsd schema. If the input syntax change deprecates existing input files, a conversion script needs to be added (see Conversion Scripts).
[x] 3. Make sure the Python code and commenting standards are respected (camelBack, etc.) - See on the wiki for details.
[ ] 4. Automated Tests should pass, including run_tests, pylint, manual building and xsd tests. If there are changes to Simulation.py or JobHandler.py the qsub tests must pass.
[x] 5. If significant functionality is added, there must  be tests added to check this. Tests should cover all possible options.  Multiple short tests are preferred over one large test. If new development on the internal JobHandler parallel system is performed, a cluster test must be added setting, in  XML block, the node <internalParallel> to True.
[x] 6. If the change modifies or adds a requirement or a requirement based test case, the Change Control Board's Chair or designee also needs to approve the change.  The requirements and the requirements test shall be in sync.
[x] 7. The merge request must reference an issue.  If the issue is closed, the issue close checklist shall be done.
[x] 8. If an analytic test is changed/added is the the analytic documentation updated/added?
[x] 9. If any test used as a basis for documentation examples (currently found in raven/tests/framework/user_guide and raven/docs/workshop) have been changed, the associated documentation must be reviewed and assured the text matches the example.

@PaulTalbot-INL Is this okay to merge now? It actually ends up fixing a test that was passing before when it shouldn't have been.

Initializing a fresh test run

@PaulTalbot-INL I pushed one more minor change and re-ran all the tests for this PR. If you could give it a once over when you have a moment, that would be awesome.

@PaulTalbot-INL I pushed one more minor change and re-ran all the tests for this PR. If you could give it a once over when you have a moment, that would be awesome.

Checklist passes, code changes fine. Merging.

OpenSUSE 15 machine is down currently; remaining tests have passed.


[WIFI-3048] Chg: make uCentral deployment use general values + bunch of sets

This PR requires https://github.com/Telecominfraproject/Toolsmith/pull/145 to be merged as it depends on new files naming


Automations always shows as triggered on logbook, even if they should not.

Make sure you run the latest version before reporting an issue. Feature requests should go in the forum: https://community.home-assistant.io/c/feature-requests
Home Assistant release (hass --version):
0.19.4
Component/platform:
automation: conditions
Description of problem:
With new action-embedded conditions, automations shows in logbook as "X was triggered", besides they finally doesn't activate because conditions.
Expected:
If finally an automation is not triggered because conditions, they shouldn't show on logbook
Problem-relevant configuration.yaml entries and steps to reproduce:
- alias: 'Movimiento -> Enciende lampara del salon'
  trigger:
    platform: state
    entity_id: binary_sensor.movyluz_3_0
    to: 'on'
  action:
    - condition: state
      entity_id: light.sofa
      state: 'off'
    - condition: state
      entity_id: light.mueble
      state: 'off'
    - condition: state
      entity_id: light.tiraled_2_0
      state: 'off'
    - condition: state
      entity_id: sensor.harmony
      state: 'OFF'
    - condition: or
      conditions:
        - condition: time
          after: '23:00'
        - condition: time
          before: '06:35'
    - service: scene.turn_on
      entity_id: scene.SubeLucesComedor

Traceback (if applicable):
16-05-12 13:10:06 INFO (Thread-2) [homeassistant.core] Bus:Handling <Event state_changed[L]: old_state=<state binary_sensor.movyluz_3_0=off; V_TRIPPED=off, icon=mdi:walk, friendly_name=Movim. Comedor, node_id=3, battery_level=0, sensor_class=motion, child_id=0, device=<IP_ADDRESS>:5003 @ 2016-05-12T13:04:49.286697+02:00>, new_state=<state binary_sensor.movyluz_3_0=on; V_TRIPPED=on, icon=mdi:walk, friendly_name=Movim. Comedor, node_id=3, battery_level=0, sensor_class=motion, child_id=0, device=<IP_ADDRESS>:5003 @ 2016-05-12T13:10:06.808596+02:00>, entity_id=binary_sensor.movyluz_3_0>
16-05-12 13:10:06 INFO (ThreadPool Worker 18) [homeassistant.components.automation] Executing Movimiento -> Enciende lampara del salon
16-05-12 13:10:06 INFO (ThreadPool Worker 18) [homeassistant.core] Bus:Handling <Event logbook_entry[L]: name=Movimiento -> Enciende lampara del salon, message=has been triggered, domain=automation>
16-05-12 13:10:06 INFO (ThreadPool Worker 18) [homeassistant.helpers.script] Script Movimiento -> Enciende lampara del salon: Running script
16-05-12 13:10:06 INFO (ThreadPool Worker 13) [homeassistant.core] Bus:Handling <Event state_changed[L]: old_state=<state group.movapert=off; order=6, entity_id=('binary_sensor.fibaro_system_fgms001_motion_sensor_sensor_2', 'binary_sensor.movyluz_3_0', 'binary_sensor.contacto_1_0', 'binary_sensor.contacto_10_0'), friendly_name=Mov./Apert. @ 2016-05-12T13:04:49.362239+02:00>, new_state=<state group.movapert=on; order=6, entity_id=('binary_sensor.fibaro_system_fgms001_motion_sensor_sensor_2', 'binary_sensor.movyluz_3_0', 'binary_sensor.contacto_1_0', 'binary_sensor.contacto_10_0'), friendly_name=Mov./Apert. @ 2016-05-12T13:10:06.894641+02:00>, entity_id=group.movapert>
16-05-12 13:10:06 INFO (ThreadPool Worker 18) [homeassistant.helpers.script] Script Movimiento -> Enciende lampara del salon: Test condition state: True
16-05-12 13:10:06 INFO (ThreadPool Worker 18) [homeassistant.helpers.script] Script Movimiento -> Enciende lampara del salon: Test condition state: True
16-05-12 13:10:06 INFO (ThreadPool Worker 18) [homeassistant.helpers.script] Script Movimiento -> Enciende lampara del salon: Test condition state: True
16-05-12 13:10:06 INFO (ThreadPool Worker 18) [homeassistant.helpers.script] Script Movimiento -> Enciende lampara del salon: Test condition state: True
16-05-12 13:10:07 INFO (ThreadPool Worker 18) [homeassistant.helpers.script] Script Movimiento -> Enciende lampara del salon: Test condition or: False

Additional info:
As you can see, time condition returns false, and automation should not trigger, but it shows on logbook

An automation rule consists of three parts: trigger, optional conditions and an action. An automation is triggered whenever the action gets called. It doesn't matter that the action consists of conditions, the only thing the automation component sees is that it called the action.
If you want to prevent the action from being called, use the condition field in the automation config (same level as trigger and action).


Passwords are generated in-place

The passwords are currently generated in-place in advance. If password-changing fails, then we have the new password in the pass store but still the old password on the website...
I have two strategies in mind:

keep generating the password in-place but carefully catch any exceptions and store the old password if necessary
only generate the password, change it and then store it into the pass-store

Somehow I prefer the 2nd solution but I'm not sure whether its possible/desirable to do so... What worries me about the 1st solution is: What happens if the user aborts somewhere - or the program crashes. Then we can never rewind even though an "error" occured...
This is related to #31 because it interferes with handling the exceptions. I'll provide a solution to this together with/after implementing the custom exceptions

Some password managers (notably pass) cannot generate passwords without storing them. This drove the current design.

(Therefore option 1 is probably good)

(Therefore option 1 is probably good)

and the only option I'm afraid... apart from using an external generator...


ANCM: Inprocess: ANCM does not use shutdownTimeLimit for inprocess mode

@jkotalik Please take a look at this issue.
We forced to shutdown the app with killing dotnet.exe if it does not shutdown by the time of shutdownTimeLimit  for the out-of-process mode.
However, in the inprocess mode, shutdownTimeLimit is not used.

Deploy app to IIS server
Set shutdownTimeLimit with 5 seconds and enable inprocess mode
Update the app with putting 20 seconds sleep time before the process is gone.
Send a request
Recycle IIS apppool which is used by the app

EXPECTED:
W3WP.exe is recycled after 5 seconds
ACTUAL:
W3WP.exe is recycled after 20 seconds

@jkotalik it proves we call managed layer shutdown which doe snot have timeout setting. We need to call managed shutdown in another thread and terminate the thread at timeout.

This issue is fixed


How do you get the network latency value tested by netperf

According to you document, I used netperf to do network performance benchmark test
below is command
netperf -H xxx.xxx.xxx.xxx -t TCP_RR

below is test result
MIGRATED TCP REQUEST/RESPONSE TEST from <IP_ADDRESS> (<IP_ADDRESS>) port 0 AF_INET to <IP_ADDRESS> () port 0 AF_INET : demo : first burst 0
Local /Remote
Socket Size   Request  Resp.   Elapsed  Trans.
Send   Recv   Size     Size    Time     Rate
bytes  Bytes  bytes    bytes   secs.    per sec

65536  87380  1        1       10.00    12507.44
65536  87380

When I read your dashboard, I found latency value required by netperl as is shown in below, but I can't found this value in netperf test result. Could you explain this? Thank you very much!

So the actual command is a little more subtle. You can find it in run_netperf.sh :
netperf -P 0 -t TCP_RR -H "$NETPERF_SERVER_HOST" -- -r 1,1 -o P50_LATENCY,P90_LATENCY,P99_LATENCY
This is what actually provides the information that we display on the dashboard.


Combining BindFactoryContract with abstract factories and parameters

As reported here, the following setup throws an exception when running ValidateCurrentScenes in Unity:
public interface IFoo {}

public class Foo : IFoo
{
    public Foo(int myInt) {}
}

public interface IFooFactory : IFactory<int, IFoo> {}

public class FooFactory : Factory<int, IFoo>, IFooFactory {}

public class FooInstaller : MonoInstaller
{
    public override void InstallBindings()
    {
        Container
            .BindFactoryContract<int, IFoo, IFooFactory, FooFactory>()
            .To<Foo>();
    }
}

as does a version which does not explicitly declare the pass-through classes:
public interface IFoo {}

public class Foo : IFoo
{
    public Foo(int myInt) {}
}

public class FooInstaller : MonoInstaller
{
    public override void InstallBindings()
    {
        Container
            .BindFactoryContract<int, IFoo, IFactory<int, IFoo>, Factory<int, IFoo>>()
            .To<Foo>();
    }
}

Surprisingly to me, the following attempt at a workaround did not succeed, also throwing an exception when validating the scene:
public interface IFoo {}

public class Foo : IFoo
{
    public Foo(int myInt) {}
}

public interface IFooFactory: IFactory<int, IFoo> {}

public class FooFactory : Factory<int, IFoo>, IFooFactory
{
    private DiContainer _container;

    public FooFactory(DiContainer container)
    {
        _container = container;
    }

    public new IFoo Create(int param)
    {
        return _container.Instantiate<Foo>(new List<object> { param });
    }
}

public class FooInstaller : MonoInstaller
{
    public override void InstallBindings()
    {
        Container
            .BindFactoryContract<int, IFoo, IFooFactory, FooFactory>();
    }
}

I may have oversimplified my example. Here is a more thorough example that I've verified does fail, and reflects the overall architecture of my system without getting super deep into the weeds. The thing to note is that what I'm trying to do is be able to call the factory Create at runtime, so even though it's claiming there's no int available to inject into Foo, that's to be expected, since I haven't actually tried to create one yet.
In this example, there is a Bar object in the scene, and a Foo prefab assigned to the FooTemplate on the FactoryInstaller, which is in turn assigned as an Installer on a SceneContext. Both the Foo and Bar prefabs have GameObjectContexts with their respective Installers assigned.
The example:
public interface IFooBehavior { }

public class FooBehavior : IFooBehavior
{
	public FooBehavior(int myInt) { }
}

public interface IFooContainer { }

public class FooContainer : IFooContainer
{
	public FooContainer(IFooBehavior fooBehavior) { }
}

public sealed class Foo : MonoBehaviour
{
	[Inject]
	public IFooContainer Container { get; private set; }
}

public class FooInstaller : MonoInstaller
{
	public override void InstallBindings()
	{
		Container
			.BindInterfacesAndSelfTo<FooContainer>()
			.AsSingle();

		Container
			.BindInterfacesAndSelfTo<FooBehavior>()
			.AsSingle();
	}
}

public interface IBarBehavior { }

public class BarBehavior : IBarBehavior
{
	public BarBehavior(IFactory<IFooBehavior> fooFactory) { }
}

public interface IBarContainer { }

public class BarContainer : IBarContainer
{
	public BarContainer(IBarBehavior barBehavior) { }
}

public sealed class Bar : MonoBehaviour
{
	[Inject]
	public IBarContainer Container { get; private set; }
}

public class BarInstaller : MonoInstaller
{
	public override void InstallBindings()
	{
		Container
			.BindInterfacesAndSelfTo<BarContainer>()
			.AsSingle();

		Container
			.BindInterfacesAndSelfTo<BarBehavior>()
			.AsSingle();
	}
}

public interface IFooFactory : IFactory<int, IFooBehavior> { }

public class FooFactory : Factory<int, IFooBehavior>, IFooFactory { }

public class FactoryInstaller : MonoInstaller
{
	public Foo FooTemplate;

	public override void InstallBindings()
	{
		Container
			.BindFactoryContract<int, IFooBehavior, IFooFactory, FooFactory>()
			.To<FooBehavior>()
			.FromSubContainerResolve()
			.ByNewPrefab<FooInstaller>(FooTemplate);
	}
}

The errors are:

ZenjectException: Unable to resolve type 'Zenject.IFactory`1[HexGame.Components.IFooBehavior]' while building object with type 'HexGame.Components.BarBehavior'.

 Could not find match for argument type 'System.Int32' when injecting into sub container installer 'HexGame.Installers.FooInstaller'

ModestTree.Assert.That (System.Boolean condition, System.String message, System.Object p1, System.Object p2) (at Assets/Scripts/HexGame.Zenject/Zenject/Internal/Assert.cs:345)
Zenject.SubContainerCreatorByNewPrefabWithParams.CreateTempContainer (System.Collections.Generic.List`1[T] args) (at Assets/Scripts/HexGame.Zenject/Zenject/Providers/SubContainerCreators/SubContainerCreatorByNewPrefabWithParams.cs:47)
Zenject.SubContainerCreatorByNewPrefabWithParams.CreateSubContainer (System.Collections.Generic.List`1[T] args, Zenject.InjectContext parentContext) (at Assets/Scripts/HexGame.Zenject/Zenject/Providers/SubContainerCreators/SubContainerCreatorByNewPrefabWithParams.cs:63)
Zenject.SubContainerDependencyProvider+c__Iterator0.MoveNext () (at Assets/Scripts/HexGame.Zenject/Zenject/Providers/SubContainerDependencyProvider.cs:47)
Zenject.IProviderExtensions.GetAllInstances (Zenject.IProvider creator, Zenject.InjectContext context, System.Collections.Generic.List`1[T] args) (at Assets/Scripts/HexGame.Zenject/Zenject/Providers/IProviderExtensions.cs:29)
Zenject.IProviderExtensions.GetInstance (Zenject.IProvider creator, Zenject.InjectContext context, System.Collections.Generic.List`1[T] args) (at Assets/Scripts/HexGame.Zenject/Zenject/Providers/IProviderExtensions.cs:75)
Zenject.PlaceholderFactory`1[TValue].Validate () (at Assets/Scripts/HexGame.Zenject/Zenject/Factories/PlaceholderFactory.cs:47)
Zenject.DiContainer.ValidateValidatables () (at Assets/Scripts/HexGame.Zenject/Zenject/Main/DiContainer.cs:291)
Zenject.SceneContext.Validate () (at Assets/Scripts/HexGame.Zenject/Zenject/Install/Contexts/SceneContext.cs:125)
Zenject.ZenUnityEditorUtil.ValidateCurrentSceneSetup () (at Assets/Scripts/HexGame.Zenject/Zenject.Editor/ZenUnityEditorUtil.cs:41)
UnityEngine.Debug:LogException(Exception)
ModestTree.Log:ErrorException(Exception) (at Assets/Scripts/HexGame.Zenject/Zenject/Internal/Log.cs:60)
Zenject.ZenUnityEditorUtil:ValidateCurrentSceneSetup() (at Assets/Scripts/HexGame.Zenject/Zenject.Editor/ZenUnityEditorUtil.cs:47)
Zenject.ZenMenuItems:m__1() (at Assets/Scripts/HexGame.Zenject/Zenject.Editor/ZenMenuItems.cs:326)
Zenject.ZenMenuItems:ValidateWrapper(Action) (at Assets/Scripts/HexGame.Zenject/Zenject.Editor/ZenMenuItems.cs:302)
Zenject.ZenMenuItems:ValidateCurrentSceneInternal() (at Assets/Scripts/HexGame.Zenject/Zenject.Editor/ZenMenuItems.cs:324)
Zenject.ZenMenuItems:ValidateCurrentScene() (at Assets/Scripts/HexGame.Zenject/Zenject.Editor/ZenMenuItems.cs:19)


Converting text to sound

Script Title - Script to convert text to sound
what will change - Adding script for that
Instructions

Create a new folder for your script and file/folder name should be appropriate.
Create a README.md in your folder for program Instructions
add requirements.txt if needed

Please add/delete options that are not relevant.

[x] Adding New Code

Programming Language

[x] Python

Happy Coding,

Hello please assign to me

Sure, go ahead :)

Can I also give it a shot?

Sorry @SaiSrichandra, Please look into other issues. This is already assigned to someone. You can create your own issue too for any practical application. Hope you understand :)


overall MAPE for STNorm on PEMS07

Thanks for sharing your excellent work!
When I estimate the performance of my work with BasicTS, I found the overall MAPE of STNorm on PEMS07 is extremely low to 8.00%, which is almost the MAPE of third horizon(7.98%). Also, I cannot reproduce the performance but get overall 8.85% for MAPE.
Is it possible that it happened to be a little digital mistake here?
Thanks again for your nice work!

After checking the training log, ST-Norm's overall MAPE is indeed problematic.
The correct number is 8.56%.
Here are the full training results:
2022-11-03 12:55:25,075 - easytorch-training - INFO - Epoch 95 / 100
2022-11-03 12:56:18,227 - easytorch-training - INFO - Result <train>: [train_time: 53.15 (s), lr: 5.00e-04, train_MAE: 18.2307, train_RMSE: 30.4419, train_MAPE: 0.0808 ]
2022-11-03 12:56:18,228 - easytorch-training - INFO - Start validation.
2022-11-03 12:56:22,276 - easytorch-training - INFO - Result <val>: [val_time: 4.05 (s), val_MAE: 19.8844, val_RMSE: 32.5213, val_MAPE: 0.0858]
2022-11-03 12:56:22,307 - easytorch-training - INFO - Checkpoint checkpoints/STNorm_100/8cb47c273d8e73528d8c8058998a0860/STNorm_best_val_MAE.pt saved
2022-11-03 12:56:26,161 - easytorch-training - INFO - Evaluate best model on test data for horizon 1, Test MAE: 17.2057, Test RMSE: 27.4493, Test MAPE: 0.0720
2022-11-03 12:56:26,164 - easytorch-training - INFO - Evaluate best model on test data for horizon 2, Test MAE: 18.3410, Test RMSE: 29.7707, Test MAPE: 0.0766
2022-11-03 12:56:26,166 - easytorch-training - INFO - Evaluate best model on test data for horizon 3, Test MAE: 19.1426, Test RMSE: 31.5075, Test MAPE: 0.0798
2022-11-03 12:56:26,169 - easytorch-training - INFO - Evaluate best model on test data for horizon 4, Test MAE: 19.7441, Test RMSE: 32.8585, Test MAPE: 0.0830
2022-11-03 12:56:26,172 - easytorch-training - INFO - Evaluate best model on test data for horizon 5, Test MAE: 20.2191, Test RMSE: 34.0451, Test MAPE: 0.0849
2022-11-03 12:56:26,174 - easytorch-training - INFO - Evaluate best model on test data for horizon 6, Test MAE: 20.6346, Test RMSE: 34.9905, Test MAPE: 0.0861
2022-11-03 12:56:26,177 - easytorch-training - INFO - Evaluate best model on test data for horizon 7, Test MAE: 21.0152, Test RMSE: 35.8100, Test MAPE: 0.0873
2022-11-03 12:56:26,180 - easytorch-training - INFO - Evaluate best model on test data for horizon 8, Test MAE: 21.3745, Test RMSE: 36.5240, Test MAPE: 0.0892
2022-11-03 12:56:26,182 - easytorch-training - INFO - Evaluate best model on test data for horizon 9, Test MAE: 21.6551, Test RMSE: 37.0576, Test MAPE: 0.0901
2022-11-03 12:56:26,185 - easytorch-training - INFO - Evaluate best model on test data for horizon 10, Test MAE: 21.9236, Test RMSE: 37.5292, Test MAPE: 0.0913
2022-11-03 12:56:26,187 - easytorch-training - INFO - Evaluate best model on test data for horizon 11, Test MAE: 22.1922, Test RMSE: 38.0034, Test MAPE: 0.0926
2022-11-03 12:56:26,190 - easytorch-training - INFO - Evaluate best model on test data for horizon 12, Test MAE: 22.5857, Test RMSE: 38.5200, Test MAPE: 0.0942
2022-11-03 12:56:26,218 - easytorch-training - INFO - Result <test>: [test_time: 3.91 (s), test_MAE: 20.5028, test_RMSE: 34.6671, test_MAPE: 0.0856]
2022-11-03 12:56:26,243 - easytorch-training - INFO - Checkpoint checkpoints/STNorm_100/8cb47c273d8e73528d8c8058998a0860/STNorm_095.pt saved

Thanks very much for reporting this issue, and I will update the results.

The results have been updated in the commit 08997c2.

Thanks for your quick response!
I also found Graph WaveNet's MAPE at Horizon 6 (8.24%) is extremely low(even smaller than which at Horizon 3 for 8.63%).
There might be similar problem for DCRNN's MAPE at Horizon 3 (6.86%), which is better than MTGNN's performance(6.89%). However, DCRNN's performance is much worse than MTGNN under other Horizons. I can't reproduce it either.

After checking the training log, ST-Norm's overall MAPE is indeed problematic. The correct number is 8.56%. Here are the full training results:
2022-11-03 12:55:25,075 - easytorch-training - INFO - Epoch 95 / 100
2022-11-03 12:56:18,227 - easytorch-training - INFO - Result <train>: [train_time: 53.15 (s), lr: 5.00e-04, train_MAE: 18.2307, train_RMSE: 30.4419, train_MAPE: 0.0808 ]
2022-11-03 12:56:18,228 - easytorch-training - INFO - Start validation.
2022-11-03 12:56:22,276 - easytorch-training - INFO - Result <val>: [val_time: 4.05 (s), val_MAE: 19.8844, val_RMSE: 32.5213, val_MAPE: 0.0858]
2022-11-03 12:56:22,307 - easytorch-training - INFO - Checkpoint checkpoints/STNorm_100/8cb47c273d8e73528d8c8058998a0860/STNorm_best_val_MAE.pt saved
2022-11-03 12:56:26,161 - easytorch-training - INFO - Evaluate best model on test data for horizon 1, Test MAE: 17.2057, Test RMSE: 27.4493, Test MAPE: 0.0720
2022-11-03 12:56:26,164 - easytorch-training - INFO - Evaluate best model on test data for horizon 2, Test MAE: 18.3410, Test RMSE: 29.7707, Test MAPE: 0.0766
2022-11-03 12:56:26,166 - easytorch-training - INFO - Evaluate best model on test data for horizon 3, Test MAE: 19.1426, Test RMSE: 31.5075, Test MAPE: 0.0798
2022-11-03 12:56:26,169 - easytorch-training - INFO - Evaluate best model on test data for horizon 4, Test MAE: 19.7441, Test RMSE: 32.8585, Test MAPE: 0.0830
2022-11-03 12:56:26,172 - easytorch-training - INFO - Evaluate best model on test data for horizon 5, Test MAE: 20.2191, Test RMSE: 34.0451, Test MAPE: 0.0849
2022-11-03 12:56:26,174 - easytorch-training - INFO - Evaluate best model on test data for horizon 6, Test MAE: 20.6346, Test RMSE: 34.9905, Test MAPE: 0.0861
2022-11-03 12:56:26,177 - easytorch-training - INFO - Evaluate best model on test data for horizon 7, Test MAE: 21.0152, Test RMSE: 35.8100, Test MAPE: 0.0873
2022-11-03 12:56:26,180 - easytorch-training - INFO - Evaluate best model on test data for horizon 8, Test MAE: 21.3745, Test RMSE: 36.5240, Test MAPE: 0.0892
2022-11-03 12:56:26,182 - easytorch-training - INFO - Evaluate best model on test data for horizon 9, Test MAE: 21.6551, Test RMSE: 37.0576, Test MAPE: 0.0901
2022-11-03 12:56:26,185 - easytorch-training - INFO - Evaluate best model on test data for horizon 10, Test MAE: 21.9236, Test RMSE: 37.5292, Test MAPE: 0.0913
2022-11-03 12:56:26,187 - easytorch-training - INFO - Evaluate best model on test data for horizon 11, Test MAE: 22.1922, Test RMSE: 38.0034, Test MAPE: 0.0926
2022-11-03 12:56:26,190 - easytorch-training - INFO - Evaluate best model on test data for horizon 12, Test MAE: 22.5857, Test RMSE: 38.5200, Test MAPE: 0.0942
2022-11-03 12:56:26,218 - easytorch-training - INFO - Result <test>: [test_time: 3.91 (s), test_MAE: 20.5028, test_RMSE: 34.6671, test_MAPE: 0.0856]
2022-11-03 12:56:26,243 - easytorch-training - INFO - Checkpoint checkpoints/STNorm_100/8cb47c273d8e73528d8c8058998a0860/STNorm_095.pt saved

Thanks very much for reporting this issue, and I will update the results.

Thanks for your quick response!
I also found Graph WaveNet's MAPE at Horizon 6 (8.24%) is extremely low(even smaller than which at Horizon 3 for 8.63%).
There might be similar problem for DCRNN's MAPE at Horizon 3 (6.86%), which is better than MTGNN's performance(6.89%). However, DCRNN's performance is much worse than MTGNN under other Horizons. I can't reproduce it either.

Thanks very much for your further suggestions.
The performance of Graph WaveNet at Horizon 6 should be 9.24%, instead of 8.24%.
Here is the training log:
2022-11-01 17:27:27,131 - easytorch-training - INFO - Epoch 180 / 200
2022-11-01 17:27:37,069 - easytorch-training - INFO - Result <train>: [train_time: 9.94 (s), lr: 2.50e-04, train_MAE: 14.0349, train_RMSE: 23.2042, train_MAPE: 0.0936]
2022-11-01 17:27:37,070 - easytorch-training - INFO - Start validation.
2022-11-01 17:27:38,466 - easytorch-training - INFO - Result <val>: [val_time: 1.40 (s), val_MAE: 14.6842, val_RMSE: 23.8004, val_MAPE: 0.1044]
2022-11-01 17:27:38,491 - easytorch-training - INFO - Checkpoint checkpoints/GraphWaveNet_200/983887ce40ff7d9ff3cf94dddf1cd557/GraphWaveNet_best_val_MAE.pt saved
2022-11-01 17:27:39,932 - easytorch-training - INFO - Evaluate best model on test data for horizon 1, Test MAE: 12.4887, Test RMSE: 19.5742, Test MAPE: 0.0803
2022-11-01 17:27:39,933 - easytorch-training - INFO - Evaluate best model on test data for horizon 2, Test MAE: 13.0537, Test RMSE: 20.7378, Test MAPE: 0.0835
2022-11-01 17:27:39,934 - easytorch-training - INFO - Evaluate best model on test data for horizon 3, Test MAE: 13.5009, Test RMSE: 21.6306, Test MAPE: 0.0863
2022-11-01 17:27:39,935 - easytorch-training - INFO - Evaluate best model on test data for horizon 4, Test MAE: 13.8442, Test RMSE: 22.3552, Test MAPE: 0.0882
2022-11-01 17:27:39,936 - easytorch-training - INFO - Evaluate best model on test data for horizon 5, Test MAE: 14.1508, Test RMSE: 22.9266, Test MAPE: 0.0902
2022-11-01 17:27:39,937 - easytorch-training - INFO - Evaluate best model on test data for horizon 6, Test MAE: 14.4118, Test RMSE: 23.4517, Test MAPE: 0.0924
2022-11-01 17:27:39,937 - easytorch-training - INFO - Evaluate best model on test data for horizon 7, Test MAE: 14.6821, Test RMSE: 23.9416, Test MAPE: 0.0937
2022-11-01 17:27:39,938 - easytorch-training - INFO - Evaluate best model on test data for horizon 8, Test MAE: 14.9098, Test RMSE: 24.3487, Test MAPE: 0.0953
2022-11-01 17:27:39,939 - easytorch-training - INFO - Evaluate best model on test data for horizon 9, Test MAE: 15.1343, Test RMSE: 24.6971, Test MAPE: 0.0969
2022-11-01 17:27:39,940 - easytorch-training - INFO - Evaluate best model on test data for horizon 10, Test MAE: 15.3347, Test RMSE: 25.0472, Test MAPE: 0.0983
2022-11-01 17:27:39,941 - easytorch-training - INFO - Evaluate best model on test data for horizon 11, Test MAE: 15.5267, Test RMSE: 25.3649, Test MAPE: 0.0992
2022-11-01 17:27:39,942 - easytorch-training - INFO - Evaluate best model on test data for horizon 12, Test MAE: 15.8140, Test RMSE: 25.7723, Test MAPE: 0.1011
2022-11-01 17:27:39,946 - easytorch-training - INFO - Result <test>: [test_time: 1.45 (s), test_MAE: 14.4043, test_RMSE: 23.3941, test_MAPE: 0.0921]
2022-11-01 17:27:39,970 - easytorch-training - INFO - Checkpoint checkpoints/GraphWaveNet_200/983887ce40ff7d9ff3cf94dddf1cd557/GraphWaveNet_180.pt saved
2022-11-01 17:27:39,977 - easytorch-training - INFO - The estimated training finish time is 2022-11-01 17:31:55

The performance of DCRNN on METR-LA is correct. DCRNN outperforms Graph WaveNet and MTGNN at short-term horizons. Here is the training log:
2022-11-02 12:24:40,516 - easytorch-training - INFO - Epoch 94 / 100
2022-11-02 12:28:57,768 - easytorch-training - INFO - Result <train>: [train_time: 257.25 (s), lr: 1.00e-06, train_MAE: 2.7032, train_RMSE: 5.5099, train_MAPE: 0.0711]
2022-11-02 12:28:57,768 - easytorch-training - INFO - Start validation.
2022-11-02 12:29:06,105 - easytorch-training - INFO - Result <val>: [val_time: 8.34 (s), val_MAE: 2.7504, val_RMSE: 5.4149, val_MAPE: 0.0765]
2022-11-02 12:29:06,115 - easytorch-training - INFO - Checkpoint checkpoints/DCRNN_100/90729dd9a08d58d63e6c9cea877b0df1/DCRNN_best_val_MAE.pt saved
2022-11-02 12:29:21,572 - easytorch-training - INFO - Evaluate best model on test data for horizon 1, Test MAE: 2.1762, Test RMSE: 3.7696, Test MAPE: 0.0519
2022-11-02 12:29:21,574 - easytorch-training - INFO - Evaluate best model on test data for horizon 2, Test MAE: 2.4718, Test RMSE: 4.5959, Test MAPE: 0.0615
2022-11-02 12:29:21,575 - easytorch-training - INFO - Evaluate best model on test data for horizon 3, Test MAE: 2.6695, Test RMSE: 5.1683, Test MAPE: 0.0686
2022-11-02 12:29:21,576 - easytorch-training - INFO - Evaluate best model on test data for horizon 4, Test MAE: 2.8237, Test RMSE: 5.6119, Test MAPE: 0.0745
2022-11-02 12:29:21,577 - easytorch-training - INFO - Evaluate best model on test data for horizon 5, Test MAE: 2.9533, Test RMSE: 5.9699, Test MAPE: 0.0797
2022-11-02 12:29:21,578 - easytorch-training - INFO - Evaluate best model on test data for horizon 6, Test MAE: 3.0657, Test RMSE: 6.2738, Test MAPE: 0.0842
2022-11-02 12:29:21,579 - easytorch-training - INFO - Evaluate best model on test data for horizon 7, Test MAE: 3.1666, Test RMSE: 6.5409, Test MAPE: 0.0882
2022-11-02 12:29:21,580 - easytorch-training - INFO - Evaluate best model on test data for horizon 8, Test MAE: 3.2560, Test RMSE: 6.7684, Test MAPE: 0.0917
2022-11-02 12:29:21,581 - easytorch-training - INFO - Evaluate best model on test data for horizon 9, Test MAE: 3.3370, Test RMSE: 6.9717, Test MAPE: 0.0950
2022-11-02 12:29:21,582 - easytorch-training - INFO - Evaluate best model on test data for horizon 10, Test MAE: 3.4080, Test RMSE: 7.1505, Test MAPE: 0.0978
2022-11-02 12:29:21,583 - easytorch-training - INFO - Evaluate best model on test data for horizon 11, Test MAE: 3.4761, Test RMSE: 7.3172, Test MAPE: 0.1006
2022-11-02 12:29:21,584 - easytorch-training - INFO - Evaluate best model on test data for horizon 12, Test MAE: 3.5426, Test RMSE: 7.4758, Test MAPE: 0.1032
2022-11-02 12:29:21,592 - easytorch-training - INFO - Result <test>: [test_time: 15.48 (s), test_MAE: 3.0289, test_RMSE: 6.2336, test_MAPE: 0.0831]
2022-11-02 12:29:21,607 - easytorch-training - INFO - Checkpoint checkpoints/DCRNN_100/90729dd9a08d58d63e6c9cea877b0df1/DCRNN_094.pt saved

I will update the results later.
Any further questions are welcome!

The results have been updated, and I have fixed a typo in DCRNN configs on metr-la and pems03 datasets. Now you should can reproduce the results.

Thanks for that! That would solved my problem.
Looking forward you checking the metrics at a proper time.

You are welcome. I have tested DCRNN on the mete-la dataset, and the results are the same the as mentioned above.
Do you have any other concerns about results and metrics?

I have no more questions for now. Thanks again for your excellent work!

 If you have any other questions, please feel free to create a new issue, and I will reply quickly.


Loses connection with HA.

Hello,
Looks like loopdeck loses connection with HA when my computer goes to sleep overnight. The solve is to restart the loopdeck application/device and everything works again.
Not sure if you have had this issue.

Adam B.

Hey @schmic any update on this bug fix?

Nope, no update. I'm currently not working on this project and I won't until they release their announced "big .NET update this summer.".

Heyo, i barley had connection issues in the past and since i updated to newest Loupedeck version i'm dealing with constant issues of the HA-Buttons not working anymore.
(Loupedeck on <IP_ADDRESS>53 and plugin on 0.8)
I'm aware you tried a fix for this in 0.8, but wanted to give you some feedback that, for me, it now work less stable then before.
Thanks for your efforts :)

Hello @St0l3nID ... could you please explain why you think there are connection problems or loose of connection? Are button presses for switches or lights not going to HA? Do they on the 2nd or 3rd try? Please explain a bit more how you experience the issues so I am able to narrow it down.
Please have a look at the logfile, probably attach it here, and check if there are any errors or messages. The readme explains where you can find the log for the new 6.0 version.

Ha.log
Sure, the log seems to call the actions, as far as i can tell it is perfectly connected in the beginning, however afer a while the buttons dont trigger the options of HA.
As you can see in the logs i try to switch lights in my kitchen (kuechenlicht_licht) and toggle scenes in my living room, but HA does simply not respond.
Unfortunatly in this log i didnt interact until it didnt work already.
Restarting loupedeck makes it work again, i cant further nail it down as to why it breaks though.
But once its broken the scene toggles and light  toggles dont work anymore and the light state is not reflected on the buttons anymore either.

Okay, there are definitely some reconnects in your log @St0l3nID ...
I'll see how this compares and behaves to mine. There is a new websocket client involved now and I think it behaves different to the old one.

i figured it out, its something within ha itself that seems to break it,  here is a video of me exploring it:
https://youtu.be/mdERYm2MBfI
Plus the log (for the second automation i trigger)
Ha.log

edit: Seems like specificly Scene:Create is killing it.

I think I can reproduce it, though I have no explanation what's happening or why ...
btw. you have a new follower 👀

Fixed by 0.8.1 - thanks for reporting and debugging.

Can confirm it worked^^
Thanks for the effort and the follow 👍


createIncrementalProgram is not emitting incrementally

Bug Report
🔎 Search Terms
createIncrementalProgram
🕗 Version
Typescript 4.5.4
💻 Code
import ts = require("typescript");

var options = {
    incremental: true,
    allowJs: true,
    sourceMap: true,
    tsBuildInfoFile: "buildInfoTestOutput",
};
let program = ts.createIncrementalProgram ({
    rootNames: ["./test1.ts"],
    options: options
});
let emitResult = program.emit();

🙁 Actual behavior
createIncrementalProgram is not emitting incrementally. Files are recompiled even though the source files are not changed. It is behaving same as createProgram.
🙂 Expected behavior
Should not recompile when no source files have been changed.

This is possibly related to https://github.com/microsoft/TypeScript/issues/47309, and possible mitigations are discussed in https://github.com/microsoft/TypeScript/issues/46677.

I am not sure what issue you are seeing. Can you share exact repro code, (Apart from your code, the source on which you are running this.) i have run this on simple repro and it works as expected.


There is no  jsoneditor-icons.png in bower_components/jsoneditor

After I do  Ember install  ember-jsoneditor
ember-cli show error
/Users/admin/www/idcos/hf-csa-web/tmp/funnel-input_base_path-VkNgXpSw.tmp/0/bower_components/jsoneditor/dist/img/jsoneditor-icons.png'

I can find a svg file in the directory

+1

Fixed by #9.


New target UNO_91H from RDA Microelectronics. (original: #7309)

Description
New target UNO_91H from RDA Microelectronics.
Pull request type
[ ] Fix
[ ] Refactor
[x] New target
[ ] Feature
[ ] Breaking change

greentea logs are attached below:
...

Softagram identified no dependency changes nor impact in this change (head commit 0b3a5b46560a)
Check the full report

Softagram Impact Report for pull/25 (head commit: 0b3a5b46560a)
Target: K64F
:star: Visual Overview
Changed elements and changed dependencies.

(Open in Softagram Desktop for full details)
:star: Change Impact
How the changed files are used by the rest of the project

(Open in Softagram Desktop for full details)
:page_facing_up: Full report

Permalink: Full report for pull/25

Give feedback of this report to<EMAIL_ADDRESS>


Publish @plone/blocks

The @plone/blocks package is currently not published on npm:
https://www.npmjs.com/search?q=%40plone%2Fblocks
While it is a pretty minimal package currently, being able to base new projects off the upcoming 'best practise' would be ideal

@JeffersonBledsoe I can do a release, but FYI this package falls into a special category that we haven't yet fully explored.
While exploring and advancing in the implementation of the PoC (Next.js, Remix, etc). I've realised of something: One can't shadow a component that it's transpiled and released. So I can't only see this solution: there will be some @plone/* packages that have to be released in the same way as Volto add-ons. Then in the build, they have to be "whitelisted" from the node_modules and the build should build them along with your code. In the future we can do this via @plone/registry, but right now you should do it via parameterizing your current build system.
@plone/slots will be in that list too.

https://github.com/plone/volto/releases/tag/plone-blocks-1.0.0-alpha.1
https://www.npmjs.com/package/@plone/blocks


Upgrading an application does not remove the existing version

Brief description of your issue
I'm torn between this being a bug, or needing to submit a feature request for an upgrade option like --remove-old.
I had OpenSCAP 1.3.4 installed on my device. winget upgrade showed that OpenSCAP 1.3.5 was available. I ran winget upgrade OpenSCAP and it ran as expected. I think checked the output of winget upgrade and it said I needed to upgrade OpenSCAP still.
I looked at my installed programs and I could see that I had both 1.3.4 and 1.3.5 installed. So even though I had installed the new version, winget will still report that I need to upgrade.
Some questions:

What's the desired/intended behaviour? Is this actually a bug?
Is this the responsibility of the application publisher to manage this behaviour?
Should winget automatically remove the old version if it's used to upgrade an application?
Is it possible for us to have winget manage this? (I'm happy to submit a feature request)

Steps to reproduce

Install version 1.3.4 from here: https://github.com/OpenSCAP/openscap/releases
Run winget upgrade, you should see an available update to 1.3.5 (possibly 1.3.6 as it was released 12h ago)
Run winget upgrade OpenSCAP to upgrade the app
Run winget upgrade, you'll see that OpenSCAP 1.3.4 is still in the upgrade list
Check your installed programs, you'll see that you have both 1.3.4 and 1.3.5 (or 1.3.6) installed
Uninstall 1.3.4, you'll see that OpenSCAP is removed from the output of winget upgrade

Expected behavior
Upgrading an application with winget should remove the old version.
Actual behavior
The new version was installed alongside the old version, so winget endlessly installs the new version if the administrator hasn't realised that it's installed the versions side-by-side.
Environment
Windows Package Manager v1.1.13405
Copyright (c) Microsoft Corporation. All rights reserved.

Windows: Windows.Desktop v10.0.22000.438
Package: Microsoft.DesktopAppInstaller v1.16.13405.0

Okay, I've thought about this a little more and I realise it would be really bad to remove previous versions, for example, you may need numerous and/or specific versions of Microsoft Visual C++ redistributables installed at the same time. Updating one shouldn't remove all previous versions.
Curious if this is still a suitable feature request though? --remove-previous-versions

We have a option for this in the manifests, upgradeBehavior: uninstallPrevious (it was designed for applications which required their previous version to be uninstalled before installing the new version). We could expose this as a command line option so the user could override the default behavior for situations like you describe.

I prefer the idea of adding a switch to --remove-previous-versions (this might be challenging to do the mapping to "all" previous versions). There are a few interesting user experience caveats here. The Windows Package Manager doesn't know about other programs that might have dependencies, so this would certainly be more of a "power user" feature.
Removing an earlier version of a package might also remove user customizations that don't get migrated to newer versions. I ran into this with Ultimaker.Cura.
The Windows Package Manager also doesn't have sufficient metadata to know if a package (.exe based) is going to install side by side or perform an upgrade. We've got some work on the backlog to test upgrade scenarios when we perform validation, that might be able to help in some of these cases.
I'm also still digging deeper into the dependency support scenarios. More thoughts / suggestions from the community are welcome.

What is OpenSCAP's upgrade behavior without using winget?
I think the least error-prone approach would be doing whatever the package defaults to when updating normally.
In fact, I thought that's what winget is already doing.
Is that not the case?

@github-account1111 it depends on whether the manifest upgrade behavior is the default or if it has been explicitly set to "uninstallPrevious".

@denelon default meaning it will install it beside the old version or do whatever the regular (non-winget) installer would do?

Yes.
If the installer is able to detect an existing version and "upgrade" it then the uninstallPrevious would not likely be necessary.
For packages that install as a "new" program even though they are a newer version, and they don't remove the previous versions, it's up to the manifest author to determine if the Windows Package Manager should remove the previous version.
In some cases, installers require you to remove a previous version before installing a newer one. In those cases, uninstallPrevious would be required.

I don't understand.
I asked if it's a or b, and your response was "yes" haha.
My guess is it's b?

In some cases, installers require you to remove a previous version before installing a newer one. In those cases, uninstallPrevious would be required.

What if it's required to keep a previous version?

For packages that install as a "new" program even though they are a newer version, and they don't remove the previous versions, it's up to the manifest author to determine if the Windows Package Manager should remove the previous version.

Unfortunately, there are many different scenarios. The default behavior for upgrade, unless the manifest has been written to change default behavior or the user has provided arguments to override is to simply run the installer after it has been downloaded and verified. Sometimes there are arguments that need to be passed to the installer for an upgrade (like Visual Studio). Those can also be provided in the manifest or via command line arguments.
It is possible in some cases that the user was intending to upgrade, but they used the "install" command, so in those cases the behavior may be incorrect since the upgrade keys in the manifest are ignored during the "install" flow. We're working on detecting if the package is already installed so that when a user runs "install", we can inform them and switch to the upgrade flow. If they really did mean to run install, they will be able to use "--force" to override our default behavior.
The Windows Package Manager is built to be very flexible. We're trying to meet developers where they are regardless of installer technology or behavior. That generally means, we need to define default behaviors and then provide mechanisms via the manifest, settings configured in the Windows Package Manager by the user (via winget settings), or by the user via command-line arguments to specify different behaviors.

Another problem with not removing the old version, is that if you want to upgrade all installed packages, it will detect the still existing old versions and install the newest version again, although that version is already installed as well. I am not sure to what absurdities you could let it devolve, but maybe one command could install the same package twice or more.
My idea would be to introduce a second flag/option in the package like "keepPrevious". If the package calls for "uninstallPrevious", it gets always removed, if it calls for "keepPrevious", it can only be explicitly deinstalled by the user and if it has neither, the choice lies with the user (maybe via a local/global config file, maybe via a parameter).
For now, there should be another check to see, whether the new version is actually needed to be installed or already there.

If the user attempts to install and the "latest" available version is already installed, we would do nothing. The exception here is if a user specifies "--force" we would run the install again.

Sounds like a reasonable way. That could solve the most annoying part of the issue. On an older machine, I tried to keep up to date, I had like 18 packages that got reinstalled each time. And some of them take quite some time.

Curious if this is still a suitable feature request though? --remove-previous-versions

Although it won't remove all previous versions, in the latest pre-release there is an experimental feature for --uninstall-previous which will uninstall one previous version before installing the new version

We've been working on the side-by-side scenarios. You could try the latest release and enable the experimental feature on:

https://github.com/microsoft/winget-cli/releases/tag/v1.8.924-preview

Please provide feedback at:

https://github.com/microsoft/winget-cli/discussions/4281

WinGet 1.8 now supports targeting a specific version for uninstalling previous versions.


Add support for include files

Can you update the README to include how this might be used?

Documentation is in place.


updates rules to add reference to wizard(s)

Hi there,
I suggest adding 'wizard(s)' as a term to create a warning about.
Thanks!

Thanks for contributing this years ago, but Joblint is now a dead project and is being archived


Add detailed LetsEncrypt instructions.

We'll likely want to follow this up with:
(1) Making Let's Encrypt the default instructions for getting a cert.
(2) Moving the "how to get an SSL cert" discussion to its own page.
(3) Eliminating the duplicate content in zulip.org/server.html
But this should definitely be an improvement!

@sharmaeklavya2 can you review this?

Looks good to me.


Helm release of Kubernetes job times out when the Job completes successfully

Output of helm version:
version.BuildInfo{Version:"v3.5.3", GitCommit:"041ce5a2c17a58be0fcd5f5e16fb3e7e95fea622", GitTreeState:"dirty", GoVersion:"go1.15.8"}
Output of kubectl version:
Client Version: version.Info{Major:"1", Minor:"20", GitVersion:"v1.20.5", GitCommit:"6b1d87acf3c8253c123756b9e61dac642678305f", GitTreeState:"clean", BuildDate:"2021-03-18T01:10:43Z", GoVersion:"go1.15.8", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.14", GitCommit:"5de7fd1f9555368a86eb0f8f664dc58055c17269", GitTreeState:"clean", BuildDate:"2021-01-18T09:31:01Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
Cloud Provider/Platform (AKS, GKE, Minikube etc.): AKS
Description

I have a helm chart that deploys a flyway migration as a kubernetes job. The chart is in local file system directory ./db-migration.

apiVersion: batch/v1
kind: Job
metadata:
  labels:
    aadpodidbinding: key-vault-identity
  annotations:
    kubernetes.io/change-cause: {{ .Values.change_cause }}
  name: {{ .Values.application_name }}-database-migration-{{ .Values.version }}
spec:
  activeDeadlineSeconds: 420
  backoffLimit: 0
  template:
    metadata:
      labels:
        aadpodidbinding: key-vault-identity
    spec:
      containers:
        - args:
            - migrate
          env:
            - name: MIGRATION_USER
              valueFrom:
                secretKeyRef:
                  key: username
                  name: database-schema-admin
            - name: MIGRATION_PASSWORD
              valueFrom:
                secretKeyRef:
                  key: password
                  name: database-schema-admin
          image: "{{ .Values.environment }}{{ .Values.application_name | replace "-" "" }}.azurecr.io/{{ .Values.application_name }}-flyway-with-test-data:{{ .Values.version }}"
          imagePullPolicy: Always
          name: flyway-migration
          volumeMounts:
            - mountPath: /flyway/conf
              name: flyway-config
            - mountPath: /mnt/secrets-store
              name: secrets-store-inline
              readOnly: true
      restartPolicy: Never
      volumes:
        - configMap:
            name: {{ .Values.application_name }}-flyway-conf
          name: flyway-config
        - csi:
            driver: secrets-store.csi.k8s.io
            readOnly: true
            volumeAttributes:
              secretProviderClass: azure-kvname
          name: secrets-store-inline

I install the release with the following command:

helm upgrade --install db-migration-release db-migration \
    --namespace migration-namespace \
    --set version=<IP_ADDRESS> \
    --set application_name=test \
    --set environment=test \
    --set change_cause=test \
    --wait --wait-for-jobs --cleanup-on-fail --timeout 7m10s

The migration runs to completion. The Job/Pod has status Complete.

Expected result
The helm release finishes successfully shortly after the Job completes.
Actual result
The helm release waits for the full timeout duration and finishes with failure due to timeout. Resources are not cleaned up after the failed release.

Hi @leiter-jakab. It is unclear to me from the ticket whether the job succeeded in the time allotted or not. I understand that it entered the Complete state. When did that occur?
As far as resources not cleaning up after the release failing, there are several flags which can assist in that matter. But by default Helm leaves the cluster in that state so the user can step in and determine what caused the release to fail. Most notable example being the --atomic flag.
Can you be more specific about why the release failed? Perhaps some --debug logs may help determine why the release failed to run to completion. Perhaps it was waiting on other resources to become ready for example.

Hi @bacongobbler, sorry for the confusion.

It is unclear to me from the ticket whether the job succeeded in the time allotted or not. I understand that it entered the Complete state. When did that occur?

The Kubernetes Job succeeded within the time allotted:

Job completed successfully.
Helm Release did not finish yet, but was running for the full timeout duration, after which it finished with a timout.

As far as resources not cleaning up after the release failing, there are several flags which can assist in that matter. But by default Helm leaves the cluster in that state so the user can step in and determine what caused the release to fail. Most notable example being the --atomic flag.

I used the --wait, --wait-for-jobs, and --cleanup-on-fail flags.

Am I correct in my expectation that this should remove resources after a failed Release?
Is this equivalent to using --atomic in combination with --wait-for-jobs?

Can you be more specific about why the release failed? Perhaps some --debug logs may help determine why the release failed to run to completion. Perhaps it was waiting on other resources to become ready for example.

Using the --debug flag I got the following logs:
history.go:56: [debug] getting history for release database-migration
install.go:173: [debug] Original chart version: ""
install.go:190: [debug] CHART PATH: /apps/database-migration

Release "database-migration" does not exist. Installing it now.
client.go:122: [debug] creating 2 resource(s)
wait.go:53: [debug] beginning wait for 2 resources with timeout of 2m0s
wait.go:194: [debug] Job is failed: moap-movie-database-migration/moap-movie-database-migration-<IP_ADDRESS>
.
.
.
wait.go:194: [debug] Job is failed: moap-movie-database-migration/moap-movie-database-migration-<IP_ADDRESS>
Error: timed out waiting for the condition
helm.go:81: [debug] timed out waiting for the condition

Additional info:

As described the job runs to completion without errors, but the release fails with timeout.
When I re-run the release with ugrade --install I see the following in the debug logs:

history.go:56: [debug] getting history for release database-migration
upgrade.go:123: [debug] preparing upgrade for database-migration
upgrade.go:131: [debug] performing update for database-migration
upgrade.go:303: [debug] creating upgraded release for database-migration
client.go:201: [debug] checking 2 resources for changes
client.go:464: [debug] Looks like there are no changes for ConfigMap "flyway-conf"
client.go:464: [debug] Looks like there are no changes for Job "database-migration-<IP_ADDRESS>"
wait.go:53: [debug] beginning wait for 2 resources with timeout of 40s
wait.go:194: [debug] Job is failed: database-migration/database-migration-<IP_ADDRESS>
.
.
.
upgrade.go:369: [debug] warning: Upgrade "database-migration" failed: timed out waiting for the condition
Error: UPGRADE FAILED: timed out waiting for the condition
helm.go:81: [debug] timed out waiting for the condition
UPGRADE FAILED
main.newUpgradeCmd.func2
	/home/circleci/helm.sh/helm/cmd/helm/upgrade.go:157
github.com/spf13/cobra.(*Command).execute
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:850
github.com/spf13/cobra.(*Command).ExecuteC
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:958
github.com/spf13/cobra.(*Command).Execute
	/go/pkg/mod/github.com/spf13/cobra@v1.1.1/command.go:895
main.main
	/home/circleci/helm.sh/helm/cmd/helm/helm.go:80
runtime.main
	/usr/local/go/src/runtime/proc.go:204
runtime.goexit
	/usr/local/go/src/runtime/asm_amd64.s:1374

The 2 resources (i.e. the Job and ConfigMap) are recognized as up by Helm as needing no changes. Despite this Helm goes into the status polling loop.

I believe this is because of BackoffLimit=0 (see https://github.com/helm/helm/issues/9637)

bingo. Yes this is because the backofflimit is set to zero.
Closing as a duplicate of #9312 which will be fixe din Helm 3.6.


Implement tokio support

Adds dependencies tokio + futures behind 'tokio' feature.
Implements tokio::UnixSeqpacketConn and tokio::UnixSeqpacketListener for supporting SEQPACKET sockets in Tokio.
Closes #1.

Much of this roughly follows how tokio UnixStream/Listeners (SOCK_STREAM) are set up, but attempts to adopt some of the existing API naming used by uds. However, I'm totally open to other naming conventions, or module organization for that matter.
My immediate use-case is for UnixSeqPacketConn, but have also introduced (perhaps a bit more roughly) the Listener type for the sake of some sort of completeness.

I'll add notes to the README and determine what needs to be done to satisfy OSX CI, but any interim reviews would be appreciated.

I went ahead and rebased the shutdown change (4d6c2c9) to go before everything else, resulting in a force-push. However, I left the existing changeset otherwise in-tact so that the subsequent fixup commits could be reviewed.

Do you plan to add more methods?

I think it might be best to leave that out for this initial pass. Not trying to push work onto you, but honestly you would probably get the tokio sockets up to parity a lot faster than I could. There's still the larger question of what to do about AsyncRead/Write traits that this PR should nail down first. And... making CI happy :(

Forgot to say that the failing test is probably flaky or maybe apple changed something.

Went ahead and rebased again to clean things up since there are only one or two remaining items. But the latest changes were:

added shutdown tests for blocking, nonblocking, and tokio socket variants.
removed AsyncRead/Write impls
reworded MSRV to exclude mentioning tokio
removed test-util feature inclusion from tokio

Let me know your thoughts how what to do about RUSTFLAGS and tokio's macros feature, or if you want to punt and figure out what works best at a later time.
Sidenote: you may be right about CI flakiness. I see the latest commit c48d0af has passed, but I didn't make any explicit changes for that to happen..

@tormol any further thoughts? Tokio is also on the verge of releasing 0.3 (which will invariably resemble 1.0 more closely than 0.2) so that is something to consider.

THANK YOU.
Sorry for being so slow to respond again.
(I lacked motivation to think about code outside of work for a while, but kept thinking I'd do it in a few days).
Now that I'm likely finished looking for an apartment to buy that will hopefully change.)

No worries at all-- know that feeling all too well, and I actually just had my first child in September, so my OSS work has been pretty inconsistent as well. Glad to see the PR make it across the finish line though!

Congratulations!
I'm hoping to test this on NetBSD and IllumOS and release this weekend.

Den 16. oktober 2020 00:49:52 CEST, skrev Jon Magnuson<EMAIL_ADDRESS>
No worries at all-- know that feeling all too well, and I actually just
had my first child in September, so my OSS work has been pretty
inconsistent as well. Glad to see the PR make it across the finish line
though!
--
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub:
https://github.com/tormol/uds/pull/2#issuecomment-709630283

0.2.0 is now released.
When I tested on Illumos, the test binaries created by cargo test --features tokio failed to start due to a missing symbol. I've just noted that in a feature table I've added to the README, but it could just be because I haven't updated that VM (OmniOS) in a while.


Suggestion: support incremental refine/refineFurther with explicit parameter

Right now it is not possible to incrementally and explicitly refine, as far as I can tell.
This is something I find needing all the time (i want to append more constraints as data travels through my application, without discarding old constraints, or having to re-type them).
I've never written any real scala3 inlines, macros or such before, so I'm sure this is not the most efficient way of solving it, but I found the following useful - maybe it can give you some ideas on how it could be done in an elegant/proper way?:
  extension [Src](value: Src)
    inline def refineZIO[C](using inline constraint: Constraint[Src, C]): IO[String, Src :| C] =
      value.refineValidation[C].toZIO

  extension [Src, Cstr](value: Src :| Cstr)
    inline def refineFurtherZIO[C](using inline constraint: Constraint[Src, C]): IO[String, Src :| (Cstr & C)] =
      (value: Src).refineZIO[C].map(_.asInstanceOf[Src :| (Cstr & C)])
    inline def refineFurtherEither[C](using inline constraint: Constraint[Src, C]): Either[String, Src :| (Cstr & C)] =
      (value: Src).refineEither[C].map(_.asInstanceOf[Src :| (Cstr & C)])
    inline def refineFurtherOption[C](using inline constraint: Constraint[Src, C]): Option[Src :| (Cstr & C)] =
      (value: Src).refineOption[C].map(_.asInstanceOf[Src :| (Cstr & C)])
    inline def refineFurther[C](using inline constraint: Constraint[Src, C]): Src :| (Cstr & C) =
      (value: Src).refine[C].asInstanceOf[Src :| (Cstr & C)]

This allows me to further refine already refined types, with a simple syntax (replace my naming and impl with something you prefer!) and append more and more refinements as I go, without discarding the original one
example
for {
  x1           <- 0.refineZIO[Less[1] // Int :| Less[1]
  _            <- foo1(x1)
  _            <- bar1(x1)

/// data travels further through the application, many method calls, storage, read from storage, etc
  x2           <- x1.refineFurtherZIO[Less[2]] // Int :| (Less[1] & Less[2])
  _            <- foo2(x2)
  _            <- bar2(x2)

/// data travels further through the application, many method calls, storage, read from storage, etc
  x3           <- x2.refineFurtherZIO[Less[3]] // Int :| (Less[1] & Less[2] & Less[3])
  _            <- foo3(x3)
  _            <- bar3(x3)
} yield ()

If there is an existing way to append constraints like this that I'm not seeing, please let me know

Indeed, the ability to refine further while still keeping current refinements looks like an important feature to me.
I think there might be a better approach to solve this problem by adding a type "proxy" for IronType.
Before:
opaque type IronType[A, C] <: A = A
type :|[A, C] = IronType[A, C]

After:
opaque type IronTypeImpl[A, C] <: A = A

type IronType[A, C] = A match
  case IronType[A, current] => IronType[A, current & C]
  case _ => IronType[A, C]

type :|[A, C] = IronType[A, C]

This way, there will be no need to create a refine...Further function for each existing refinement method:
val x: Int :| Positive = 5.refine
val y: Int :| (Positive & Odd) = x.refine //(Int :| Positive) :| Odd <=> Int :| (Positive & Odd)

Source compatibility should not break and AFAIK (I need to check it) binary should also be ok since all these types desugar to A post compilation. TASTy compatibility might break however. I will have to test it.
Would this approach solve your problem or am I missing something?

Indeed, the ability to refine further while still keeping current refinements looks like an important feature to me.
I think there might be a better approach to solve this problem by adding a type "proxy" for IronType.
Before:
opaque type IronType[A, C] <: A = A
type :|[A, C] = IronType[A, C]

After:
opaque type IronTypeImpl[A, C] <: A = A

type IronType[A, C] = A match
  case IronType[A, current] => IronType[A, current & C]
  case _ => IronType[A, C]

type :|[A, C] = IronType[A, C]

This way, there will be no need to create a refine...Further function for each existing refinement method:
val x: Int :| Positive = 5.refine
val y: Int :| (Positive & Odd) = x.refine //(Int :| Positive) :| Odd <=> Int :| (Positive & Odd)

Source compatibility should not break and AFAIK (I need to check it) binary should also be ok since all these types desugar to A post compilation. TASTy compatibility might break however. I will have to test it.
Would this approach solve your problem or am I missing something?

I guess I have two goals:

being able to specify the refinement explicitly (instead of inferred through a cast/assignment like your example)
Ensuring the runtime refinement cost/test is only executed for the additionally refined condition

goal 1 is the most important (since I will be creating domain specific constraints on my types, for example proofs that domain objects have reached certain states)

This should be possible with the type alias/proxy solution. I will also need to experiment to ensure the compiler infers the right types.

This should be possible with the type alias/proxy solution. I will also need to experiment to ensure the compiler infers the right types.
I will first finish #66 before experimenting but I'm pretty confident if will work.

Thanks! :)

After experimentations, it seems that this cannot be resolved using a match type because A and A :| B are not disjoincts, causing the compiler to fail. Implicit conversions work but are not powerful enough to emulate this behaviour.
I your first approach (using refineFurther) is probably the best at the moment.

The PR is almost done but I have some concerns about the name of some methods. For example how would you call a method that turns A :| C1 :| C2 to A :| (C1 & C2) and its reciprocal?

I need an example of a use case of refineFurthers methods for the documentation.


Add f32 support

Right now we only offer f64 support.
f64 take a large amount of space and are often overkill.
(Interestingly the gcd trick are possibly helping today.)
f32 without any compression would be a nice improvement

In contrast, support for Decimal would provide larger compression gains and provide a more explicit API for the user. It's more effort though.


add EPIK in Apps/Websites

still in beta  https://epik.com.pe/ created in vue js.
is a platform for local experiences in tourism

close due to in-activity.
if you hope push this PR again, you can push it as new PR. :)


execute-job/wrong-comment-on-pr/d0771f8a-6afd-483d-9876-4627089b24c9

PR for testing 991b17eade60d4b2969d21dfd43a5867754661bb

/canonical/self-hosted-runners/run-workflows 991b17eade60d4b2969d21dfd43a5867754661bb


Uncaught TypeError: Cannot read property 'file' of null

Describe the bug
Using vue-filepond with the pdf preview plugin, sometimes the stacktraced error occurs on route change.
To Reproduce

Uploading an image
Waiting for it to succeed
Navigating away from the current route
Going back to upload view
Remove current image
Upload new image
Navigate away from current route

Expected behavior
No console TypeError.
Stacktraces
Uncaught TypeError: Cannot read property 'file' of null
    at filepond-plugin-pdf-preview.js:181
    at filepond.js:1445
    at filepond.js:1096
    at Array.forEach (<anonymous>)
    at Object._write (filepond.js:1095)
    at filepond.js:1125
    at Array.forEach (<anonymous>)
    at Object._write (filepond.js:1123)
    at filepond.js:1125
    at Array.forEach (<anonymous>)

Information about your project:
Windows 10
Google Chrome - Version 89.0.4389.90 (64-Bit)
"filepond": "^4.21.1"
"filepond-plugin-file-encode": "^2.1.9"
"filepond-plugin-image-preview": "^4.6.4"
"filepond-plugin-file-validate-type": "^1.2.6"
"filepond-plugin-image-transform": "^3.7.6"
"filepond-plugin-pdf-preview": "^1.0.2"
"filepond-plugin-file-validate-size": "^2.2.4"
"vue-filepond": "^6.0.3"

Additional context
I tried "resolving/catching" this error in the beforeDestroy() life-cycle hook of the component holding vue-filepond, but the this.$refs is already undefined at that point.

I can't reproduce it. Maybe is something in your Vue Project.
It happens when you upload a image or a pdf? Looks like is something that do Vue on the variable context when the route change.
Sure If we put a null check in the error line 181 Will be another null error elsewhere. The thing is know why It execute the method when the route change, maybe It shouldn't at this point, and why is null because there are a file on it, isn't it?
Anyway I can’t reproduce it, I think is something that should be fixed in your project environment, If you find a reason and a robust fix for it we can change the library.

@Adri-Glez
Happens to me when I upload a PDF
Using React

can confirm this issue using vue

It happens to me too. Hard to reproduce, but the issue exists. See attached error log.

Same here on svelte.

Same here on Vue + nuxt

@Adri-Glez looks like quite a few people have this issue - maybe reopen this?

same issue vue + nuxt 😕

issue still there ... when changing the route the error is triggered ... vue 2 (Nuxt) 😕


Unable to use @clerk/tanstack-start w/ Cloudflare deployment or local development

Preliminary Checks

[X] I have reviewed the documentation: https://clerk.com/docs

[X] I have searched for existing issues: https://github.com/clerk/javascript/issues

[X] I have not already reached out to Clerk support via email or Discord (if you have, no need to open an issue here)

[X] This issue is not a question, general help request, or anything other than a bug report directly related to Clerk. Please ask questions in our Discord community: https://clerk.com/discord.

Reproduction
https://github.com/pdlug/tanstack-start-clerk
Publishable key
pk_test_bWFzc2l2ZS10ZXJtaXRlLTY1LmNsZXJrLmFjY291bnRzLmRldiQ
Description
I'm using Tanstack Start with the @clerk/tanstack-start package and trying to get it to work when deploying to Cloudflare. It works great running locally with the vinxi dev server but I cannot get it to pick up the publishable and secret keys when running locally with Cloudflare's wrangler CLI or in an actual deployment even though I have them set.
Steps to reproduce:
See this minimal repo that reproduces the issue: https://github.com/pdlug/tanstack-start-clerk

Created a new Tanstack Start project using the instructions
Added @clerk/tanstack-start with config updates specified in the docs
Run local development server, visit http://localhost:3000 and see that Clerk auth and components work as expected
Build and use wrangler pages dev to preview the app locally. Loading the app you'll get a 500 error with @clerk/tanstack-start: Clerk: no secret key provided in the console.

I've added the CLERK_PUBLISHABLE_KEY and CLERK_SECRET_KEY to .dev.vars which is what wrangler uses to set environment variables running locally but that didn't work. I also set them as environment variables (secrets) in a Cloudflare deployment and deployed. I get the same error when running live in Cloudflare as I do locally.
It looks like I can pass the publishable and secret keys to createClerkHandler but I haven't figured out how to actually get those from the environment. Cloudflare exposes them on the context which is called when the handler runs so I'm thinking the issue may be that the code in @clerk/tanstack-start seems to rely on it being in import.meta.env at the time it's loaded.
Tanstack Start uses Vinxi with Nitro to abstract the webserver. My knowledge of both is rather limited.
Expected behavior:
Expected @clerk/tanstack-start to pick up the configured environment variables running in a Cloudflare deployment.
Actual behavior:
500 error with the console message:
✘ [ERROR] [nitro] [request error] [unhandled] @clerk/tanstack-start: Clerk: no secret key provided

Environment
System:
    OS: macOS 15.1
    CPU: (14) arm64 Apple M4 Pro
    Memory: 150.61 MB / 48.00 GB
    Shell: 5.9 - /bin/zsh
  Binaries:
    Node: 22.11.0 - ~/.asdf/installs/nodejs/22.11.0/bin/node
    npm: 10.9.0 - ~/.asdf/plugins/nodejs/shims/npm
    pnpm: 9.14.2 - ~/.asdf/installs/pnpm/9.14.2/bin/pnpm
    bun: 1.1.36 - ~/.bun/bin/bun
  Browsers:
    Chrome: 131.0.6778.86
    Safari: 18.1
  npmPackages:
    @clerk/tanstack-start: ^0.5.2 => 0.5.2
    @tanstack/react-router: ^1.82.2 => 1.82.2
    @tanstack/start: ^1.82.5 => 1.82.5
    @types/react: ^18.3.12 => 18.3.12
    @types/react-dom: ^18.3.1 => 18.3.1
    @vitejs/plugin-react: ^4.3.3 => 4.3.3
    prettier: ^3.3.3 => 3.3.3
    react: ^18.3.1 => 18.3.1
    react-dom: ^18.3.1 => 18.3.1
    typescript: ^5.6.3 => 5.6.3
    unenv: ^1.10.0 => 1.10.0
    vinxi: ^0.4.3 => 0.4.3
    wrangler: ^3.89.0 => 3.89.0

I can confirm that the SDK is not able to read env vars in both CF deployment and local dev with wrangler. Thanks for reporting!