Protocol Decoding & Discussion

I've been working extensively on reverse engineering the IQ2020 firmware from a hex file found online, and have identified the RS485 address (0x1F) of their internet gateway and some of the functions. Followed you on Twitter, and happy to collaborate.
Commands from address 0x1F to IQ2020 at 0x01 have 2 bytes to identify the command, and optional additional bytes. I've identified the following so far.
Get Versions
0x01 0x00
[18:50:41][I][iq2020.component:097]: sendCmd_ 1F -> 01 Length:02 Operation:40 Data:01:00 Checksum:63
[18:50:41][I][iq2020.component:349]: readline_ Full Packet 01 -> 1F Length:17 Operation:80 RESP Data:01:00:57:52:34:2E:30:34:64:65:31:63:45:30:30:32:44:4B:34:2E:30:30:06 DataS:..WR4.04de1cE002DK4.00. Checksum:Valid 4D=4D

Get/Set Timestamps
0x02 0x4C Seconds ...
if Seconds >60, don't set, just return values
[18:50:41][I][iq2020.component:097]: sendCmd_ 1F -> 01 Length:03 Operation:40 Data:02:4C:FF Checksum:B0
[18:50:41][I][iq2020.component:349]: readline_ Full Packet 01 -> 1F Length:0A Operation:80 RESP Data:02:4C:37:12:11:0B:08:D1:07:01 DataS:.L7....... Checksum:Valid C1=C1 SinceLast:0.3s

Get lots of data
0x02 0x55
[18:50:41][I][iq2020.component:097]: sendCmd_ 1F -> 01 Length:02 Operation:40 Data:02:55 Checksum:B9
[18:50:41][I][iq2020.component:349]: readline_ Full Packet 01 -> 1F Length:75 Operation:80 RESP Data:02:55:00:08:40:04:00:00:06:04:00:0A:06:20:72:13:00:20:1C:20:1C:20:1C:84:03:60:54:00:00:00:00:31:30:32:46:10:66:09:00:C0:1C:06:00:61:A5:D9:02:1D:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:00:21:A3:06:00:02:00:2C:A4:D9:02:00:00:00:00:00:00:00:00:31:30:30:46:31:30:32:46:78:00:07:01:07:01:00:00:00:00:00:00:00:00:00:00:00:00:3A:00:00:00:00:00<EMAIL_ADDRESS>r.. . . ...`T....102F.f......a............

Oh nice. Thanks. I spent a lot of the last 48 hours on this and made a lot of progress. You can see all of my protocol decoding work here. I got lights, temperature, jet pumps and more decoded and working.
I also made progress on the ESP32 integration and got lights and temperature working really well. I how have my Home Assistant automatically change the temperature based on electricity rates and recording the temperature graph.

I still have a few more days to go before it's something others can use. I have to work a bit more to get both Celsius and Fahrenheit modes working well. More importantly, the Spa Connection Kit polls for the data every minute and so, I just listen to that traffic, no need to add more traffic to the RS485 bus if I don't need to. I am coding it so the ESP32 will detect if a Spa Connection Kit device is already present and will not poll, but if it's not present it will do the polling itself.
Are you a Home Assistant user? I would go buy a M5Stack+RS485+Wires right away (it takes a while to deliver and it's cheap). If your not using Home Assistant, let me know what setup your looking to integrate with.
Also, if you have a music module on your Spa, let me know. I would absolutely love to get a traffic capture of the audio device. I would to impersonate it so I can control Home Assistant audio from the spa display,

@wolfson292 - Just tried your commands and yes, they do work. Just added them to the protocol document.

FYI for everyone, more progress this morning. I got a lot more features supported and exposed to Home Assistant.

0x02, 0x55 and 0x02, 0x56 have the same handler, but 0x56 includes additional fields. Everything up to the length of 0x55 is the same in 0x56 response.
To tell if the device is in C or F mode, you can check byte 15, bit 6, of the response (1 = C, 0 = F).

@wolfson292 What are C and F modes?

Excellent work, ordered the hardware and looking forward to try it out.
I am from Belgium and have a Hotspring Envoy 2017 with an ace saltwater system

Thanks @npinguin. Hopefully you can help me test and give me feedback.
Today I am working on getting jets that have multiple speeds working nicely in Home Assistant. I also need to add data polling for this ESP-Home component to work for people that don't have the SPA connection kit, but I should get the done in the next few days.

Great work! Following closely with much interest! Just the thing I have been looking to implement for a while now. Parts are on order & hopefully arriving soon.

Thanks @fricker-ben. Nice to hear this. Certainly motivating me to code. I can't wait to hear your feedback.

Thanks to the information provided by @wolfson292. I added the version string as a sensor.

I may have found the water heater wattage value. 3866W in the picture below. I am polling this every minute anyway and so, a good value for HA graphs.

Ok. I added the polling support long with a bunch more things and updated all the documentation. I think this is pretty much a v1.0 at this point, it's probably something most geeks/nerds can use.

Ok, one more feature. The integration will now show when the hot tub is heating in the climate user interface.

@wolfson292 What are C and F modes?
Oh... Celsius or Fahrenheit probably. I can see that when I read the temperature. However, good to know.

I was originally determining this like you were seeing if the temp string ended with F, but having the status bit seemed cleaner.
The third temperature is the heater outlet temperature. You'll see it spike when the heater turns on.
Your identification of these functions has allowed me to significantly improve my disassembly of the controller firmware, identifying many more global variables in the code for various statuses. Unsure on the legality of publicly sharing the current disassembly, but it's based off a version WR4.04 and DK4.00 hex file of the firmware for the controller and wireless display that was available on a public website. I was previously doing a MiTM setup on the RS422 interface with the wireless display, but this RS485 interface is far simpler and doesn't impact the ability to use the display itself.
Plan to upload my version of an IQ2020 component shortly to GitHub. Started on it before I found yours.

Hi @wolfson292. To be clear, I do not want to ever see or be exposed to 3rd party code and please don't post any details about other code or binary from any sources here or on any forum. API's and interfaces are ok. I do appreciate the enthusiasm and that your even capable of doing this, that is absolutely insane! But yes, please keep it legal and respect other's work.
In any case, yes, thanks for the information on exhaust temp. I will add the outlet temperature to my component right away and the C/F bit to the documentation.

I just added the heater outlet temperature to the integration. One more thing to graph!

Hi @wolfson292. To be clear, I do not want to ever see or be exposed to 3rd party code and please don't post any details about other code or binary from any sources here or on any forum. API's and interfaces are ok. I do appreciate the enthusiasm and that you are even capable of doing this, that is absolutely insane! But yes, please keep it legal and respect other's work.
In any case, yes, thanks for the information on exhaust temp. I will add the outlet temperature to my component right away and the C/F bit to the documentation.

Completely understand having worked on many commercial reverse engineering projects.
Identified a few more of the data fields from the 0x02, 0x55/0x56 command.
Data[35-38] Heater Total Runtime Seconds
Data[39-42] Jets 1 Total Runtime Seconds
Data[55-58] Jets 2 Total Runtime Seconds
Data[59-61] Jets 3 Total Runtime Seconds
Data[67-70] Lights Total Runtime Seconds
Data[73-76] Lifetime Runtime Seconds
Data[43-46] Lifetime Runtime Seconds (6 minutes higher than previous value, 1.5yrs, for my own Spa)
Data[47-50] Unknown Counter, 30 on my own Spa
Data[63-66] Unknown Seconds Timer, 0 on my own Spa
Data[63-66] Unknown Seconds Timer, 0 on my own Spa
Data[77-80] Unknown Seconds Timer Related to Jets 1, 0 on my own Spa
Data[81-84] Unknown Seconds Timer Related to Jets 2, 0 on my own Spa

Wow @wolfson292 that is amazing data! I will add these runtime sensors in the next few hours. I also am reassured you know this business.
By the way, if you ever find a command to control light brightness and colors, that would be a fun one. I could add the brightness control on the existing light. I can read the lights brightness (0 to 5) using 0x0256 but can't control it.
Any other tricks we can do we the spa would be great!!

I just added your counters to the C# DataViewer app. I index starting from the start of the packet.
int HeaterTotalRuntime = getIntFromByteArray(data, 40);
int Jets1TotalRuntime = getIntFromByteArray(data, 44);
int LifetimeRuntimeSeconds1 = getIntFromByteArray(data, 48);
int UnknownCounter1 = getIntFromByteArray(data, 52);
int Jets2TotalRuntime = getIntFromByteArray(data, 60);
int Jets3TotalRuntime = getIntFromByteArray(data, 64);
int UnknownCounter2 = getIntFromByteArray(data, 68);
int LightsTotalRuntime = getIntFromByteArray(data, 72);
int LifetimeRuntimeSeconds2 = getIntFromByteArray(data, 78);
int UnknownCounter3 = getIntFromByteArray(data, 82);
int UnknownCounter4 = getIntFromByteArray(data, 86);

These are my own values, very similar to yours.

Done. Total runtime sensors are now included. I just added the ones we know what they indicate.

I am starting to graph temperature with my electricity time-of-use rules since last night. I had a bad data point added during development, but in general, I like this.

@wolfson292. I just added the protocol decoding to get/set the current time. This time is also at the end of the 0x0256 response.
Get Current Time
--> 01 1F 40 024C
<-- 1F 01 80 024C3431011400D40701
                 SSMMHHDDMMYYYY
Encoded as:
SS:MM:HH Seconds (0 to 59), Minutes (0 to 59), Hours (0 to 24).
DD:MM:YYYY Days (1 to 31), Month (0 to 11), Year (2 byte Big-Endian).

Set Current Time
--> 01 1F 40 024C2C2B140405E807
<-- 1F 01 80 024C2C2B140405E80701

Same encoding as Get Current Time.

Just for kicks, I added a new "Set Current Time" to the Data Viewer to set the spa time to your computer's clock.

Would it be possible to transform the timers in seconds in  yyy/mm/dd hh:mm:ss in home assistant?

@npinguin I will look into that.

@npinguin I just added a new document with Home Assistant Sensor Templates. This allows you to change the seconds into any format you like. I provide a full example to show the following:

Hope that helps.

More data on my hot tub power usages over 2 days. Temperature is automatically adjusted. Where I live the prices are approximately:

Off-peak = 8c / KWh
Mid-peak = 14c / KWh
On-peak = 43c / KWh

So this makes a huge difference if I can heat only during off-peak. It's also good for the grid and the environment.

I just fixed the problem with controlling jets that have a low/high settings. Works great now.

Just installed it, so far so good. Filling up the tub to see if the pumps work etc.
just noticed that power on counter is not visible for me. Tried a new install but did not see the sensor

Good catch @npinguin . I just updated the configuration yaml file on the main page to add the power_on_counter value in the sensor section. That should work now.

Let let us know how it goes.

I received a private email from someone trying to use the RS485MAX module. They can send commands but not receive any. I think the module looks like this:

I just updated the code to support flow_control_pin in the iq2020 section of the configuration yaml.
iq2020:
   uart_id: SpaConnection
   flow_control_pin: GPIO0
   port: 1234

If you use this specific board, I think you need to set flow_control_pin to the DE GPIO pin. However, I will need confirmation that it work, I don't have this setup.

Filled up the tub and all is working perfectly!
Absolutely great work, very happy with this.
how did you get the software version from the spa?

Thanks @npinguin for reporting back. If you can, please let me know what your software version and spa model is. I will keep a running list of tested versions and models. Also, do you happen to have any of the extra modules? Music/ACE/Freshwater?
For the version, I used the following command given by @wolfson292 above.
--> 01 1F 40 0100
<-- 1F 01 80<PHONE_NUMBER>2E30346465316345303032444B342E303006

Thanks @npinguin for reporting back. If you can, please let me know what your software version and spa model are. I will keep a running list of tested versions and models. Also, do you happen to have any of the extra modules? Music/ACE/Freshwater?
For the version, I used the following command given by @wolfson292 above.
--> 01 1F 40 0100
<-- 1F 01 80<PHONE_NUMBER>2E30346465316345303032444B342E303006

It is a Hotspring Envoy 2017 with an ace system.
Regarding the command to get the software version, how can I execute it and display it in home assistant? Is it possible to add it as a sensor ?

Thanks. Yes, I have it as a sensor called 'version' already. Are you not seeing it?

No don’t see it, is also not listed in the yaml?

I just added it at the end of the yaml on the main page. Add it and let me know if it works.

Top, just added it and confirms that my firmware is old i guess

Wow. That is super impressive.

anyone experience with firmware upgrades of the tub and what it could bring ? If it ain't broken do not upgrade is the typical recommendation. But i like keeping things up to date

@npinguin  - Yes, getting the firmware change log for different versions would be great. I wonder if dealers have that.

@wolfson292 - I sent all 64k possible commands to the IQ2020 from 0x1F looked for responses. I made the following table and if you have any guesses as to what some of the unknown commands are, please let me know. Thanks!
0100 - Get version string
0109 - Change temperature
0241 - Unknown (02413C001E0000)
024C - Unknown (024C1B0C110300E80701)
0255 - Get status short
0256 - Get status long
0B02 - Jets 1 & Lights on/off - (0B028F)
0B03 - Jets 2 on/off - (0B038E)
0B04 - Jets 3? on/off - (0B0400)
0B07 - Jets 4? on/off - (0B0700)
0B1C - Summer Timer
0B1D - Spa Lock
0B1E - Temperature Lock
0B1F - Clean Cycle
0B20 - Unknown (0B2001)
0B27 - Unknown (0B278C)
1702 - Lights on/off
1705 - Read lights status
1900 - Unknown (190015)
1901 - Unknown Periodic Polling<PHONE_NUMBER>00000B0004010000)
1D07 - Unknown Periodic Polling (1D07FFFF)
1E02 - Freshwater Salt System - Set Power / Start Test
1E03 - Get FreshWater Salt Module Data

Just published YouTube videos on this integration:

I just put in more updates. When changing settings, the ESP32 will auto-retry a command 3 times at 1/5 second interval. This is needed since there is a tiny chance that two commands would collide on the bus or there is an error and the IQ2020 controller did not get the command.
Also, I added a new sensor with the salt system power level (0 is off and 10 is max power). Only useful if you have this system. I could also remotely change the setting, I am not sure what Home Assistant control I could use for it.
Lastly, if anyone has this document... that would be super useful.

Finally home after a week away. All plugged in & working nicely on my Hot Spring Sovereign!
Is there a way to set & view the current temperature in 0.5°C increments?

@fricker-ben - Yes, Fahrenheit is set in 1.0F increments and Celsius in 0.5C increments. I have tested Fahrenheit a lot more than Celsius. You need to change some _f_ to _c_ in the ESPHome device config. In the view below, hit +/- and it should change by 0.5. If it does not, let me know, I will work to fix.

I have changed the f to c, but when pushing the plus/minus button it's going in 1°C increments. Also the temp display only shows to 0 decimal places.
The heater outlet temp displays to 0.1°C resolution.

Ok. I will do some testing and fixing on this later today. This is an essential feature of this integration.

@fricker-ben I see the problem, working on it.

I have been testing the new version with the salt water, unfortunately it is not reusing the ace system interfaces which I was hoping it would. Would be great to have the ace system integrated, let me know what you need

@fricker-ben I see the problem, working on it.

Thank you for your very quick response!

@npinguin ACE uses completely different commands. I don't have a ACE system, however, I will open a new issue for you, just like I did for #4. If you feel up to it, you can send me traffic captures and I will add support for it. I will finish up with @fricker-ben  and get that done.

@fricker-ben I just put in a fix for Celsius for the client control. I just updated the main page. In the climate section of the ESP home configuration, add the line celsius: true like this.
climate:
  - platform: iq2020
    name: Temperature
    celsius: true

Since I just did the change in GitHub, you can force the ESP Home add-in to check for updates by adding refresh: 30s to the external_components section like this:
external_components:
  - source: github://ylianst/esp-iq2020
    refresh: 30s

Put in the two lines, update your ESP32 device over WIFI and it should start working. It's not just the .5 step, the min/max temperatures in Celsius are also a bit different and needed fixing.

Also, can I have your hot tub model, version string and what is any extra modules you have? ACE/Freshwater/Music? I am trying to keep track of what is working and what is not. Let me know if this fix works for you. Thanks!

@npinguin - Seems you like having access to a lot of sensors. I added instructions to add the WIFI signal strength sensors. Maybe you are interested in this.

@Ylianst - Thank you, the temperature set point now acts as expected.
It's a 2015 Hot Spring Sovereign, no extra modules.
Version: 1.08.D0172E002HL1.02

@npinguin - Seems you like having access to a lot of sensors. I added instructions to add the WIFI signal strength sensors. Maybe you are interested in this.

Thanks :-)
wifi signal is indeed poor waiting for my outdoor Unifi access point to improve the signal ;-)

Excellent. Thank you @fricker-ben. That was a really important fix and thank you for that version string, your the oldest version of the firmware I know about that it still works! This is excellent news. I was concerned that older firmwares would not work or need changes, but it seems not. Nice!!
@npinguin - I just added #6 for you. Let me know if you have any questions. I hope my instructions are clear and it should be simple.

@Ylianst - All seems to be working very well, although my life time runtime is displaying as approx 29.22 years (921453056 seconds), which I think it a little off! All other runtimes seem reasonable for the age of the Hot Tub.

@fricker-ben That is interesting. I will take note of this. The lifetime encoding is very basic with not much room for error, basically, the value is directly obtained from the hot tub. If you even want to investigate this further, you can send me a packet capture like #4 to see what is going on, but not something that is crucial.

@Ylianst - Certainly not crucial! I will dig out a windows laptop at some point & do a packet capture for you. Always interesting to see what's going on, especially on the different (much older in my case) firmware versions.

You would be the first one to provide a v1 firmware package capture... interesting to see what change have been made since.

I just installed the OEM Hot Springs connected spa kit on a 2023 HS Flair and am able to log into its hub and it does let me download several python scripts and driver files for accessing the hot tub serial interface.  Would that be helpful?
Looking at the logs, here are all the settings and sensors it seems to expose:
('compatibility_byte', 0),
('control_box_error_zc_signal', False),
('control_box_error_flow_signal_open', True),
('control_box_error_flow_error', False),
('control_box_error_probe_error', False),
('control_box_error_config_error', False),
('sys_state_temp_lock', True),
('sys_state_spa_lock', False),
('sys_state_jet_1_high', False),
('sys_state_jet_2_high', False),
('sys_state_clean', False),
('sys_state_summer_time_mode', False),
('sys_state_water_care', True),
('sys_state_navigator_sync', False),
('sys_state_jet_1_low', False),
('sys_state_jet_2_low', False),
('sys_state_circ', True),
('sys_state_blower', False),
('sys_state_jet_3_high', False),
('sys_state_heater', 0),
('peripheral_status_adv_lite', False),
('peripheral_status_sp_audio', False),
('peripheral_status_i_cast', False),
('peripheral_status_free_pad', False),
('peripheral_status_cd_player', False),
('peripheral_status_rheodyne', False),
('peripheral_status_sp_audio_2', False),
('peripheral_status_mp3', False),
('peripheral_status_video', False),
('peripheral_status_chlorine', True),
('peripheral_status_networking', True),
('peripheral_status_frog', False),
('peripheral_status_water_feature', False),
('peripheral_status_mz_light', False),
('peripheral_status_hpc', False),
('peripheral_status_two_way_audio', False),
('peripheral_status_ace_gen_3', False),
('peripheral_status_wireless_audio', False),
('lls_set_bits', 10),
('model_type', 8),
('output_configuration_multi_zone_exterior', False),
('output_configuration_2_pump_system', True),
('output_configuration_multi_zone_themes', False),
('output_configuration_3_pump_system', False),
('output_configuration_blower', False),
('output_configuration_heater_service_mode', True),
('output_configuration_blower_service_mode', False),
('output_configuration_three_pump_concurrency', False),
('pump_configuration', 241),
('feature_configuration_pump_1_speed', False),
('feature_configuration_pump_2_speed', True),
('feature_configuration_24_hour_operation', False),
('feature_configuration_display_blanking', True),
('feature_configuration_extended_auxillary_key', False),
('feature_configuration_degree_celcius', False),
('feature_configuration_temp_dsp_set', False),
('mz_light_configuration_zone_1', False),
('mz_light_configuration_zone_2', True),
('mz_light_configuration_zone_3', True),
('mz_light_configuration_zone_4', False),
('pump1_timeout', 7200),
('pump2_timeout', 7200),
('pump3_timeout', 7200),
('blower_timeout', 900),
('light_timeout', 21600),
('pump1_speed', 0),
('pump2_speed', 0),
('pump3_speed', 0),
('blower_speed', 0),
('high_limit_temperature', '101F'),
('heater_operation_seconds', 1770297),
('pump1_operation_seconds', 588704),
('time_in_service', 18416413),
('lost_line_counter', 9),
('stm_state', 0),
('spa_lock_state', 0),
('temp_lock_state', 1),
('clean_lock_state', 0),
('pump2_operation_seconds', 149370),
('pump3_operation_seconds', 0),
('blower_operation_seconds', 0),
('light_operation_seconds', 4366929),
('spa_state', 3),
('spa_light_state', 0),
('circ_pump_operation_seconds', 18416308),
('jet_1_low_operation_seconds', 232900),
('jet_2_low_operation_seconds', 0),
('ctrl_head_set_temperature', '100F'),
('ctrl_head_water_temperature', '101F'),
('voltage_l1', 122),
('voltage_heater', 251),
('voltage_l2', 251),
('voltage_jet3', 0),
('current_l1', 0),
('current_heater', 0),
('current_l2', 0),
('current_jet3', 0),
('power_l1', 0),
('power_heater', 62),
('power_l2', 18),
('power_jet3', 0),
('daily_clean_cycle', 1),
('filter_time_1', 60),
('filter_time_2', 30),
('hawk_status_econ', False),
('hawk_status_circ', False),
('pcb_temperature', 97),
('peripheral_current', 101),
('rtc_seconds', 49),
('rtc_minutes', 18),
('rtc_hours', 4),
('rtc_date', 17),
('rtc_month', 6),
('rtc_year', 2001),
('rtc_status', 1),
('lls_ready_flash', False),
('lls_power_and_ready_flash', False),
('lls_power_flash', False),
('lls_power_and_ready_on', True),
('lls_power_on', False),
('lls_power_and_ready_alter', False),
('lls_power_and_ready_ace_err', False),
('sys_state_light_intensity', 0),
('hawk_status_filter', 0)

'mz_POL_light_intensity': 0,
'mz_POL_color_state': 1,
'mz_Exterior_light_intensity': 0,
'mz_Accent_loop_speed': 0,
'mz_24_hour_light_status': 1,
'mz_Accent_wheel_status': 0,
'mz_Main_loop_speed': 0,
'mz_Main_wheel_status': 0,
'mz_Main_light_intensity': 0,
'mz_Accent_light_intensity': 0,
'mz_model': 0,
'mz_Exterior_color_state': 1,
'mz_system_status': 0,
'mz_Exterior_loop_speed': 0,
'mz_version': 0,
'mz_POL_wheel_status': 0,
'mz_Accent_color_state': 1,
'mz_POL_loop_speed': 0,
'mz_Main_color_state': 1,
'mz_Exterior_wheel_status': 0

'audio_radio_signal_strength': 0,
'audio_play_pause_status': 0,
'audio_power': 0,
'audio_wireless_channel': 1,
'audio_source_selection': 4,
'audio_balance': 0,
'audio_treble': 0,
'audio_bass': 0,
'audio_bluetooth_pairing': 0,
'audio_subwoofer_volume': 11,
'audio_volume': 25

@Ylianst Oh nice, I was seriously considering your diy approach but wasn't quite ready to commit to that as I primarily use Hubitat and haven't tinkered with home assistant too much.  But good to know I could use both, may still try that!
To log in to the hub, just find its IP address, and then login in a browser with the username "python" and password written on the bottom of the hub.  Once in, there is a file browser and you can download the python scripts used to connect the hub to the xbee spa dongle as well as to Watkins servers.  The Watkins stuff isn't even compiled and shows their api key in the code... so I bet you could even get your script to connect to their app if you really wanted.  The event and python logs are also pretty descriptive, which is where I got that channel listing in my earlier message.  The logging is set to debug by default and the cpu load can be pretty high on that thing, disabling debug logging might actually increase performance.
The reason I looked into the hub in the first place is that my connected spa kit was misconfigured by hot springs from the box and wouldn't connect...  They had set the autorun python script command as "dia.py and dia.yml" rather than "dia.py dia.yml" so it was looking for a config file called "and", failing, not installing the xbee driver, and rebooting the hub.  Pretty dumb... but I fixed that and it started loading the correct configuration and working correctly.

@rabidfurball Thanks for all that valuable information. Wow, this is so much more interesting than I ever expected. Since you are not using Home Assistant, one idea I was thinking of is at some point hosting a tiny web site on the ESP32 itself with the data and control. The ESP32 is very limited so, I would probably put CSS and JavaScript files on GitHub in this repo so ESP32 would not need to serve them. In any case, I will take a look at the hub, absolutely fascinating.

@rabidfurball Wow. I logged in to the hub and indeed, saw the logs being written just as you described. I was able to improve my protocol decoding a lot based on this new information. I could capture my on decoding vs the decoding in the log for the exact same packet and compare, it's wonderful. I will be adding a bunch more diagnostics sensors in the next few days.

I just added a bunch of new sensors to the integration. New run times, voltage, currents, power, temperature. The one thing I wonder is there is a "heater" voltage/current/power that is around 70 watts all the time for me. This is clearly not the main heater, but the official decoder does call it a "heater". I am not sure what that is. Is there a second small heater in the hot tub? Or is this a mislabeled sensor? In any case, many more values to look at. If you Jet 1 does not have a low setting, your Jet 1 low lifetime will be zero.
sensor:
    circulation_pump_total_runtime:
      name: Circulation Pump Runtime
    jet1_low_total_runtime:
      name: Jets 1 Low Runtime
    jet2_low_total_runtime:
      name: Jets 2 Low Runtime
    power_l1:
      name: Pump Power
    power_heater:
      name: Small Heater Power
    power_l2:
      name: Heater Power
    voltage_l1:
      name: Voltage L1
    voltage_heater:
      name: Voltage Heater
    voltage_l2:
      name: Voltage L2
    current_l1:
      name: Current L1
    current_heater:
      name: Current Heater
    current_l2:
      name: Current L2
    pcb_temperature:
      name: Controller Temperature

works nicely, just do not understand the flag for celcius. I get the pcb temperature in farenheit and the other temps remain as they are

Internally the PCB temperature is given in Farenheit, however, I have to look into this because I should try to give the temperature in the same units as the others, even if I have to convert it.

Ok. Just made the PCB sensor a _f_ or _c_ type of sensor, so you can configure it.
sensor:
    pcb_f_temperature:
      name: Controller Temperature

Top, unfortunately with the latest update of esphome, the configuration does not work/compole anymore

The latest ESP Home add-in has a breaking change. If you are getting an error around the ota area, you need the add the line - platform: esphome in that section.
Old:
ota:
  password: "xxxxxxxxxxxxxx"

New:
ota:
  - platform: esphome
    password: "xxxxxxxxxxxxxx"

Keep the password as-is, just add the platform and move the password line two spaces right. Let me know if that helps.

top working perfectly

Ok... I did it. I got the ESP32 to emulate the Music module and route buttons back to Home Assistant. I just got this working hours ago and I am sure I will evolve this a lot more. Instructions here, feedback appreciated.

Works perfectly, just had to switch power off and on

Thanks @npinguin. I just updated the instructions to mention that you need to reset the remote, there is a demonstration video here. That could help and does not require a full spa reset.

I just put in a major update for the audio module emulation. I lot more sensors now. The "audio_volume" sensor is now removed and the integration needs the following sections:
select:
number:
text:

Updated audio instructions here.

Just added one more sensor that track the logo lights (green and blue) in front of the hot tub. You can see information on this new sensor at the top of the extras page.

My hot tub lights button flips through three stages, high med and low brightness and can also change colors. Any idea how to control this?

@joshs85 If your talking about the green/blue logo, I can't control that, I can only tell you what it's showing currently. If you are talking about the lights within the hot tub that can be controlled by the remote, I can turn them on/off (which should work) and I can tell you exactly which light is what color and brightness, but I can't control it (yet). It's possible there is a command to control lights colors and brightness, but have not found this yet.

@joshs85 If your talking about the green/blue logo, I can't control that, I can only tell you what it's showing currently. If you are talking about the lights within the hot tub that can be controlled by the remote, I can turn them on/off (which should work) and I can tell you exactly which light is what color and brightness, but I can't control it (yet). It's possible there is a command to control lights colors and brightness, but have not found this yet.

Its the lights inside of the hot tub.  The switch in HA turns them on and off just fine.  There's a setting on the control panel for brightness within settings (also the light button on the control panel cycles through the brightness low, med, high + off).. Then you can also set the color in the settings.

Great work on this by the way.. I had hooked up an atom lite + the rs485 base several months ago and finally had time to start reversing this.. but i did one last search for it and you had already done it.. so I deployed it and it works really well.

@joshs85 Ha yes, for the inside lights, I can see the exact status of them. I have not created any sensors for them yet, but I could. For example, you could make it so that other lights around your house match the color of the lights in your hot tub. I can't control them, but I could add these sensors so you can see the values if you like. Feel free to open a new issue requesting these if needed.
Ha thanks. Once I found the RS485 pins and did confirm that it worked and got the basic packet format and checksum, but did go ahead and buy the spa connection kit so I could get this project moving forward. I don't plan on ever using the connection kit, but it was well worth it for the benefit of the community. ESP-Home is amazing, i used it for my Rheem water heater and that got me started. There are still unknowns on the protocol. I don't have a real packet capture of the music module and I wish I did, I had to make up some stuff and in scanning for all commands, I did find a few I still don't know what they do. However, the basics are working really well.

@joshs85 Ha yes, for the inside lights, I can see the exact status of them. I have not created any sensors for them yet, but I could. For example, you could make it so that other lights around your house match the color of the lights in your hot tub. I can't control them, but I could add these sensors so you can see the values if you like. Feel free to open a new issue requesting these if needed.
Ha thanks. Once I found the RS485 pins and did confirm that it worked and got the basic packet format and checksum, but did go ahead and buy the spa connection kit so I could get this project moving forward. I don't plan on ever using the connection kit, but it was well worth it for the benefit of the community. ESP-Home is amazing, i used it for my Rheem water heater and that got me started. There are still unknowns on the protocol. I don't have a real packet capture of the music module and I wish I did, I had to make up some stuff and in scanning for all commands, I did find a few I still don't know what they do. However, the basics are working really well.

Yeah, if you could add the sensors that'd be cool.  Which Rheem water heater do you have?  I have a newer Rheem heat pump water heater with EcoNet.. I would love to dump their cloud service and move over to an esp32.  Do you happen to have a link to that project?
For a sniffer, you can just use the uart port on an atom with the rs485 base.  If you set dummy device you can have it log all packets in both directions to the logs easily.. This is what I had setup on the same hardware hooked up to my hot tub before I deployed your code to it.
esphome:
  name: hot-tub-controller
  friendly_name: Hot Tub Controller

esp32:
  board: esp32dev
  framework:
    type: arduino

# You need a UART to talk to the RS485 bus
uart:
  id: my_uart
  baud_rate: 38400
  rx_pin: GPIO22
  tx_pin: GPIO19
  debug:
    direction: BOTH
    dummy_receiver: True
    after:
      delimiter: "\n"
    sequence:
      - lambda: UARTDebug::log_hex(direction, bytes, ',');

select:
  - id: change_baud_rate
    name: Baud rate
    platform: template
    options:
      - "2400"
      - "9600"
      - "38400"
      - "57600"
      - "115200"
      - "256000"
      - "512000"
      - "921600"
    initial_option: "115200"
    optimistic: true
    restore_value: True
    internal: false
    entity_category: config
    icon: mdi:swap-horizontal
    set_action:
      - lambda: |-
          id(my_uart).flush();
          uint32_t new_baud_rate = stoi(x);
          ESP_LOGD("change_baud_rate", "Changing baud rate from %i to %i",id(my_uart).get_baud_rate(), new_baud_rate);
          if (id(my_uart).get_baud_rate() != new_baud_rate) {
            id(my_uart).set_baud_rate(new_baud_rate);
            id(my_uart).load_settings();
          }

# Enable logging
logger:

# Enable Home Assistant API
api:
  encryption:
    key: ! secret api_encryption_key

ota:
  - platform: esphome
    id: ota_password
    password: !secret ota_password

wifi:
  ssid: !secret wifi_ssid
  password: !secret wifi_password

  # Enable fallback hotspot (captive portal) in case wifi connection fails
  ap:
    ssid: "Hot-Tub-Controller"
    password: !secret fallback_wifi_pw

captive_portal:

button:
  - platform: uart
    name: "UART Bytes Output"
    data: [0x58,0x4d,0x53,0x00,0x03,0x6b,0x00,0x10,0x01,0x76]

Ha. Funny you ask about the Rheem Heat Pump water heater, that is exactly what I have and I made a video about it, the project is EspHome-EcoNet and yes, it works so well, I don't use the cloud app anymore. I am a huge fan of that project and they use my video on their main page.
For the RS485 sniffer, yes, that is exactly how I got started with dummy_receiver: True, however I quickly needed something more powerful. You may have noticed that this integration as a built-in RS485-to-TCP relay and I built a C# app to decode the traffic. Take a look for the first post of #6 to see how it works. The C# app allowed me to send/receive data and conduct all sorts of experiments like trying all possible commands (64k of them), etc.

Do you know if this takes over control of the hot tub completely when its hooked up and running?  I don't think my cleaning cycle has come on automatically since I hooked it up.  it used to come on all the time by itself.

This integration does not take control and the cleaning cycle will happen as usual. I will note that is you have "Summer Timer" turned on, my understanding is that the cleaning system stop not happen, but not sure. In any case, you should see the 'cleaning cycle' turn on in the logs each time it happens.

@joshs85 I added the light intensity and color sensors, take a look at the extras here for information on adding them. Let me know if it works for you.

@joshs85 I added the light intensity and color sensors, take a look at the extras here for information on adding them. Let me know if it works for you.

hmm.. it works.. but when my lights are on the color cycle setting, I see 7 for the color instead of 8.   The intensity seems to work.  Here's what I added to the config
    lights_underwater_intensity:
      name: Underwater Light Intensity
    lights_underwater_color:
      name: Underwater Light Color

@joshs85 Can you speed up the polling rate like this:
iq2020-dev:
   uart_id: SpaConnection
   polling_rate: 10                          <----

Then, cycle thru all the colors and see what values to read for each of them? I will do the same later today and see if I the table I created is not right. I think these are my values... but will check again.
1 = Violet
2 = Blue
3 = Cyan
4 = Green
5 = White
6 = Yellow
7 = Red
8 = Color Cycle

@joshs85 Can you speed up the polling rate like this:
iq2020-dev:
   uart_id: SpaConnection
   polling_rate: 10                          <----

Then, cycle thru all the colors and see what values to read for each of them? I will do the same later today and see if I the table I created is not right. I think these are my values... but will check again.
1 = Violet
2 = Blue
3 = Cyan
4 = Green
5 = White
6 = Yellow
7 = Red
8 = Color Cycle

I already have my polling time set to 5s.  These are my results below.  There's a separate setting for color loop 1-3 which doesn't affect the color that's set as far as I can tell.

7 = Violet
1 = Blue
2 = Cyan
3 = Green
4 = White
5 = Yellow
6 = Red

@joshs85 Color loops? Here is my color screen on my remote. Can you port what yours looks like?

Arg. This is very different indeed. If you install the DataViewer all and look at the lights status packet, you may have different values. If you decode your own usage, I could add ESP-Home sensors where needed and/or update the documentation to match your model.
<-- 01 1F 40 1705
<-- 1F 01 80<PHONE_NUMBER>0202000000000000000005010101010000
                 FFAABBCCDD                WWXXYYZZOO

FF = Flags (0x01 = Lights Timer On)
AA = Underwater light intensity (0x05 = High ... 0x01 = Low, 0x00 = Off)
BB = Bartop light intensity
CC = Pillow light intensity
DD = Exterior light intensity
WW = Underwater light color     (0x01 = Violet, 0x02 = Blue, 0x03 = Cyan, 0x04 = Green, 0x05 = White, 0x06 = Yellow, 0x07 = Red, 0x07 = Color Cycle)
XX = Bartop light color
YY = Pillow light color
ZZ = Exterior light color
OO = 00 for lights are off, 01 for lights are on.

Eh... the windows client seems to crash as soon as I connect and get a single data packet.

If I connect really fast and then disconnect it seems to work
Lights on.  Color loop 0
--> 01 1F 40 1705
<-- 1F 01 80<PHONE_NUMBER>0000000000000000000003010101010000
Lights on Color Loop 1
--> 01 1F 40 1705
<-- 1F 01 80<PHONE_NUMBER>0000010000000100000003010101010000
Lights on Color Loop 2
--> 01 1F 40 1705
<-- 1F 01 80<PHONE_NUMBER>0000010000000200000003010101010000
Lights on Color Loop 3
--> 01 1F 40 1705
<-- 1F 01 80<PHONE_NUMBER>0000010000000300000003010101010000

@joshs85 Arg! Thanks for letting me know. I bet I try to parse a packet and crash doing it. I will add more exception handling to fix this. I will need to add a new sensor for your color loop.
<-- 1F 01 80<PHONE_NUMBER>0202000000000000000005010101010000
<-- 1F 01 80<PHONE_NUMBER>0000010000000300000003010101010000
                 FFAABBCCDD        LL      WWXXYYZZOO

FF = Flags (0x01 = Lights Timer On)
AA = Underwater light intensity (0x05 = High ... 0x01 = Low, 0x00 = Off)
BB = Bartop light intensity
CC = Pillow light intensity
DD = Exterior light intensity
LL = Color Loop (NEW)
WW = Underwater light color     (0x01 = Violet, 0x02 = Blue, 0x03 = Cyan, 0x04 = Green, 0x05 = White, 0x06 = Yellow, 0x07 = Red, 0x07 = Color Cycle)
XX = Bartop light color
YY = Pillow light color
ZZ = Exterior light color
OO = 00 for lights are off, 01 for lights are on.

FYI. This integration is now in the ESP-Home devices list.

FYI. This integration is now in the ESP-Home devices list.

Yeah, I saw your pull request.  Nice!

I used your other integration to hook up my hot water heater this morning.  Works great!

FYI, The hot tub I have is the HotSpring HotSpot SX.. if it matters.

@joshs85 Oh yes, ESP-home Econet is amazing, just like my hottub, I use it to dynamically change temperature levels to avoid using electricity at high prices and maximize use at cheap prices.
I added your hot tub to the tested list here. What is your version string? What ESP32 do you use? Also, if there is anything you want to add to the tested page, let me know.

@joshs85 Oh yes, ESP-home Econet is amazing, just like my hottub, I use it to dynamically change temperature levels to avoid using electricity at high prices and maximize use at cheap prices. My graphs for both integrations below.
I added your hot tub to the tested list here. What is your version string? What ESP32 do you use? Also, if there is anything you want to add to the tested page, let me know.

Version string is: 112T b5e1A002 v3.0
Using the m5stack atom lite
https://shop.m5stack.com/products/atom-lite-esp32-development-kit
With the atomic rs485 base
https://shop.m5stack.com/products/atomic-rs485-base

@joshs85 I got two updated for you. First, a new sensor in the latest integration update that should match your loop value. Let me know if this works.
sensor:
    lights_main_loop_speed:
      name: Lights Main Loop Speed

Also, I updated the DataViewer to display decoding errors instead of crashing.

Awesome.  I was able to pull in the extra sensor and its working correctly.   I'll check out the windows app later.  for what its worth it wasn't even giving me an error code before but just exiting.

<-- 1F 01 80<PHONE_NUMBER>0000000404000A0A2040160E201C201C201C840360540000000020393646DA1B9B00E0C60802BF4AD2146200000000000000000000000000000000000000AF4F8B00030000000000CEB0F101000000003130324631303146E700E700E700000000000000000000000000000000000000003C001E00006C00000000000000DC0701
System.ArgumentOutOfRangeException: Year, Month, and Day parameters describe an un-representable DateTime.
at System.DateTime.DateToTicks(Int32 year, Int32 month, Int32 day)
at System.DateTime..ctor(Int32 year, Int32 month, Int32 day, Int32 hour, Int32 minute, Int32 second)
at DataViewer.MainForm.processData(Byte[] data, Int32 len)

@joshs85 Oh! Makes sense. I will add code to handle this case.

@joshs85 Oh! Makes sense. I will add code to handle this case.

Can you post this as a new github project with an esp home component for the tcp listener?  I'd love to use this CS app to reverse my PoolSync (to control my pool pump, heater, and salt cell)   and may compile it for .net on osx if possible.

For the RS485 to TCP in ESP-Home, take a look at esphome-stream-server. It's super easy to get yourself just the RS485 relay. For the C# app, it's located in this GitHub repo here. I have not cleaned up the code, it's a bit ugly. I don't think I need to put it in a separate repo, the entire source code is there. Let me know if that helps.

@joshs85 For the RS485 to TCP in ESP-Home, take a look at esphome-stream-server. It's super easy to get yourself just the RS485 relay. For the C# app, it's located in this GitHub repo here. I have not cleaned up the code, it's a bit ugly. I don't think I need to put it in a separate repo, the entire source code is there. Let me know if that helps.

Ok, cool.. that works for me.

@joshs85 I just updated DataViewer with your fix. Thanks for reporting this and the exact error.

Just wanted to confirm that this is working great with a 2022 Caldera Niagara(Utopia Series). Reports power and everything..

@dvanderb Thanks for confirming. Can you also provide your version string? It's something like "WR2.02ad1bE002DK2.02". I will add it to the tested list.

@dvanderb Thanks for confirming. Can you also provide your version string? It's something like "WR2.02ad1bE002DK2.02". I will add it to the tested list.

EU2.039018E0020.00.0

Thanks @dvanderb for the contribution, you are the first one with a Caldera. I just updated the tested list.

I am getting close to having every possible feature for this integration I can think of. I just added one more, the ACE Module Emulation feature. It allows you to see this screen on your spa remote:

You can then use added Home Assistant sensors in your automatons to do anything you like.
I did have to change the audio_buttons sensor to buttons. So, if you get an error, just remote audio_ and that will fix it.
sensor:
  - platform: iq2020
    buttons:
      name: Buttons

I did that since both the audio emulation and ACE emulation use the buttons sensor.

I see the new ace emulation with boost. Did you also manage to get boost working/decoded for tubs with ace?
One feature I am really missing is the color setting. That would really help for doorbell scenarios

@npinguin I just managed to get ACE boost decoded and probably working, but I can't truly test it because I don't have a real ACE module. I updated the documentation and the 3 ACE sensors are below. The salt_power did not change, still 0 to 10. salt_content did change, it's not a value from 0 to 7 where 3 or 4 are ideal. This is the yellow/green/red indicator on the remote. Lastly, salt_system_boost is the new one and should indicate if you are in boost mode or not and allow you to change you boost setting from Home Assistant. If anyone can confirm this works. Note that if you change the boost setting, you need to go back to the main page and return to ACE on the remote to see the updated state.
number:
  - platform: iq2020
    id: salt_power
    name: Salt Power
    datapoint: 5

sensor:
    salt_content:
      name: Salt Content

switch:
  - platform: iq2020
    name: Salt Boost
    id: salt_system_boost
    datapoint: 8

I'm having some issues turning the lights on and off now for some reason.. When I turn them on, they turn on physically but the switch in HA goes back to turned off almost instantly.  Any idea what could cause that?

FYI. I just figured out how to change light brightness and colors! I am still paying around with this.
--> 01 1F 40 17020102
                 NNAA
NN = Light Number 0x00 to 0x03 for individual lights, 0x04 for all lights.
AA = Action
  02 = Lower Brightness (0 to 5)
  03 = Increase Brightness (0 to 5)
  04 = Previous color (1 to 7 and cycles back)
  05 = Next color (1 to 7 and cycles back)
  06 = Previous Color Loop ?
  07 = Next Color Loop ?

Full control over the light colors, intensity and cycle speed has been added to the integration. You can look at the top of the extras section for details on how to add this. Note that I only tested this on my hot tub and that I may need to fix things for other hot tub models. So please let me know if it does not work right.

Once nice extra is that I can now control the cycle speed of the lights, something I can't do from the remote.

Full control over the light colors, intensity and cycle speed has been added to the integration. You can look at the top of the extras section for details on how to add this. Note that I only tested this on my hot tub and that I may need to fix things for other hot tub models. So please let me know if it does not work right.

Very cool.. my Caldera only has one controllable RGB light(it simply affects all lights in the tub).. so do we think that would just be data point 1?

Working on a Hot Springs Limelight Flash 2018 here.  Version is 1.14D6ce5E002.  Light control colors are off by one (violet=blue, etc.) and wheel doesn't cycle, although the control panel flashes the light icon like it does when cycling normally.  No audio or salt systems to test.

@dvanderb Yes, that's correct. Only use select with data point 1. That would be ok for you. Let me know what your exact model and version number is, I will add it to the tested page with that note.
@jbwalkup920 Thanks for reporting this. It does seem like there is a lot of variations on what colors are set to what number for different tubs. If you send back your exact color order yaml, I will add it to the tested page. I will also work on cycling at some point to get that working, but I may need some data capture from various tubs.

Just tested the lights on my Hot Spring Sovereign. The lights are off by one colour & the descriptions are also off (Exterior control pillows, bar top control water fall etc)
This is the current working YAML with everything correct:
select:
  - platform: iq2020
    name: Color Underwater
    id: lights1_color
    datapoint: 1
    options:
      - Blue
      - Cyan
      - Green
      - White
      - Yellow
      - Red
      - Magenta
      - Cycle
  - platform: iq2020
    name: Color Waterfall
    id: lights2_color
    datapoint: 2
    options:
      - Blue
      - Cyan
      - Green
      - White
      - Yellow
      - Red
      - Magenta
      - Cycle
  - platform: iq2020
    name: Color Bar Top
    id: lights3_color
    datapoint: 3
    options:
      - Blue
      - Cyan
      - Green
      - White
      - Yellow
      - Red
      - Magenta
      - Cycle
  - platform: iq2020
    name: Color Pillow
    id: lights4_color
    datapoint: 4
    options:
      - Blue
      - Cyan
      - Green
      - White
      - Yellow
      - Red
      - Magenta
      - Cycle
  - platform: iq2020
    name: Color Cycle Speed
    id: lights_cycle_speed
    datapoint: 5

number:
  - platform: iq2020
    id: lights1_intensity
    name: Intensity Underwater
    datapoint: 7
    maximum: 5
  - platform: iq2020
    id: lights2_intensity
    name: Intensity Waterfall
    datapoint: 8
    maximum: 5
  - platform: iq2020
    id: lights3_intensity
    name: Intensity Bar Top
    datapoint: 9
    maximum: 5
  - platform: iq2020
    id: lights4_intensity
    name: Intensity Pillow
    datapoint: 10
    maximum: 5

Also, not sure what 'Small heater power' actually is, but mine sits at an average of 115W, except for then the pump is on 'high' then its approx 50W. The 115W reading cannot be correct, as I have power monitoring on the supply to the Hot Tub & it reads around 85W when just the small filter pump is running (the rating of the pump), so unsure what that data point is.
Also on my Hot Spring Sovereign, heater power & pump power both work correctly, but heater current shows on L2 & pump current on L1, 'Current heater' never reads above 0

@fricker-ben That is interesting. I will take note of this. The lifetime encoding is very basic with not much room for error, basically, the value is directly obtained from the hot tub. If you even want to investigate this further, you can send me a packet capture like #4 to see what is going on, but not something that is crucial.

The response to an 'Ask Status 0x256' command:
1F 01 80<PHONE_NUMBER>0400000404000A0900405600201C201C201C840360540000000033352E30A17AB7065752A3004C9F31374000000000000000000000000000000000000000B3FB830003003B652A37AFCC35000000000033352E3033352E30F200F200F200000000000000000000000000760000000000003C001E0000664500240F0613FFD00701

@fricker-ben Thanks for the reports. I just added your light yaml to the tested page, but now I am thinking I am going to create a new page just for controlling lights as this is going to be a crazy topic.
For the power/voltage/amps sensors... that too seems to vary between tubs and it's a still a mystery to me. The files I have seen show the following sensors in command 0x0256 in this order:
voltage_l1
voltage_heater
voltage_l2
voltage_jet3
current_l1
current_heater
current_l2
current_jet3
power_l1
power_heater
power_l2
power_jet3

But for me, l2 is clearly the main heater and I have not idea what "heater" is, so, I called it "small heater" as a stand it until it's figured out. For your 0x0256 command, in DataViewer, the last 0x0256 command is always decoded in the "State" tab. In my case, it looks like this:

If you can figure out what the values map to in your hot tub, that would be wonderful.

This is mine currently, I still can't work out what the 'Small heater power' value could be, is it definitely a power figure, or could it be something else? Possibly heater flow/pressure?
Hex:
1F01868002560008000400000404000A0900405600201C201C201C840360540000000033352E30A17AB7065752A3003DED31374000000000000000000000000000000000000000B3FB830003002CB32A37AFCC35000000000033352E3033352E30F000F000F000000000000000000000000000790000000000003C001E00006544000D2D110206E80701DA1C
0008     - Unknown
00       - Flags
04       - Flags
00000404000A090040 - Unknown
5600     - Flags, 0x4000 is Celsius
201C201C201C - Unknown
84036054 - Unknown
00000000 - Unknown
33352E30 - Exaust Temperature string, "35.0"
A17AB706 - Heater total runtime in seconds, 112687777
5752A300 - Jets 1 total runtime in seconds, 10703447
3DED3137 - Lifetime in seconds, 926018877
40000000 - Power on / Boot counter
00000000 - Flags
00000000 - Jet 2 runtime
00000000 - Jet 3 runtime
00000000 - Blower runtime
B3FB8300 - Lights runtime
0300     - SPA state & Light state
2CB32A37 - Circulation pump Lifetime
AFCC3500 - Jets 1 low operation Lifetime
00000000 - Jets 2 low operation Lifetime
33352E30 - Target Temperature String, "35.0"
33352E30 - Current Temperature String, "35.0"
F000     - Voltage L1
F000     - Voltage Heater
F000     - Voltage L2
0000     - Voltage Jet3
0000     - Current L1
0000     - Current Heater
0000     - Current L2 (Heater)
0000     - Current Jet3
0000     - Power L1
7900     - Power Heater
0000     - Power L2 (Heater)
0000     - Power Jet3
003C001E00006544 - Unknown
000D2D11 - Time hh:mm:ss, 17:45:13
0206E807 - Date, 2-7-2024
01       - Clock Status

So, I just tested out the ability to change the lights for the underwater lights.  If I set cycle speed to Pause, it does nothing (it was normal previously). If I change to fast it works.. as soon as I change the color from whatever it was, the lights stop cycling.  The cycle speed stays on normal.. I can't get the lights to change back to cycling even if I set the color to Cycle or change the speed.  Also, the colors don't match the UI

The light colors seems to be very different depending on the model, so, changing the options to the right value is going to be needed. For cycling, I coded the integration to work for my hot tub, but to debug get the DataViewer working and use "Ask Lights Status" in the "Commands" menu to ask for the current status.

The light status starts with 1F 01 80 1705... like here:
 <-- 1F 01 80<PHONE_NUMBER>0000000000000000000001010101000000

The lights status seems to decode like this:
<-- 1F 01 80<PHONE_NUMBER>0000010000000300000003010101010000
                 FFAABBCCDD        LLMMNNOOWWXXYYZZOO

FF = Flags (0x01 = Lights Timer On)
AA = Underwater light intensity (0x05 = High ... 0x01 = Low, 0x00 = Off)
BB = Bartop light intensity
CC = Pillow light intensity
DD = Exterior light intensity
LL = Underwater light cycle speed - Main loop speed
MM = Bartop light cycle speed
NN = Pillow light cycle speed
OO = Exterior light cycle speed
WW = Underwater light color     (0x01 = Violet, 0x02 = Blue, 0x03 = Cyan, 0x04 = Green, 0x05 = White, 0x06 = Yellow, 0x07 = Red, 0x08 = Color Cycle)
XX = Bartop light color
YY = Pillow light color
ZZ = Exterior light color
OO = 00 for lights are off, 01 for lights are on.

However, if you can get the possible valid states for your tub tub, I will fix the integration.

My tub only has underwater lights.  Here's the decode.
Version:<PHONE_NUMBER>54202062356531413030322076332E30200A
Blue.  Paused:
17050003000000000000000000000001010101010000
Cyan Paused:
17050003000000000000000000000002010101010000
Green Paused:
17050003000000000000000000000003010101010000
White Paused:
17050003000000000000000000000004010101010000
Yellow Paused:
17050003000000000000000000000005010101010000
Red Paused:
17050003000000000000000000000006010101010000
Violet Paused:
17050003000000000000000000000007010101010000
Cycle Speed 1:
17050003000000010000000100000005010101010000
Cycle Speed 2:
17050003000000010000000200000005010101010000
Cycle Speed 3:
17050003000000010000000300000005010101010000
Intensity 1 Paused:
17050001000000000000000000000001010101010000
Intensity 2 Paused:
17050002000000000000000000000001010101010000
Intensity 3 Paused:
17050003000000000000000000000001010101010000
These are the light color values that seemed to work earlier:
1 = Blue
2 = Cyan
3 = Green
4 = White
5 = Yellow
6 = Red
7 = Violet

There's also an LCD intensity setting with values 1-3.  Not sure where that value lives.  It doesn't seem to affect the light status.

@joshs85 Wow! Thank you. Yes, based on your data, I have a good idea on how to fix the integration. I will put in a fix in the next few days. Question: It looks like you can enabling cycling for any color? In your data, you changed the cycling speed while having yellow selected... but I imagine any color could be selected?
Or when cycling, is the "05" at this location always present?
17050003000000010000000100000005010101010000
                                                 ^^

@joshs85 Wow! Thank you. Yes, based on your data, I have a good idea on how to fix the integration. I will put in a fix in the next few days. Question: It looks like you can enabling cycling for any color? In your data, you changed the cycling speed while having yellow selected... but I imagine any color could be selected?
Or when cycling, is the "05" at this location the only possible value?
17050003000000010000000100000005010101010000
                              ^^

No, any color will work it seems.
This is me setting it to loop 2 when it was red
17050003000000010000000200000006010101010000

@joshs85 - Excellent. Got it. Thanks. I need to redesign my controls a bit because it's possible to cycle while adjusting the colors in your case... so having a "cycle" color is not going to work. I think I need to have a "Off" on the cycle selector instead and have a cycle selector for each light group.

Can you just make it so that if the color is changed while a cycle is running the cycle setting also gets set to 0?

@joshs85 - Ok, I got a new version of the integration ready.
select:
  - platform: iq2020-dev
    name: Audio Source
    id: audio_source
    datapoint: 0
  - platform: iq2020-dev
    name: Color Underwater
    id: lights1_color
    datapoint: 1
    options:
      - "Violet"
      - "Blue"
      - "Cyan"
      - "Green"
      - "White"
      - "Yellow"
      - "Red"
#     - "Cycle"
  - platform: iq2020-dev
    name: Color Bartop
    id: lights2_color
    datapoint: 2
    options:
      - "Violet"
      - "Blue"
      - "Cyan"
      - "Green"
      - "White"
      - "Yellow"
      - "Red"
#     - "Cycle"
  - platform: iq2020-dev
    name: Color Pillow
    id: lights3_color
    datapoint: 3
    options:
      - "Violet"
      - "Blue"
      - "Cyan"
      - "Green"
      - "White"
      - "Yellow"
      - "Red"
#     - "Cycle"
  - platform: iq2020-dev
    name: Color Exterior
    id: lights4_color
    datapoint: 4
    options:
      - "Violet"
      - "Blue"
      - "Cyan"
      - "Green"
      - "White"
      - "Yellow"
      - "Red"
#     - "Cycle"
  - platform: iq2020-dev
    name: Color Cycle Speed
    id: lights_cycle_speed
    datapoint: 5
    options:
      - "Off"
      - "Slow"
      - "Normal"
      - "Fast"

@joshs85 - Ok, I got a new version of the integration ready. The key difference now is that if you do not specify the 8th color and only put 7 colors in the options, the code will assume that you can system on any color and change the Color Cycle Speed selector to make it do that. If you specify the 8th color, the integration will assume that you can only cycle when the 8th color is selected. Give that a try, remove the 8th color from all options and let me know if it works.
select:
  - platform: iq2020-dev
    name: Audio Source
    id: audio_source
    datapoint: 0
  - platform: iq2020-dev
    name: Color Underwater
    id: lights1_color
    datapoint: 1
    options:
      - "Violet"
      - "Blue"
      - "Cyan"
      - "Green"
      - "White"
      - "Yellow"
      - "Red"
#     - "Cycle"
  - platform: iq2020-dev
    name: Color Bartop
    id: lights2_color
    datapoint: 2
    options:
      - "Violet"
      - "Blue"
      - "Cyan"
      - "Green"
      - "White"
      - "Yellow"
      - "Red"
#     - "Cycle"
  - platform: iq2020-dev
    name: Color Pillow
    id: lights3_color
    datapoint: 3
    options:
      - "Violet"
      - "Blue"
      - "Cyan"
      - "Green"
      - "White"
      - "Yellow"
      - "Red"
#     - "Cycle"
  - platform: iq2020-dev
    name: Color Exterior
    id: lights4_color
    datapoint: 4
    options:
      - "Violet"
      - "Blue"
      - "Cyan"
      - "Green"
      - "White"
      - "Yellow"
      - "Red"
#     - "Cycle"
  - platform: iq2020-dev
    name: Color Cycle Speed
    id: lights_cycle_speed
    datapoint: 5
    options:
      - "Off"
      - "Slow"
      - "Normal"
      - "Fast"

My light intensity only goes up to 3.  Is there a way to set a max?

Closing this since it's long and things have gotten pretty stable. Please open a new issue with any new data, suggestions or bug reports.


Write a blog post comparing performance of different methods

I'm curious how these methods compare for different values of n_samples, n_features, and fraction_missing. If there's a very obvious trend we should provide a wrapper function that switches between underlying implementations. Also if anything is uniformly slower than the reference implementation we should drop it (since I implemented these for fun and they're not all necessarily useful).

Not actually sure this would be interesting, especially since overall performance would improve way more from Cython than algorithmic tricks in pure-Python implementations.


Launcher changes

Some general clean up and checking before adding the XX:MaxPermSize by default.

Windows Build 1237 is now running using a merge of 881cb7c02068abafb21592f6ca10f88f368b21d0

Core - Full Integration Build 838 is now running using a merge of 881cb7c02068abafb21592f6ca10f88f368b21d0

Linux Build 1587 is now running using a merge of 881cb7c02068abafb21592f6ca10f88f368b21d0

Linux Build 1587 outcome was SUCCESS using a merge of 881cb7c02068abafb21592f6ca10f88f368b21d0
Summary: Tests passed: 3098, ignored: 66 Build time: 0:19:21

Windows Build 1237 outcome was SUCCESS using a merge of 881cb7c02068abafb21592f6ca10f88f368b21d0
Summary: Tests passed: 3098, ignored: 66 Build time: 0:20:20

Core - Full Integration Build 838 outcome was FAILURE using a merge of 881cb7c02068abafb21592f6ca10f88f368b21d0
Summary: Tests failed: 1 (1 new), passed: 2856, ignored: 412 Build time: 0:49:03
Build problems:
Failed tests detected
Failed tests
org.jboss.as.test.integration.ejb.stateful.passivation.PassivationTestCase.testPassivationMaxSize: <no details avaliable>

Re: the first commit, have you checked with whoever the known users are re: potential breakage from this change?
I think it's good and we don't want to leave it out of 9 and then make a breaking change in 10. But if anyone will be affected, giving them a heads up would be good.
TBH I doubt anyone uses this. :)

Yeah I should have linked the JIRA. I'm not sure why I didn't clarify in my comments, but 9 is the first release with the launcher so I think we should remove them before final.
And yeah I doubt anyone uses it except for me :)


Support embedded scene in png

I believe the workflow with the plugin currently requires having a separate excalidraw file per exported png. Would it be possible to take advantage of excalidraw's embedded schene in png feature?

Sadly this feature is not available in the npm package :/

Looks like embedded scene support is included in latest version of the plugin (not yet in VsCode extensions marketplace).
The toggle exportEmbedScene should apply to both .svg and .png as per below:
https://github.com/excalidraw/excalidraw/blob/1c11df011a2f1eaf7f1164f475c94fc3ebe9f36b/src/data/index.ts#L80

@pomdtr Do you know when a new version will be released in the VsCode marketplace?
I have been looking to use the embedded scene option for .svg export but VsCode marketplace version is still v1.2.0

Yeah I'm a bit lost in refactoring hell right now, I'll try to merge my changes this weekend 😅

:woah: If the vscode plugin had the capability to also open a png with an embedded excalidraw scene, and then save directly back to the png, that would make for an amazing workflow since all diagrams could live in git repos as PNGs only.

This looks achievable with the latest version of the excalidraw library. However, I would need to implement a customEditorProvider inside VSCode, which looks harder to implement than the exisiting customTextEditorProvider


service/daemon error

aegeusd -reindex
/usr/include/c++/4.8/debug/safe_iterator.h:264:error: attempt to
dereference a past-the-end iterator.
Objects involved in the operation:
iterator "this" @ 0x0x7fffdbb63090 {
type = N11__gnu_debug14_Safe_iteratorISt17_Rb_tree_iteratorISt4pairIK7uint25620CMasternodeBroadcastEENSt7__debug3mapIS3_S5_St4lessIS3_ESaIS6_EEEEE(mutable iterator);   state = past-the-end;   references sequence with typeNSt7__debug3mapI7uint25620CMasternodeBroadcastSt4lessIS1_ESaISt4pairIKS1_S2_EEEE' @ 0x0x117bfc0
}
Aborted (core dumped)

Which flavor of Linux is this and release version?

Was in a rush, sorry;
Ubuntu 16.04
Latest 1.2 release

That's alright we'll try to reproduce this and determine the cause

Is this the pre-built release binary, or did you build this from source?

Some context, this is after the service dies, attempting to fix it. There are no other aegeusd processes in proc list, any attempt at using aegeusd with or without arguments results in the error after a few seconds

Please type this command out as the user that runs aegeusd and owns your wallet and paste the results:
ls -al ~/.aegeus

root@aegMN7:~/.aegeus# ll
total 10688
drwxr-xr-x 6 root root    4096 Jun 14 23:27 ./
drwx------ 5 root root    4096 Jun 14 23:26 ../
-rw-r--r-- 1 root root     310 May 24 06:01 aegeus.conf
-rw------- 1 root root       6 Jun 15 03:13 aegeusd.pid
drwx------ 2 root root    4096 Jun 15 03:13 backups/
drwx------ 3 root root    4096 May 24 06:01 blocks/
-rw------- 1 root root      61 Jun 14 23:27 budget.dat
drwx------ 2 root root    4096 Jun 15 03:13 chainstate/
drwx------ 2 root root    4096 Jun 14 23:28 database/
-rw------- 1 root root       0 May 24 06:01 db.log
-rw------- 1 root root  452322 Jun 15 03:13 debug.log
-rw------- 1 root root    1902 Jun 14 23:27 fee_estimates.dat
-rw------- 1 root root       0 May 24 06:01 .lock
-rw-r--r-- 1 root root     264 May 24 06:01 masternode.conf
-rw------- 1 root root 6544200 Jun 14 23:27 mncache.dat
-rw------- 1 root root 2489019 Jun 14 23:27 mnpayments.dat
-rw------- 1 root root  743306 Jun 15 02:58 peers.dat
-rw------- 1 root root  663552 Jun 15 03:12 wallet.dat
root@aegMN7:~/.aegeus#

And are you running aegeusd as root or another user?

As per the install script the service is started as root
[Service]
User=root
Group=root
Manual attempts to start it are also under root
I'll be working on my own script to change this but as of now all of mine are configured to run as root and are started as root

I was just able to reproduce this error and am working diligently on it.  Please have faith that we will resolve this.

Please try this new build and let us know if issues still persist: https://github.com/AegeusCoin/aegeus/releases/download/1.2/Aegeus-1.2-Linux-64-bit.tar.gz

Syncing chain now, will update later


銀魂乱舞 [PCSG01073]

App summary

App name: 銀魂乱舞
App serial: PCSG01073
App version: 1.00

Vita3K summary

Version: v0.2.0
Build number: 3608
Commit hash: https://github.com/vita3k/vita3k/commit/53741632
CPU backend: Dynarmic
GPU backend: Vulkan

Test environment summary

Tested by: FayyazMaru
OS: Windows 11
CPU: AMD Ryzen 5 3500U with Radeon Vega Mobile Gfx     2.10 GHz
GPU: AMD Radeon (TM) Vega 8 Graphics
RAM: 8 GB

Issues

When selecting the option on the menu, the game suddenly crash and forced close. Also it said Exception Access Violation when press the option
When changing the GPU Backend to OpenGL, the game is fine but some image are missing and in the gameplay, it's fine in the beginning but suddenly crash a few seconds later and forced close

Screenshots

Log
vita3k.log
Recommended labels

Shaders Bug
Forced Close
Ingame -
Crash

https://github.com/Vita3K/compatibility/assets/171367603/09c66a4e-1019-4708-b1e0-0c5308cea3f9


Is there a way to represent batch of graphs as one large graphs?

❓ Questions & Help

Hi,
Do we have any specific API or library in PyTorch-geometric that represents a batch of graphs as one large graph?
Also, can we use this library on machines with CPU only?

Batch.from_data_list.

Can  Batch.from_data_list be used with Pytorch data object?

You just pass in a list of data objects: Batch.from_data_list([data1, data2]).

@rusty1s Can Batch.from_data_list be used with PyTorch Tensors?

I'm not sure what that means. Batch.from_data_list expects a list of torch_geometric.data.Data objects. However, the collate_fn of the torch_geometric.data.DataLoader provides reasonable defaults for handling PyTorch tensors.


docker-compose up error

Attaching to designpatternsphp_php_1
php_1  | Installing composer...
php_1  | Installer corrupt
php_1  | Could not open input file: composer-setup.php
php_1  |
php_1  | Warning: unlink(composer-setup.php): No such file or directory in Command line code on line 1
php_1  | Downloading dependencies using composer...
php_1  | Could not open input file: composer.phar
php_1  | Running tests...
php_1  | /opt/php/docker/test_runner.sh: line 15: ./vendor/bin/phpunit: No such file or directory
php_1  | Tests failed :(

@Isengo1989 is now fixed in current master, the problem here has been a checksum problem.


Add collar color selection and Dev Container support

I've added color support to the Collar light entity.
I manually mapped color codes for my dog's Fi3 color to RGB colors. From a HA perspective, the light supports RGB. Euclidean distance is used to find the closest supported color.
Brightness is not supported but HA expects this for all lights with color control so brightness is simply ignored.
Risks:
I have only tested this with FI3. It's possible other collars have other colors and could act in unexpected ways.
Alternatives considered:
The color matching may work unexpectedly  for users as the UI does not tell the user which colors are supported. Instead we could use an additional selection control or light effects in HA so the UI is more clear.
I've also added dev container support for VSCode. As this is a cloud-integration I find this a much easier way to test changes and it does not require messing with my actual HA instance

Thank you for the contribution @prestomation. I think the list of colours can be retrieved from a property availableLedColors in the pytryfi library (I believe). Appreciated the contribution!

Clearly I didn't explore the library enough! I see some improvements to make, look for another PR


Further improvements of HAMT

@ruslansennov To get the most out of List used in HAMT we should do the following:

use prepend() and prependAll() instead of append() and appendAll(). Note: I hope the order of entries does not matter in a HAMT!? However, the iterator() will then return the entries in a different order. Does this matter for HashMaps/HashSets?
use findFirst() instead of filter().headOption()
in mergeLeaves() please replace List.empty() with this.entries() and delete the appendAll(this.entries) below

Additionally I have created two issues to improve removeIndx() and iterator().

What if we put the entries in a (Sorted)Set instead? E.g. containing Tuple2 with compared by ._1/key?

As I know nobody promises that the return set of keys should be in certain order. We can do it as our feature, but I'm not sure. All another your proposals are very good.

Using List, we have O(n), using Set it is O(log_2(n)) for operations on entries.
I have to take a look on other HAMT impls and need also to understand, why a HAMT backed by a Set (RedBlackTree) performs better than using just a RedBlackTree (containing Entries compared by key). How do Scala and Closure store the entries in a HAMT?
Interesting!

Good reading: http://lampwww.epfl.ch/papers/idealhashtrees.pdf

just for clarify: HashMap.keySet() should be return Set and only Set. But sorted set or hash set, that is the question

Our API will declare return type Set, that makes it flexible for other impls.
Our implementation may use SortedSet for performance reasons, if it makes sense.

(minor thing) We could make the number of slots (currently 32) a constant SLOTS, just for clarity.

@ruslansennov Is there still something to do here? https://github.com/javaslang/javaslang/blob/master/src/main/java/javaslang/collection/HashArrayMappedTrie.java#L208-L216

Of course no! After last List improvements It can be changed to

{ List.removeAt() } else { List.set() }

I'll fix it later

I thought the idea to change javaslang.collection.List<Tuple2<K,V>> on java.util.TreeMap<K,V> in HashArrayMappedTrie.LeafNode...
 In all other cases java.util.TreeMap will require more memory but did not give us a big win in CPU

thanks for investigating! then let's stay with our List.

Finished with #457


[BUG] Verify StronglyEntangling Layers Ansatz

From PR #20; StronglyEntanglingLayers is implemented, but maybe needs some extension because in the original paper, the two-qubit gates feature trainable parameters.

close as not relevant


Change the Container tests to use a local backend negating the need for command dispatch in PRs

@stack72 I know we parked this PR.  Could you refresh my memory with a comment below on why this change didn't work?


hfarray from set of touchstone

can you give  an minmal example  of how to create a hfarray from a set of touchstones?
i would like to pull values out of each touchstone's filename for the additional axis.

here is what i got ,
http://nbviewer.ipython.org/gist/anonymous/7739162f341d33ff7072
i can convert the filenames to datetime objects, but need a to store this on an axis.

I did a quick example available at:
http://nbviewer.ipython.org/gist/jstenar/80db55224cc45d603f98
I did not use touchstone files but the principles should be the same.

Very interesting to draw smith chart by Python. I try to run Smith.py but I error at line 6 in OS X and Window.  I used Python 2.7.10 (OSX) and 3.5.2 (Windows)
\smith.py", line 6, in  from hftools.plotting.helper import HFToolsAxes
ImportError: No module named 'hftools.plotting'


Make juttle-service-client a module.

Move most of the implementation of juttle-service-client a module so
it can be easily re-used in juttle-engine-client, which has most of
the same capabilities.
This module exports the following functions:

init: override the name of the option that specifies the
daemon (juttle-service or juttle-engine). Can also provide
additional commands and/or options (this will be used to add the
rendezvous options when used by juttle-engine).
usage: usage for the program.
handle_command: the switch block that does all the work based on command.

juttle-service-client is now limited to option capturing and calling handle_command.
This also removes the rendezvous functionality. We'll add it back in
juttle-engine, which has a app to which we want to push programs/watch
programs.
@demmer @go-oleg @VladVega

arg, the coverage thresholds dropped after I added everything to lib. Lemme make some unit tests.

Mike and I discussed in person and he gave a verbal +1 with the globals change and also testing for one of the built-in commands in usage. So I'll merge.


Include used options for each codec

Could you provide a table with compression/decompression options that were used with each codec for benchmarking?

All codecs are benchmarked using the default options; only the level varies.

So it just occurred to me that you may be referring to compiler options, not the codec options.  If so, https://quixdb.github.io/squash-benchmark/#faq-compiler-flags
That will change a bit for the next release; I'll drop -march=native -mtune=native since that's not an option for most people.  I'll also have to drop -flto because of a bug in CMake.  In the past it wasn't a problem, but we're trying to integrate library's build systems where possible, which means we end up linking plugins to a static library, and for LTO that requires using gcc-ar instead of ar, so we can't just add -flto to our cflags anymore.  So basically, we're back to just -O3 until CMake gets fixed.

Actually I was referrring to the codec options. Thanks for the explanation.


fix unit test

apparently not having a sensor can return noise. SO fix or remove.

Check - https://github.com/Arduino-CI/arduino_ci/blob/master/REFERENCE.md#wire

Check - https://github.com/Arduino-CI/arduino_ci/blob/master/REFERENCE.md#wire

Unit test is minimal but running now, Still to elaborate but that is true for all libraries.

Unit test is minimal but running now, Still to elaborate but that is true for all libraries.


parseDate and formatDate are not exposed in V3 when using require

In the documentation it says that parseDate and formatDate are exposed, but since v3 Flackpickr is initialised through a helper function, and therefor these methods are not reachable when using var flatpickr = require('flatpickr').

Seems like a have the same issue: the calendar never appears. Gonna try the 2nd version

Same here, it's a serious bug here, i can't use here.

Same issue for me :'(

I'm not sure what is the issue for you, but in my case the Flatpickr is working perfectly. It would just be more convenient to be able to access those two methods without creating an instance first. This feature was present in v2 but is missing in v3.

Bram-I is the same issue that you have reported, for us we need to access in those two methods.

Hi all,
Thanks for the notice.
I'll have these methods restored by v3.0.7 very soon.

Hi,
We still have the issue.
We have updated flatpickr. The package.json file, states that we have dowloaded the version 3.0.7, but the flatpickr.js file is the 3.0.6 version.

@chmln It looks like this commit was not included in the 3.0.7 release. I noticed that currently the library is being rewritten to Typescript. While I see this would be a great step forward, I also assume this will be a quite serious overhaul. Would you consider releasing a version including this fix prior to the new TS version?

Any news here?

same here: no way to access parseDate etc from Typescript.
These are the only exported objects, none of which have the static methods:
  import  {Instance, Options, CustomLocale, Locale} from 'flatpickr'

On it. The static methods will be a part of the next release :)

thanks, EXCELLENT plugin, and outstanding response.
btw: for now I'm using moment.js ,
e.g: given:
   config = {
      enableTime: true,
      dateFormat: 'm/d/Y at h:i K'
    }

I can get a Date like this:
      const date:Date = moment(x, ''MM/DD/YYYY at hh:mm a').toDate()

would be super cool if the same formatting string could be used !!
btw - I'm using vue-flatpickr-component, which also doesn't export the static methods (not a surpise).
my use case requires sending a standard Date object to and from the server, so I need some plumbing to both parse and format the date each way.

Yeah,  these methods slipped by in the big rewrite but no worries. And if you're just using moment for parsing or formatting, you should be able to ditch it for the built-in static methods.

actually I think I'm not using this component correctly at all.
The goal is to display a JSON stringified Date using the the format above,
then send the changed input back again as a JSON stringified Date.
I'm not letting users manually edit the component, so really there shouldn't be any parsing, just formatting for display, both what's being set both programatically, and manually, correct?
Dave

ok, so the part that's broken is th: vue-flatpickr-component
If I pass in a Date object it displays using the above format, but it return a string, ie: the exact string in the text box. Again, it's possible that I'm not using this correctly, but that seems odd.

@davidm-public flatpickr has a date format specifically for JSON-stringified ISO dates, you should be able to use that. flstpickr also stores the selected Date objects if you need them. Take a look at the docs if you're curious

btw, here is my solution, just to show you how it's handled in the Vue world:
<template>
  <flat-pickr v-model="date"
              :config="config"
  ></flat-pickr>
</template>

<script lang="ts">
  import Vue from 'vue'
  import { Component, Prop } from 'vue-property-decorator'
  import { Options } from "flatpickr"
  import moment from 'moment'

  @Component
  export default class DateTime extends Vue{

    @Prop({required: true}) iso_date_string: string

    config: Options = {
      enableTime: true,
      dateFormat: 'm/d/Y at h:i K',
    }

    readonly moment_format = 'MM/DD/YYYY at hh:mm a'

    get date(): Date{
      return this.iso_date_string ? new Date(this.iso_date_string) : null
    }

    set date( formatted_string ){
      const iso_string = moment(formatted_string, this.moment_format).toDate().toString()
      this.$emit('update:iso_date_string', iso_string)
    }
  }
</script>

<style>
</style>

Then it's used like this:
        <date-time :iso_date_string.sync="OPENING_TIME"></date-time>

now the variable OPENING_TIME is bound to whatever is set by flatpickr, and is always a json iso string.

Is it fixed? Cause I still can't see how to use these methods


add missing DCO file to repo root

We should have DCO in repo root in each repo. Here it is missing, add it.

/remove lgtm

/approve

/cc @lentzi90


[vm/ffi] Updating double-conversion causes std::cout to segfault in FFI calls on MacOS

Updating double-conversion causes std::cout to segfault in ffi calls.

https://dart-review.googlesource.com/c/sdk/+/221884
https://logs.chromium.org/logs/dart/buildbucket/cr-buildbucket/8829006337907998913/+/u/test_results/new_test_failures__logs_

Minimal repro (any FFI call to native code with std::cout).
import 'dart:ffi';

import 'ffi_test_helpers.dart';

void main() {
  final takeMaxUint8 = ffiTestFunctions
      .lookupFunction<IntPtr Function(Int8), int Function(int)>("TakeMaxUint8");
  takeMaxUint8(0xFF);
}

Initial investigation:

The FFI Trampolines are identical in IL and machine code.
Setting a breakpoint on TakeMaxUint8 gives a very similar looking registers and stack. So we don't have a misaligned stack or wrong stack height.
Using the the libffi_test_functions.dylib from the other (top of tree vs CL) does not make a difference. So the problem should be in the Dart VM, not in the dylib.

Earlier encounters with std::cout segfaults

https://github.com/dart-lang/sdk/issues/43075
misaligned stack, causing compiled C++ code which is supposed to refer to a pointer read garbage and try to read from wrong memory address (which is my hunch on what is happening here, but I don't see it.)

Blocking:

https://github.com/dart-lang/sdk/issues/38587

cc @rmacnak-google

I investigated the top two frames of the segfault and their machine code.
dart!std::__2::collate<wchar_t>::do_hash(wchar_t const*, wchar_t const*) const
dart!std::__2::__num_put<char>::__widen_and_group_int(char*, char*, char*, char*, char*&, char*&, std::__2::locale const&)

do_hash loops from the first to the second pointer. We segfault because 0x1 is passed in to the second argument, causing an infinite loop until we cross the memory page boundary causing invalid access.
__widen_and_group_int computes the argument with
    0x100019747: 48 63 05 9a c9 3f 02     movslq 0x23fc99a(%rip), %rax     ; std::__2::numpunct<char>::id + 8
    0x10001974e: 48 8d 50 ff              leaq   -0x1(%rax), %rdx

where id + 8 is 0x2 so the argument becomes 0x1. That also seems reasonable.
What does not seem reasonable is the call instruction leading to do_hash. When I open the source code in buildtools/mac-x64/clang/include/c++/v1/locale there is no call to do_hash in __widen_and_group_int.
I wonder if we have two different versions of libc++ linked somehow.
Then it would be likely that that is also the cause of std::cout issues in

https://github.com/dart-lang/sdk/issues/43075

This might actually be a bug in our toolchain.
Compiling a dynamic library which invokes std::cout in a separate CMake project on MacOS works fine when using XCode's clang version:
$ /Applications/Xcode13.1.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/c++ --version
Apple clang version 13.0.0 (clang-13<IP_ADDRESS>)

Adding the following the flags to the CMake invocation causes exactly the error that we're seeing:
-DCMAKE_CXX_COMPILER=/Users/dacoharkes/dart-sdk/sdk/buildtools/mac-x64/clang/bin/clang++
-DCMAKE_CXX_FLAGS="-Wl,/Users/dacoharkes/dart-sdk/sdk/buildtools/mac-x64/clang/lib/libc++.a"

Only adding the first flag makes the compilation fail with std::__2 symbols missing:
Undefined symbols for architecture x86_64:
  "std::__2::locale::use_facet(std::__2::locale::id&) const", referenced from:
      std::__2::ctype<char> const& std::__2::use_facet<std::__2::ctype<char> >(std::__2::locale const&) in mylib_dylib.cc.o
...

We started passing in a libc++.a manually in 73f6d15665a3c9a7cb521c90f7257c606c53171a.
Last clang roll was in ba0b0d8cc6d497cc1963c381f6ffcbc457b89c5e.
$ buildtools/mac-x64/clang/bin/clang++ --version
Fuchsia clang version 14.0.0 (https://llvm.googlesource.com/a/llvm-project e3a7f0e2f9ab566bd9b71fb54fe77e947b061a12)

@rmacnak-google why do we have to pass in lib/libc++.a in our build?

I'm not sure why we're explicitly passing libc++.a. IIRC, this also created some problems with the sanitizers, which disable it. I'll try building with the default.

Another thing I tried but didn't document yet: don't link in the standard library to the dynamic library and resolve symbols at runtime with -undefined dynamic_lookup. However, that makes lookup of std symbols fail when trying to load the dynamic library.
Unhandled exception:
Invalid argument(s): Failed to load dynamic library '/Users/dacoharkes/dart-sdk/sdk/xcodebuild/DebugX64/libffi_test_functions.dylib': dlopen(/Users/dacoharkes/dart-sdk/sdk/xcodebuild/DebugX64/libffi_test_functions.dylib, 0x0001): symbol not found in flat namespace '__ZNSt3__217bad_function_callD1Ev'
#0      _open (dart:ffi-patch/ffi_dynamic_library_patch.dart:12:43)
#1      new DynamicLibrary.open (dart:ffi-patch/ffi_dynamic_library_patch.dart:23:12)
#2      main (file:///Users/dacoharkes/dart-sdk/sdk/tests/ffi/aliasing_test.dart:14:32)
#3      _delayEntrypointInvocation.<anonymous closure> (dart:isolate-patch/isolate_patch.dart:297:19)
#4      _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:192:12)

已收到您的邮件，我会尽快回复。

If I try to use the default standard library, I get missing symbols at link time. Maybe the Clang we get from Fuchsia is using different headers by default?

I think this was fixed by b8ee3a999676b0e502c0a2a9cba9730374d9bf83.
New roll attempt in https://dart-review.googlesource.com/c/sdk/+/325543.


IR: How to represent control flow

We want to produce output in formats that require structured control flow (e.g., SPIR-V for Vulkan), and would also prefer readable output in text-based formats (HLSL, GLSL). As such it seems like we need to retain structured control-flow information in our IR while also retaining the ability to do optimizations.
There are a variety of ways we could try to accomplish this:
Build a CFG, but retain high-level structural information
We could lower functions to a conventional CFG of basic blocks (with predecessors/successors) but try to retain enough information on terminator instructions to allow the high-level control flow to be intuitied.
For example, we might have a dedicated branch instruction to represent an if that includes not only the true and false destinations, but also the "join point" where control flow resumes after the alternatives (this would be the same as the false destination in a one-sided if).
This should allow many textbook instruction-level optimizations to be applied using the CFG. We would need to be very careful about any optimizations that can change the CFG, to ensure that they don't break the properties we rely on.
Build an AST-like structure, but retain use-def information
The most important thing we want from an IR is really use-def information, ideally in SSA form so that we can simplify dataflow optimizations.
It is entirely possible to define an AST-like structure where the leaf "statements" are SSA instructions, and compound statements are expressed explicitly by recursively nesting statements.
In this setting any control-flow optimizations would have to be expressed at the AST level - potentially limiting what can be expressed, but guaranteeing that the results should be a valid AST.
Optimizations that require a more complete CFG might require that we build a CFG on-demand.
Other options?
There are of course many other options we could try, including various points on the spectrum between AST-like and CFG-like solutions. The big question is how will these choices affect our ability to do the main optimizations we want:

Specialization of callee functions to know values for logically-constant arguments, which seems to require some of the following
SSA-ification of local variables
Dead code elimination
Constant folding
Copy propagation?
Inlining?

Asides
One thing to keep in mind when designing an IR is that inlining can potentially create control flow that can't be expressed in the current high-level shading languages. This is because a return is effectively a "multi-level break" that can escape multiple levels of control-flow. If a function with a deeply-nested return gets inlined, then there may be no way to output that return as ordinary control-flow structures (in some cases you could wrap the inlined function in a dummy do { ... } while(false); and turn the return into a break, but that obviously won't work if there are intervening loops or switches).
As a result, if we go ahead and perform our own inlining, we may find that to generate output source code we need to either "outline" things again, or introduce extra Boolean values to direct control flow.
Another item to think about is whether to represent SSA via the conventional phi functions, or via a "blocks with parameters" approach. Blocks with parameters means that whenever a terminator instruction references a target block, it also provides a list of argument values to use as the parameters to that block. Such a representation directly encodes the "assignments on the edge" that are required to actually execute a program in SSA form. Blocks-with-parameters hilights the relationship between SSA form and continuation-passing style, and it also makes the entry block of a function seem like less of a special case (now all blocks have parameters that must be provided when jumping to them).
I don't have enough experience with any of these alternatives to make an educated choice, so I'd welcome feedback.

I've got some work in flight on this (in a branch that I hope to push today). I ended up doing the first option, where we have a more-or-less ordinary CFG with "terminator" instructions at the end of basic blocks, but where there are special cases of terminators to represent structured control flow.
Right now, I have the two basic unstructured cases:

unconditionalBranch <targetBlock>
conditionalBranch <condition> <trueBlock> <falseBlock>

(I haven't gotten to switch statements yet).
Those operations seldom get used directly, and instead most use sites involve one or more "derived" instructions that carry more information:

derived from unconditionalBranch:

break <target>
continue <target>
loop <target> <breakBlock> <continueBlock> represents the start of a structured loop, where breakBlock is the block after the loop, and continueBlock is where a continue will branch

derived from conditionalBranch

loopTest <condition> <loopBody> <breakBlock> represents the conditional used to implement the loop test in a for, while or do while loop
if <condition> <trueBlock> <afterBlock> represents a one-sided if statement
ifElse <condition> <trueBlock> <falseBlock> <afterBlock> represents a two-sided if statement

Note that in each case the derived ops use start with the same operands and given them the same meaning as the base cases, so in cases where we don't care about structured control flow we can just ignore the extra data. The added operands represent additional blocks we need to know about for structuring purposes.
We could conceivably add more specialized ops to capture other cases, but these seem to be the bare minimum required to re-discover structure during high-level code emit.

PR #186 added the control flow constructs listed above. Looking at the code, though, it seems that another pass is going to be needed here.
We probably can't special-case break or continue, because simple control flow simplifications (e.g., eliminating jumps to jumps) may create cases where a break isn't a standalone terminator instruction, but rather one of the two destinations of a conditional branch.
A similar argument says that loopTest isn't special (it is just an ifElse where we know one side branches to the break label outside the loop), and that having a distinct if and ifElse doesn't really help.

Doing the right thing here probably means tracking a set of "captured" labels during high-level code emission, and rewriting branches to these captured labels as break, continue, etc. as needed.

The IR control-flow representation is decent enough for our purposes right now. The next step would likely need to be a more careful "restructuring" process where we build explicit high-level control structures before emitting HLSL/GLSL, and then also add logic to fix up control flow that can't be directly supported in the downstream language (e.g., desugar multi-level breaks into ordinary breaks plus flags).
I'm closing this issue. We can open specific ones for next steps.


Add a way to see probe-rs logs.

There's no way to see logs from probe-rs.

--verbose doesn't work
RUST_LOG=trace doesn't work.

This is a blocker in helping people coming to the probe-rs matrix channel asking for help with flashing issues etc. There's no way for them to provide the logs we need without patching and recompiling probe-run.
This is due to this check here https://github.com/knurling-rs/probe-run/blob/ca0b3c53e28bf607a9db03c01798778a17a967c8/src/main.rs#L117-L121

Hi @Dirbaio, using #171 you should be able to get all logs by passing -vv to probe-run. Does this match your expactations?

Yes! Works great, thank you! :)


[TST] Add tests for ConfusionMatrix base class

Unit tests need to be implemented for the base-class used in the modelling of the confusion matrix [mmu.models.base].
Base class of BetaBinomialConfusionMatrix and  DirichletMultinomialConfusionMatrix

Moved to MMUR


Save data with encoding (utf-8)

I recently use this module for store user information, one of this is the customer name, first/last name and this fields store weird characters (latin and others). So when I call the service and get the json values, they look good, like this:
{
  "firstName": "Ingeniería",
}

And if I use this data with angular to render the view, works great (appear the 'í' char)
When I get the value from local storage I get this:
{
  "firstName": "Ingenier�a",
}

Any ideas, workarround, config?

My mistake... the rest service haven’t content-type header with the encoding.


Refactor PR 622

Partially reposted as part of PR #648 (except for the simplification around the handling of non_carbon_curves and carbon_curves, which requires some more careful thought).


Add the ability to search users with unwrapped string

Resolves #22
Copies getFollowers.js

This should be considered carefully because this can break old codes. It will not be merged right now.

This should be considered carefully because this can break old codes. It will not be merged right now.

Am I misunderstanding the function?
Wouldn’t all old code pass an object to getFollowers, and therefore be unchanged by this PR?

Sorry, I mischecked stuffs because I was on mobile.