Unable to set I2S samplerate under 2441Hz - ADC (IDFGH-995)

Environment

Development Kit:      ESP32-Gateway
Module or chip used:  ESP32-WROOM-32
IDF version (run git describe --tags to find it):
// v3.3-release

Problem Description
I want to use the I2S with internal or external 16bit ADC with samplerate of 1000 Hz. But based on the i2s driver the samplerate cannot be set under 2441 Hz.
In the line 317 from the i2s.c module this can be found:
    double clkmdiv = (double)I2S_BASE_CLK / (rate * factor);

    if (clkmdiv > 256) {
        ESP_LOGE(I2S_TAG, "clkmdiv is too large\r\n");
        return ESP_ERR_INVALID_ARG;
    }

With I2S_BASE_CLK=160MHz, factor=256 and based on the equation above 2441Hz will be the lowest which can be set.
I tried to use APLL with different fixed_mclk without success.
adc | sample_rate | fixed_mclk |
-- | -- | -- | --
internal | 2500 | 8000000 | works
internal | 2500 | 2500 | works
external | 2500 | 2500 | works
external | 2500 | 5000 | works
external | 1250 | 5000 | not work
external | 1250 | 50000 | not work
external | 1250 | 500000 | not work
external | 1250 | 600 | not work
internal | 1250 | 600 | not work
internal | 1250 | 5000 | not work
internal | 1250 | 5000000 | not work
Is this a harware issue? Or is it required a different configuration if we want to reach slower speeds?
Code to reproduce this issue
  if(type == I2S_EXTERNAL_ADC)
  {
    i2s_config_flags = (i2s_mode_t) (I2S_MODE_MASTER | I2S_MODE_RX);

    i2s_in_pin_config =
    {
      bck_io_num: ADC_SCLK,
      ws_io_num: ADC_CONVST,
      data_out_num: -1, //Not used
      data_in_num: ADC_DOUT
    };
  }
  else
  {
    i2s_config_flags = (i2s_mode_t) (I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_ADC_BUILT_IN);
  }

i2s_config_t i2s_config =
  {
    //.mode = (i2s_mode_t)(I2S_MODE_MASTER | I2S_MODE_RX | I2S_MODE_TX | I2S_MODE_ADC_BUILT_IN),  // I2S receive mode with ADC
    .mode = i2s_config_flags,  // I2S receive mode with ADC
    .sample_rate = 1000,                                                          // sample rate
    .bits_per_sample = I2S_BITS_PER_SAMPLE_16BIT,                                 // 16 bit I2S
    .channel_format = I2S_CHANNEL_FMT_ONLY_RIGHT,                                 // only the right channel+
    .communication_format = I2S_COMM_FORMAT_I2S_MSB,                                  //
    //.communication_format = I2S_COMM_FORMAT_I2S,                                  // I2S format
    .intr_alloc_flags = 0,                                                        // none
    //.intr_alloc_flags = ESP_INTR_FLAG_LEVEL1,                                                        // none
    .dma_buf_count = 16,                                                           // number of DMA buffers, 128 is the max
    .dma_buf_len = U8_NUM_SAMPLES,                                                   // number of samples, 1024 is the max
    //.use_apll = 0,                                                                // no Audio PLL
    .use_apll = true,                                                                // no Audio PLL
    .tx_desc_auto_clear = 1,     //I2S auto clear tx descriptor if there is underflow condition (helps in avoiding noise in case of data unavailability)
    .fixed_mclk = 1000
  };

  // install I2S 0 driver, using event queue
  i2s_driver_install(I2S_NUM_0, &i2s_config, 1, &i2s_event_queue);

Debug Logs
E (2765) I2S: clkmdiv is too large

E (2766) I2S: /Users/ficeto/Desktop/ESP32/ESP32/esp-idf-public/components/driver/i2s.c:844 (i2s_set_sample_rates):bits_per_sample not set
Guru Meditation Error: Core 1 panic'ed (IntegerDivideByZero). Exception was unhandled.

Hi,
Maybe your fixed_mclk is too small. can you try to set fixed_mclk = 40000000 ?

Any suggestion? Or would somebody link a documentation for APLL configuration?

Hi,@rob-bits
Do you need to change the sampling rate frequently?

Hi, @koobest
Frequently means each time when user want to change the acqusition speed. In the end software the user will have control on the samplerate. And probably the samplerate will be set to one value for minutes/hours but if the end-user want to measure at different sampling rate then he will change it.

Hi, @rob-bits
Can you accept the following solutions?  when the sampling rate is greater than 2441, use i2s_set_sample_rates to set the clock. when the sample rate is less than 2441,  you can configure the clock as follows(change the value of  clkm_div_num and tx_bck_div_num):
    i2s_stop(0);
   rtc_clk_apll_enable(1, 32, 32, 2, 31);
    I2S0.clkm_conf.clkm_div_num = 40;
    I2S0.clkm_conf.clka_en = 1;  //enable APLL
    I2S0.clkm_conf.clkm_div_b = 0;
    I2S0.clkm_conf.clkm_div_a = 1;
    I2S0.sample_rate_conf.tx_bck_div_num = 40;
    I2S0.sample_rate_conf.rx_bck_div_num = 40;
    i2s_start(0);

Hi, @koobest,
Thanks for the suggestion! I will try it out and get back to you.
If it is working then I will close this issue.

Hi @koobest,
I am trying your suggestion and seems it work. But I would like to control the requested frequency. Do you know how can I calculate the output frequency by equation? I tried the equation from the ESP32 technical reference manual from the chapter "12.3 The Clock of I2S Module" but I got different values. So the measured acqusition frequency and the calculated frequency are not equal.
Thank you in advance!

Hi @koobest, it's true that a low sample rate is not supported before release/v4.4, now the algorithm of APLL has been optimized, you can refer to clk_ctrl_os.c on master for the latest APLL algorithm and line 929/980 in i2s.c for the calculation of ADC clock.

@rob-bits Thanks for reporting, would you please help share if any updates for the issue? Thanks.

Thanks for reporting, will close due to short of feedback, feel free to reopen with more updates.


Anytype Docs - initial Hungarian translation - v0.1

[X] I understand that contributing to this repository will require me to agree with the CLA

Description
This PR adds translation for Anytype Docs: initial release of Hungarian language.
Please perform a check by the translated README.md, SUMMARY.md, vault-and-key.md, connect-with-us.md and views.md if the created fork was made properly.
What type of PR is this? (check all applicable)

[ ] 🍕 Feature
[ ] 🐛 Bug Fix
[X] 📝 Documentation Update
[ ] 🎨 Style
[ ] 🧑💻 Code Refactor
[ ] 🔥 Performance Improvements
[ ] ✅ Test
[ ] 🤖 Build
[ ] 🔁 CI

Related Tickets & Documents

Mobile & Desktop Screenshots/Recordings

Added tests?

[ ] 👍 yes
[X] 🙅 no, because they aren't needed
[ ] 🙋 no, because I need help

Added to documentation?

[ ] 📜 README.md
[X] 📓 tech-docs
[ ] 🙅 no documentation needed

[optional] Are there any post-deployment tasks we need to perform?

Hey @NoteMyBrain, thanks for the contribution.
Unfortunately, I can't accept this right now as it rewrites English version of the docs. Please check fr-FR and zh-CN folders to have a reference of how docs translation should be organized.


python312Packages.pylance: 0.17.0 -> 0.18.2

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python312Packages.pylance is: Python wrapper for Lance columnar format
meta.homepage for python312Packages.pylance is: https://github.com/lancedb/lance
meta.changelog for python312Packages.pylance is: https://github.com/lancedb/lance/releases/tag/v0.18.0
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.18.0 with grep in /nix/store/c4rf56k3rzw59017bly20izq4xmf56zb-python3.12-pylance-0.18.0
found 0.18.0 in filename of file in /nix/store/c4rf56k3rzw59017bly20izq4xmf56zb-python3.12-pylance-0.18.0

Rebuild report (if merged into master) (click to expand)

7 total rebuild path(s)

7 package rebuild(s)

First fifty rebuilds by attrpath

python311Packages.graphrag
python311Packages.lancedb
python311Packages.pylance
python312Packages.graphrag
python312Packages.lancedb
python312Packages.pylance

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/c4rf56k3rzw59017bly20izq4xmf56zb-python3.12-pylance-0.18.0 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python312Packages.pylance https://github.com/r-ryantm/nixpkgs/archive/e2a373bdda5da76797b22a6e78ea5ad6d73c6ee8.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/e2a373bdda5da76797b22a6e78ea5ad6d73c6ee8#python312Packages.pylance

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/c4rf56k3rzw59017bly20izq4xmf56zb-python3.12-pylance-0.18.0
ls -la /nix/store/c4rf56k3rzw59017bly20izq4xmf56zb-python3.12-pylance-0.18.0/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

8 packages failed to build:

python311Packages.graphrag
python311Packages.graphrag.dist
python311Packages.lancedb
python311Packages.lancedb.dist
python312Packages.graphrag
python312Packages.graphrag.dist
python312Packages.lancedb
python312Packages.lancedb.dist

4 packages built:

python311Packages.pylance
python311Packages.pylance.dist
python312Packages.pylance
python312Packages.pylance.dist

Maintainer pings
cc @natsukium for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.

Automatic update generated by nixpkgs-update tools. This update was made based on information from passthru.updateScript.
meta.description for python312Packages.pylance is: Python wrapper for Lance columnar format
meta.homepage for python312Packages.pylance is: https://github.com/lancedb/lance
meta.changelog for python312Packages.pylance is: https://github.com/lancedb/lance/releases/tag/v0.18.2
Updates performed

Ran passthru.UpdateScript

To inspect upstream changes
Impact
Checks done

built on NixOS
The tests defined in passthru.tests, if any, passed
found 0.18.2 with grep in /nix/store/p5i02g91wlmrdic08px3j9iksn35f6k7-python3.12-pylance-0.18.2
found 0.18.2 in filename of file in /nix/store/p5i02g91wlmrdic08px3j9iksn35f6k7-python3.12-pylance-0.18.2

Rebuild report (if merged into master) (click to expand)

7 total rebuild path(s)

7 package rebuild(s)

First fifty rebuilds by attrpath

python311Packages.graphrag
python311Packages.lancedb
python311Packages.pylance
python312Packages.graphrag
python312Packages.lancedb
python312Packages.pylance

Instructions to test this update (click to expand)

Either download from Cachix:
nix-store -r /nix/store/p5i02g91wlmrdic08px3j9iksn35f6k7-python3.12-pylance-0.18.2 \
  --option binary-caches 'https://cache.nixos.org/ https://nix-community.cachix.org/' \
  --option trusted-public-keys '
  nix-community.cachix.org-1:mB9FSh9qf2dCimDSUo8Zy7bkq5CX+/rkCWyvRCYg3Fs=
  cache.nixos.org-1:6NCHdD59X431o0gWypbMrAURkbJ16ZPMQFGspcDShjY=
  '

(The Cachix cache is only trusted for this store-path realization.)
For the Cachix download to work, your user must be in the trusted-users list or you can use sudo since root is effectively trusted.
Or, build yourself:
nix-build -A python312Packages.pylance https://github.com/r-ryantm/nixpkgs/archive/cc7df8855f09b603314117242629fb1e7c60dbde.tar.gz

Or:
nix build github:r-ryantm/nixpkgs/cc7df8855f09b603314117242629fb1e7c60dbde#python312Packages.pylance

After you've downloaded or built it, look at the files and if there are any, run the binaries:
ls -la /nix/store/p5i02g91wlmrdic08px3j9iksn35f6k7-python3.12-pylance-0.18.2
ls -la /nix/store/p5i02g91wlmrdic08px3j9iksn35f6k7-python3.12-pylance-0.18.2/bin

Pre-merge build results
We have automatically built all packages that will get rebuilt due to
this change.
This gives evidence on whether the upgrade will break dependent packages.
Note sometimes packages show up as failed to build independent of the
change, simply because they are already broken on the target branch.
Result of nixpkgs-review run on x86_64-linux 1

8 packages failed to build:

python311Packages.graphrag
python311Packages.graphrag.dist
python311Packages.lancedb
python311Packages.lancedb.dist
python312Packages.graphrag
python312Packages.graphrag.dist
python312Packages.lancedb
python312Packages.lancedb.dist

4 packages built:

python311Packages.pylance
python311Packages.pylance.dist
python312Packages.pylance
python312Packages.pylance.dist

Maintainer pings
cc @natsukium for testing.

[!TIP]
As a maintainer, if your package is located under pkgs/by-name/*, you can comment @NixOS/nixpkgs-merge-bot merge to automatically merge this update using the nixpkgs-merge-bot.

Add a :+1: reaction to pull requests you find important.


The problem appears when I run tmux

/home/adminyk/.tmux.conf:16: unknown option: focus-events
/tmp/tmp.JE0lKlhKpK.in:2: invalid or unknown command: bind-key          " split-window -c '#{pane_current_path}'

CentOS without X

Sorry, please close this issue - I had old version tmux - 1.8. Now is Okay.


Tabs refactor

Style cleanup (I can reverse it if needed)

Don't assume $ and leverage jQuery instead.

Works even before DOM is completely ready!

Switched to hashchange listening instead of delegation
We still check that the given hash exists as a href within a valid Skeleton tabs markup, but you can now send links around that focus on tabs.
As an example, if I tweet out mypage.com/#activetab, it will check that #activetab is a valid href in an anchor within an <ul class="tabs"> element, and if so, make that tab active.

In addition, you can have multiple anchors in the page that trigger tabs by simply setting the appropriate href.

Leverage closest and find instead of parent, siblings and children (code simplification).

Thanks so much for the contribution man!


Test times out - Foreign Key constraint test on Postgres tables with 800k+ rows

I am unable to get Postgres to finish running foreign key constraint on two tables with <1M rows runs.
The current implementation uses a subquery to identify pk-fk mismatches.  I assume that Snowflake handles these subqueries w/out a hitch; Postgres struggles to do so.  Some brief testing shows me that it works fine for tables with 50k rows, but @ 100k+ it's not functional.
Could this could be refactored to reflect the implementation of the relationships test built-in to dbt-core?
Looks like they started with the same method but eventually moved to a left join several years ago - see here and here
It's not clear whether the left join approach is equally beneficial for snowflake.  I will make a pull request with this change - can the owners of this repo let me know if the change should be made under default__test_constraints.sql or only for postgres via the adapter dispatch (i.e. postgres__create_constraints.sql ?

I should first point out that you can use the relationship test to create all your fk constraints unless you need to create a compound constraint with multiple columns. That is the main advantage over the out of the box test. I am traveling home from a vacation but I will review the links you provided on the outer join this weekend. Snowflake is ok with that way of testing a constraint in an out join.

Thank you Dan!  Good to know re: single- and multi-column FK constraints.  This package has seriously improved my quality of life :-]


Update C3, C4 and DSE dependencies by Stargate

Due to work needed to resolve CVE-2021-44521 (see f.ex https://nvd.nist.gov/vuln/detail/CVE-2021-44521), we need to upgrade dependencies Stargate has. New versions to use are:

Cassandra-3.11: 3.11.12 (or later)
Cassandra-4.x: 4.0.3 (or later)
DSE: 6.8.20 (or later)

Initial findings suggest that neither 3.11.12 nor 4.0.2 work without patching of Stargate: both upgrade introduce new test failures. :-(

Note: bumped 4.0.2 -> 4.0.3 since that appears to be the real target.

https://issues.apache.org/jira/browse/CASSANDRA-17248 changed some stuff in prepared statements, changes from that might be the cause.

@JeremiahDJordan Could be. @ivansenic added detailed description of observed issue on #1647 -- it almost sounds like "fix to the incomplete fix" might have produced another race condition somehow... :-(

Done in the three PRs above.


update deps

Release "Mildly Choice Yeti"

Project
New Version

membrane_core
0.6.0

unifex
0.3.0

membrane_common_c
0.5.0

proudly automated with Sebex

Done in #12


proc_open(): fork failed - Cannot allocate memory

Using version ^0.1.12 for hirak/prestissimo
./composer.json has been updated
Loading composer repositories with package information
Updating dependencies (including require-dev)

Installing hirak/prestissimo (0.1.12)
Downloading: 100%
The following exception is caused by a lack of memory and not having swap configured
Check https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors for details
The following exception is caused by a lack of memory and not having swap configured
Check https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors for details
PHP Fatal error:  Uncaught exception 'ErrorException' with message 'proc_open(): fork failed - Cannot allocate memory' in phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:940
Stack trace:
#0 [internal function]: Composer\Util\ErrorHandler::handle(2, 'proc_open(): fo...', 'phar:///usr/loc...', 940, Array)
#1 phar:///usr/local/bin/composer/vendor/symfony/console/Application.php(940): proc_open('stty -a | grep ...', Array, NULL, NULL, NULL, Array)
#2 phar:///usr/local/bin/composer/vendor/symfony/console/Application.php(748): Symfony\Component\Console\Application->getSttyColumns()
#3 phar:///usr/local/bin/composer/vendor/symfony/console/Application.php(709): Symfony\Component\Console\Application->getTerminalDimensions()
#4 phar:///usr/local/bin/composer/vendor/symfony/console/Application.php(639): Symfony\Component\Console\Application->getTerminalWidth()
#5 phar:///usr/local/bin/composer/src/Composer/Console/Application.php(231): Symfony\Component\Console\Application->renderException(Object(ErrorException), Ob in phar:///usr/local/bin/composer/vendor/symfony/console/Application.php on line 940

Fatal error: Uncaught exception 'ErrorException' with message 'proc_open(): fork failed - Cannot allocate memory' in phar:///usr/local/bin/composer/vendor/symfony/console/Application.php on line 940
ErrorException: proc_open(): fork failed - Cannot allocate memory in phar:///usr/local/bin/composer/vendor/symfony/console/Application.php on line 940
Call Stack:
0.0004     223440   1. {main}() /usr/local/bin/composer:0
0.0480     408808   2. require('phar:///usr/local/bin/composer/bin/composer') /usr/local/bin/composer:25
0.1498    3767264   3. Composer\Console\Application->run() phar:///usr/local/bin/composer/bin/composer:43
0.1668    4119528   4. Symfony\Component\Console\Application->run() phar:///usr/local/bin/composer/src/Composer/Console/Application.php:98
73.8030  208712688   5. Composer\Console\Application->renderException() phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:127
73.8038  208712864   6. Symfony\Component\Console\Application->renderException() phar:///usr/local/bin/composer/src/Composer/Console/Application.php:231
73.8039  208713216   7. Symfony\Component\Console\Application->getTerminalWidth() phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:639
73.8039  208713216   8. Symfony\Component\Console\Application->getTerminalDimensions() phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:709
73.8039  208713216   9. Symfony\Component\Console\Application->getSttyColumns() phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:748
73.8039  208715176  10. proc_open() phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:940
73.8045  208716312  11. Composer\Util\ErrorHandler::handle() phar:///usr/local/bin/composer/vendor/symfony/console/Application.php:940

Please read the documentation. This is not a bug.
https://getcomposer.org/doc/articles/troubleshooting.md#proc-open-fork-failed-errors

fafadsfd


Feature request: Disable gnucash numpad

Steps to reproduce the behaviour
Do something that pulls up the custom gnucash numpad (say, add a transaction, and click the amount box)
Expected behaviour
The gnucash onscreen numpad doesnt pop up, allowing me to type in a number with my physical keypad, which should also be autoset to number input mode.
Actual behaviour
The gnucash custom numpad appears, taking up half the screen. When i try to enter numbers on the physical numpad (i use a galaxy folder 2, one of the few android flipphones), it defaults to text-input, causing me to have to switch input mode.
The suggested solution would be a switch in the settings disabling the custom numpad, and if toggled, instead when you click a number box, just telling the android keyboard to pull up a number input (and not pulling up the gnucash one), which should allow us few smart-flipphone (or blackberry, i guess) users to use our keypads.
Software specifications

GnuCash Android version: 2.4.0
System Android version: 7.1.1
Device type: Galaxy folder 2 G1650

AFAIK the custom keyboard was used to deal issues with older default Samsung Keyboard (pre-TouchWiz / Samsung Experience 6.0; equivalent to Android Marshmallow for high-end Galaxy devices or Nougat for others) where punctuation marks cannot be entered from the numeric keypad.

But as some custom keyboards allow advanced inputs in their numeric keypad, I also agree to include this feature. By the way, Samsung has already fixed this issue, though.


Update DC/OS to track Mesos 1.8.x release branch.

High-level description
In preparation for the upcoming 1.13 release of DC/OS,
update the Mesos package to track the Mesos 1.8 release
branch which will be the version included in the release.
Corresponding DC/OS tickets (obligatory)
These DC/OS JIRA ticket(s) must be updated (ideally closed) in the moment this PR lands:

DCOS-51386 Update DC/OS to track the Mesos 1.8 release branch

Checklist for all PRs

[ ] Added a comprehensible changelog entry to CHANGES.md or explain why this is not a user-facing change:
[ ] Included a test which will fail if code is reverted but test is not. If there is no test please explain here:
[x] Read the DC/OS contributing guidelines
[x] Followed relevant code rules Rules for Packages and Systemd

Checklist for component/package updates:
If you are changing components or packages in DC/OS (e.g. you are bumping the sha or ref of anything underneath packages), then in addition to the above please also include:

[ ] Change log from the last version integrated (this should be a link to commits for easy verification and review): example
[ ] Test Results: [link to CI job test results for component]
[ ] Code Coverage (if available): [link to code coverage report]

PLEASE FILL IN THE TEMPLATE ABOVE / DO NOT REMOVE ANY SECTIONS ABOVE THIS LINE
Instructions and review process
What is the review process and when will my changes land?
All PRs require 2 approvals using GitHub's pull request reviews.
Reviewers should be:

Developers who understand the code being modified.
Developers responsible for code that interacts with or depends on the code being modified.

It is best to proactively ask for 2 reviews by @mentioning the candidate reviewers in the PR comments area. The responsibility is on the developer submitting the PR to follow-up with reviewers and make sure a PR is reviewed in a timely manner. Once a PR has 2 ship-it's, no red reviews, and all tests are green it will be included in the next train.

@mesosphere-mergebot bump-ee

@mesosphere-mergebot sync

@mesosphere-mergebot label Ready For Review


Can I use  CompreFace API to detect faces and extract vectors so I can then add then in FAISS to search a lot of faces?

Can I use  CompreFace API to detect faces and extract vectors so I can then add then in FAISS to search a lot of faces?
Is there an API call to get a pure vector?

yes, you can use a detection service and use face_plugins=calculator parameter

What is dimension/size of a vector that it returns?
Have anyone tried to store it in Faiss or milvus?

FaceNet and ArcFace r100 vectors are 512 and the MobileNet ArcFace vector is 128 long.
I don't see any problems why they can't be stored in Faiss or Milvus

FaceNet and ArcFace r100 vectors are 512 and the MobileNet ArcFace vector is 128 long. I don't see any problems why they can't be stored in Faiss or Milvus

Thanks!)


Add validation test for etcd backup and restore with k8s upgrade

Issue:

https://github.com/rancher/qa-tasks/issues/406
Note:

This validation test takes around 20 mins to complete which is more than default timeout for the test that's why we need to  use -timeout option with value either greater than 20mins or 0. Similar to below command
go test providers.go etcd_bkp_restore_test.go -v -timeout 0
Also it will also require at least two k8s version in ascending order in the cattle config file and at least one CNI provider as per below:

provisioningInput:
   kubernetesVersion:
    - v1.22.13+rke2r1
    - v1.23.10+rke2r1
  cni:
    - calico

A few nitpicks and comments, but overall looks good

@caliskanugur @igomez06, I've added back the CreateDeployment helper function back, since it was getting used in other tests.

@caliskanugur @igomez06, I've added back the CreateDeployment helper function back, since it was getting used in other tests.

I don't see it, and I think it's better not to have it. It looks like you did your tests just fine using the templates instead.

@caliskanugur, @igomez06 I've shared the latest jenkins run, can you please check.

Made the changes requested in the backport PR here as well.

Backport: #39636


Detect Computer

Hi,
Your gem is just awesome. I just want to know how can i detect if the request was initiated from a computer.
At the moment i can see methods that detect it its a mobile, tablet, console, tv etc etc but i cant find computer there.
Can you please help with this.

If it's none of the others, I guess you can assume it's a computer?

So, detecting things like computer? or web? or app? is really hard because there are too many things to consider (all platforms, all devices, etc).
I don't think we can do this, but I'm open to suggestions.


Link broken for ConvertAnITKGrayScaleImageToCVMat example

http://public.kitware.com/pub/itk/Examples/src/Video/BridgeOpenCV/ConvertAnITKGrayScaleImageToCVMat/ConvertAnITKGrayScaleImageToCVMat.zip

The dashboard build that generates the output now builds with ITKExamples_USE_OpenCV:BOOL=ON, and the file is available.


Blank page on connection

Hello!
I'm trying to run Hauk with your docker image but I always end up on a blank page and when I try to start sharing my location with my phone I get an an error. Here are my logs:
docker logs -f hauk
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using <IP_ADDRESS>. Set the 'ServerName' directive globally to suppress this message
AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using <IP_ADDRESS>. Set the 'ServerName' directive globally to suppress this message
[Thu Aug 29 16:37:41.131733 2019] [mpm_prefork:notice] [pid 14] AH00163: Apache/2.4.38 (Debian) PHP/7.3.8 configured -- resuming normal operations
[Thu Aug 29 16:37:41.131782 2019] [core:notice] [pid 14] AH00094: Command line: '/usr/sbin/apache2 -D FOREGROUND'
<IP_ADDRESS> - - [29/Aug/2019:16:37:47 +0000] "GET / HTTP/1.1" 200 1378 "-" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0"
<IP_ADDRESS> - - [29/Aug/2019:16:37:47 +0000] "GET /style.css HTTP/1.1" 200 1446 "https://my.super.address/" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0"
<IP_ADDRESS> - - [29/Aug/2019:16:37:47 +0000] "GET /main.js HTTP/1.1" 200 3064 "https://my.super.address/" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0"
<IP_ADDRESS> - - [29/Aug/2019:16:37:47 +0000] "GET /dynamic.js.php HTTP/1.1" 200 679 "https://my.super.address/" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0"
<IP_ADDRESS> - - [29/Aug/2019:16:37:47 +0000] "GET /assets/logo.svg HTTP/1.1" 304 126 "https://my.super.address/" "Mozilla/5.0 (X11; Fedora; Linux x86_64; rv:68.0) Gecko/20100101 Firefox/68.0"

I used your config.php just changed password (and it seems to work as if I put the wrong password in the mobile app I get a proper error) and the public url to match mine.
Here is a mobile screenshot:

That's strange. Apparently the server is correctly sending the handshake reply to the app, but the app is interpreting it as an error. It's a pretty easy client-side fix, but I'm still curious about what exactly is being returned here.
First, I'd like to rule out the app as the faulty part. Is it possible for you to try the app against the demo server? If that succeeds, something is wrong with the server stack (either the image itself, or some proxy in front of it). If it fails, the app is broken.
Next, if the demo server does work - is there any chance you could run this command against your own server in your terminal and paste the output here? It will give me a hexdump of the entire HTTP handshake reply, but with your hostname removed.
Just replace YOUR_PASSWORD and YOUR_SIDE_ADDRESS accordingly.
curl -X POST https://YOUR_SITE_ADDRESS/api/create.php -F pwd=YOUR_PASSWORD -F dur=10 -F int=1 -D - 2>/dev/null | sed -e 's/YOUR_SITE_ADDRESS/redacted-hostname/g' | hexdump -C
If you want to see what data you're sending me first, just remove | hexdump -C from the end. The hexdump is so I can see if there are any stray bytes in the response that could arise from something funky going on with Apache.
Finally, the blank page - could you check if there are any errors in the Network or Console tabs of developer tools (F12)? Might have to reload the page with those open for something to show up.

That's strange. Apparently the server is correctly sending the handshake reply to the app, but the app is interpreting it as an error. It's a pretty easy client-side fix, but I'm still curious about what exactly is being returned here.
First, I'd like to rule out the app as the faulty part. Is it possible for you to try the app against the demo server? If that succeeds, something is wrong with the server stack (either the image itself, or some proxy in front of it). If it fails, the app is
broken.

The mobile app is working against the demo server so we can rule that out.

Next, if the demo server does work - is there any chance you could run this command against your own server in your terminal and paste the output here? It will give me a hexdump of the entire HTTP handshake reply, but with your hostname removed.
Just replace YOUR_PASSWORD and YOUR_SIDE_ADDRESS accordingly.
curl -X POST https://YOUR_SITE_ADDRESS/api/create.php -F pwd=YOUR_PASSWORD -F dur=10 -F int=1 -D - 2>/dev/null | sed -e 's/YOUR_SITE_ADDRESS/redacted-hostname/g' | hexdump -C

00000000  48 54 54 50 2f 32 20 32  30 30 20 0d 0a 63 6f 6e  |HTTP/2 200 ..con|
00000010  74 65 6e 74 2d 74 79 70  65 3a 20 74 65 78 74 2f  |tent-type: text/|
00000020  68 74 6d 6c 3b 20 63 68  61 72 73 65 74 3d 55 54  |html; charset=UT|
00000030  46 2d 38 0d 0a 64 61 74  65 3a 20 46 72 69 2c 20  |F-8..date: Fri, |
00000040  33 30 20 41 75 67 20 32  30 31 39 20 30 37 3a 35  |30 Aug 2019 07:5|
00000050  32 3a 34 33 20 47 4d 54  0d 0a 73 65 72 76 65 72  |2:43 GMT..server|
00000060  3a 20 43 61 64 64 79 0d  0a 73 65 72 76 65 72 3a  |: Caddy..server:|
00000070  20 41 70 61 63 68 65 2f  32 2e 34 2e 33 38 20 28  | Apache/2.4.38 (|
00000080  44 65 62 69 61 6e 29 0d  0a 76 61 72 79 3a 20 41  |Debian)..vary: A|
00000090  63 63 65 70 74 2d 45 6e  63 6f 64 69 6e 67 0d 0a  |ccept-Encoding..|
000000a0  78 2d 70 6f 77 65 72 65  64 2d 62 79 3a 20 50 48  |x-powered-by: PH|
000000b0  50 2f 37 2e 33 2e 38 0d  0a 63 6f 6e 74 65 6e 74  |P/7.3.8..content|
000000c0  2d 6c 65 6e 67 74 68 3a  20 31 30 37 0d 0a 0d 0a  |-length: 107....|
000000d0  0a 4f 4b 0a 65 36 66 61  61 33 32 38 64 36 33 36  |.OK.e6faa328d636|
000000e0  33 65 31 35 35 64 65 33  65 65 39 30 64 30 36 64  |3e155de3ee90d06d|
000000f0  65 63 66 33 64 30 62 63  30 31 66 33 37 37 64 38  |ecf3d0bc01f377d8|
00000100  37 66 61 33 34 32 39 32  30 33 38 34 62 37 39 31  |7fa342920384b791|
00000110  33 35 39 62 0a 68 74 74  70 73 3a 2f 2f 72 65 64  |359b.https://red|
00000120  61 63 74 65 64 2d 68 6f  73 74 6e 61 6d 65 2f 3f  |acted-hostname/?|
00000130  36 36 50 37 2d 53 57 43  53 0a                    |66P7-SWCS.|
0000013a

If you want to see what data you're sending me first, just remove | hexdump -C from the end. The hexdump is so I can see if there are any stray bytes in the response that could arise from something funky going on with Apache.
Finally, the blank page - could you check if there are any errors in the Network or Console tabs of developer tools (F12)? Might have to reload the page with those open for something to show up.

Okay, something interesting here, here is what I get in my dynamic.js.php:

Warning:  Cannot modify header information - headers already sent by (output started at /etc/hauk/config.php:78) in /var/www/html/dynamic.js.php on line 7

var TILE_URI = "https:\/\/{s}.tile.openstreetmap.org\/{z}\/{x}\/{y}.png";
var ATTRIBUTION = "";
var DEFAULT_ZOOM = 14;
var MAX_ZOOM = 19;
var MAX_POINTS = 100;
var VELOCITY_DELTA_TIME = 2;
var TRAIL_COLOR = "#d80037";
var VELOCITY_UNIT = {"havMod":45871200,"mpsMod":3.6,"unit":"km\/h"};

Thanks a lot for the helpful hexdump and error message! Looks like there is indeed a stray newline in the output:
0x000000cc: 0d 0a 0d 0a 0a 4f 4b
                        ^^

And this also connects to the warning you see in dynamic.js.php - there's a newline apparently coming from after the ?> tag at the end of config.php; Hauk tries to send some headers after the config file has started outputting this newline.
Might be possible to solve this if the PHP tag isn't closed by that tag. Can you try just removing the ?> at the end of the config.php file and see if that fixes the issue?

It fixes it. And yes there was a trailing line.
Thank you very much !
Out of curiosity, how did you get that:
0x000000cc: 0d 0a 0d 0a 0a 4f 4b
                                         ^^

Awesome! I'll patch the default config so other don't get this issue.

Out of curiosity, how did you get that:

That was a snip from your hexdump:

HTTP headers are delimited by the body with "\r\n\r\n" or 0x0d 0x0a 0x0d 0x0a. The start of line 0x000000d0 shows an extra 0x0a; the newline that wasn't supposed to be there, which was then included as part of the body.


更新版本后数据问题

Question
你们版本更新为什么获取的数据会被添加output 从2.4.0到3.0.1版本更新后出现的问题 数据格式改了嘛？

An online translator gives the following translation:
Data problems after updating the version
Why is the data obtained by your version update added to the output? The problem occurred after the update from version 2.4.0 to 3.0.1. Has the data format changed?
(Not an optimal translation but maybe we can say something in this regard)

@admins123C Maybe this migration information here helps, as the output indeed has changed (hence the SemVer major release): https://github.com/geostyler/geostyler-sld-parser/releases/tag/v3.0.0


Update FairnessDashboard for named argument MetricFrame

A recent update to Fairlearn has started the process of moving to requiring named arguments for MetricFrame. Update FairnessDashboard to match.
Closes #770

And I've run the two modified notebooks, and checked that the dashboard is no longer emitting warnings.


Added cross button in color gradient platte

PR Description 📜
Please include summary related to the issue you have fixed and describe your PR in brief over here by specifying the issue number on which you were working below -
Fixes issue #84 , added cross button in gradient palette, even if both colors are selected and we press this cross button, those colors will not be changed. There is the apply gradient button for that. This cross button correctly closes the gradient palette without any problem.

Mark the task you have completed ✅

[x] I follow CONTRIBUTING GUIDELINE & CODE OF CONDUCT of this project.
[x] I have performed a self-review of my own code or work.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] My changes generates no new warnings.
[x] I have followed proper naming convention showed in CONTRIBUTING GUIDELINE
[x] I have added screenshot for review.

## Add your screenshots(Optional) 📸
Screenshot is added in comments

Thank you soo much for contributing to our repository 💗

Here is the screenshot for the feature and proper working/testing has been performed by me before submitting PR.
Please label it under Gssoc and assign a level and merge into main branch @Durgesh4993

@Durgesh4993 where should i add the demo video?

Yes

@Durgesh4993 added changes.

@Durgesh4993 please merge this PR under Gssoc24

@Durgesh4993


Update: Remove dao dependency from txn table

@robert-zaremba Please review it.

@robert-zaremba Can you approve it?


error : You must provide a private key with -key-name or -key-file to execute this command

hello,
since few hours, i can't execute any command except wake.
i have this error:
You must provide a private key with -key-name or -key-file to execute this command
my command line is:
./lib/tesla_api/tesla-control/tesla-control -key-file ./lib/tesla_api/TeslaKeys/privatekey.pem -token-file .temp_token -debug -vin XP7XXXXXXNBXXX123 charging-start'
before, all worked good, i could change amps, stop and start the charge
with debug arrg, i have this:
no error, Server returned 200: OK:
except the last line:
2024-02-23T13:12:00+01:00 [warn ] No session available for DOMAIN_INFOTAINMENT
thank you for your help
jerome

It looks like the problem is that the client ignores handshake responses from the server if there's too much latency. This is by design. But it still behaves as though the server completed its handshake and continues, which is a bug.
You can increase these two timeout values as a work-around until the bug is fixed. They should probably be higher, anyway.
Thanks for your report!

Hello, i have increase these Two timeout values (5 -> 15), it's ok, i can again send orders to my car
thank you
jerome


remove redundant nesting

This removes some redundant if nesting.
Edit: Is this inteantional? I found that in a lot of places

Thanks for diving into this. These are defects and they need to be corrected.


MB-15671 Replace MakeMove helpers

Jira ticket for this change
Summary
This ticket replaces some of the Make*MoveWithShipment helper functions. Since we no longer use selectedMoveType, most of these can be consolidated. The old MakeHHGMoveWithShipment allowed move assertions to be passed in, and the new BuildMoveWithShipment function calls BuildMTOShipmentWithMove (which doesn't allow move customizations). So we added a new function to remove the customizations of a specific type from the customs slice.
Setup to Run Your Code

Run make server_test and make sure tests pass
Run make db_dev_e2e_populate and make sure the db populates correctly

Verification Steps for Author
These are to be checked by the author.

[ ] Tested in the Experimental environment (for changes to containers, app startup, or connection to data stores)
[ ] Request review from a member of a different team.
[ ] Have the Jira acceptance criteria been met for this change?

Verification Steps for Reviewers
These are to be checked by a reviewer.

Frontend

[ ] User facing changes have been reviewed by design.
[ ] There are no aXe warnings for UI.
[ ] This works in Supported Browsers and their phone views (Chrome, Firefox, IE, Edge).
[ ] There are no new console errors in the browser devtools
[ ] There are no new console errors in the test output
[ ] If this PR adds a new component to Storybook, it ensures the component is fully responsive, OR if it is intentionally not, a wrapping div using the officeApp class or custom min-width styling is used to hide any states the would not be visible to the user.

Backend

[ ] Code follows the guidelines for Logging
[ ] The requirements listed in Querying the Database Safely have been satisfied.

Database
Any new migrations/schema changes:

[ ] Follows our guidelines for Zero-Downtime Deploys
[ ] Have been communicated to #g-database
[ ] Secure migrations have been tested following the instructions in our docs

Screenshots

Messages

:book:
:link: MB-15671

  Generated by :no_entry_sign: dangerJS against d491d45746c025795c68be69ae6c4e2d1c42786f


Missing binary sensors

Home assistant core version: 2022.11.4
Thermiq: 2.0.5
FW: 1.4?
After the latest update all my thermiq binary sensors got replaced by regular sensors. An example:
sensor.thermiq_mqtt_vp1_boiler_6kw_on
This sensor used to be binary_sensor type previously. It seems that the new sensor is working and updating On and Off values correctly.
Is this the way it should be working from now one or is this a bug? Similar issues for someone else?

This is intended for now. The binary sensor has been replace by a normal sensor with 0 and 1 as valid values.
Are there any drawbacks by doing it like this?
Rgds
/&&

Ok, thanks for the info.
I am surely not an expert not an expert to answer for this. I took the opportunity to ask from discord channel and here is an answer: "(binary sensor) restricts the range of possible values, so don't need to worry about getting random values. binary_sensor also has a bunch of standard translations based on device_class that sensor doesn't have"
Based on this I would say that the latter could be a possible advantage of using binary sensors instead of just sensors.
And the out of scooe values could be a problem but impossible to say without knowing how HW works.
Some more info on developer portal: https://developers.home-assistant.io/docs/core/entity/binary-sensor/

I would also suggest that all sensors that have a state of 1 or 0 should be a binary sensor. IMO it is the most correct :)

Ok, I'll put it on the backlog but with low priority until base functionality is in place and verified by some more people

@ThermIQ any news on this :)

@ThermIQ any news :)

No, feel free to implement and push a change

Another point that increases the priority is that groups of sensors are not working as they are expecting numerical values not on/off states.

A new beta is available with binary_sensors, @tomrosenback please check it out

You are referring to 2.1.1? It seems to work but looking at the code it seems like the BinarySensor entity is missing the is_on property which makes the states be strings instead of booleans.

Aha,That makes sense, I thought behaviour was odd, will fix later todayRgds/&&Sent from my Galaxy
-------- Original message --------From: Tom Rosenback @.> Date: 08/01/2024  10:25  (GMT+01:00) To: ThermIQ/thermiq_mqtt-ha @.> Cc: ThermIQ @.>, Mention @.> Subject: Re: [ThermIQ/thermiq_mqtt-ha] Missing binary sensors (Issue #43)
You are referring to 2.1.1? It seems to work but looking at the code it seems like the BinarySensor entity is missing the is_on property which makes the states be strings instead of booleans.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>

Tested 2.1.2 now, works perfect! Thank you!

Great! Summary of how time was spent:

1 Year stalling the task
4h to get the development environment running again adding some internal docs on How-to
2h Updating the development environment to latest HA
15min Coding of feature
1h Debug
-- Release and repeat
15min Coding and debug

With my new internal How-to I expect that I can shorten the development time for a feature to an hour or so. Still no idea how to address the 1 Year stalling time :-)
/&&

Also running version 2.1.2 here without any issues so far:) great work :)


Can’t open the tutorial file

Hello！
Why can’t I open the tutorial file directly from the README anymore? It shows a 404 error. Is there an issue?
Thanks!

Hi, it was because the package has been moved to the lab's github page, and only supports for v1.0.4. I have hyperlink in the readme page, it should work now and can direct you to the new tutorial.


Could not load local module, with RuntimeError: Missing implementation that supports: loader(*('/tmp/tfhub_modules/mobilenet_module',), **{})

Hi,
I tried this tutorial with a little bit change: https://www.tensorflow.org/hub/tutorials/image_retraining
I download the mobilenet module(mobilenet_v2_1.4_224.tgz) and decompressed under the folder /tmp/tfhub_modules/mobilenet_module.
Run the retrain.py with "python retrain.py --image_dir flower_photos --tfhub_module /tmp/tfhub_modules/mobilenet_module", but got following error:
Traceback (most recent call last):
File "retrain.py", line 1318, in
tf.app.run(main=main, argv=[sys.argv[0]] + unparsed)
File "/usr/local/lib/python3.5/dist-packages/tensorflow/python/platform/app.py", line 125, in run
_sys.exit(main(argv))
File "retrain.py", line 1001, in main
module_spec = hub.Module(FLAGS.tfhub_module)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_hub/module.py", line 144, in init
self._spec = as_module_spec(spec)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_hub/module.py", line 33, in as_module_spec
return load_module_spec(spec)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_hub/module.py", line 58, in load_module_spec
return registry.loader(path)
File "/usr/local/lib/python3.5/dist-packages/tensorflow_hub/registry.py", line 45, in call
self._name, args, kwargs))
RuntimeError: Missing implementation that supports: loader(*'/tmp/tfhub_modules/mobilenet_module',), **{})
How can I fix it? Thanks!
BR,
Jacky

I just tried it in a Docker container and it worked, so it might be a macOS specific issue?

You should change all directory access permissions under tfhub to rwxr-xr-x.

@htjacky  Any updates please ?

Closing as it is in "awaiting response" status for more than 3 days. Feel free to add comments(if any) and we will reopen. Thanks!

I noticed that /google/tf2-preview/mobilenet_v2/feature_vector/2 is missing tfhub_module.pb
$ ls
assets/		saved_model.pb	variables/

It's expected that the tf2-preview modules no longer contain a tfhub_module.pb file: for TF2, Hub modules will become more like standard SavedModels.

I am using tf version 2.0.0-dev20190504 (by printing the version in my code) and hub version 0.4.0.  As suggested above, deleted all the files from the cache directory:
C:\Users\XX\AppData\Local\Temp\tfhub_modules
When I tried to run the following:
module = hub.Module("https://tfhub.dev/google/tf2-preview/mobilenet_v2/classification/2")
print(module.get_signature_names())
I received the same error as above:
RuntimeError: Missing implementation that supports: loader(*('C:\Users\XX\AppData\Local\Temp\tfhub_modules\6f74ba811a9cabfa24e78922ad6ca96a0ddfe094',), **{})
BTW, after running the above code, the directory "6f74ba811a9cabfa24e78922ad6ca96a0ddfe094" was created under "tfhub_modules" and it does have the saved_model.pb file (there are two more files: assets and variables). There is also a 6f74ba811a9cabfa24e78922ad6ca96a0ddfe094.descriptor.txt file at the same level as the "6f74ba811a9cabfa24e78922ad6ca96a0ddfe094" directory.
Any suggestions for a fix please?

Hey @hw1,
You could try this: https://github.com/tensorflow/hub/issues/212#issuecomment-451923229

@alberduris: Thanks for your reply. I have seen your remedy/post on stackoverflow as well. As I mentioned above, I had deleted the directory per your suggestion above, but that did not help in my case (mine is windows CPU version). IMO, the problem is that code might be expecting a tfhub_module.pb  file  in the directory which I believe is no longer provided. Instead, a saved_model.pb is made available (see the post by @arnoegw above). However, the code may not be compatible with this change.

I'm getting this problem locally (macOS 10.14.1) when trying to load Universal Sentence Encoder with Tensorflow version 1.12.0, Hub version 0.2.0. However, on Kaggle, it's using Tensorflow version 1.11.0-rc1 (also version 0.2.0 of Hub) and loading USE works.
I can't check if using those versions fixes things, because Tensorflow version 1.11.0-rc1 isn't a valid version for macOS. One potential problem is that I am using Pipenv locally. @htjacky, is this happening in a virtualenv?

The problem is that you need to have USE model extracted in that directory ('/var/folders/l3/2_gkqqs57rl2l1vk0zg978jw0000gn/T/tfhub_modules/1fb57c3ffe1a38479233ee9853ddd7a8ac8a8c47').
If it is not then use :
# For Colab !mkdir -p [folder_name] !cd [folder_location] !curl -L "https://tfhub.dev/google/universal-sentence-encoder-large/3?tf-hub-format=compressed" | tar -zxvC [folder_location]

hey @hw1
with tf version
pip install tensorflow==1.13.1
pip install tensorflow-hub
script
https://github.com/tensorflow/hub/blob/master/examples/image_retraining/retrain.py
Try
for "float-point version"
--tfhub_module https://tfhub.dev/google/imagenet/mobilenet_v1_100_224/feature_vector/3
"instrumented for quantization version"
--tfhub_module https://tfhub.dev/google/imagenet/mobilenet_v1_100_224/quantops/feature_vector/3

I got the same error message for a different reason. However, the cause for my error was different.
I tried to run an older TF script using tf-hub's hub.Module call (TensorFlow V1). Once replaced with hub.load(...) (TensorFlow V2, TF 1.15 or >TF2.0), everything works ok.

It's expected that the tf2-preview modules no longer contain a tfhub_module.pb file: for TF2, Hub modules will become more like standard SavedModels.

@hw1 @arnoegw  Hi guys, I am facing the same problem and want some help. I am trying to get the immediate output of Bert, and according to the solution in this issue, it seems like I have to use tensorflow 1.0 right? But when I change to tf 2.0, as you pointed out, I got the error because I do not have the file tfhub_module.pb, so what can I do? Any suggestions will be highly appreciated!

The two different file formats are explained here:
https://www.tensorflow.org/hub/tf1_hub_module
https://www.tensorflow.org/hub/tf2_saved_model

Hi,
None of the above-mentioned solutions worked for me. So I just replaced hub.Module with hub.KerasLayer  and it worked. Just so:
Change
embed = hub.Module(<your_model_path>)
to
embed = hub.KerasLayer(<your_model_path>)

@Remorax That is correct. The initial issue was developed under TF 1.x which required the model to be loaded as a graph with hub.Module. Since then, tf.hub evolved and supports now TF2.x which allows eager execution with the hub.KerasLayer. Thank you for the correction!


The behavior of series[i:j] with an integer-dtype index is deprecated.

FutureWarning: The behavior of series[i:j] with an integer-dtype index is deprecated. In a future version, this will be treated as label-based indexing, consistent with e.g. series[i] lookups. To retain the old behavior, use series.iloc[i:j]. To get the future behavior, use series.loc[i:j].

Thank you @revanthstrakz
It has been resolved on another PR: https://github.com/bukosabino/ta/pull/309


Add version: pnpm.pnpm.6 version 6.31.0

[X] Have you signed the Contributor License Agreement?
[X] Have you checked that there aren't other open pull requests for the same manifest update/change?

[ ] Have you tested your manifest locally with winget install --manifest <path>?
[X] Does your manifest conform to the 1.1 schema?

Microsoft Reviewers: Open in CodeFlow

@wingetbot run

@wingetbot waivers Add Validation-Executable-Error


C SDK support for Symmetric key attestation

The document says "The SDK currently provides TPM and X.509 support for devices running on Windows or Ubuntu implementations." AAFAIK it also now supports Symmetric Key!

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1eee2ad9-8af7-8781-cbaa-c0d7987a4528
Version Independent ID: 81c16b29-772f-e407-3798-10b735a587db
Content: Tutorial - Set up device for the Azure IoT Hub Device Provisioning Service
Content Source: articles/iot-dps/tutorial-set-up-device.md
Service: iot-dps
GitHub Login: @wesmc7777
Microsoft Alias: wesmc

You are correct. This topic is actually being depreciated and replaced this week as we update the table of contents and tutorials with a better structure pointing to updated content

@wesmc7777 Is there any update on this issue, can this issue be closed?

Can close this. We have depreciated that topic


[boost] Boost uses non-CONFIG mode and doesn't work in CONFIG mode?

Hi,
I believe the Boost port in vcpkg doesn't work in CMake's CONFIG mode. After installing Boost, vcpkg outputs something like "Use find_package(Boost REQUIRED COMPONENTS ...)andtarget_link_libraries( PRIVATE Boost::boost Boost::.....)` (sorry I haven't screenshot this and don't know where vcpkg outputs this to recreate it exactly.
When I then try to use Boost in CONFIG mode: find_package(Boost REQUIRED CONFIG COMPONENTS system filesystem program_options), then it fails to find Boost.
As far as I am aware, more recent versions of Boost support the CONFIG mode, e.g. https://gitlab.kitware.com/cmake/cmake/-/issues/19658, https://gitlab.kitware.com/cmake/cmake/-/issues/19402.
I believe the CONFIG mode of find_package is the strongly preferred way of finding packages, so vcpkg should support it? Especially if a library already supports it?

Yes.
Generally, boost use MODULE mode file FindBoost.cmake which provided by cmake.
But the boost now provides cmake submodule: https://github.com/boostorg/cmake.


NodeJS detection breaking pdf.js when used with Angular CLI and Electron

Attach (recommended) or Link to PDF file here: Any PDF file
Configuration:

Web browser and its version: Electron 2/3
Operating system and its version: All
PDF.js version: 2.0.550
Is a browser extension:

Steps to reproduce the problem:

Use a working example of pdf.js within an Angular 6 App
It works in the browser
It doesn't work in Electron

What is the expected behavior? (add screenshot)
It should download the PDF files to render them, but it doesn't download them.
What went wrong? (add screenshot)
Since Electron contains the process variable (https://github.com/mozilla/pdf.js/blob/master/src/shared/is_node.js), pdf.js assumes that it needs to use the default http package from NodeJS to download the PDF file. But Angular will silently fail the import of http by returning an empty object (causing it to fail loading PDF files: See issue here https://github.com/VadimDez/ng2-pdf-viewer/issues/358).
I'm not certain why NodeJS is even considered here, but I think that a deeper feature detection would be useful to detect a clean NodeJS environment. My suggestion would be to check the existence of require("http").request and require("https").request before assuming they can be used.
I know this is very specific, but it would help us out to avoid dirty quick fixes such as destroying the process variable that might be otherwise useful to keep in Electron.

I'm not certain why NodeJS is even considered here, but I think that a deeper feature detection would be useful to detect a clean NodeJS environment. My suggestion would be to check the existence of require("http").request and require("https").request before assuming they can be used.
I know this is very specific, but it would help us out to avoid dirty quick fixes such as destroying the process variable that might be otherwise useful to keep in Electron.

This all suggests that the actual problem here is that some libraries/frameworks are pretending to be Node.js, despite not actually containing all the methods/functionality of a proper Node.js environment.
It might be possible to improve the Node.js detection in https://github.com/mozilla/pdf.js/blob/master/src/shared/is_node.js, but if/how that should be done someone else will have to chime in on. @yurydelendik Any ideas/opinions here?

I reality, Electron is a proper Node.js environment, it's just that (I'm speculating) Angular's Webpack configuration will silently fail. If I were to do the same with window["require"]("http"), it would work perfectly fine.

This has still not been addressed.
I saw a commit merged that addresses a similar issue for nw.js (https://github.com/mozilla/pdf.js/commit/13b0ca6b2afc1bae46f6757425e54e97b087ba9f)
Could the same be done for Electron?

Fixed by the pull request above.


Fix TF 1.x deprecation speech training notebook

Google Colabs removed support for the %tensorflow 1.x directive in August 2022. This notebook still relies on 1.x functionality, and isn't easy to adapt, as noted in #1426, so I've suggested a change that switches to the 1.15 CPU version of the framework. This doesn't take advantage of the GPU, so it's a bit slower, but only increases the time to about two hours instead of one in my experiments, so I think it's the least-worst option.

Retrying the CLA


Documentation and examples for multithreaded/multicore parallelism using XLA:CPU

The parallelism documentation is mostly focused on multiple TPU/GPU devices. It would be great if jax team add concrete explanations on how jax uses parallelism when the runtime system is composed of multiple CPU cores. Ideally, the documentation may contain both within-op parallelism (what happens when x = jnp.ones((5000, 5000)); jnp.dot(x, x) is performed) as well as parallel directive pmap.
Specifically, if one allocates a certain number of cores, what is the most meaningful way of using pmap, e.g. does one need to specify the same number of XLA devices before importing jax? e.g.
import os
os.environ["XLA_FLAGS"] = "--xla_force_host_platform_device_count=_NUM_ALLOCATED_DEVICES"

However, this will cause an issue if the size of the mapped dimension of the input array is larger than the number of allocated devices. In such a case, are you supposed to reshape the input to fit to the number of XLA devices? Which also begs the question what to do for parallelizing the inner dimension (a nested pmap?) I have seen a discussion in another thread but I can't get a concrete conclusion. I also tried to use other mapping options. In my use case, however, they take much longer than setting the  xla_force_host_platform_device_count to a large enough number, so pmap doesn't complain about the dimension size exceeding the number of XLA devices.
As another example, is it expected to get good performance if only one physical core is allocated, but multiple XLA devices are defined before importing jax? -- in my tests, this way results in a considerably faster runtime in comparison with performing the operation sequentially (looping over the input mapped dimension).

Hey @amir-saadat
Have you tried using jit(vmap(f), backend='cpu') without pmap?  In my use case, weirdly, jit(vmap(f)) results in using all the available cores at 100% utilization (according to htop at least) but using pmap results in using one core only for a long time and finally leads the process to being killed (I assume because of memory issues). So my questions are:

How does pmap do in comparison to jit(vmap) on cpu?
Does pmap on cpu copies broadcasted items for each core? (Let's say we have forced the number of XLA cores to be the same as XLA cores)

I wish pmap -- or something like it -- would just figure out how to split up your thousands/millions of array elements across the number of devices (cpu cores) you have, instead of having to manually reshape the input array with an extra dimension matching those devices/cores (and guarantee your array length is a multiple of that). It seems to me the docs could make this behavior more clear at least.

I am using my CPU as a test, since I don't have access to a nvidia gpu. Can someone tell me if pmap will also require me to know exactly how many threads are on the GPU, and split the problem up like this?
FWIW, so far, jit(vmap) with backend='cpu' is much faster than pmap with --xla_force_host_platform_device_count=501. (Obviously I don't have 500 cpu cores, but my test case has 500 array elements to parallelize.)
If pmap works (albeit slowly) on my cpu while spoofing the number of cores, can I count on it working when I pay for access to a GPU? The full array will have hundreds of thousands of elements, do I need to break them up into batches for the GPU to work properly?

@doctor-phil you would need more than 1 GPU for pmap to be useful.

@cgarciae I must be misunderstanding the documentation. If I only have 1 gpu, I only need to use vmap?

@doctor-phil correct


There is no 'Clone' action for emulator blobs

Storage Explorer Version: 1.30.0
Build Number: 20230613.2
Branch: main
Platform/OS: Windows 10/Linux Ubuntu 20.04/MacOS Ventura 13.4 (Apple M1 Pro)
Architecture: x64
Azurite Version: 3.24.0
How Found: From running test cases
Regression From: Previous release (1.30.0)
Steps to Reproduce

Install and run Azurite 3.24.0 -> Launch Storage Explorer.
Expand Emulator & Attached -> Storage Accounts-> (Emulator - Default Ports) (Key).
Blob Containers -> Create a blob container.
Check whether there is a 'Clone' action on the toolbar.

Expected Experience
There is a 'Clone' action.

Actual Experience
There is no 'Clone' action.

Additional Context
There is no 'Clone...' action for one emulator blob container.

@v-kellyluio please consider this a by design behavior. We have removed old code that existed only for the purpose of supporting clone for Azurite/emulator.


Framehandle package update

Spaces replaced with tabs.

getFrameByName was replaced with getFrame
createFrameByType was replaced with overloaded createFrame
setVisibleFor was replaced with overloaded setVisible
showFor, hideFor were replaced with overloaded show and hide
name was replaced with getName
added enable disable functions
minMax was replaced with setMinMax
added hotdocs to setLevel
hideOriginFrames(bool) was repcalsed with setOriginFramesVisible(bool)
added hideOriginFrames and showOriginFrames
boolean keyword was replaced with bool

_Hadles package now imports in alphabet order

Some things may depend a bit on taste, like "boolean keyword was replaced with bool", but overall seems useful.
Thanks 👍
We should probably add some layout helpers that can be used for multiboard and framedefs or so, as I postulated back then for trokkin's multiboard PR.


Fixes issue-1455 MetricRegistryListener should be invoked before adding meter to the registry

Each on*Added method returns a meter corresponding to the one passed in, making it possible to wrap/proxy a meter before it's added to the registry. A common use case for this is interoperability with another metrics library.
Also got rid of MetricRegistryListener.Base since we can use default methods in Java 8+, and don't need an abstract adapter class.
Added unit tests.

Seems like codeclimate check is failing on existing code, and in a way that doesn't make sense.

How do you handle when a listener is added after some metrics have already been added to the registry? It looks like you ignore the return result form the listener.... which creates a weird behavior in that the return value of the listener may or may not actually be used, and there's no way for the listener to know.
Similar with register() and registerAll().... both of these allow an application to pass in a metric that they've already created and are using. The listeners wouldn't be able to wrap these either.

@baharclerode How’s the case of adding a listener after adding a metrics handled in the existing code?

Currently, when you add a listener, it is invoked for all existing metrics already added to the listener. This ensures that any listener registered with a MetricRegistry always knows about every metric in the registry: https://github.com/dropwizard/metrics/blob/4.1-development/metrics-core/src/main/java/com/codahale/metrics/MetricRegistry.java#L367-L369
This greatly simplifies the API usage / avoids timing issues about when metrics are registered vs when listeners are registered.

@asarkar Please rebase this PR on the current 4.1-development branch. Thanks!

How do you control the order in which listeners are invoked? Previously this did not matter since listeners can't modify the metrics when they're added, but now they can so order is important/matters. It is often difficult to control all the points where listeners are added (some may be added by bundles, some may be added by the application code itself, some may be added dymanically at runtime, etc., which makes it difficult to control.
I'm still curious how you plan to handle adding a listener after metrics are already added to the registry. Right now it means that listeners have two different behaviors, and there's no way to know which will be used-- which makes for a very bad API.
IMHO the proper way to achieve what you're looking for is to create a separate metric registry B, install a metric listener into registry A, and wrap all the metrics before adding them to registry B, and then expose registry B however you want; Or subclass the MetricRegistry and override register(String,T) to wrap all metrics as they're added. Changing MetricListener into MetricMutator is a bad design choice.

I'm actually also curious about this:

A common use case for this is interoperability with another metrics library.

How does this help interoperability with another metrics library? Wrapping a metric when it is added to the registry does not, for example, allow you to reliably intercept calls or method invocations on that metric, or know when that metric has been updated.
You would actually need a pretty massive overhaul of all of the current metrics to support something like that (each metric would have to have it's own listener type); Even with wrapped metrics, the best you can do right now is poll the metrics periodically and export their aggregate state to another metrics library, which can be done with the existing MetricListener API.

@dropwizard/metrics @dropwizard/committers How should we continue with this branch?


call create project api failed

when I ran the create project command from swagger "curl -X POST --header 'Content-Type: text/plain' --header 'Accept: application/json' -d 'projectname' 'http://localhost/api/projects'  " ,
I received an invalid json request error

please check your json string.
curl -X POST --header 'Content-Type: text/plain' --header 'Accept: text/plain' -d '{
"project_name": "myproject_0001",
"public": 0
}' 'http://youripaddress/api/projects'

it's my mistake that I didn't input data in the correct  format,the value of the project parameter should be a json not a string


php82: add sysvsem

Similar to https://github.com/NixOS/nixpkgs/pull/193921

Should this be enabled by default ?

It is not there without. Any of my Symfony with Lock component projects explode without it :)

Right.
To be honest, this is the first time I see this extension.
IMHO, I don't think it needs to be enabled by default, unless there is something I miss.
If I were you, I would use a custom version of PHP including sysvsem instead. It's easy to do.
If you share your flake.nix file, I can assist you.
Cheers.

Should this be enabled by default ?

@jtojnar ^^

If it is enabled by default in Nixpkgs, then we should do the same.

It's enabled indeed. This PR is therefore legit. Validating it.

@shyim Do you plan to upgrade your PR ?

@drupol I have added it to all versions :)

All good, thanks !


[Feature request] Allow to specify ENV for the command

Description
Allow to set env during command sending process
Current behavior
Error, as specified in the docs
OP help
Unfortunately, not possible as of now

Do you know that you can use tempfile plus the openssh::Builder to override the ssh-config and specify the env within it?

I know it. Unfortunately I need to change the env on each command :/

Hmmm seems that you would definitely need openssh support for this.
As an alternative, you can change command to be run to env, then pass env as A=B, then pass the original cmd and args.

As an alternative, you can change command to be run to env, then pass env as A=B

I've been aware of this option. Overall it could be a solution. On the other hand I consider it as a huge, dreadful hack. At least doing it manually. It either requires me to write an additional wrapper around library, or perform these changes on every command execution. Both of these options are... I don't like them xD

I agree, I will investigate to see if there's better solution.


Implementation of thin-walled / double-sided materials

It looks like the only implementation for thin_surface nodes is in MDL. Missing both a GLSL and OSL implementation. Unfortunately, without the GLSL implementation, we cannot use this node in either MaterialX View or Usdview with Storm. And the OSL implementation would be needed to use with Usdview with Prman.

Hi @klucknav, I just want to add some context on this.
We have refrained from working on an implementation of thin_surface so far, because its definition is debatable. When we defined the PBR library it seemed like a natural way to separate closed surfaces with volume (the surface node) from thin-walled surfaces (the thin_surface node). But in hindsight it's not a good approach.
The first problem is that there is no natural way to have a user controlled switch between the two, for example as needed for an uber shader. You could use a surfaceshader mix node to switch between the one or the other, but then you introduce the concept of having a surface that could be 50% closed and 50% thin-walled, which doesn't makes sense.
In addition, we now have the "unlit" version of surface, which then by definition becomes a closed volume unlit surface. So we would then need an "unlit" version of thin_surface as well, which is quite messy.
So we have come to the conclusion that we need a better way to define closed vs. thin-walled surfaces. A proposal we are considering is to move the definition of closed vs thin-walled to the material level. Adding inputs on the surfacematerial constructor node for both frontfacing and backfacing surfaceshaders, as well as a boolean switch to say if it's thin-walled or not.
The thin_surface node would then be removed and replaced with this more robust definition.

Here is a proposal to discuss for the surfacematerial node to make it support thin-walled / double-sided materials:
  <!-- Surface material -->
  <nodedef name="ND_surfacematerial" node="surfacematerial" nodegroup="material">
    <input name="surface" type="surfaceshader" value="" />
    <input name="backface" type="surfaceshader" value="" />
    <input name="thin_walled" type="boolean" value="false" />
    <input name="displacement" type="displacementshader" value="" />
    <output name="out" type="material" />
  </nodedef>

If thin_walled is enabled the material is thin-walled and double-sided, where surface and backface defines the surfaceshaders on each side respectively. If thin_walled is disabled (the default) the material is a closed surface with an interior volume, where surface defines the single surfaceshader and backface is ignored.
Upgrading documents from the old surfacematerial node is straight forward since the default behavior is the same. The upgrade just need to rename the inputs.

@niklasharrysson This proposal looks great to me, and I'll forward the suggestion from a parallel thread that this should be a new surfacematerial_doublesided node, rather than a modification of the existing surfacematerial node.  This should make it more straightforward for renderers to detect when a node graph references double-sided materials, which they may wish to handle with a separate code path.

I am pleased to see that clarification and thought around the issues of double-sided materials is under way.
I'd like to explore @spiffmon's comment that in USD, sidedness is a property of geometry.
In general, I hold that statement as a maxim, not just for USD. Hence the "parallel thread" suggestion on introducing a new material node that's explicitly meant for double sided geometry, for those cases where this thin-walled/double-sided notion is absolutely required.
I want to raise the issue that if MaterialX allows geometry properties (sidedness) to be affected in the material node, it means that a renderer has to do extra work to discover if any node in a graph is sided, and if so, the network the node participates in can be polluted with sidenedness, and thus all geometry bound to the network will itself become polluted with sidedness. In the worst case limit, if a single node forces double sided rendering, it might force useless double rendering of everything. The alarm bell is that it has the potential of an artist introducing the One Node that pessimizes all render times, and even if a conformance tool discovers the issue, the One Slow Node might be part of the One Holy Network That Production Can't Change.
That's where the proposal for a new material node comes from. An explicit node would mean that a potentially pessimizing choice must be deliberately made and easily identified, rather than introducing a requirement that renderers must introspect every node to discover its pessimizing potential.
So in conclusion, I don't think that geometry should be modified by a material, and if it must, I think it should be done in a highly visible and controlled manner. Furthermore, propagation rules may be required to specify how far the double-sided requirement reaches.

Thanks for the great feedback @spiffmon and @meshula
@spiffmon You are precisely right that the thin_walled input is a bit redundant in this sense. The main reason it was added to this proposal is that there are existing shading models that have such a user-controlled switch. Examples include Autodesk Standard Surface and glTF PBR, which we want to describe in MaterialX. In addition, Nvidia MDL also has such a switch in its material definition.
But I agree we should not introduce redundancy unless there is a very good reason. Looking more closely at what the thin-walled property does in practice for the mentioned use-cases above, it alters the transmissive properties of the material, disabling refraction and disabling volumetric scattering and absorption. I think this could be implemented explicitly in a BSDF graph instead, for the shading models that need this switch. For example, one could mix/switch between different BSDF nodes based on this property. This might actually be a better solution since each shading model may have slightly different opinions on how thin-walled should be handled. So in other words we could probably remove the thin_walled property in this proposal, and let each shading model handle it in their implementation graph.
I also agree with your second point that we should provide guidance that for double-sided materials (however we introduce this) the BSDF's on both sides must be of the same kind. At least the transmissive properties must be the same.
@meshula this is a great point, and I think we all agree that if we need double-sided materials it should be introduced as a separate material node for easier identification.
However I think we could use some more information on how this is handled on geometry level in USD. Are you able to assign different materials to each side when the geometry is tagged as double-sided?
I'm not sure we could rely on having such a material assignment workflows in applications that want to adopt MaterialX outside of USD. But it's definitely something to discuss more.
Thanks.

There's practice, and there's specifications :) Without addressing USD, here's what MaterialX has to say:
It is common for shading models to differentiate between thick and thin surfaces. We define a thick surface as an object where the surface represents a closed watertight interface with a solid interior made of the same material. A typical example is a solid glass object. A thin surface on the other hand is defined as an object which doesn’t have any thickness or volume, such as a tree leaf or a sheet of paper.
For a thick surface there is no backside visible if the material is opaque. If a backside is hit by accident in this case, the shader should return black to avoid unnecessary computations and possible light leakage. In the case of a transparent thick surface, a backside hit should be treated as light exiting the closed interface. For a thin surface both the front and back side are visible and it can have different materials on each side. If the material is transparent in this case the thin wall makes the light transmit without refracting, like a glass window or bubble.
Two nodes in our material model are used to construct surface shaders for these cases. The <surface> node constructs thick surfaces and is the main node to use since most objects around us have thick surfaces. The <thin_surface> node may be used to construct thin surfaces, and here a different BSDF and EDF may be set on each side. See the Shader Nodes section below for more information.
```

There is lee-way for a pessimistic interpretation here. One may have a different BSDF and EDF set on each side, and also it says "it can have different materials on each side". In order to achieve different materials in the general case, the geometry would have to be implicitly promoted, or the spec could say "it can have different materials on each side, if the geometry is double sided", to close the loophole. :)

I like "it can have different materials on each side, if the geometry is double sided" but pedantically would suggest going further, avoiding use of "material" altogether, since Material is a) the thing to which a primitive can bind, b) a container hosting (weak) primvars, c) an encapsulation point.  Which is not what we're suggesting, here, so I'd suggest sticking with something like "it can have different a BSDF/EDF for each side, if the geometry is double sided"

I sense we may have been talking about different kinds of sidedness above, so first I want to clarify that the material sidedness we want to describe in MaterialX was never intended to alter or take precedence over geometric sidedness. Geometry set to single-sided, invisible from the backside, would always be invisible even if a two-sided material is assigned to it. So a renderer would not need to parse any shading networks to find out if a surface is visible or not from the backside.
If we make the change and remove the explicit thin_walled parameter from the proposal (which I think is the right move the more I think about it), then I will rewrite the whole section about thin and thick surfaces. The specification would no longer dictate what a thin vs thick surface is, and instead each shading model is free to implement this as needed in a BSDF graph. I will also clarify that a two-sided material is only meaningful on double-sided geometry.

If there is no thin_walled parameter, how would the renderer decide whether to apply refraction or not, if, for example, a dielectric_bsdf with a ior = 1.5 is used? Would that mean that you will add a thin_dielectric_bsdf?

@proog128 You could flip between two dielectric_bsdf's, using a mixer with the thin_walled parameter as mixing weight, so it's either one with refractions or the other without refractions. This is what I meant by having each shading model decide how to handle it in its BSDF graph.
Something like this would have to be done in the generated code anyway in order to implement the thin-walled property if we had it specified as part of the specification. So we're moving the implementation details from the code generator into the hands of the shading model BSDF graph creator. An advantage is that the graph creator can then also do custom handling and tweaks if needed for the specific shading model.
Please let us know of any concerns with this approach. Thanks!

At the moment, thin_dielectric_bsdf and similar nodes are not available in the MaterialX PBRSpec. thin_dielectric_bsdf could be emulated, for example via layer(top = dielectric_bsdf(ior = 1.5, "R"), bottom = dielectric_bsdf(ior = 1.0, "T")), but that increases complexity in the BSDF graph and makes shading less efficient and complicates the implementation (if I am thinking about writing a native MaterialX implementation in a renderer). Therefore, I think adding thin versions of all potentially refractive BSDFs is required if there is no thin_walled flag.
One concern I have is that this doesn't map well to MDL and would be difficult to support in the MDL code generator. Somehow the code generator has to guess the value for MDL thin_walled from the BSDFs used in the graph. For instance, if it finds thin_dielectric_bsdf, it would set thin_walled to true. In this particular case it makes the code generator more complicated.
Another problem with removing an explicit thin_walled property (being it a thin_surface node or a thin_walled bool in surfacematerial) is that it creates ambiguity in the material definition. What if a graph contains thin_dielectric_bsdf and dielectric_bsdf, and both are active at the same time? Is it a thin or a volumetric material? Thinking again about the MDL code generator, should it set material.thin_walled to true or false?
Finally, the thin_walled property may serve as a hint whether the material has to be put onto a stack of nested dielectrics or not. Only if thin_walled = false the material has to be pushed onto the stack, because only then there is a volume below the surface. Alternatively, this information could be derived from the BSDF (i.e., dielectric_bsdf has a volume, thin_dielectric_bsdf not), but then we again end up with ambiguity: a graph could be, at the same time, a volume and a thin surface.

@proog128 , I thought it was redundant as part of the material definition. Question: is there a meaningful scenario in which we would name a back_surface bsdf but did not intend the material to be thin?
if not, then the render-prepper or code generator could infer thin_walled not from the presence of certain bsdfs, but by how they are used/combined, as specified by the OM.

@proog128 , my concern and questioning was for the user-level object model that we would expose in UsdShadeMaterial, and those concerns may indeed be different from those of the code generator.  On UsdShadeMaterial, there will be a backSurface “relationship” that can target a shader node/prim representing a bsdf, but its default state is “empty”.  So, at the UsdShade level it is trivial to know whether you have a back surface or not, no need to compare graphs.
But if you all decide that it makes for more efficient MaterialX codegen to separately encode thin-sidedness, then we can arrange to have HdMtlx reconstitute the bit based on presence of a backSurface network.  I just wanted to make sure we wouldn’t be losing generality in removing from the user model what seemed like redundant information.

This is a great discussion. But I think there are two parallell questions being discussed:
A) Should thin-walled be a feature added to this twosided surfacematerial node?
B) How should the thin-walled feature be enabled on user-level on this surfacematerial node?
B is only relevant if the answer to A is Yes, so I think we should resolve A before discussing B.
For A my proposal above was that thin-walled could be kept out from the new surfacematerial node.

From a data model point of view it's redundant because it's really just a short-cut override for users to disable refraction and absorption on upstream BSDF's. This can be set on the BSDF level directly instead.

From an implementation point of view if we do add this override feature it means every MaterialX back-end and renderer will need to build support for it. @proog128 you are right that MDL has the thin_walled switch build into its data model, so it's easy to support there. But I'm afraid it's a lot harder everywhere else. Not only in our code generators but also for renderer that doesn't use code generation and have their own shading systems.

Instead it can be implemented as logic directly in the BSDF graph, for the shading models that requires thin-walled, as suggested above. Then it would automatically be supported by all since it's just a normal MaterialX BSDF graph.

Note that with this proposal the new twosided surfacematerial node would be trivial to support, since it's just a backfacing switch between two surfaceshaders.

Perhaps I'm missing something about the thin-walled feature that is not expressible the way I'm imagining? But let's at least discuss question A first, before/if we get to B :)

@niklasharrysson Solving the questions in this order makes total sense, thanks for structuring the discussion :-).

Instead it can be implemented as logic directly in the BSDF graph, for the shading models that requires thin-walled, as suggested above. Then it would automatically be supported by all since it's just a normal MaterialX BSDF graph.

Do you mean that the flag should become a per-BSDF switch (dielectric_bsdf has a new parameter "thinWalled") or a per-surface switch (surface constructor has a new parameter "thinWalled")? I think my concerns only apply if thinWalled becomes a per-BSDF switch.

I mean the flag could be added only to the nodedef's of shading models that requires this, and then used as needed in the corresponding BSDF nodegraphs.
There is already an example of this in the Autodesk Standard Surface shading model:
The nodedef for standard_surface has a thin_walled flag:
https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/libraries/bxdf/standard_surface.mtlx#L93
This flag is used in the corresponding implementation nodegraph. First converted to float:
https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/libraries/bxdf/standard_surface.mtlx#L221
..and then used to select a different SSS model if thin-walled is enabled:
https://github.com/AcademySoftwareFoundation/MaterialX/blob/main/libraries/bxdf/standard_surface.mtlx#L223
What is missing in this graph currently is to also select between two different models for transmission (with/without refraction, absorption, scattering) depending on this flag, but this is something we should fix to make the standard_surface MaterialX implementation more complete.
I think you should be able to do the same thing with the gltf_pbr shading model. If the thickness input is > 0.0 you can select a different model for transmission in the nodegraph.

this is something we should fix to make the standard_surface MaterialX implementation more complete.

Which BSDF do you want to use in Standard Surface to disable refraction? This is the part that troubles me. I cannot think of a good way to express this with the current set of BSDFs for glTF PBR. Of course to disable refraction I could set the IOR to 1 (or 0, because of special case), but then I am describing one interface between two volumes with an IOR of 1. However, what I actually want to describe is two interfaces between three volumes: the volume on the one side, the (infinitesimally) thin volume described by the current node graph in the middle, and the volume on the other side.
Maybe I am overcomplicating, maybe I am misunderstanding the relationship between surfaces and volumes in MaterialX. As far as I understand, IORs in MaterialX are relative, i.e., they describe the relative IOR at an interface, as opposed to IORs in MDL, which are absolute, requiring the renderer to track the nested volumes (touching the problem of nested dielectrics). But besides that, I had always understood the difference between surface and thin_surface to be that the former is one interface and the latter is two interfaces. It would be great if the MaterialX PBR node spec would include a few more details on this. I am struggling to understand how to set up the IORs and volume parameters such that nesting of volumes works across all code generation targets. Is the MDL code generation in combination with Iray a good reference? Which code generation/renderer gives the correct solution?

Yes for the transmission part I would use a dielectric_bsdf with ior = 1.0. But for the reflection part we would still have the user-controlled ior.
BSDF thin_walled_dielectric = layer(top = dielectric_bsdf(ior = 1.5, "R"), base = dielectric_bsdf(ior = 1.0, "T"))
So it may be a hack but the whole concept of thin-walled is a hack., and I think this would be good enough. But I could definitely be wrong :)
What I like about this approach is that it uses constructs that are already available in MaterialX, so there is no update at all needed to the specification or to all renderers and back-ends.
If I where to implement thin_walled_dielectric in vanilla OSL code this is the approach I would have to take anyway. But even if thin_walled_dielectric was implemented as an atomic BSDF in C++, would it handle this very differently?

Maybe I am overcomplicating, maybe I am misunderstanding the relationship between surfaces and volumes in MaterialX.

Not at all, the description of volumetrics and things like nested dielectrics is not defined well enough in MaterialX yet. I know we've had conversations about this befor in other threads. That part needs to be worked on. And certainty from a reference implementation point of view. I'm afraid there have been no updates on that since we last spoke about it. However that is a much larger topic, and with thin-walled there is no volume, so my hope was that we don't need that here.

Thanks for the additional details @proog128, these are some great points.
To my mind they are strong enough arguments that we may consider A to be resolved, with the conclusion that we do need explicit support for thin-walled in the specification. If others agree I think we can move on to look at B, how to define this in the spec.
There is a practical issue with the initial proposal I made above where the surfacematerial node has the thin_walled switch. The problem is that our "ubershader" nodes, where we want to have control over this (standard_surface, gltf_pbr, etc.), are all nodes of type surfaceshader and not type material. So there is no practical way to wire this in their implementation graph that outputs a surfaceshader, when the input to control it is on the material level outside the graph.
One option would be to change the ubershaders to output a material type instead, but then displacement becomes part of the ubershaders as well (displacement is a material property), so it doesn't match their original definition.
Another solution is to separate thin_walled from the front/back assignments, keeping only assignments on material level and having thin_walled specified on surfaceshader level. This gives a better separation of concerns.
Here's a new proposal to discuss with these changes made:
 *** REMOVED NODE ***
 <!--
   Node: <thin_surface>
   Construct a surface shader from scattering and emission distribution functions for non-closed "thin" objects.
 -->

 *** NEW NODE ***
 <!--
   Node: <surfacematerial_twosided>
   Constructs a material with different surfaceshaders on each side of geometry,
   if the geometry is double sided. If only one of the sides are connected this
   surfaceshader is used on both sides.
 -->
 <nodedef name="ND_surfacematerial_twosided" node="surfacematerial_twosided" nodegroup="material">
   <input name="frontsurfaceshader" type="surfaceshader" value="" />
   <input name="backsurfaceshader" type="surfaceshader" value="" />
   <input name="displacementshader" type="displacementshader" value="" />
   <output name="out" type="material" />
 </nodedef>

 *** UPDATED NODE ***
 <!--
   Node: <surface>
   Construct a surface shader from scattering and emission distribution functions.
 -->
 <nodedef name="ND_surface" node="surface" nodegroup="pbr">
   <input name="bsdf" type="BSDF" value="" />
   <input name="edf" type="EDF" value="" />
   <input name="opacity" type="float" value="1.0" />
   <input name="thin_walled" type="boolean" value="false" />
   <output name="out" type="surfaceshader" />
 </nodedef>

There is a possibility to have two surfaceshaders, one thin and one thick, be mixed or assigned on front vs back on a material, which doesn't make sense. But this we could resolve by letting thin take precedence and override the other in such cases.

To my mind they are strong enough arguments that we may consider A to be resolved, with the conclusion that we do need explicit support for thin-walled in the specification.

That's great, thank you @niklasharrysson.

Here's a new proposal to discuss with these changes made:

The proposal looks good to me, it will definitely work perfectly with the glTF PBR and its thin_walled toggle.

Glad to hear that @proog128.
@spiffmon, @meshula just let us know if there are any concerns from USD and Pixar side.
Thanks for all feedback!

This new proposal satisfies my hope to make front-shader/back-shader available on a special node rather than an option that can be introduced anywhere in a graph, and it also makes the use of thin-surface explicit in terms of describing the distribution functions. So it addresses the concerns I raised :)

Excellent! Thanks @meshula

I think I follow why it is important for codegen and the underlying math to have an explicit declaration of thin_walled behavior.  But I am unsure if it is advantageous to propagate that up into the user-model, vs. inferring it from the presence of a backsurface  shader terminal connection.  In UsdShade there is only a single Material definition/schema, and following the proposal here to the letter would mean introducing a second TwoSidedMaterial schema, which, if we wanted the specificity of front vs back, would not even share an inheritance relationship with Material, which already defines a simple surface terminal.  That's kind of a big deal.
I will discuss with Pixar internal stakeholders, but I'm very curious to hear from Autodesk, Substance, and other toolmakers whether the idea of exposing a different material type to users is clarifying or cluttering.  For sure, you could key off of it to do GUI validation on the shaders you connect to it, to ensure they are thin-walled, and so don't get into the inconsistent state Niklas mentioned, but I think you could also enable such validation when the backsurface terminal becomes connected, also?

Thanks @spiffmon for the additional feedback. I think there are two questions here that need further discussion:

Could thin-walled be inferred from the presence of a backsurface connection on material level?
Is there a need for a separate material node for two-sided materials?

For (1), first of all I think it's a good separation of concerns if we keep the distinction between material assignment (i.e. material nodes) and surface appearance (i.e. surfaceshader nodes). Since thin-walled is a surface appearance feature I think it belongs as a switch in the surfaceshader domain.
But there is also the practical aspect that some shading models have thin-walled as a user controlled switch in their parameterization (e.g. standard_surface and gltf_pbr). We can't encapsulate these shading models as atomic surface shader nodes, with a thin-walled input in their parameterization, if thin-walled is solely inferred from how the shader is assigned on the material node downstream.
However, we could enforce that having both front-surface and back-surface assigned would result in the shaders getting thin-walled enabled, no matter what the user has set, to avoid the inconsistent state you mention.
For (2) I agree it doesn't make sense to introduce a second TwoSidedMaterial schema in USD. Having this as a separate node in MaterialX was proposed in response to @meshula's concern that introducing two-sidedness may be problematic for renderers if not done through an easily identifiable new node. I wonder though if that concern was not mainly about geometry sidedness? Now that we have clarified that this is strictly about material sidedness which doesn't affect geometry sidedness, is this still a concern? Or would it be possible to just add backsurface to the existing material node and avoid the secondary two-sided specialization?

My concern was that due to existing wording, introducing a two-sided material could be interpreted as implicitly "upgrading" geometry to two-sided - in other words it was not explicitly clear whether a single sided mesh must be rendered double sided if a "back" material was introduced. If the rule is that materials cannot affect geometry, then my concern is alleviated. I proposed the separate node to accommodate an "upgrade" have it be blazingly obvious in the graph, and have the "upgrade" propagate to other mesh nodes that must be rendered jointly with the same material.

That's great to hear! To my mind it would make sense then to keep a single surfacematerial node. I will rewrite the section in the specification that deals with this so there is no risk of confusion what two-sided means in this context.

Just to follow up, @meshula and I discussed with our RenderMan colleagues and we're all pretty happy with a single Material definition that can optionally be fed a back surface shader.  Thanks so much @niklasharrysson and @proog128 !

@niklasharrysson Now that we're approaching the release of MaterialX 1.39, I wanted to check in to see whether we might be able to implement this change to MaterialX in the time frame of our upcoming release.
As of today, we have the new specification language for double-sided surface materials (with separate inputs for front and back surfaces), but not yet an implementation of this feature that artists and developers can use.
Let me know what your thoughts are!

This has now been addressed in #1866 and #1890. The definition files have been changed to reflect the 1.39 spec, and the implementation for MDL is done.
Implementations remains to be done for OSL and GLSL, but the stage is now set for any team to pick up those tasks if there is an urgent need. Otherwise we'll plan for implementing these targets in the near future.

Thanks, @niklasharrysson!  I would agree that this has been resolved for the upcoming MaterialX 1.39 release, and I'll close out this issue.