Question about QoS levels for State topic.

I see that the Connection topic was originally QoS level zero, and was changed to 1 per the following issue: https://github.com/VDA5050/VDA5050/issues/24
We are wondering why we don't utilize QoS level one for the State topic as well? As is, state information can conceivably be lost, such as if a vehicle is taken from automatic into manual and thereafter back into automatic. If we would like to have some functionality in the traffic controller take place when a vehicle is placed into manual mode, we'd have to build an abstraction on top of the protocol to ensure this which is inelegant.

QoS level 1 has more communication overhead compared to QoS level 0. Still tests showed us, that the overhead is minimal and we fully support the QoS for the state topic as well. Actually we would suggest to implement it for every topic.

Update from WG01 meeting on 2023/11/08:
Changing to QoS 1 only ensures broker -> MC connection. It would still not be guaranteed the message makes it from the truck to the broker. It is already the case in VDA5050 that state messages can be swallowed if the states change very rapidly.
The implicit intention of VDA5050 is that if you want the full historical state data of a vehicle, you must ask the vehicle itself for this outside of the scope of VDA5050.
I will take this info back to my team and then update/close this issue as necessary.

Some thoughts to start the discussion again:

The QoS 1 ensures that a MQTT message is received at least once by the addressee. This is done by resending the message if the sender doesn’t receive an acknowledge in time. Because the sender ensures the order of the messages this might lead to sending more than one message again. E.g. Broker sends message “A”, “B”, “C”, “D” and gets no acknowledge for “B” the broker will send “B”, “C” and “D” again (in this order). The addressee will get message “C” and “D” twice (or more).
Typically the connection to a broker is handled by a TCP/IP connection and the TCP/IP Stack ensures that the application (the MQTT client or broker) sees a correct data stream. Handling of lost or out-of-order Ethernet packages are handled by the TCP/IP stack. The only way to lose MQTT messages is a broken TCP/IP connection. Typically this will happen when the MQTT client or broker received no "heartbeat" messages and closes the TCP/IP connection (the TCP/IP stack itself of very patient before giving up a connection).
To receive outstanding messages of a QoS 1 connection the client has to use single MQTT session id. Otherwise the broker cannot map the outstanding message to the reconnected client. Deserted sessions also stresses the broker, because for all sessions with QoS 1 (or higher) messages have to be buffered (and MQTT V3.1.1 defines no end-of-life for a session).

Using QoS 1 can be tricky (I speak from experience.).

Using QoS 1 for the connection topic makes sense, because messages on these topics will only send once (these messages have an event nature).
The state message contains the complete state of the vehicle (not a singe event) and is send at least ever 20 - 30 sec (with zones more often) and can activity requested by MC. No harm is done if some message is lost. Maybe it should be best practices to send a state message immediately after a reconnect to the broker.
The visualization message: "Master control must not use the info messages for logic, it must only be used for visualization and debugging purposes." and why (re-)sending outdated messages.
Using QoS 1 for the instant actions and order messages may useful to unburden MC with a logic to re-transmit an instant action or order message (the MQTT protocol will do it).  In this case the handling of the 'headerId' has to be defined precisely, because messages can be received twice or more. Make it sense to introduce this now for these topics, because today MC should have a mechanism to send instant actions and order messages again?

I dont think that introducing QoS 1 for instant actions and orders would remove the need to resend said messages. QoS 1 would only ensure that these messsages would reach the broker but when the AMR is disconnected at that time the message would never reach it. (I assume that at the time of sending the messages MC isnt aware of the AMR being disconnected)
In that case MC still needs to resend these messages after the AMR reconnects and does not reflect them in its state.

QoS 1 is only effective if each client use it; especially the client connected via WiFi to the broker. If this is the case and the client (in our use case the AGV) loses the connection to the broker, the broker buffers each message for the client and send the message again until the broker receives an acknowledge form the client. Therefore it is crucial that the client sticks to his session id thereby the broker can detect the reconnection and resend the message.
Because of the necessary buffering QoS1 leads to a higher resource consumption (especially at the broker) and if the client uses a new session id for every connect it can be really messy on broker side, because the broker has to buffer messages for open sessions (MQTT V3 doesn't specify the lifetime of a session).


close() closes immediately; Add new closeGracefully()

Fixes #370

Codecov Report

Merging #383 (17dafe1) into main (3561175) will increase coverage by 0.80%.
The diff coverage is 54.05%.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #383      +/-   ##
==========================================
+ Coverage   46.88%   47.69%   +0.80%
==========================================
  Files         110      110
  Lines        8988     9033      +45
==========================================
+ Hits         4214     4308      +94
+ Misses       4774     4725      -49

Files Changed
Coverage Δ

...es/PostgresNIO/Connection/PostgresConnection.swift
39.50% <0.00%> (+3.95%)
:arrow_up:

Sources/PostgresNIO/New/PSQLEventsHandler.swift
72.41% <ø> (ø)

Sources/PostgresNIO/Postgres+PSQLCompat.swift
0.00% <0.00%> (ø)

Sources/PostgresNIO/New/PSQLError.swift
71.67% <38.88%> (-2.16%)
:arrow_down:

...nection State Machine/ConnectionStateMachine.swift
57.87% <63.63%> (+1.38%)
:arrow_up:

...urces/PostgresNIO/New/PostgresChannelHandler.swift
77.02% <66.66%> (+5.54%)
:arrow_up:

... and 4 files with indirect coverage changes


Update hashicorp-vault.md

Updated the docs to support vaultToken as an alternative to vaultTokenMountPath. This addresses issue #1354
Thank you for helping make the Dapr documentation better!
Please follow this checklist before submitting:

[x] Read the contribution guide
[ ] Commands include options for Linux, MacOS, and Windows within codetabs
[ ] New file and folder names are globally unique
[ ] Page references use shortcodes instead of markdown or URL links
[ ] Images use HTML style and have alternative text
[ ] Places where multiple code/command options are given have codetabs

In addition, please fill out the following to help reviewers understand this pull request:
Description
This updates the docs to support vaultToken in the Hashicorp Vault secret store.
Issue reference
PR will close: #1354

@DarqueWarrior is this in the context of https://github.com/dapr/components-contrib/issues/794? is this documenting a change in the code or is it relevant to v1.0? (currently this doc update is targeting v1.0)

Correct the change was just made and is not released. Should I close this and target it somewhere else?

Correct the change was just made and is not released. Should I close this and target it somewhere else?

Yes please. If it was part of last week's v1.1 release re-target to the v1.1 branch. If it's still unreleased target the v1.2 branch

Ok I moved to 1.2.


fix(vue-renderless): [radio] fix handleKeydown blocking default events and bubble logic

… events and bubble logicbug
Fix blocking default events and bubble logic
PR
PR Checklist
Please check if your PR fulfills the following requirements:

[x] The commit message follows our Commit Message Guidelines
[ ] Tests for the changes have been added (for bug fixes / features)
[ ] Docs have been added / updated (for bug fixes / features)

PR Type
What kind of change does this PR introduce?

[x] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] CI related changes
[ ] Documentation content changes
[ ] Other... Please describe:

What is the current behavior?

Issue Number: N/A
What is the new behavior?
Does this PR introduce a breaking change?

[ ] Yes
[x] No

Other information

Summary by CodeRabbit

Bug Fixes

Improved keyboard navigation for radio groups by ensuring consistent handling of keydown events.

你好，感谢您的PR，请问这个PR是解决哪个issue单的问题呢？有没有issue单的链接呢？

你好，感谢您的PR，请问这个PR是解决哪个issue单的问题呢？有没有issue单的链接呢？

你好，没有相关的 issue，这个问题是我在看源码的过程中发现的，对比上下文发现应该是个笔误。

@all-contributors please add @Nowitzki41 for code.


Rename models --> plugins in manifest files

Types of changes

[x] Enhancement (project structure, spelling, grammar, formatting)
[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[x] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[x] I have updated the documentation accordingly.
[ ] I have added tests to cover my changes.
[x] All new and existing tests passed.

A description of the changes proposed in the Pull Request

Resolves #407
renames any models to plugins to follow consistent naming convention across the repositories
updates initialize in manifest files to initialize/plugins
updates all example manifest files to use plugins instead of models

@jmcook1186 I created this PR to resolve #407 . I updated tests and examples accordingly. When I run the demo.yml w/ impact-engine command I see ImplValidationError, referring '"initialize.models" parameter as required'. I updated both validateImpl at util/validations and impactEngine in index.js.  I see this error coming from path: /usr/local/lib/node_modules/@grnsft/if/build/ Is there another place with model in the codebase?

Hello, sorry I missed the notifications for this - I think this will get naturally resolved as part of our big refactoring task currently underway. Encourage you to look for help-wanted tags in the issues board or reaching out to the core team if you'd like to find other ways to contribute. Thanks a lot for your efforts.


Add key differences to solidity to readme

What is wrong?
The first question I get when I tell people about Fe is "how is Fe different than Solidity?" I believe the readme should clearly list some of the key differences between it and Solidity. In other words, why should I use Fe instead of Solidity? The "features & goals" section is good, but a newcomer might not understand what makes it special.
There could be a political problem in calling out Solidity. If so, feel free to ignore this issue.
How can it be fixed
Add a list of key differences between Fe and Solidity to the readme.

There could be a political problem in calling out Solidity. If so, feel free to ignore this issue.

As a Solidity team member, I think I can safely say - don't worry :) We're cool with an objective comparison even if it points out shortcomings in the language. We're well aware of many and constantly working on improving it. Just please make it fair and be sure you're comparing latest releases and not latest Fe with ancient Solidity :)


Replaces dataview._fetch by dataview.fetch

Because _fetch wasn't really needed and it made things a bit more confusing (IMO). And, it was being used externally (by deep-insights).
@xavijam what do you think?
There's a corresponding PR in deep-insights.js (CartoDB/deep-insights.js#1079)

Fine by me!

restest this please


[Question]Process a 2second wav to text cost around 1 minute

insanely-fast-whisper --file-name .\tmp\1701954781.1698363.wav
It cost around 1 mintue to show the following message
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
The BetterTransformer implementation does not support padding during training, as the fused kernels do not support attention masks. Beware that passing padded batched data during training may result in unexpected outputs. Please refer to https://huggingface.co/docs/optimum/bettertransformer/overview for more details.
🤗 Transcribing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0:00:02
Voila! Your file has been transcribed go check it out over here! output.json

Any ideas on this?
OS: Windows
insanely-fast-whisper Version: 0.0.8
python -V Python 3.11.7

Fifty minutes of voice to text conversion from acc takes two hours, compared to fast Whisper which only takes five minutes
❯ ffmpeg -i out.aac
ffmpeg version 4.2.7-0ubuntu0.1 Copyright (c) 2000-2022 the FFmpeg developers
  built with gcc 9 (Ubuntu 9.4.0-1ubuntu1~20.04.1)
  configuration: --prefix=/usr --extra-version=0ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --arch=amd64 --enable-gpl --disable-stripping --enable-avresample --disable-filter=resample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librsvg --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-nvenc --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 31.100 / 56. 31.100
  libavcodec     58. 54.100 / 58. 54.100
  libavformat    58. 29.100 / 58. 29.100
  libavdevice    58.  8.100 / 58.  8.100
  libavfilter     7. 57.100 /  7. 57.100
  libavresample   4.  0.  0 /  4.  0.  0
  libswscale      5.  5.100 /  5.  5.100
  libswresample   3.  5.100 /  3.  5.100
  libpostproc    55.  5.100 / 55.  5.100
[aac @ 0x55b5a6e57240] Estimating duration from bitrate, this may be inaccurate
Input #0, aac, from 'out.aac':
  Duration: 00:49:48.09, bitrate: 87 kb/s
    Stream #0:0: Audio: aac (LC), 44100 Hz, stereo, fltp, 87 kb/s
At least one output file must be specified

    ~/workspace/fleet/whisper ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── INT ✘  base   at 03:05:39 
❯ insanely-fast-whisper --file-name out.aac --flash True
You are attempting to use Flash Attention 2.0 with a model initialized on CPU. Make sure to move the model to GPU after initializing it on CPU with `model.to('cuda')`.
Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
🤗 Transcribing... ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1:52:08
Voila! Your file has been transcribed go check it out over here! output.json

    ~/workspace/fleet/whisper ──────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────── took 1h 52m 53s   base   at 04:58:34 

Hi @mikeyang01 - The 1-minute time includes downloading the model (when used for the first time), initialising it and preparing it for Flash attention/ Better Transformer, preparing the audio, and finally transcribing.
The time that we track for benchmarks is only for generations since the rest would be quite small for long audios anyway.
I hope that helps clear out the confusion.

Hi @james-curtis - I'm not 100% sure but there's something definitely wrong with the environment. Ideally with the default arguments, you should get a OOM since the default batch-size is 24 which would mean roughly 13-14GB VRAM utilisation.
Can you try a fresh installation after power-cycling your GPU?

Hi @james-curtis - I'm not 100% sure but there's something definitely wrong with the environment. Ideally with the default arguments, you should get a OOM since the default batch-size is 24 which would mean roughly 13-14GB VRAM utilisation.
Can you try a fresh installation after power-cycling your GPU?

Sorry, I didn't re-use insanely-fast-whisper, because I admit that I don't have such a large amount of video memory on my machine, and even if it runs successfully, it will be slower due to frequent memory swapping. Anyway, thank you and the project team for your time.

Hi @mikeyang01 - The 1-minute time includes downloading the model (when used for the first time), initialising it and preparing it for Flash attention/ Better Transformer, preparing the audio, and finally transcribing.
The time that we track for benchmarks is only for generations since the rest would be quite small for long audios anyway.
I hope that helps clear out the confusion.
Thanks for the reply,
I would like to use it for realtime voice to text.
is it possible to use it to ASR small size audio in realtime?
thanks


Move the detail window to player.gui.left

As the title suggests. Currently having the detail window open will push any Gui's (I.E Yarm, Fat Controller when open) attached to .left down the screen.  If it is moved to left side (what I term the "info" side in my GUI) it will stay closer to the same spot without moving other things down.

Seems like a solid idea.

Implemented in r81.


Add Purdy - A useful pretty printer for debugging

Pretty print objects in real purdy colors. Allows clearer visualization of objects than you get from most pretty printers due to colors. It will also print out the complete path to an object, something that's extremely useful for debugging. Purdy will also print out the path to access a variable using Hoek/_.get format making it useful on accessing values.

Thanks for the suggestion. I'm unfortunately going to pass on this one. I have a lot of PRs to review, so I'm not able to write a custom response.

🙄


Locally build the conda package in an OSX platform

Comment:
Hi, I wonder if you guys have any experience with regard to do a local build of the OSX-64 conda package?
I have been using build-locally.py script to build the package for linux-64 on a linux machine and it worked fine.
But if I tried to do the same thing on my MacOS laptop, I got the following error.
-- The C compiler identification is Clang 15.0.7
-- The CXX compiler identification is Clang 15.0.7
-- The Fortran compiler identification is GNU 12.3.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - failed
-- Check for working C compiler: $BUILD_PREFIX/bin/x86_64-apple-darwin13.4.0-clang
-- Check for working C compiler: $BUILD_PREFIX/bin/x86_64-apple-darwin13.4.0-clang - broken
CMake Error at /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_build_env/share/cmake-3.20/Modules/CMakeTestCCompiler.cmake:66 (message):
  The C compiler

    "/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_build_env/bin/x86_64-apple-darwin13.4.0-clang"

  is not able to compile a simple test program.

  It fails with the following output:

    Change Dir: /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/work/build/CMakeFiles/CMakeTmp

    Run Build Command(s):/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_build_env/bin/make -f Makefile cmTC_f9850/fast && /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_build_env/bin/make  -f CMakeFiles/cmTC_f9850.dir/build.make CMakeFiles/cmTC_f9850.dir/build
    make[1]: Entering directory '/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/work/build/CMakeFiles/CMakeTmp'
    Building C object CMakeFiles/cmTC_f9850.dir/testCCompiler.c.o
    /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_build_env/bin/x86_64-apple-darwin13.4.0-clang   -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/include -fdebug-prefix-map=/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/work=/usr/local/src/conda/ambertools-23.3 -fdebug-prefix-map=/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p=/usr/local/src/conda-prefix  -arch x86_64 -mmacosx-version-min=10.9 -MD -MT CMakeFiles/cmTC_f9850.dir/testCCompiler.c.o -MF CMakeFiles/cmTC_f9850.dir/testCCompiler.c.o.d -o CMakeFiles/cmTC_f9850.dir/testCCompiler.c.o -c /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/work/build/CMakeFiles/CMakeTmp/testCCompiler.c
    Linking C executable cmTC_f9850
    /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_build_env/bin/cmake -E cmake_link_script CMakeFiles/cmTC_f9850.dir/link.txt --verbose=1
    /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_build_env/bin/x86_64-apple-darwin13.4.0-clang -march=core2 -mtune=haswell -mssse3 -ftree-vectorize -fPIC -fPIE -fstack-protector-strong -O2 -pipe -isystem /Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/include -fdebug-prefix-map=/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/work=/usr/local/src/conda/ambertools-23.3 -fdebug-prefix-map=/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p=/usr/local/src/conda-prefix  -arch x86_64 -mmacosx-version-min=10.9 -Wl,-search_paths_first -Wl,-headerpad_max_install_names -Wl,-pie -Wl,-headerpad_max_install_names -Wl,-dead_strip_dylibs -Wl,-rpath,/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib -L/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_p/lib  CMakeFiles/cmTC_f9850.dir/testCCompiler.c.o -o cmTC_f9850
    ld: library not found for -lSystem
    clang-15: error: linker command failed with exit code 1 (use -v to see invocation)
    make[1]: *** [CMakeFiles/cmTC_f9850.dir/build.make:100: cmTC_f9850] Error 1
    make[1]: Leaving directory '/Users/zwu/src/amber-feedstock/miniforge3/conda-bld/ambertools_1698769639923/work/build/CMakeFiles/CMakeTmp'
    make: *** [Makefile:127: cmTC_f9850/fast] Error 2

Do you have xcode installed in your machine?

Yes, I think so
(base) zwu@OXFHCW936KX2W amber-feedstock % xcode-select --install
xcode-select: error: command line tools are already installed, use "Software Update" in System Settings to install updates

Ok, apparently the OSX_SDK_DIR has to be absolute path


Add support for Video

Add widgets for recording video and storing locally on the device until they are submitted. The recording should dictate the resolution, quality level, and time allotted to reduce transfer time during submission.

I've determined that the Camera2 api does not have a support library and since we need to support API levels before 21, we will move forward with the older (and deprecated) standard camera api.

This is in version 1.2


[How to / Enhancement?] Get frame rates list and pixel format from capture device?

I can get a list of resolutions supported by the capture device. But I can not figure out how to get a list of frame rates.
Similarly, I'm looking for how to set a pixel format (for example uyvy422). Is it possible to set this up using WPF.MediaKit?

See https://stackoverflow.com/questions/49280327/


Droppers can draw and inject locked containers

Description
Bit odd to put chems out and in locked can without damaging it, no? Droppers are not syringes after all.
Reproduction

Get soda can and dropper.
Make sure soda is closed.
Draw with dropper

Should syringes be able to inject and draw past seals?
I think it would be interesting to poison a soda.
But I agree, droppers should not be able to draw from closed containers

Does this behavior happen because the dropper is of the injector type?
If injectors should still be able to inject and draw from closed containers(makes sense because they can pierce cans and stuff) Should a DropperComponent be created with a new behavior SharedDropperSystem that doesn't allow drawing and injecting into closed containers

Yes

Does this behavior happen because the dropper is of the injector type? If injectors should still be able to inject and draw from closed containers(makes sense because they can pierce cans and stuff) Should a DropperComponent be created with a new behavior SharedDropperSystem that doesn't allow drawing and injecting into closed containers

Yes it does. Anything with the injector component does (did, if my pr gets merged) not check if the container was closed before injecting/drawing. My pr adds a check to see if the injector can ignore seals and if not, check if the container is closed.

My pr adds a check to see if the injector can ignore seals and if not

Yours is a good solution. I was thinking of a whole new Component


Z2-Converter API Mainnet : GetMinimumGasPrice

Issue Summary:
Calling GetMinimumGasPrice on the ZQ1 and ZQ2 has the following diff.
Here are the URLs for ZQ1 and ZQ2.
ZQ2 : http://<IP_ADDRESS>:4201/
ZQ1 : http://<IP_ADDRESS>:4201/
Result:
Missing keys:

Different Keys:
Values that differ between ZQ2 and ZQ1:
  root['result']:
    ZQ2:<PHONE_NUMBER>
    ZQ1:<PHONE_NUMBER>

Observations:
There is a slight difference in gas price.
Conclusion:
If the above differences are expected and align with the migration to ZQ2, no further action is required. However, if these differences are unexpected, they may indicate issues that must be addressed.

ZQ2 uses an eth_getPrice of 4,761.9048 gwei which corresponds to 2,000,000,016 / 420. Any ZQ1 apps that use a hardcoded 2,000,000,000 gas price must be adjusted otherwise I don't see any reason not to accept this difference. If a ZQ2 node (aka super archive node) is about to re-execute ZQ1 transactions at a later point, we can implement an exemption.

Closing based on Zoltan's comment


Dev/2.0 twxentities

fixes #29

@it-hms I've addressed the core of issue #47 in this PR. I did not add the full steps to building a .jar file, as that is included in the Ewon development environment documentation. I've added the steps to get a full clone of the repository with the libraries that can be built following those instructions.


Cannot read property 'Player' of undefined

Hello,
I have just compiled the shaka player, trying first 'example' from here:
https://shaka-player-demo.appspot.com/docs/tutorial-player.html
but getting: Uncaught TypeError: Cannot read property 'Player' of undefinedinitPlayer @ shaka.html:26
Is this a problem with example Or I have built some wrong version of shaka-player?
Thank you

Still problem, even if i'm using version 1.6.x. My code is the following:

TurtleTube - Basic Test

      Your browser does not support HTML5 video.

@Pierpaolo1992, there is nothing obviously wrong with the code, but if you are loading Shaka Player v2, this will not work.  shaka.player.Player in v1 became shaka.Player in v2.
When you open this page in the browser, open the JS debugger.  Type this to see what version you have loaded:
shaka.player && shaka.player.Player.version || shaka.Player.version

That will display the version number, both for v1 and v2 releases.


Update Tor-Browser to version 8.5

Tor-Browser version 8.5 has just released. I think they add win64 to the binary name, that's why checkver doesn't work.
32 bit build seems to be still available in this link though

{
    "description": "Web browser that connects to the internet via the Tor anonymity network",
    "homepage": "https://www.torproject.org/",
    "version": "8.5",
    "##": "Multiple components under various open source licenses. Included HTTPS Everywhere extension is the strictest.",
    "license": "GPL-3.0-or-later",
    "architecture": {
        "64bit": {
            "url": "https://dist.torproject.org/torbrowser/8.5/torbrowser-install-win64-8.5_en-US.exe#/dl.7z",
            "hash": "077032e7b0d9849e95917c42364b0f9e7c81090d2106113c843a6d67c49afc3c"
        },
        "32bit": {
            "url": "https://dist.torproject.org/torbrowser/8.5/torbrowser-install-8.5_en-US.exe#/dl.7z",
            "hash": "119da848876ed7fa786807e0214f7250c93045b4e4d314d2e3a7ad60962b4916"
        }
    },
    "extract_dir": "Browser",
    "bin": [
        [
            "firefox.exe",
            "tor-browser"
        ]
    ],
    "shortcuts": [
        [
            "firefox.exe",
            "Tor Browser"
        ]
    ],
    "checkver": {
        "url": "https://www.torproject.org/download/",
        "re": "torbrowser/(\\d+(\\.\\d+)*)/torbrowser-install-win64-\\1_en-US.exe"
    },
    "autoupdate": {
        "architecture": {
            "64bit": {
                "url": "https://dist.torproject.org/torbrowser/$version/torbrowser-install-win64-$version_en-US.exe#/dl.7z"
            },
            "32bit": {
                "url": "https://dist.torproject.org/torbrowser/$version/torbrowser-install-$version_en-US.exe#/dl.7z"
            }
        },
        "hash": {
            "url": "https://dist.torproject.org/torbrowser/$version/sha256sums-signed-build.txt"
        }
    }
}

Here's working autoupdate, but 7z isn't able to extract 64bit installer, looking for input on how we can extract it.

#2191


Peering updates

WIP
Here is what is done though.

Bug fix for issues with peering. Registry was being pulled out in an inconsistent format. This could led to a connect->disconnect loop.
Added direct disconnection.
Added disconnecting from cloud to zetta hub via API requests.

This is ready for review if someone can take a look at it. Going over it multiple times would probably be best. This thing is kind of large.

cc: @kevinswiber @AdamMagaluk

After that long process i think its ready to be green lit. Merge away.
Signed-off: Adam T Magaluk


feat: add dinero incentives markets program

Context
DINERO supply incentives for the apxETH/WETH market on Mainnet

Hey @0xKubko, we are waiting for you to send the funds to 0x330eefa8a787552DC5cAd3C3cA644844B1E61Ddb with 0x7f6494D4fBEA1c06daC2250A3FCa81003bF8D20C on mainnet to merge the PR.


Call onFileUploaded when the upload was succesful.

Looks to me like onFileUploaded was not used, and the documentation was out of sync. I needed a post-upload handler, so I figured I'd use the unused handler.

Dank u!


Deployed staging admin interface doesn't allow changes/saves/posts cuz CSRF

Use http, not https, and it works.


how to add words char-composition to ner_dstc2

Hi!
I've trained ner_dstc2 component on my data (russian dialogs).
But it performs more or less good only if user's answers have more then one word in response.
As I see this caused by that the ner is based only on words and there is now char-composition accounting.
I found that ner_rus (based on Conll2003) uses char embeddings.
Is it the right way to stack char-analysing components in the same way and to use its ner model component?

You can just use the ner_rus config or copy the char level parts from it to train the components from scratch. If you want to use pretrained parts of the ner_rus the easiest way would be just to pass the predictions from the pretrained network to the new network. However, the ner_rus is trained on the news data which is almost long context.

Hi!
When i trained ner_rus I found that the directory download\total_rus (where ner_rus takes data from) contanined train.txt, test.txt, valid.txt in english.
So when I use yours prebuilt model it gives more or less right tags for russian texts.
But if I use the model trained using ner_rus it gives 'o'-tags for every token in any russian text.


Discrepancy in skating ratio: HumanML3D dataset vs. OmniControl results

Dear Authors,
While working with OmniControl, I've noticed some issues regarding the skating ratio that I'd like to bring to your attention:
1.Skating ratio in the HumanML3D dataset:

The paper seems to assume a skating ratio of 0 for the HumanML3D dataset.
However, when testing with the code in the repository, I'm getting a result of about 0.075.
Does this discrepancy indicate a potential issue with the code? Or does the original dataset indeed have some degree of skating?

2.Skating ratio in OmniControl generated results:

I've observed that the motions generated by OmniControl actually have a lower skating ratio(0.0547) than the ground truth motions.
Is this an expected outcome? If so, could you explain the reasoning behind this?

3.Skating ratio calculation method:

Does the current calculation method account for potential issues like clipping or floating?
Is there a more precise method to calculate the skating ratio, especially when dealing with potential baseline shifts?
Thank you for your time and consideration. Your insights will greatly help in better understanding and utilizing OmniControl.

Hi Qingyuan,
Thanks for asking. The Ground Truth motion is not perfect so it's reasonable to have foot skating. A better way might be also show the GT skating ratio in the table as a reference.
It's also possible that the predicted motion has a lower skating ratio or even lower FID than Ground Truth as the GT motion sequences are captured using some mocap methods that aren't flawless and may contain some errors.
Our calculation method exactly follows GMD code (https://github.com/korrawe/guided-motion-diffusion/blob/main/data_loaders/humanml/utils/metrics.py#L204). Not sure if it includes issues like clipping. As far as I know, there are no other public code scripts that can calculate the skating ratio.
Thanks


Added "advanced" .capabilities scanning

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Still in testing, opened PR to track.

Will move this to a new PR since it'll become a standalone feature.


Adding Porter to sandbox

Pre-submission checklist:
Please check each of these after submitting your pull request:

[x] Are you only including a repo_url if your project is 100% open source? If so, you need to pick the single best GitHub repository for your project, not a GitHub organization.
[x] Is your project closed source or, if it is open source, does your project have at least 300 GitHub stars?
[x] Have you picked the single best (existing) category for your project?
[x] Does it follow the other guidelines from the new entries section?
[x] Have you included a URL for your SVG or added it to hosted_logos and referenced it there?
[x] Does your logo clearly state the name of the project/product and follow the other logo guidelines?
[x] Does your project/product name match the text on the logo?
[x] Have you verified that the Crunchbase data for your organization is correct (including headquarters and LinkedIn)?
[x] ~15 minutes after opening the pull request, the CNCF-Bot will post the URL for your staging server. Have you confirmed that it looks good to you and then added a comment to the PR saying "LGTM"?

Build failed because of:
No cached entry, and Porter has issues with logo: porter-sh.svg, Error in parsing SVG: Attribute without value
Line: 50
Column: 89
Char: >

Deploy preview for landscape failed.
Built with commit ca734b0db22a5ff0134b8a26fd3f69fdf82a198a
https://app.netlify.com/sites/landscape/deploys/5f80aae34db57c0007a8edb0

LGTM


Fix system framework localizations and add notification

Fix issue with system frameworks not loading localized strings (https://github.com/cmaftuleac/BundleLocalization/issues/1)
Set "AppleLanguages" user defaults to allow system frameworks to load localized strings in the same language that is set using setLanguage:
Send a notification when the localization is changed using setLanguage:

this works. it even solve the problem when integration with 3rd party UI elements such as zopim.


fix: now markdown has common styles

Description
Adds some default styles for markdown renderer in StarSearch. It fixes headings and lists. @isabensusan, feel free to tweak my suggestions in globals.css.
Related Tickets & Documents
Fixes #3337
Mobile & Desktop Screenshots/Recordings
Before

After

Steps to QA

Ask Star-Search a question, e.g. "who is nickytonline and what has he written about recently?"
Inspect the HTML
See that the markdown was rendered and it has some base styling now (headings, lists)

Tier (staff will fill in)

[ ] Tier 1
[ ] Tier 2
[ ] Tier 3
[x] Tier 4

[optional] What gif best describes this PR or how it makes you feel?

Sidenote @jpmcb, but I don't think we need those  s you added in a previous PR, or maybe not in all cases.

I don't think we need those  s you added in a previous PR, or maybe not in all cases.

cc @nickytonline - I honestly have no idea: the model seems to send empty data: chunks when it wants a new line. But I can't figure out a good way for it to render that via react-markdown since line breaks in markdown aren't really a well standardized thing.
So yeah, up to you how we approach this, already looks way way better in this PR.


How to get the last messages of a user using a bot?

Is there any method to get the last messages of of a chat using the chat_id?

No.

No.


Android add image compress and use SimpleCameraX

This repositorie already have compress and camerax dependencies
  // image compress library (Not necessary)
  implementation 'io.github.lucksiege:compress:v3.11.1'

  // simple camerax library (Not necessary)
  implementation 'io.github.lucksiege:camerax:v3.11.1'

add option compressSize : Do not compress when the origin image file size less than compressSize value, default 10Kb

Usage:
import { openPicker } from '@baronha/react-native-multiple-image-picker';

openPicker({
    mediaType:'image',
    maxSelectedAssets:9,
    compressSize: 20480 //Image file lager than 20Kb will be compressed
}).then((response) => {
    console.log("picker",response);
}).catch(() => {
    console.log("user cancel...");
});

need more test.


svg2png: changing image size

For resolution > 8k image's width and height are changed, but the change was not applied to the image itself. Now fixed.

btw, just wonder how it should have 8k limit.... @_@?

btw, just wonder how it should have 8k limit.... @_@?

Here https://github.com/thorvg/thorvg/pull/1279, Maybe heap overflow occurred (issue : https://github.com/thorvg/thorvg/issues/1278)


Rename method ConfigurationPath.GetSectionKey() to GetChildKey()

Current name is unintelligible as said in https://github.com/aspnet/Configuration/pull/461/files/100a4a6ad7a4b371b3068e623da2bb498936851b#r66742090

I don't particularly like this name, but its your unintelligible name :)  See https://github.com/aspnet/Configuration/pull/389
I chose a functional name for it originally, I called it GetLastSegment() which is what its doing...

I don't particularly care if came up with it originally.

This rename doesn't meet the bar for 1.0.0 and therefore won't happen later either. We should make sure the API documentation helps clarify the purpose of the API.


Remove duplicate OOM event reports in OOM tracker (1.15)

Backport of #1368 to release-1.15 branch.
/sig scalability
/cc mm4tt

/lgtm
/approve


webcam liveness detction

HI,
Thank you very much for posting the various types of face recognition solution.
Livenessdetection in flask is possible with the code you have provided in testing_webcam_livenessdetection.py ? can we directly pass the arguments in the code? Please suggest this. Thankyou

Got it.


Add new Felix configuration parameters

Description
Add new Felix configuration parameters.

Todos

[ ] Tests
[ ] Documentation
[ ] Release note

Release Note

None required

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Docs: https://github.com/projectcalico/calico/pull/2546


Testing flags: Migrate from gflags to abseil's flags

Currently there are ~36 uses of gflags (docs) under the /test subdirectory. This is to track changing them to use Abseil Flags (docs), which aligns with migrations that have already occurred in the Google monorepo. Abseil's Flags is an evolution of gflags, requiring use of getter and setter methods, and providing, via helper libraries, support for additional types.

Mark, I started some work on this and filed this issue to track it, so you can assign it to me.

Nathan, can you please create the PR again with the workaround in my test PR (#24560)? Basically workaround is just using the shortened help message which is not that bad until the Abseil team manages to fix this properly.

Certainly. It'll be tomorrow though. Thanks for chasing down the issue!


Initial rfc for pip install buildpack

This might be a dumb question, but why would we want to ignore previously installed packages (with the --ignore-installed flag)? Wouldn't we want to make use of anything we've already installed?

@sophiewigmore In my understanding --ignore-installed means don’t uninstall before reinstalling.


So, how do I use it ?
I tried the following, but it didn't work, presumably because npm did not install the module in a central place. Running npm i as root does also not yield the right result.
Is there anything else I need to do ? I presume I might need to add a path where the actual module is compiled or something, but if so, it might be worthwhile pointing that out in the README.md for idiots like me :)
node-rpi-rgb-led-matrix$ npm i
# ... compiles and stuff

node-rpi-rgb-led-matrix$ cat > testit.js
var LedMatrix = require("node-rpi-rgb-led-matrix");

var matrix = new LedMatrix(16);
matrix.fill(255, 50, 100);
matrix.setPixel(0, 0, 0, 50, 255);
^D

node-rpi-rgb-led-matrix$ sudo nodejs testit.js
module.js:340
    throw err;
          ^
Error: Cannot find module 'node-rpi-rgb-led-matrix'
    at Function.Module._resolveFilename (module.js:338:15)
    at Function.Module._load (module.js:280:25)
...

Yes, it is a nodejs beginner error. It is an error about the require function.
I'm not remember perfectly the rules about the require function but it is something like that:

require check the module name passed as parameter is nodejs module. You can find them onto nodejs doc: https://nodejs.org/docs/latest-v5.x/api/ for the latest version (the 5.x).
For example: var fs = require("fs"); load into fs the filesystem nodejs module
If the parameter is not a nodejs module, it checks if exists in node_modules directory. The node_modules directory contains all dependencies your project have.
If it is not in node_modules, require try to load from the path. For example: var module = require("./myawesomemodume.js")

So, in summary:

you should put node-rpi-rgb-led-matrix directory into a node_modules directory
or instead of use var LedMatrix = require("node-rpi-rgb-led-matrix");, you should use var LedMatrix = require("./index.js");

Hope my explanations are clear. Good luck ! ;)

Ah, thanks. Testing with ./index.js worked!


News module not working

I just pulled the new version today and uploaded it and the news module isn't working
I just changed the rss feed to "http://www.reddit.com/r/news/.rss", as I've done with many previous versions and not changed any internal settings.

"No feed results for: https://query.yahooapis.com/v1/public/yql?format=json&q=select * from rss where url%3D'http%3A%2F%2Fwww.reddit.com%2Fr%2Fnews%2F.rss'"
Failed to show a news story 20 times, stopping any attempts

I get an error when I click on your second link
{"error":{"lang":"en-US","description":"Query syntax error(s) [line 1:62 mismatched character ' ' expecting ''']"}}
You're sure the url is correct? Maybe reddit has some issues in their rss feed? Using a different feed it works for me.

@jerbookins Where you able to solve this? Can I close this issue?

I am getting the same error as @jerbookins after replacing the feed with the AP feed
news: {
        feed: 'http://hosted2.ap.org/atom/APDEFAULT/cae69a7523db45408eeb2b3a98c0c9c5'
    }

{"error":{"lang":"en-US","description":"Query syntax error(s) [line 1:98 mismatched character ' ' expecting ''']"}}

It's due to the usage of the yahoo api. In a previous version we did not use it. I'm not very happy with the Yahoo solution, but don't have the time to make a new solution for it. A pull request would be kindly appreciated.

This will be solved in V2.


captcha module

Abstract interface over other captcha providers based on config

@NathanFlurry I am interested in working with the captcha module, could u provide some more info regarding the issue so that I could start working around it?


Custom code actions

Greetings,
I'm looking for a way to extend currently available code actions with the custom ones. For instance, I need a way to to add my own refactoring that should be available on C# code in a Razor markup file.
A little research of the source code revealed the following "hard-coded" logic that filters available code actions by name:
https://github.com/dotnet/razor-tooling/blob/8a81847851689218a64f40c586bc4a2ab99d3801/src/Razor/src/Microsoft.AspNetCore.Razor.LanguageServer/CodeActions/CodeActionEndpoint.cs#L300-L317
Am I right that this is impossible at the moment?

@Seteh love that you're interested in adding your own type of code actions that also work in Razor files! To add some clarity as to why we hardcode a list of "allowed" code actions: CodeActions can pretty much do anything, add project references, add usings, remove/delete code so we were left with a difficult problem of "what happens if a CodeAction tries to delete auto-generated boiler plate code that's required" or "what happens if a code action wants to add new members" and how does that translate to a Razor document. More over "what" a code action does is actually a separate functional behavior that should a code action be available. Soooo in Razor's case we can show certain code actions but on selection of a code action it's possible that bogus edits get applied to the document leading us to feel that we should have never showed that code action to begin with.
To account for the above we built an "allow list" of code actions that are understood to not do unsupported operations. Ultimately this is just a stop-gap until we are able to build the infrastructure to detect unsupported scenarios better though.
Ok so with that background out of the way we're in an interesting predicament where we need to decide how we want to handle custom code actions. Given the right infra things should "just work" however; that's quite a hard problem leaving us in an interesting situation where how can we enable this earlier for folks. Some thoughts that have been talked about in the past is having a marker on code actions that say "I can work in Razor".
What sort of code actions are you looking to build?

Hi, @NTaylorMullen,
Thank you for your reply and please pardon my late response.
I'm working for DevExpress in the IDE Productivity Tools team. Our team's main product is CodeRush, that you may have heard about. Approximately a year ago we announced support for Razor editor in Visual Studio 2019 and shipped ported versions of existing C# features along with new Razor specific ones, e.g., Extract Razor component code provider. That was a quite big task for us and we still remember how resistant to modifications the old editor was, so we can understand your decision to add such a stop-gap at the early stage of development when the proper infrastructure is not yet completely ready.
However, since our users already asked about new razor editor support for CodeRush me and my colleagues are now looking for recommended extensibility points, if any. You've mentioned an approach with markers:

Some thoughts that have been talked about in the past is having a marker on code actions that say "I can work in Razor".

From our perspective it is a good idea. We don't see any issues with that approach and will be happy to assist you. Just let us know what we can help you with.

What sort of code actions are you looking to build?

As one of the CodeRush developers I'm looking for a way to bring Razor support for an existing subset of features, but as an Visual Studio extension developer, I'm looking for more general thing: a proper extensibility point of the new Razor tooling. With that we will be able to accomplish requests from our users without any dirty hacks or anything like that.

@Seteh oh wow that Extract Razor component code provider must have been quite challenging to construct!! Very impressive 😄.
So I'm not entirely sure how you extended the legacy Razor editor to enable that functionality; however, the new editor should actually make the expansion a bit easier given it seems that you're installing a VS extension (you don't need to wait for an "allow list"). The new editor utilizes LSP to provide code actions and by default LSP allows multiple language servers to operate on a single document. What this means is you should be able to write a language server, hook it into Visual Studio like we do our own Razor language server, support the code action LSP endpoint and then walla you should be able to return light bulbs that mutate the parent document while simultaneously introspecting the Roslyn Workspace. If you're curious this is the C# Razor language server (there's a separate one for Razor only because normal C# documents aren't fully LSP yet).
Some interesting decision points that we may want to talk through more is how you all determined "when" to show the light bulb in the old world + how you ensured things didn't corrupt pre-existing editor state without crashing VS. Thankfully for the latter LSP should take care of ensuring things don't explode / break. For the prior understanding "when" might be a bit more difficult because we don't use projection buffers anymore, meaning knowing when you're selecting C#/HTML etc. is a bit more challenging.
Oh and with this model it means your light bulbs could show up / function perfectly in LiveShare with minimal/no effort.

Anyhow, does the above make sense? I'm psyched this is a path you all are interested in pursuing 😄

Hi @NTaylorMullen,
Yet again thank you for the reply. And for your kind words as well, 😌.
You're right - writing our own razor-tooling almost from scratch is not a way we are looking for. Honestly speaking, we expected the same extensibility model that TypeScript language server already has. It allows you to register your own plugin that will be executed in-proc. Will the Razor language server receive the same extensibility model?

However, we tried the 'Just write own ILanguageClient implementation' approach you've suggested and it seems like this statement isn't true:

by default LSP allows multiple language servers to operate on a single document

According to our research, Visual Studio uses the first (or the default) language client available for a particular content type.
At least, we did manage to populate the Light Bulb menu with stab code actions from our ILanguageClient implementation.
Any help is greatly appreciated, 🙂.

Will the Razor language server receive the same extensibility model?

We don't currently plan to support a similar model. Our current plans are to piggy back on LSPs built-in extensibility provider model. If you'd be willing to share how your plugin interacts with the C# workspace I'd be willing to (or find someone who knows to) point you in the right direction for how to do similar things in Visual Studio.

According to our research, Visual Studio uses the first (or the default) language client available for a particular content type

Hmm, that definitely shouldn't be the case. I just took a peak at the decompiled code of ILanguageClient bits and they do seem to be requesting light bulbs from all ILanguageClient implementations. Maybe something else is falling over in the process. What content type are you targeting?

Hi @NTaylorMullen,
I apologize for the delayed reply.

If you'd be willing to share how your plugin interacts with the C# workspace I'd be willing to (or find someone who knows to) point you in the right direction for how to do similar things in Visual Studio.

This is exactly what I'm currently stuck with - how do I get the workspace information on my endpoint?

Mind expanding on what stab code actions are?

I created a new VSIX project where I tried to implement a simplified version of our Add Parameter code provider for razor documents. In short, my goal was to update all references of a method in order to add a parameter to its definition and an argument to its every call site.
Based on the information from your previous replies, I created my own ILanguageClient descendant and added the CodeActionEndpoint type. However, I could not obtain a current document. I reviewed the Handle method of your CodeActionEndpoint class to understand how you accomplish that. Then, I tried to re-implement this approach on my side, however, it resulted in cloning more than a half of your repository code: ProjectSnapshotManagerDispatcher, DocumentResolver, RazorDocumentMappingService, etc. since almost all of the types are marked as internal.
This way is hard to follow since it requires syncing your changes with my code with every release, and it's definitely not what I am looking for.
Please let me know if I correctly understand you and whether I am going in the right direction.

This is exactly what I'm currently stuck with - how do I get the workspace information on my endpoint?

Ah you can import the VisualStudioWorkspace (example of an area where we do it)

I created a new VSIX project where I tried to implement a simplified version of our Add Parameter code provider for razor documents. In short, my goal was to update all references of a method in order to add a parameter to its definition and an argument to its every call site.

Ah having access to the workspace above should enable you to do that functionality I believe. The gist is that in an LSP world you'll calculate all of the edits that need to happen and then return them as a workspace edit in your code action. For instance this  is C#'s code action resolve handler which gets called when a user selects a code action, its responsibility is to resolve "what" will happen.

This way is hard to follow since it requires syncing your changes with my code with every release, and it's definitely not what I am looking for.

Ya absolutely don't want you to have to do that (and you shouldn't need to). SO a language server can be written in many different ways and ultimately doesn't need to have really any dependencies. In the end it's just a server which can receive and respond to requests based on specific LSP contracts. On the Razor side we use the O# language server framework  to construct our language server whereas others, for instance Roslyn, does it manually with StreamJsonRpc  (note the "JsonRpcMethod" attributes on Roslyn's impl) or products like TypeScript use NodeJS. The gist is that language servers can be written in any language and are expected to abide by the LSP specification to ensure clients know how to communicate with them. So in your case you're looking to add a new light bulb as an option. To do that you'll need to have a language server that cares about:

Text document events, this enables you to know when documents open/close/what's their content
Code Actions, this allows you to provide light bulbs & resolve what they do when a user selects them

Other than these two endpoints your language server doesn't need any other magic. However, filling out "how" to calculate code action logic is where the tricky part may come in. In your textDocument/codeAction handler (this is the "provide" part of light bulbs) you'll probably want to import the VisualStudioWorkspace and then lookup corresponding documents for the currently active light bulb request to decide if you should provide yours. Then you'll next want to fill out your codeAction/resolve handler to also pull in a workspace but this time you'll be handed what code action was selected at which point you'll need to calculate "what happens". Similar to what Roslyn does in their code action handler I'd imagine.

@NTaylorMullen hi!
While I'm not sure that my delayed replies bothering you, however, I still feel myself ashamed for allowing @msftbot to break this discussion flow. My bad.
I'm not working for DevExpress anymore and is going to close this issue.
Thank you for spelling out the whole process in stupefying detail. I think the remaining members of the CodeRush team will find this information useful.
Happy New Year and may all your wishes come true!

@Seteh delayed responses don't bother me at all! Love that you started this conversation and for any future CodeRush person feel free to repoen then or create new (whatever works for you) and we can continue the convo.
Happy new years!


Optionally enable multiple SANs

This PR unfortunately does not seem very helpful. Even if we disregard the fact that the implementation is not ideal (it should probably be a generator expression instead for debug mode only, but that’s also a problem in the current setup; these lines were a temporary thing and were never supposed to be in the main repo anyway), sanitisers are only really helpful for developers, and developers know how to enable them if they want to use them.
Currently, there is another reason why sanitisers are not desirable: The project is currently undergoing fundamental changes that turn entire systems upside down, and memory leaks etc. are irrelevant for the time being so long as it works. Only when we’re done implementing a feature would it make sense to enable ASAN etc. to stomp out bugs, memory leaks, etc. I last did that a few weeks/months ago. Most of the time during development, however, they’d mostly be a nuisance. Just enabling sanitisers unfortunately doesn’t fix anything and instead can actually impair development if there are more important matters at hand.
Furthermore, as was already discussed, sanitisers are not available on all platforms/compilers; all in all this really doesn’t seem like it needs to be a part of the main CMake file. We might want to temporarily enable sanitisers sometimes, but other times they really wouldn’t be that useful. These lines are not intended to be in there for ever and are certainly not supposed to be a documented option or part of the API or anything like that. I simply forgot to remove them.
Finally, the tone of this pr seems a bit off to me. First, the comments you added seem a bit condescending to me; we know what sanitisers are. Secondly, ‘guessing’ CMake flags such as an apparently non-existant ‘ARM’ flag doesn’t help anyone. CMake is already confusing enough what with its multitude of options, what with the fact that for anything you could ever possibly think of doing, there seem to be 7 different ways of doing that thing, and what with the confused, whitespace-sensitive syntactical mess that is generator expressions... The simpler our CMake config remains, the better. Complexity is needed sometimes, but adding options guarded by flags intended to be used only by developers, when debugging, and only sometimes, doesn’t seem like it would be particularly helpful unfortunately...


How does one define an "epoch" when the data loader is infinite?

I just started playing with pytorch-lightning API to decide whether to make the switch for my own speech processing project.
I have an infinite DataLoader that wraps an IterableDataset that basically extracts random audio chunks from a large speech corpus, forever. As such, it has no __len__ and never raises StopIteration exception.
Therefore, calling trainer.fit(...) stays at Epoch 1 forever as the batch number keeps increasing.
I am looking for a way to tell trainer.fit(...) that it should consider that an epoch is completed every N batches. Is there any?

set __len__ to return n

Thanks for your quick answer, @s-rog. It does the trick.

Hello,
I'm reopening this issue  with a follow-up question. Setting __len__ indeed does the trick but it also restarts the infinite IterableDataset at the end of each epoch.
For illustrating the problem, let's imagine a dataset from which we will get information from a pseudo-infinite number of files:
class InfiniteDataset(torch.utils.data.IterableDataset):
    def __init__(self, n_files, n_steps):
       self.n_files = n_files
       self.n_steps = n_steps    # defines an epoch -> thus how often we check the callbacks, compute the val_loss, etc.

    def __iter__(self):
       while True:
           for x in range(self.n_files):
               yield x    # this correspond to the index file we would like to work with and do whatever

    def __len__(self):
       return self.n_steps

In the first epoch, I expect to use the files from [:n_steps] and then, for the next epochs the files from [n_steps*current_epoch:n_steps*current_epoch+n_step]. However at each epoch I only have access to the [:n_steps] files. Here's a running example:
class DumpModel(pl.LightningModule):
    def __init__(self):
        super().__init__()
        self.automatic_optimization = False
        self.some_weights = nn.Sequential(nn.Linear(28 * 28, 64), nn.ReLU(), nn.Linear(64, 3))

    def training_step(self, batch, batch_idx):
        print(batch)    # check the info given by the dataloader
        return

    def validation_step(self, batch, batch_idx):
        return

    def configure_optimizers(self):
        return torch.optim.Adam(self.parameters(), lr=1e-3)

ds_train = InfiniteDataset(n_files=999, n_steps=3)
model = DumpModel()
trainer = pl.Trainer(max_epochs=3)
trainer.fit(model, train_dataloaders=ds_train)

Outputs
Epoch 0:   0%|                                                                                | 0/3 [00:00<00:00, 4604.07it/s]
0    # expected file index
Epoch 0:  33%|████████████████████████████████████████████▋                                                                                | 1/3 [00:00<00:00, 2099.25it/s, v_num=0]
1
Epoch 0:  67%|█████████████████████████████████████████████████████████████████████████████████████████▎                                            | 2/3 [00:00<00:00, 1800.39it/s, v_num=0]
2
Epoch 1:   0%|                                                                                | 0/3 [00:00<00:00, 1973.79it/s, v_num=0]
0   # expected file index -> should be 3
Epoch 1:  33%|████████████████████████████████████████████▋                                                                                | 1/3 [00:00<00:00, 1093.41it/s, v_num=0]
1   # expected file index -> should be 4
Epoch 1:  67%|██████████████████████████████████████████████████████████████████████████████████████████                                             | 2/3 [00:00<00:00, 985.20it/s, v_num=0]
2   # expected file index -> should be 5
Epoch 2:   0%|                                                                                | 0/3 [00:00<00:00, 2790.62it/s, v_num=0]
0   # expected file index -> should be 6
Epoch 2:  33%|████████████████████████████████████████████▋                                                                                | 1/3 [00:00<00:00, 1700.16it/s, v_num=0]
1   # expected file index -> should be 7
Epoch 2:  67%|█████████████████████████████████████████████████████████████████████████████████████████▎                                            | 2/3 [00:00<00:00, 1552.68it/s, v_num=0]
2   # expected file index -> should be 8
Epoch 2: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 3/3 [00:00<00:00, 53.14it/s, v_num=0]

Is there an easy way of defining the n_steps of each epoch for an IterableDataset without restarting it?
Thanks!

I am also currently looking into this as I am facing a similar issue in my own project.
As far as I understand, the data loader is not actually reset/restarted between each epoch. There is an option for that (reload_dataloaders_every_n_epochs (int)  in the trainer), but it is not set per default.
However, the iteration is restarted for each epoch, so a new iterator is created by calling __iter__ of the IterableDataset again, and this is where you are misunderstanding what's happening. You need to keep track of where you stopped during the last training  iteration yourself inside of your InfiniteDataset implementation. An intuitive way in your minimal example might look like this:
class InfiniteDataset(torch.utils.data.IterableDataset):
    def __init__(self, n_files, n_steps):
       self.n_files = n_files
       self.n_steps = n_steps    # defines an epoch -> thus how often we check the callbacks, compute the val_loss, etc.
       self.current_file = 0

    def __iter__(self):
       while True:
           yield self.current_file
           self.current_file += 1
           if self.current_file >= self.n_files:
               self.current_file = 0

    def __len__(self):
       return self.n_steps

I personally prefer to manually handle the iteration instead of using yield though, especially because it allows me to properly implement an iterable dataset that can leverage multiple data loader worker threads. In your minimal example, that would look like this:
class InfiniteDataset(torch.utils.data.IterableDataset):
    def __init__(self, n_files, n_steps):
       self.n_files = n_files
       self.n_steps = n_steps    # defines an epoch -> thus how often we check the callbacks, compute the val_loss, etc.
       self.current_file = 0

    def __iter__(self):
        self.current_file -= 1
        return self

    def __next__(self):
        self.current_file += 1
        if self.current_file >= self.n_files:
            self.current_file = 0
        return self.current_file

    def __len__(self):
       return self.n_steps


/uses catHeading covers navBar

On the /uses route, the .uses_catHeading covers the header when scrolling down the page due to the same z-index value (10).

Not sure if intentional! Quick fix: changing z-index to 9 for .uses_catHeading
.uses_catHeading {
  position: relative;
  top: 2.5rem;
  z-index: 9;
}

Thank you for taking the time to report the issue with such detail!


Change API surface for integration to make more sense

The API is confusing today and hard to reason what does what.
Look at something like below for the methods.
DefineModel => DefineProjection
DefineImport => DefineEventEmitters / DefineEvents
DefineImportMapping =>  DefineInputModelMapping
In addition the builder for DefineEventEmitters could be clearer as well:
builder.OnPropertyChanged(s => s.Property)....

for multiple properties:
builder.OnPropertiesChanged(s => s.Property, s => s.OtherProperty)....

The IImportBuilderFor can also be simplified, not taking more than the target model. We are not interested in the input model at this point.
The callback for .AppendEvent() should have a better context as well. Today it requires navigation through the changeset and incoming, which does not make sense. Instead we could give the dev the incoming instance.
Multiple overloads could be for this as well, when one need more context.
Another option is also to provide an overload that gives you a mapper builder that tells how to map from the projected version to an event.

I also think IFromBuilder/ISetBuilder can be merged into a IMapBuilder or something similar, as the current syntax of Set/To is (at least to me) very confusing with regards to how the values flow (is "Set()" defining the target or the source?)
    public override void DefineModel(IProjectionBuilderFor<MyProjectionModel> builder)
    {
        builder.From<SomeEvent>(_ => _
                .Set(_ => _.Value1)
                .To(_ => _.Value1)
                .Set(_ => _.Value2)
                .To(_ => _.Value2));
    }

I propose a simpler interface, with just a Map() method, with parameters in a flow oriented order (source => target).
    public override void DefineProjection(IProjectionBuilderFor<MyProjectionModel> builder)
    {
        builder.From<SomeEvent>(_ => _
                .Map(_ => _.Value1, _ => _.Value1)
                .Map(_ => _.Value2, _ => _.Value2)
    }

(perhaps the overridable method should be called ProjectionBuilder()? 🤔)

As for the method currently known as DefineImport(), where I suggested DefineEventEmitters() (which at least describes what it performs, although I'm sure an even better name can be found);
Here I would want an IEventBuilder which defines a .Map() method similar to the proposed IMapBuilder , as the ChangeSet here seems like something I as a developer often will not need to know about when defining the events to be emitted.
It should also be possible to set hard-coded values here.
So something like this (I've also renamed IImportBuilderFor to IEventBuilderFor):
    public override void DefineEventEmitters(IEventBuilderFor<ForsendelsesDetaljer> builder)
    {
        // Trigger on spesific property/propierties change
        builder.OnPropertyChange(s => s.ProjectionValueProperty1)
            .AppendEvent<MyBasicEvent>(_ => _
                    .Map(projection => projection.ProjectionProperty1, @event => @event.ProjectionProperty)
                    .Map(projection => projection.Id, @event => @event.SomeIdentifierProperty)
                    .Map("MyApplication", @event => @event.Origin.Application);

        // Trigger on a changed property in s.ProjectionReferenceProperty, e.g. s.ProjectionReferenceProperty.SomeValue
        builder.OnPropertyChange(s => s.ProjectionReferenceProperty)
            .AppendEvent<MyFancyEvent>(_ => _
                    .Map(projection => projection.ProjectionReferenceProperty.SomeValue, @event => @event.SomeValue)
                    .Map(projection => projection.Id, @event => @event.SomeIdentifierProperty);

        // Do automapping mixed with (followed by) defined mapping
        builder.OnPropertyChange(s => s.ProjectionValueProperty1)
            .AppendEvent<MyBasicEvent>(_ => _
                    // First automatically map properties from the projection to event, where name and type are same.
                    .AutoMap()
                    // Then automap from Subclass properties directly to event
                    .AutoMap(projection => projection.Subclass, @event => @event)
                    // Finally, any manually defined mappings
                    .Map(projection => projection.Id, @event => @event.SomeIdentifierProperty)

    }
    ```

Great input and samples. Love the idea about the .AutoMap() for nested objects.
The definition API for the projection is the same API used for regular projections.
The reason for the division of operations today (.Set().To()) is for readability and flexibility. There are today methods for doing more than just setting it to a value from the event. There are things like .ToEventSourceId() and we could have things like .ToCurrentDate() or other operations. With a single .Map() method it becomes rigid and less readable as you can only work on overloads.
It could be a rename of .Set() to .Map() but I’m unsure if that reflects accurately on what we’re trying to do.
The projections API in particular has been an attempt at mimicking a query language in a way, a cross between SQL and LINQ in mindset. from EventA, EventB, EventC select { PropA = EventA.Prop, PropB = EventB.Prop, PropC = EventC.Prop }.

Notes from a Slack discussion.
A possibility would be to have .Set() take an expression that gets parsed rather than splitting it into a fluent interface, making it more intuitive to work with:
        builder.From<SomeEvent>(_ => _
                .Set((m,e) => m.Value1 = e.Value1)
                .Set((m,e) => m.Value2 = e.Value2)

This would allow for literals in the same way:
        builder.From<SomeEvent>(_ => _
                .Set((m) => m.Value1 = 42)
                .Set((m) => m.Value2 = "Magic String")

For things in the EventContext or other formalized values these could be given as part of the expression:
        builder.From<SomeEvent>(_ => _
                .Set((m) => m.Value1 = EventContext.Occurred)
                .Set((m) => m.Value2 = DateTimeExpressions.UtcNow)

The expressions would be C# Expression<> and we would need to validate them with the operations we support. They are not to be mistaken as callbacks. Clear error messages to the developer would be necessary.

Closing this, as we will circle back to integration at a later stage.
The goal is to let this be a Kernel service, and the clients just specifying how it should do it in a declarative manner, similar to what we have, but not calling the client. This would enable the server to have these as jobs.


Lazy import of sklearn and xgboost

Importing xgboost just before using it is needed is useful on systems where it is not installed.
Since installation of XGBoost can be tricky, I think this is a good idea.

Hi @yoavram, let me see if I've understood your issue.
You want to say that in current release version you need to have installed both scikit-learn and xgboost, even if you want to use only one of them?

Yes it seems that when I import dtreeviz I get entries about bad xgboost install

I've checkout your PR and uninstalled xgboost.

If we cannot install xgboost or have problems importing it, we still will have issues.
Even if we have the error in pycharm  (xgboost module cannot be found), we can still render scikit-learn visualizations... (because the way python interpreter works)

If we switch the ShadowXGBDTree initialization block and we want to render scikit-learn visualizations, we will receive the "ModuleNotFoundError: No module named 'xgboost'" exception.
Since get_shadow_tree() is called from any visualization methods.... I don't have an idea right now how to implement the "lazy" import. Any suggestions ?

I believe I did lazy import somewhere. hang on. Yep, i've done this kinda stuff. would this be helpful?
    if 'numpy' not in sys.modules:
        import numpy
    np = sys.modules['numpy']
    if not isinstance(data,np.ndarray):
        return " "

No I think that’s an overkill. What I mean by “Lazy import” Is that you only import when you need it, not that you import if it hasn’t been imported. The latter is taken care of by Python automatically.

@tlapusen I’ll take a closer look at what you wrote later.

Meaning that the user has to do the import? isn't that the way it is now?

hi @yoavram
do you continue with this or should we close this PR ?

@tlapusan I was thinking that we should do this conditional install that somebody fixed for me in https://github.com/parrt/tensor-sensor
That fixes what is required to be installed. Still, inside the code we have to avoid using types that don't exist. We tweaked the code also. See https://github.com/parrt/tensor-sensor/blob/master/tsensor/analysis.py#L459

@yoavram  my understanding is that those imports would slow things down but I'm not sure it matters, depending on how many times you call the function.  I think it's easiest if we asked for the class names and module names to check for types.

@parrt  I'm gonna look on those conditional imports in the next days, thanks.

I'm laughing because I had no idea you could do conditional pip installs...

@parrt I've looked over conditional imports and found also in pip docs about it. Like you said, we still need to check into code if the libraries are installed or not.
I suppose that the default installation (pip install dtreeviz) should install all requirements and the custom pip installations should be very well documented in readme.md page, else the people will be confused.
I started to work on lightgbm implementation and I would like to work on this fix (#99) later. (happy if someone else could do this meanwhile)

@tlapusan lightgbm is definitely more important :)

I think all we need is something like the following in the set up (from tensor-sensor):
    install_requires=['graphviz>=0.14.1','numpy','IPython', 'matplotlib'],
    extras_require = {'all': all_requires,
                      'torch': torch_requires,
                      'tensorflow': tensorflow_requires
                     },

then check for the modules:
if v.shape.__class__.__module__ == "torch": ...

hi @yoavram, please follow this PR #115 if you are still interested in making xgboost or pyspark lazy imported.

hi @yoavram, please follow this PR #115 if you are still interested in making xgboost or pyspark lazy imported.

Sorry I fell off this issue. Hopefully someone can pick it up.

Sorry I fell off this issue. Hopefully someone can pick it up.

@tlapusan  should we close this one in favor of #115?  I'm trying to figure out which is the PR to focus on.

@tlapusan  should we close this one in favor of #115?  I'm trying to figure out which is the PR to focus on.

Closing since I merged #115.

Closing since I merged #115.


[Ruby] Examples are showing incorrect construction of client objects

It looks like the Ruby examples were recently changed from, e.g.:
require "google/cloud/tasks/v2beta2"
cloud_tasks_client = Google::Cloud::Tasks::V2beta2.new
#...

To
require "google/cloud/tasks"
cloud_tasks_client = Google::Cloud::Tasks::V2beta2.new(version: :V2beta2)
#...

This is incorrect. It needs to be:
require "google/cloud/tasks"
cloud_tasks_client = Google::Cloud::Tasks.new(version: :v2beta2)
#...

Note the TWO corrections:

Need to call new on the service module, not the version module, if you want to pass a version argument.
The version argument is misspelled; it should not be capitalized.

See https://github.com/GoogleCloudPlatform/google-cloud-ruby/pull/2186 for an example.

Do you think this could have anything to do with the synth tools you're using? I don't know much about how they work, but I just regenerated this locally and got this in the examples sections:
require "google/cloud/tasks/v2beta2"
cloud_tasks_client = Google::Cloud::Tasks::V2beta2.new

I could change it to use the symbol, and change the case of the symbol, but that doesn't explain what's in that PR where the two ways to create a client are being mixed incorrectly.

The synth tool for that directory isn't doing any postprocessing. I suppose it's possible synth is using a different version of gapic-generator than you're running locally?

Do you know what version you're using? I've tried with a few versions but haven't been able to reproduce it.

As I understand it, it's using the current latest artman docker image googleapis/artman:latest. See https://github.com/GoogleCloudPlatform/synthtool/blob/master/synthtool/gcp/gapic_generator.py

I just tried running synthtool locally (which runs artman from that image) and I'm getting the same weirdly "mixed" results.

Thanks @dazuma I was able to reproduce it. It was something unrelated on my side but seeing that synthtool script was pretty useful anyway.
Turns out this is because of a bad regex and it is happening because it's v2beta2 vs. something like v2, which isn't covered by our tests. I should be able to fix that part fairly quickly.

Tested artman 0.14.1. It looks like the second bullet point is fixed:

The version argument is misspelled; it should not be capitalized.

but not the first:

Need to call new on the service module, not the version module, if you want to pass a version argument.

Ops, I didn't mean to auto close this. @dazuma we should have a new release later today.

Great, looking forward to testing it!

@dazuma is this still an issue?
After my last change I checked locally for the cloud tasks library and it seemed to be generating the names correctly.

It looks good now. I've tested a number of cases today. Closing.


Add GitHub actions to run tests

This adds GitHub actions that run the tests on every push to master and for PRs. It also runs them daily so we see when something breaks because HLTV changed their page. (I'm not sure who gets notified how).
Obviously the benefit is limited since there are only tests for the upcoming matches right now.

Looks good. Not sure who will get notified when test failed either. But we will see.


Moving events lasting less than a minute

Fix issue #2314
When working with events that can have only a few seconds difference between end and start, dragging is visually broken because the end date is set the next day.

+1 it's important

@cutterbl add this correction please, it's very important

Someone will have to update the branch with the latest updates from master, and provide working examples in Storybook for proper testing, before this can be approved.


Add Travis matrix for macOS ChainerX tests

Adds a new Travis matrix: macOS x ChainerX.
Only the native backend will be tested.
Note: Error fixes can be done in separate PRs I think. Specifically #5776 will be required.

#5776 is now merged but we're getting an error trying to uninstall oclint when it's not available. 🤔

Could you take a look at this new Travis failure?

I'll try to reproduce it locally.

Reproduced it. I'll take a look.

Seems like RuntimeError is raised instead of the expected chainerx.ChainerxError 🤔

This is only an issue when building with setuptools (e.g. pip install or python setup.py) so it's probably about some option that's not configured.

This happens both when installing via setuptools and directly with cmake.

Can you fix the conflicts?

The above failure should be fixed by https://github.com/chainer/chainer/pull/5936.


Typescript definition doesn't include types for all the exported members.

E.g.  setAttr.  pointsAtCell etc

Fixed in v1.3.0-beta.1


Trigger setup.php to update settings-frontend.php

Apply values from local.php to settings-frontend.php, which seems to be necessary since Nominatim 3.6
Please see conversation regarding issue #177

Thanks! I think we can remove
https://github.com/mediagis/nominatim-docker/blob/6238cb6e89a78bf765693cf442cefd552011fd33/3.6/Dockerfile#L80
from the Dockerfile in that case, since we do it on every startup with your PR to reflect the changes from the local.php.

@rebos could you please also add that change (removal from Dockerfile) to your PR? I'll merge after that change, thanks!

@all-contributors please add @rebos for infrastructure and code.


Collection cli

:books: Documentation preview :books:: https://mrmoe--171.org.readthedocs.build/en/171/

Codecov Report

Merging #171 (0457945) into master (fbb11d0) will decrease coverage by 0.03%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #171      +/-   ##
==========================================
- Coverage   97.93%   97.90%   -0.04%
==========================================
  Files          35       35
  Lines        1990     1909      -81
==========================================
- Hits         1949     1869      -80
+ Misses         41       40       -1


refactor: some changes

What has changed?

added editorconfig vscode extension to extension recommendations because project contains .editorconfig file
updated discord svg
added hover animations
fixed hover issue with the mail icon in socials component

I may have missed the hover animations in some parts.

Although I like all of these changes, I don't think I'll merge this unless you remove the scale animation from those components. I find it cool but don't really need them on my personal website at the moment, probably never will. Update your fork and remove those changes, start a vote on Change.org or wait for me to close this PR in a couple days if there's no activity :sadge:


CI: builds on master fails if the author isn't registered on buildkite

https://buildkite.com/scionproto/scionproto/builds/3515#0d07e8ac-0a8f-4ef0-93ba-03b3afa8db7b

422 Builds cannot be triggered from this pipeline as there is no Buildkite user linked to this build. Ensure that a user has added "<< redacted<EMAIL_ADDRESS>as a verified email address on their account. (Attempt 1/60 Retrying in 5s)

Created issue with buildkite: https://github.com/buildkite/agent/issues/1050

This happened again with #3164

Issue solved. Buildkite turned off verification of user for all builds.


How to convert duktape varargs to C varargs?

For a format function I'd need a way to convert the values passed on the stack to a va_list so that I can send that to sprintf. I looked at the logger code but that doesn't use a format string for the JS log calls, so it is no help here. There's also duk_push_sprintf() but that seems to be meant for duktape/C calls only (since it doesn't take va_args). Which other options do I have? Is it possible at all?

duk_push_vsprintf() takes a va_list directly.  Not sure if that's what you're looking for?

Ah yes, very good. Still the question remains how to go from the value stack  to a va_list.

I might be wrong but AFAIK there's no portable way to populate a va_list dynamically :-/

Yes, it's really tricky and only possible with (probably non-portable) hacks. Fortunately, the NodeJS format implementation is just a cheap imitation of the C printf API, so it's easy to implement it manually. I've taken NodeJS' JS format function and converted that to duktape/C. Let me know if you want the code. Would be useful for duktape's print and console impl (which currently ignore format strings).

That code would make a nice Howto :)

OK, here we go :-) It's C++ code so you would need massage it a bit to become plain C.
/**
 * A very simple format function similar to printf which does not support any of the flags, width, precision or length
 * parameters that printf can handle.
 * Modeled after https://github.com/nodejs/node/blob/master/lib/util.js#L65.
 */
static duk_ret_t format(duk_context *ctx) {
  // Stack can vary here. There can be no format string or the format string doesn't contain (enough) format specifiers
  // for all parameters. In that case they are simply stringified and appended to the result string.
  // stack: [ optional-format-string <varargs> ] but at least one of the 2 must be present.
  duk_int_t top = duk_get_top(ctx);
  if (top == 0) {
    duk_error(ctx, DUK_ERR_TYPE_ERROR, "format string expected");
    return 0;
  }

  std::string result;

  if (top == 1 || !duk_is_string(ctx, 0)) {
    // No format string or no additional parameters.
    for (duk_int_t i = 0; i < top; ++i) {
      if (!result.empty())
        result += " ";
      result += duk_to_string(ctx, i);
    }
  } else {
    std::string format = duk_get_string(ctx, 0);
    duk_int_t currentArgument = 1;
    duk_int_t lastPosition = 0;

    for (duk_int_t i = 0; i < format.size(); ) {
      if (format[i] == '%' && (i + 1 < format.size())) {
        switch (format[i + 1]) {
          case 'd':
          case 'f':
            if (currentArgument >= top)
              break;
            if (lastPosition < i)
              result += format.substr(lastPosition, i - lastPosition);

            // Make the argument (might be a string, object, null, number etc.) into a number.
            // Then convert that to a string for the result.
            duk_to_number(ctx, currentArgument);
            result += duk_to_string(ctx, currentArgument++);
            lastPosition = i = i + 2;
            continue;

          case 'j':
            if (currentArgument >= top)
              break;
            if (lastPosition < i)
              result += format.substr(lastPosition, i - lastPosition);

            result += duk_json_encode(ctx, currentArgument);
            ++currentArgument;
            lastPosition = i = i + 2;
            continue;

          case 's':
            if (currentArgument >= top)
              break;
            if (lastPosition < i)
              result += format.substr(lastPosition, i - lastPosition);
            result += duk_get_string(ctx, currentArgument++);
            lastPosition = i = i + 2;
            continue;

          case '%':
            if (lastPosition < i)
              result += format.substr(lastPosition, i - lastPosition);
            result += '%';
            lastPosition = i = i + 2;
            continue;
        }
      }
      ++i;
    }

    if (lastPosition == 0)
      result = format;
    else if (lastPosition < format.size())
      result += format.substr(lastPosition);

    while (currentArgument < top) {
      result += std::string(" ") + duk_to_string(ctx, currentArgument++);
    }
  }

  duk_push_string(ctx, result.c_str());

  return 1;
}


Key Operations on CDN

Issue on page:        https://github.com/opentelekomcloud-docs/cloud-trace-service/blob/main/umn/source/supported_services_and_operation_lists/network/key_operations_on_cdn.rst
CDN Service is not available on OTC and not planned. Please remove

This page was removed with the latest update https://github.com/opentelekomcloud-docs/cloud-trace-service/pull/35.
You can find the list of Supported Services and Operations on page https://docs.otc.t-systems.com/cloud-trace-service/umn/user_guide/supported_services_and_operations.html


Add length limit for the generated names used in tests

ObjectNameForTest is used to generate names for objects (like service) used in tests. It calls kmeta.ChildName which allows max name length is 63 (supported by k8s). However some non-k8s implementation has a smaller limit than k8s.
This PR proposes to limit the length of generated name to 50, which is sufficient in knative tests.

/assign @chizhg

To hook into this, since we have a similar problem down at Openshift: What exactly is this fixing? Is the limitation of your system (I assume Cloud Run?) imposed on the names themselves or are you using a different domainTemplate (i.e. $name-$ns) and this fixes that?

To hook into this, since we have a similar problem down at Openshift: What exactly is this fixing? Is the limitation of your system (I assume Cloud Run?) imposed on the names themselves or are you using a different domainTemplate (i.e. $name-$ns) and this fixes that?

The max length of Cloud Run supports is less than 63.

/retest

@markusthoemmes @julz ping

/lgtm
/approve

/retest


ImportString fails to generate JsonSchema - in fastapi openapi docs generation

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
We use pydantic with FastApi and have a model which contains an ImportString field associated with an endpoint and found that the openapi docs generation failed due to ImportString. Underlying issue is that pydantic could not create JsonSchema for the object, the error looks like:
 raise PydanticInvalidForJsonSchema(f'Cannot generate a JsonSchema for {error_info}')
pydantic.errors.PydanticInvalidForJsonSchema: Cannot generate a JsonSchema for core_schema.PlainValidatorFunctionSchema ({'type': 'no-info', 'function': <function import_string at 0x105a2c360>})

A workaround was to create a type-annotated version:
ImportString = Annotated[
    pydantic.ImportString, pydantic.WithJsonSchema({'type': 'string', 'description': 'fully qualified class name'})
]

I'm not sure if this is intended, but the docs don't seem to explicitly mention the json schema serialization.
Example Code
# Issue was in FastApi docs generation, but the below produces an error that
# looks the same
import pydantic

pydantic.TypeAdapter(pydantic.ImportString).json_schema()

Python, Pydantic & OS Version
python -c "import pydantic.version; print(pydantic.version.version_info())"
             pydantic version: 2.7.1
        pydantic-core version: 2.18.2
          pydantic-core build: profile=release pgo=true
                 install path: /Users/arik/.miniconda3/lib/python3.11/site-packages/pydantic
               python version: 3.11.5 (main, Sep 11 2023, 08:31:25) [Clang 14.0.6 ]
                     platform: macOS-13.5-arm64-arm-64bit
             related packages: typing_extensions-4.9.0 fastapi-0.108.0
                       commit: unknown

@amitschang,
Ah, nice catch. We'll want to add json schema support to the ImportString type. PR welcome!

@sydney-runkle , I made a PR - I can't seem to add the necessary labels to it (ready for review and relnotes-fix), but it is there


about training device

Hello, thank you very much for your work, I would like to ask about the equipment you use when training your code. And why is the fc_weight file given in your paper only 4kb, but the model file I trained is 1~2GB?

They may uploaded a git-lfs pointer file instead of a full pretrain model...
And on Tesla P40 I need 6~7s for a single iteration that caused about ~20 min to see the first loss output.
Obviously if you don't have a a100*8 device nor lot of time, train it from the scratch is a impossible misson.
Also, in default hyper parameters it takes non-less than 13.5G VRAM to train.

They may uploaded a git-lfs pointer file instead of a full pretrain model... And on Tesla P40 I need 6~7s for a single iteration that caused about ~20 min to see the first loss output. Also, in default hyper parameters it takes non-less than 13.5G VRAM to train on GenImage SD-V1.4 dataset with bs256. For 1 epoch, ~1.5hrs needed.

UPD: but validation result are not so good. idk if i get anything wrong in training process? i peformed training by adress fake/real_list_path manually in command args and just let it run and save the early stop checkpoints. and it just give me very low ap score on genimage valset.

I would like to ask, why does it take two hours to run one epoch? Is this normal? How did you perform training by addressing fake/real_list_path manually in the command args?

They may uploaded a git-lfs pointer file instead of a full pretrain model... And on Tesla P40 I need 6~7s for a single iteration that caused about ~20 min to see the first loss output. Also, in default hyper parameters it takes non-less than 13.5G VRAM to train on GenImage SD-V1.4 dataset with bs256. For 1 epoch, ~1.5hrs needed.

UPD: but validation result are not so good. idk if i get anything wrong in training process? i peformed training by adress fake/real_list_path manually in command args and just let it run and save the early stop checkpoints. and it just give me very low ap score on genimage valset.
I would like to ask, why does it take two hours to run one epoch? Is this normal?

IDK. But it may be normal because no runtime exception was thrown in training process.

How did you perform training by addressing fake/real_list_path manually in the command args?

You should modify data/datasets.py, add a data mode like 'manually'.
elif opt.data_mode == 'manually':
    real_list = get_list( os.path.join(opt.real_list_path) )
    fake_list = get_list( os.path.join(opt.fake_list_path) )

Because there is no .pickle in this path so it just triggered recursive searching in your image dataset path.
Address your real and fake image path and it works soon.

They may uploaded a git-lfs pointer file instead of a full pretrain model... And on Tesla P40 I need 6~7s for a single iteration that caused about ~20 min to see the first loss output. Also, in default hyper parameters it takes non-less than 13.5G VRAM to train on GenImage SD-V1.4 dataset with bs256. For 1 epoch, ~1.5hrs needed.

UPD: but validation result are not so good. idk if i get anything wrong in training process? i peformed training by adress fake/real_list_path manually in command args and just let it run and save the early stop checkpoints. and it just give me very low ap score on genimage valset.

I would like to ask, why does it take two hours to run one epoch? Is this normal?

IDK. But it may be normal because no runtime exception was thrown in training process.

How did you perform training by addressing fake/real_list_path manually in the command args?

You should modify data/datasets.py, add a data mode like 'manually'.
elif opt.data_mode == 'manually':
    real_list = get_list( os.path.join(opt.real_list_path) )
    fake_list = get_list( os.path.join(opt.fake_list_path) )

Because there is no .pickle in this path so it just triggered recursive searching in your image dataset path.
Address your real and fake image path and it works soon.

Thank you very much for your answer. When you validate with your trained .pth file, do you encounter this error: RuntimeError: Error(s) in loading state_dict for Linear: Missing key(s) in state_dict: 'weight', 'bias'. Unexpected key(s) in state_dict: 'model', 'optimizer', 'total_steps'?


[Feature request] Endpoint to get the current configured services

In the same way we have the "remove" or "reconfigure" endpoints it would be very useful to have another endpoint to see which is the current configuration. For example:
curl "$(docker-machine ip node-1):8080/v1/docker-flow-proxy/get-configuration

Which could return a JSON encoded data with the current endpoints and some simple info about each configured backend service.
Another option could be to offer an alternative endpoint which could return the HAProxy gui data directly.

At the moment, there is an option to retrieve HAProxy config (https://github.com/vfarcic/docker-flow-proxy#config.
Right now, I'm trying to keep the HAProxy stateless (besides mandatory HAProxy config) so I don't store information from reconfigure and remove requests. That can be changed and I can store it in memory which would be retrievable as JSON.
Initially, I thought that the actual proxy config is more valuable since it provides an accurate image of how the proxy works.
Please confirm that the config is not enough and I'll add the option to the the JSON representation of all configured services.

The output format is hardly parseable, then it's difficult to create a decent UI over it.
But it's enough to see what's configured, without the need of do manually the "cat" of the config file inside the container.