board: refresh boards after update LIBCANUTILS dependency

Summary
board: refresh boards after update LIBCANUTILS dependency
refresh boards after update LIBCANUTILS dependency
depends on: https://github.com/apache/nuttx-apps/pull/2786
Impact
refresh boards
Testing
CI

[Experimental Bot, please feedback here]
No, this PR does not meet the NuttX requirements.
Missing Information:

Summary:

Lacks a clear explanation of why the change is necessary. What issue does updating the LIBCANUTILS dependency address?
Needs a more detailed description of the functional code changes.

Impact:

While it mentions "refresh boards," it doesn't elaborate on what this means or how it affects users, builds, hardware, documentation, etc.

Testing:

Provides no details about the testing environment (build host, target architecture, board configuration).
Lacks testing logs both before and after the change.

Recommendations:

Expand the summary to clearly articulate the problem and solution.
Provide specific details about the impact on all relevant aspects (users, builds, hardware, documentation, security, compatibility).
Include comprehensive testing information, including the testing environment and logs demonstrating the change's effect.


Evaluation Metric

Hi, this is an interesting work! Could you show more details and the code about your evaluation metrics? especially FDS and DS. Thanks.

Sorry I could not find the evaluation script anymore. But here are my implementations in MATLAB:
FDS: First, calculate TSNE features (let's say 2 features, you can select the number as your preference) from the synthetic image and true images (let's say we have 100 of each of them.
"https://www.mathworks.com/help/stats/tsne.html"
Now we have two matrices, each of them are 2x100. Then we calculate the KL between these two matrices:
"https://www.mathworks.com/matlabcentral/fileexchange/20688-kullback-leibler-divergence".
DS: Use for loops to calculate the ssim for every synthetic image and all other synthetic images. For example, we have 3 synthetic images, the ssim matrix is 2x3. Each synthetic image has the neighbor with the highest ssim (so this pair is the most similar pair). Then we compute an average of all the highest ssim. The idea is that if even the most similar pair has low ssim, that mean the synthetic images are different.


junkins_plugin ContentLengthMismatch for remote url

I have plugins uploaded to S3 and I'm running into this quite frequently. Not sure if it's an issue with S3 or what, I was able to curl the URL and download it on my machine, and the sha256 matches the official one. Other remote_file uses in this chef run that hit this same bucket don't have any issue. This is running on a local VirtualBox VM via Vagrant. Any ideas?
==> default: Resource Declaration:
==> default: ---------------------
==> default: # In /tmp/vagrant-chef-3/chef-solo-1/cookbooks/mycorp-jenkins/recipes/jenkins_plugins.rb
==> default:
==> default:  15:       jenkins_plugin plugin["name"] do
==> default:  16:               source "#{node['mycorp-jenkins']['plugin_source_url']}/#{plugin['name']}/#{plugin['version']}/#{plugin['name']}.hpi"
==> default:  17:               version plugin["version"]
==> default:
==> default:  18:               install_deps false
==> default:  19:       end
==> default:  20: end
==> default:
==> default: Compiled Resource:
==> default: ------------------
==> default: # Declared in /tmp/vagrant-chef-3/chef-solo-1/cookbooks/mycorp-jenkins/recipes/jenkins_plugins.rb:15:in `block in from_file'
==> default:
==> default:
==> default: jenkins_plugin("git-client") do
==> default:   action :install
==> default:   retries 0
==> default:   retry_delay 2
==> default:   guard_interpreter :default
==> default:   cookbook_name :"mycorp-jenkins"
==> default:   recipe_name "jenkins_plugins"
==> default:   source "https://s3-us-west-1.amazonaws.com/my-bucket/jenkins-plugins/git-client/1.12.0/git-client.hpi"
==> default:   version "1.12.0"
==> default: end
==> default:
==> default:
==> default:
==> default: [2015-01-27T22:31:00+00:00] INFO: Running queued delayed notifications before re-raising exception
==> default: [2015-01-27T22:31:00+00:00] DEBUG: Re-raising exception: Chef::Exceptions::ContentLengthMismatch - jenkins_plugin[git-client] (mycorp-jenkins:
:jenkins_plugins line 15) had an error: Chef::Exceptions::ContentLengthMismatch: remote_file[/var/chef/cache/jenkins-cli.jar] (dynamically defined) had an error
: Chef::Exceptions::ContentLengthMismatch: Response body length 179864 does not match HTTP Content-Length header 751490.
==> default: /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.16.2/lib/chef/http/validate_content_length.rb:102:in `validate'
==> default:   /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.16.2/lib/chef/http/validate_content_length.rb:60:in `handle_stream_complete'
==> default:   /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.16.2/lib/chef/http.rb:236:in `block in apply_stream_complete_middleware'
==> default:   /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.16.2/lib/chef/http.rb:234:in `each'
==> default:   /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.16.2/lib/chef/http.rb:234:in `inject'
==> default:   /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.16.2/lib/chef/http.rb:234:in `apply_stream_complete_middleware'
==> default:   /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.16.2/lib/chef/http.rb:174:in `block in streaming_request'
==> default:   /opt/chef/embedded/lib/ruby/gems/1.9.1/gems/chef-11.16.2/lib/chef/http/basic_client.rb:74:in `block in request'

I'm getting the same error with http

Same error with the official http://updates.jenkins-ci.org/download/plugins

I'm able to reproduce the issue outside of my network on an EC2 instance

Seems to download just fine when using Chef::HTTP::Simple manually:
uri = URI('https://s3-us-west-1.amazonaws.com/my-bucket/jenkins-plugins/git-client/1.12.0/git-client.hpi')
http = Chef::HTTP::Simple.new(uri.host)
tmp_file = http.streaming_request(uri.to_s)

I am not able to reproduce this with your version of Chef.
[ec2-user@ip-172-31-22-112 ~]$ chef-apply -s
remote_file '/tmp/whee' do
  source 'https://updates.jenkins-ci.org/download/plugins/git-client/1.12.0/git-client.hpi'
end
^D
Recipe: (chef-apply cookbook)::(chef-apply recipe)
  * remote_file[/tmp/whee] action create
    - create new file /tmp/whee
    - update content in file /tmp/whee from none to f585fa
    (new content is binary, diff output suppressed)
[ec2-user@ip-172-31-22-112 ~]$ chef-apply --version
Chef: 11.16.2

A gist of the full output of chef-client -l debug showing all the debug header information and everything else associated with the resource might help, along with all the logging that all the Chef::HTTP middleware does.  Getting tcpdump logs of the stream might show that a RST is coming back from s3 or something like that indicating a bad TCP teardown.  Its possible that the server is gzip compressing the file and it has a smaller size in the payload but the server failed to indicate gzip compression (but since the server is s3 i'd think we'd have a plague of bug reports--which we don't have).  I tried replicating this last time and similarly failed to replicate the issue.

Whoops, I wasn't going far enough back in the exception. It looks like the download of git-client.hpi is actually working fine. The trouble seems to be when it grabs https://jenkins.mycorp.com//jnlpJars/jenkins-cli.jar. Looks like maybe because it's being gzipped but the content-type and content-length headers are not reflecting that?
Here's a gist of the debug output: https://gist.github.com/fletchowns/e6ae51aa20eaddc775ee

s3 isn't sending back gzip headers and we're doing the noop gzip inflator which means its not gzipped.  and the number of bytes downloaded this time is different - 605848 vs 179864 - which is not consistent with gzipped content length vs ungzipped content length.
about all i can think of is to take out any sysctl settings that you have around tcp/ip tuning and to turn off all the possible tcp offload settings...
ethtool --offload  eth0  rx off  tx off sg off tso off ufo off gso off gro off lro off rxvlan off txvlan off rxhash off
(untested command, no idea if all those options are supported on your box)

Thanks Lamont! Couldn't get that command to work (and don't know much about ethtool) but now that I know the actual file download it's failing on I was able to reproduce it with curl.
Downloading jenkins-cli.jar through nginx reverse proxy:
[vagrant@jenkins ~]$ curl -versbose https://jenkins.mycorp.com//jnlpJars/jenkins-cli.jar -o jenkins-cli.jar
* About to connect() to jenkins.mycorp.com port 443 (#0)
*   Trying <IP_ADDRESS>... connected
* Connected to jenkins.mycorp.com (<IP_ADDRESS>) port 443 (#0)
* Initializing NSS with certpath: sql:/etc/pki/nssdb
*   CAfile: /etc/pki/tls/certs/ca-bundle.crt
  CApath: none
* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA
> GET //jnlpJars/jenkins-cli.jar HTTP/1.1
> User-Agent: curl/7.19.7 (x86_64-redhat-linux-gnu) libcurl/7.19.7 NSS/<IP_ADDRESS> Basic ECC zlib/1.2.3 libidn/1.18 libssh2/1.4.2
> Host: jenkins.mycorp.com
> Accept: */*
> Referer: rsbose
>
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0< HTTP/1.1 200 OK
< Server: nginx/1.6.2
< Date: Wed, 28 Jan 2015 18:36:26 GMT
< Content-Type: application/java-archive
< Content-Length: 751490
< Connection: keep-alive
< Content-Disposition: attachment; filename=jenkins-cli.jar
< Last-Modified: Wed, 28 Jan 2015 03:52:35 GMT
< Expires: Wed, 28 Jan 2015 03:52:35 GMT
< Accept-Ranges: bytes
< X-Content-Type-Options: nosniff
<
{ [data not shown]
* transfer closed with 276719 bytes remaining to read
 63  733k   63  463k    0     0  4278k      0 --:--:-- --:--:-- --:--:-- 16.7M* Closing connection #0

curl: (18) transfer closed with 276719 bytes remaining to read

And the nginx error log shows this:
2015/01/28 18:42:02 [crit] 1005#0: *21 open() "/opt/nginx-1.6.2/proxy_temp/0/01/0000000010" failed (13: Permission denied) while reading upstream, client: <IP_ADDRESS>, server: jenkins, request: "GET //jnlpJars/jenkins-cli.jar HTTP/1.1", upstream: "http://<IP_ADDRESS>:8080//jnlpJars/jenkins-cli.jar", host: "jenkins.mycorp.com", referrer: "rsbose"

The permissions to /opt/nginx-1.6.2/proxy_temp looked ok but it turned out that an unrelated script was mucking with the permissions of the parent directory.
Thanks so much for your time, really appreciate it. I'll go ahead and close this one out.


Support async execute in SqlAlchemy API

Looking at polling in pyhive https://github.com/dropbox/PyHive#db-api-asynchronous, seems like a clean way would be to submit execute() with async_=True and use poll().operationState.

What is the actual problem and i.e. what thing we want?
And we need to integrate async_=Ture and poll().operationState here https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py#L232 ?

What is the actual problem and i.e. what thing we want?
And we need to integrate async_=Ture and poll().operationState here https://github.com/cloudera/hue/blob/master/desktop/libs/notebook/src/notebook/connectors/sql_alchemy.py#L232 ?

It would be simpler to start first with https://github.com/cloudera/hue/issues/1682
Yes, we want to add an is_async property to sqlalchemy dialects, so that we can chose to execute statements in non blocking mode.

It would be simpler to start first with https://github.com/cloudera/hue/issues/1682
Yes, we want to add an is_async property to sqlalchemy dialects, so that we can chose to execute statements in non blocking mode.


Updated new unit tests to work on Travis

Only the most recent commit is relevant here.

Thanks for fast fix @mpenkov :+1:

Fix for #212


SSO User Registration Cache Issue

Hi @ShellyXueHan:
@matthieu-foucault eloquently described the issue we're encountering as follows:
The issue was reproduced (sometimes) by

clicking on our registration button, which is a link to https://sso.pathfinder.gov.bc.ca/auth/realms/pisrwwhx/protocol/openid-connect/registrations?client_id=cas-ciip-portal&response_type=code&scope=openid&redirect_uri=https%3A%2F%2Ftest.ciip.gov.bc.ca%2Flogin%3Fauth_callback%3D1
complete the form to create a new user
click on register and get redirected to the /login?auth_callback=1 route in our application
the keycloak express middleware kick in and calls the accessDenied method, resulting in a 403 HTTP response

At this point, the user can go back to the home page of the application, and log in with the credentials they entered in 2). We implemented a custom accessDenied response to indicate to them that they can just log in if that issue occurs, working around the issue. This is however a less-than-ideal path to registration.
This behavior could not be reproduced consistently between users or between attempts: I managed to trigger the issue twice in a row from an application running in localhost, using the sso-dev keycloak instance, and then it did not do it again. Some of our users however have encountered the issue consistently (we tried deleting the user in the realm and make them register again, and the issue was reproducible for them).
The issue does not seem to occur when we clicking our "Login" button (which calls the keycloak.connect() middleware), and then clicking on the "Register" button from there, so this may have to do with routing the user directly to the registration page.
Note that we tried to add the state query string parameter in the "Register" redirection URL, as it is missing when we compare the URL in step (1) to the one that keycloak.connect() creates (the value for state was generated by the UUID() function from the keycloak nodejs adapter). However this did not solve the issue.

Network requests reproducing issue available as har file.

Update from RedHat:
There is no specific configuration to replicate user registration cache, we will need to wait for the next release to have this added, which is still a date-to-be-determined. On the other side, as Alec has found a message from RedHat sometime ago about using the registration endpoint. This does not apply to this as we are running a stateless KeyCloak instance on Openshift, no sticky session is available to support. As a result, for now you will have to either go with the normal auth flow or use the custom accessDenied response you mentioned. And it's either that the next release has this available and we upgrade to include ot, or that this application has moved on to use an identity provider instead of keycloak local users as expected.

Waiting for SSO to include as a feature.
Plan is to move away from local users. IDP Users

@brycereid73 here's the RedHat support ticket: https://access.redhat.com/support/cases/#/case/02713060 The team has follow up with some more options, please try out and provide feedback!

@arcshiftsolutions  @junminahn is it still a thing we should add in our backlog?

@zsamji We haven't had any other teams ask about local users that I know. As noted our focus has been providing IDP access with our new shared realm approach. I think we can close this item and re-open if we have further requests.


Scroll bar not efficient

I have checked the following:

[X] I use the newest version of bruno.
[X] I've searched existing issues and found nothing related to my issue.

Describe the bug
Hi
Scroll bar is not very well functioning if we had many collection in the bar. We are struggling scrolling or selecting services if it goes next page.
.bru file to reproduce the bug
No response
Screenshots/Live demo link
None

Hey, if your struggling with performance and don't need all the Pro features, you can checkout my fork: https://github.com/Its-treason/bruno/releases/tag/nightly It includes a lot of performance optimazations.
Here's a quick demo. The differences can easly be tracked via the CPU usage bar in the bottom.
https://github.com/user-attachments/assets/9e60f56e-566f-4a05-ad81-b75031640d2d

Hey @ykburda,
Thanks for reporting the issue. We're aware of it and are working on it.
Best,
Ganesh

Hey @ykburda,
if possible could you share any video/screenshot.
Looking forward to your response.


No email dispatch from Froxlor possible

The image is missing the exim4 package to send out emails.

Currently no clue how to fix this ^^"

Currently no clue how to fix this ^^"


It isn't good that appending the default Content-Type in the http header

Now, all the POST/PUT/? method append the default Content-Type(Content-Type: application/x-www-form-urlencoded) in the header field. And I don't find the setting to replace this default value. It's realy not good to me because we use the "Content-Type: application/json".
I have to replace the content-type by hand when I change the api.
Where can I set the default content-type?

There's no such setting. It is the same behaviour how it always was in the app. But earlier it was covered by encoding drop-down.
Adding the header into the list is a handy feature. 1) it notice developer that there is some change and he should react if required 2) the developer don't need to do it manually.
There are a lot of different mime types. For you most common mime type for the request is application/json, for others it will be something else. I used the most basic and generic web content type for data sending and you can always change it.
I also decided not to complicate application more so I'm not planning adding another setting. It may change in the future but current status will be wontfix.


Bluetooth: Mesh: Add brg_cfg_cli commands to shell

Adds the Bridge Configuration Client commands to Mesh shell.

Hi @carlescufi, this is ready for merge.


Consider renaming/rescoping the project to also deal with upcomming 1.0.0 ?

I think this tool is wonderfull but I would like to use it not just for RC1-RC2 but also for RC1/2-1.0.0. The new version 1.0.0 is currently scheduled for 27th of June.
I just looked at the prelimary list of changes 1.0.0 which currently numbers about 15 breaking changes in addition to new version numbers. I therefore think this tool could be quite useful for changing to 1.0.0 also ?

Thanks - I agree. All of the upgrade logic so far is basically contained within a set of IProjectUpgradeAction, ICsharpCodeUpgradeAction and ISolutionUpgradeAction's. Together, the set of those is the RC1 - RC2 upgrade. I;d most likely extend the tool to only Apply this set of actions if the current global.json was RC1. I'd then define the new set for the RC2 - 1.0.0 upgrade, and get the tool to run that set after the RC1 --> RC2 upgrade completes.

I will revisit this when 1.0.0 is released, and most likely rename this repository after that support is added ;)

The word on the street is that VS will auto upgrade from RC2 to RTM.

@RehanSaeed perfect. If only they could have done that for RC1 to RC2 :(

Will close this for now, on the assumption that Microsoft are indeed taking care of this

@RehanSaeed Where have you heard this? I also heard that there would be no breaking changes but now I see there are plenty.

Pretty sure it was mentioned in the community standup video.

Looks like its not exactly seemless.
At a minimum I could extend this tool to:

Upgrade the versions of packages used from RC2 to 1.0.0
Upgrade global.json from preview1 to preview2
Some of this stuff for those using EF: https://docs.efproject.net/en/latest/miscellaneous/rc2-rtm-upgrade.html

Very, very nice!


DC Demographic Vaccine Scraper (Race)

source: https://coronavirus.dc.gov/data/vaccination
They surface demographic data for sex/age/ethnicity, but I couldn't find it in the returned JSON. I'll look more tomorrow/Monday (3/21/21).
For now, I wanted to open this just to have something up

Thanks @smcclure17
This was a scraper for sex, so I made that change. Looking forward to more!

hmm, the Demographics  table was (i'm pretty sure) returning data for race before. I also think that the dashboard's default view was data by race, instead of by sex as it is now, which might have something to do with it.
I can keep an eye on it, and keep looking for the other data


CordovaPayTM plugin call but not redirect paytm page

Hi ,
I am trying to implement paytm integration in my hybrid app. I have set the required parameters and yet its not calling the server.
While installing the plugin have given the required variables; checksum generate url , checksum verify url, webiste, mid , industry type id and on button click am calling window.plugins.paytm.startPayment(orderId, custid, 'email', mobile, amount, successCallback, failureCallback) function with proper params.
on calling, but not get any message or any page.
This is for android platform. I have tried putting window.plugin code in device ready function and it is of no help.
How can I set the call for staging server and production server. ?
Can you please help in this.

same problem give any suggestion Please ?

any update on this one. trying to use this plugin with ionic 3 .. always getting 'network not available'

@ranjitseth123 I am not use currently this plugin.
I have build own login and use InAppBrowser Plugin to open paytm page.

Oh. thats fine. thank you. but is there a straight forward way to use any
plugin for ionic/cordova .. i am new to mobile apps.
On Tue, Jun 5, 2018 at 12:49 PM, sagargheewala<EMAIL_ADDRESS>wrote:

@ranjitseth123 https://github.com/ranjitseth123 I am not use currently
this plugin.
I have build own login and use InAppBrowser Plugin to open paytm page.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/iamisathya/CordovaPayTM/issues/7#issuecomment-394607245,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AmF5KqQ6I4cmrJujH6mOXOgJF96Ugk6wks5t5jDtgaJpZM4OY7P3
.

@ranjitseth123  Sorry, no idea.

@ranjitseth123 I am not use currently this plugin.
I have build own login and use InAppBrowser Plugin to open paytm page.

Hi , Could you please share the code to show how have u used InAppBrowser. I am unable to redirect to patym page using InAppBrowser. Stuck at it from a long time. Any help would be great use.


Enhancing the About Page

Category of documentation update
Fixing the background and format of the page.
Describe the change you think might work
The about page of the website looks unattractive. Instead of containing all the the information at one place , I propose to make separate boxes for information about each thing of the website and would also like to change the background color, font color and font style.
Kindly assign me this as I would like to work on it and I request you to add gssoc and level tags to this issues so that my prs get recognized under gssoc'24.
Checklist

[X] I have checked the existing issues.
[X] I follow Contributing Guidelines of this project.
[ ] I am a IWOC'24 contributor.
[ ] I am a JWOC'24 contributor.
[ ] I am a SWOC'24 contributor.

Code of Conduct

[X] I agree to follow this project's "Code of Conduct".

Hi. Under GSSOC'24, I would like to work on this issue.
Please Assign me this task.

I am thrilled to be participating in GirlScript Summer of Code 2k24 as a passionate computer science student with a keen interest in web development. I am eager to contribute to your project and leverage my skills to make meaningful contributions. I look forward to collaborating with you and the team to create impactful solutions.
@Rakesh9100  Sir, I would really like to work on this issue under Gssoc'24, so please assign it to me.

Hey, can you please assign me this issue if it is not solved? This is my first time making an open-source contribution, and I really want to showcase my skills. I am very familiar with languages like Java, HTML, CSS, and JS. Apart from coding, I'm an ux/ui freelancer as well.

Pls provide any supporting screenshot @arpitarout01


⚠ SceneTime has degraded performance

In 96ccc24, SceneTime ($ST) experienced degraded performance:

HTTP code: 200
Response time: 1179 ms

Resolved: SceneTime performance has improved in b85c4f4 after 6 minutes.


ERROR: Could not send video packet: -1094995529

[x] I have read the FAQ.
[x] I have searched in existing issues.

Environment

OS: Lubuntu
scrcpy version: 1.12
installation method: snap
device model:
Android version: 5.1.1

INFO: scrcpy 1.12 <https://github.com/Genymobile/scrcpy>
/usr/local/share/scrcpy/scrcpy-server:...shed. 3.6 MB/s (26196 bytes in 0.007s)
error: closed
ERROR: "adb reverse" returned with value 1
WARN: 'adb reverse' failed, fallback to 'adb forward'
INFO: Initial texture: 720x1280
ERROR: Could not send video packet: -1094995529
ERROR: Could not process frame
WARN: Device disconnected

How do I fix this?

Android version: 5.1.1

Hmm, are you sure? The error suggests the device runs Android 4.

uhh
yes it is android 5.1.1

Oh, is scrcpy connected to your device over wifi?
If this is the case, could you test over USB please?

no it is over usb

so uhhh
my phone lastest version is only 4.3.3 so i installed a custom rom
could this be the reason?
would it work if i switch to other rom, i meant stock rom only 4.3.3 so it wont work
and would it work if i switched to other custom rom?

What is the result of:
adb reverse localabstract:scrcpy tcp:27100

?

i flashed another rom and it is working now
it might be because i dint wipe the system before flashing rom
Thanks for your help.

Sorry for bumping closed issued, because it happen to me.
┌─[sumar][x240][~]
└─▪ scrcpy
INFO: scrcpy 1.16 <https://github.com/Genymobile/scrcpy>
/usr/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 116.5 MB/s (33622 bytes in 0.000s)
[server] INFO: Device: Xiaomi Redmi 5 Plus (Android 10)
INFO: Renderer: opengl
INFO: OpenGL version: 3.0 Mesa 20.2.3
INFO: Trilinear filtering enabled
INFO: Initial texture: 1080x2160
ERROR: Could not send video packet: -1094995529
ERROR: Could not process frame
WARN: Device disconnected

ADB over USB

ERROR: Could not send video packet: -1094995529

$ printf '%x'<PHONE_NUMBER> | xxd -r -p | rev
INDA

The error is AVERROR_INVALIDDATA. It seems that device encoder does not produce valid data :/
What if you just record and play the file with VLC?
scrcpy --no-display --record file.mkv

Also try with another encoder (take this version: https://github.com/Genymobile/scrcpy/pull/1827#issuecomment-725933512).

What if you just record and play the file with VLC?

┌─[sumar][x240][~]
└─▪ scrcpy --no-display --record file.mkv
INFO: scrcpy 1.16 <https://github.com/Genymobile/scrcpy>
/usr/share/scrcpy/scrcpy-server: 1 file pushed, 0 skipped. 149.0 MB/s (33622 bytes in 0.000s)
[server] INFO: Device: Xiaomi Redmi 5 Plus (Android 10)
INFO: Recording started to matroska file: file.mkv
ERROR: The first packet is not a config packet
ERROR: Could not record packet
ERROR: Could not send packet to recorder
ERROR: Could not process frame
INFO: Finishing recording...
ERROR: Recording failed to file.mkv
WARN: Device disconnected

Also try with another encoder (take this version: #1827 (comment)).

┌─[sumar][x240][~]
└─▪ scrcpy --encoder _
scrcpy: unrecognized option '--encoder'
┌─[sumar][x240][~]
└─▪ scrcpy --version
scrcpy 1.16

dependencies:
 - SDL 2.0.12
 - libavcodec 58.91.100
 - libavformat 58.45.100
 - libavutil 56.51.100

unrecognized option '--encoder'

You need to download the version from the comment.

┌─[sumar][x240][~]
└─▪ SCRCPY_SERVER_PATH=/home/sumar/Downloads/scrcpy-server scrcpy --encoder _
scrcpy: unrecognized option '--encoder'
┌─[sumar][x240][~]
└─▪

I continue to get the ERROR: Could not send video packet: -1094995529 error. I already read the crashes section:
ERROR: Could not send video packet: -1094995529
ERROR: Could not send config packet to sink 0
ERROR: Could not process packet
WARN: Device disconnected

Installation details shown by scrcpy -v:
scrcpy 1.24 <https://github.com/Genymobile/scrcpy>

Dependencies (compiled / linked):
 - SDL: 2.0.22 / 2.0.22
 - libavcodec: 59.18.100 / 59.18.100
 - libavformat: 59.16.100 / 59.16.100
 - libavutil: 57.17.100 / 57.17.100
 - libusb: - / 1.0.26

The device encoder does not produce a valid stream. Try with another encoder: https://github.com/Genymobile/scrcpy#encoder

The device encoder does not produce a valid stream. Try with another encoder: https://github.com/Genymobile/scrcpy#encoder

Running scrcpy --encoder _ or scrcpy --encoder OMX.qcom.video.encoder.avc outputs this:
[server] ERROR: Exception on thread Thread[main,5,main]
com.genymobile.scrcpy.InvalidEncoderException: There is no encoder having name '_"
	at com.genymobile.scrcpy.ScreenEncoder.createCodec(ScreenEncoder.java:222)
	at com.genymobile.scrcpy.ScreenEncoder.internalStreamScreen(ScreenEncoder.java:83)
	at com.genymobile.scrcpy.ScreenEncoder.streamScreen(ScreenEncoder.java:74)2022-07-31 23:01:40.300 scrcpy[3161:4473247] WARN: Device disconnected

	at com.genymobile.scrcpy.Server.scrcpy(Server.java:101)
	at com.genymobile.scrcpy.Server.main(Server.java:335)
	at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
	at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:340)
[server] ERROR: Try to use one of the available encoders:
[server] ERROR:     scrcpy --encoder 'OMX.amlogic.video.encoder.avc'

However, even running the suggested one (scrcpy --encoder 'OMX.amlogic.video.encoder.avc') brings to tha same crashes.

how fix this issue

The device encoder does not produce a valid stream. Try with another encoder: https://github.com/Genymobile/scrcpy#encoder

Running scrcpy --encoder _ or scrcpy --encoder OMX.qcom.video.encoder.avc outputs this:
[server] ERROR: Exception on thread Thread[main,5,main]
com.genymobile.scrcpy.InvalidEncoderException: There is no encoder having name '_"
	at com.genymobile.scrcpy.ScreenEncoder.createCodec(ScreenEncoder.java:222)
	at com.genymobile.scrcpy.ScreenEncoder.internalStreamScreen(ScreenEncoder.java:83)
	at com.genymobile.scrcpy.ScreenEncoder.streamScreen(ScreenEncoder.java:74)2022-07-31 23:01:40.300 scrcpy[3161:4473247] WARN: Device disconnected

	at com.genymobile.scrcpy.Server.scrcpy(Server.java:101)
	at com.genymobile.scrcpy.Server.main(Server.java:335)
	at com.android.internal.os.RuntimeInit.nativeFinishInit(Native Method)
	at com.android.internal.os.RuntimeInit.main(RuntimeInit.java:340)
[server] ERROR: Try to use one of the available encoders:
[server] ERROR:     scrcpy --encoder 'OMX.amlogic.video.encoder.avc'

However, even running the suggested one (scrcpy --encoder 'OMX.amlogic.video.encoder.avc') brings to same crashes.

For me, running following cmd worked:
scrcpy --encoder 'OMX.MTK.VIDEO.ENCODER.AVC'
scrcpy --encoder 'OMX.google.h264.encoder'


Adding the python bitinfo to numcodecs

I've been looking into modifying the numcodecs bitround codec to accept a user defined function to determine the number of bits to round. namely xbitinfo.bitinformation. This would streamline the process of chunk-wise bitrounding to something like
from numcodecs import Blosc, BitRound
compressor = Blosc(cname="zstd", clevel=3)

filters = [BitRound(custom_function())]

encoding = {"precip": {"compressor": compressor, "filters": filters}}
ds.to_zarr(<file_name>, encoding=encoding)

where custom_function would essentially wrap get_bitinformation and get_keepbits. All in all, that may not offer much over your current chunk-wise approach, except enabling us to use bitinformation in pangeo-forge compression and rechunking pipelines.
Alternatively, we could add some stripped down bitinfo implementation to numcodec and avoid the need for custom_function(). I'd be happy to help with that, but I don't want to advance that without permission from the xbitinfo team. Both projects have MIT license.

Sounds great to me, I let Hauke comment though.

Hi @thodson-usgs,
Thanks for reaching out! I like your proposal. The challenges I can see and which also emerge from your issue at numcodecs is that we would like to have as few implementations as possible to keep the algorithms maintainable as we develop them further. I therefore favour your custom_function suggestion over a new implementation into numcodecs. It should also be noted also that the custom_function needs only to be applied in the writing process and not in the reading process. The filters argument in the output file could therefore remain the same. In this case the user would not need to have xbitinfo or any other dependencies installed that provide custom_function.

Exactly right. I'll proceed with custom_function for now and see where we get.

https://github.com/zarr-developers/numcodecs/pull/503#issuecomment-1919916377
Unfortunately, I'm not sure this will work.
I'm by no means the expert, but as I look through the respective code bases, a reimplementation might be the simplest option, and could refer users to xbitinfo for the lastest and greatest version. Even if passing custom_function did work, xbitinfo's whole implementation is really built to leverage xarray. In general, I like that design choice. However, for this purpose, I'm finding myself trying to unravel some of that into a simpler function that operates on arrays.
I didn't double check, but the only dependency might be numpy.

...true, this might reduce direct traffic to xbitinfo, but I also think it's a valuable step into mainstream adoption.

@martindurant, I'm bringing you into this discussion with the xbitinfo team before continuing on the numcodecs side.
The two options I'd like both camps to mull over are:

Reimplementing a basic bitinfo algorithm in numcodecs. I think the only dependency is numpy.
Having xbitinfo as an optional dependency in the bitround codec.

Rather than including this in bitround, lets create a bitinfo codec that calls bitround.
Here's my draft proposal:
https://github.com/thodson-usgs/numcodecs/blob/adaptive-bitrounding/numcodecs/bitinfo.py

Okay works now. Going for multithread.

Going for multithread.

Zarr is commonly used with dask, so it should be enough to have your algorithms not hold the GIL

Thanks @thodson-usgs for all your work! If I understand it correctly you copy/reimplement part of the xbitinfo package to add to a numcodec algorithm. Can't we find a different solution, where e.g. the xbitinfo package provides an entry point to numcodecs? This will require xbitinfo to get installed to provide the codec but as mentioned above this should only be necessary for users who write data and not for those that read it and we make it easier for new features to be implemented across the board.

I sense that could get complicated, but I defer to the respective maintainers for guidance.

@observingClouds,
I'm also new to numcodecs, but I think it's designed to prohibit our initial proposal (probably a sign of a good codec).
But you too can join the ranks of zarr-devs, if you can fix this or any other typo :)
def _cdf_from_info_per_bit(info_per_bit):
    """Convert info_per_bit to cumulative distribution function"""
    # I suspect something is wrong with tol
    #tol = info_per_bit[-4:].max() * 1.5
    #info_per_bit[info_per_bit < tol] = 0
    cdf = info_per_bit.cumsum()
    return cdf / cdf[-1]

Maybe register_codec is the route to adding the codec to xbitinfo.
For example, https://github.com/cgohlke/tifffile/blob/master/tifffile/numcodecs.py

That looks indeed promising. Thanks for the pointer!

I take that back, register_codec is the wrong approach here. I believe the plan is:

implement a numpy-based codec in numcodec (basically done)
that codec could be migrated to xbitinfo such that the bitinfo codec imports xbitinfo, but we can assess the practicality of that after the codec is setup.

Please let me know if I've misjudged the necessity of a numpy-based codec, otherwise we can jump to 2 and import the dask-xarray version in numcodecs


Use Proper Encapsulation

In the NetStorage class, you have:
public String hostName;
public String username;
public String key;
public Boolean useSSL;

This, along with the fact that there is no interface, provides a nightmare for when you want to share this thing in your application with other folks.  These fields can be modified by anyone.

moved to using a credential bean

This is now inline with the edge-grid signers who utilize com.akamai.auth.ClientCredential interface. The netstorage specific interface (com.akamai.auth.DefaultCredential) has specific requirements that differ from the edge-grid auth model so it does require the Netstorage class to be aware of the differences when calling the Signer. This is unfortunate but I'll clean that up at some future time.


Write a clean up script

Summary

Write a script to clean up all provisioned resources and their access rights.

Implemented with #8.


Show message for missing metadata for file types

This needs to go in for release 1.10

@hyi Thanks for the quick review.


Consider using identifier names directly

Currently Chinese identifier names are transformed into pinyin, which might cause a redefinition if the same pinyin is also defined using wenyan, like this:
Both ES2015 and Python 3 support Unicode identifier names, there shouldn't be much problem using it directly.

Or set a more sophisticate variable naming machanism. Chinese include a vast amount of characters, simply difference words with its pronouciation and tone is not enough. Maybe do it like url encode function?

Hi there,
Thanks for the Issue,

Firstly there is actually an option to turn off the automatic Romanization. In CLI it is --roman false, in JavaScript it's parser.compile(js, input, {romanizeIdentifiers:false}). I'll probably add a GUI toggle for it on the online IDE too.

Currently, If two different identifiers have the same pinyin, a _ character will be appended to the latter (or more _'s will be appended if name clash remains), as you see in your example (the second JIA_). So there shouldn't be any problem if you only use Chinese characters.

Which brings us to the third line where you used the pinyin directly, and my current policy is to leave non-Chinese characters as they are. Your example made a good point, and now I'm rethinking that policy. :-P

I'm planning to add another mechanism, as DrMerxer mentioned. Probably unicode entry points? so 「甲」 would be _u7532. But that would decrease the readability of the compiled code.

Please feel free to let me know your thoughts! Thanks.

@LingDong- we can keep "甲" in the compiled file without any modification.

Prefer using Han characters directly

By keeping the original Han character name in compiled result, the compiled program would be able to keep best readability.
We just need to output the compiled result in UTF-8 encoding, so everything should work, and no one will be harmed.
So, IMO, this should be the default way to handle variable names.

In case output ASCII if needed
In case someone want to compile into ASCII source code, using U4F60U597D would be enough (which is easy to implement and works fine).
Why not using pinyin

Han characters are not only used in Chinese. Pinyin is not the only way to romanise Han characters.
Characters reading in Classical Chinese may variant with part of speech, may be different to stander mandarin readings nowadays.
Unihan includes >80k characters, and most of them wont have a stander reading at all.

@tiansh, congrats, you've convinced me :)
I will change using original identifier names to default, and add an option to use unicode entry points.
Thanks.

This issue is now fixed:
34119f806c2e62b9543d7e692bd3bd5e9b406edd
Original identifiers are used by default in all components. GUI for optionally selecting romanization method will be added in the next GUI update.
Thanks everyone for the discussion!


STATIC_URL not defined in templates

I'm trying to use the map widget and I noticed in the template {{ STATIC_URL}}. It seems that only gets added if you add django.template.context_processors.static to your context processors.  The Django docs seems to suggest using the static tag which doesn't require the context processor.
So instead of:
background-image: url("{{ STATIC_URL }}floppyforms/gis/img/move_vertex_on.png");
it should be:
background-image: url("{% static "floppyforms/gis/img/move_vertex_on.png" %}");
with {% load static %} up at the top.
So, I guess you could leave it as, but I didn't notice anything in the docs saying that you needed to have the above mentioned context processor added for some of the default templates to work.  The static tag works without any changes to settings.

Yes, you are right. The {% static %} approach is prefered over the one using the context processor. floppyforms comes from an age where only {{ STATIC_URL }} existed.
We should start dropping the support for those ancient releases.
I'm happy to accept a Pull Request that changes the static use, if you like to prepare one.

I'm a little confused about compatibility with older releases...  The context processor you included in your tests seems to have been added in 1.8 (looking at github, it seems they used to be in django.core.context_processors).  How are the tests passing for version 1.4 - 1.7 if they're including context processors that aren't there? (I'm referring to them being referenced in tests/settings.py)


fix fix ColumnFixedLenghtObjectTest

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.76 seconds
stream load tsv:          530 seconds loaded<PHONE_NUMBER>9 Bytes, about 134 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select:          29.7 seconds inserted 10000000 Rows, about 336K ops/s
storage size:<PHONE_NUMBER>5 Bytes

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.63 seconds
stream load tsv:          537 seconds loaded<PHONE_NUMBER>9 Bytes, about 132 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          28.9 seconds inserted 10000000 Rows, about 346K ops/s
storage size:<PHONE_NUMBER>9 Bytes

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 48.27 seconds
stream load tsv:          532 seconds loaded<PHONE_NUMBER>9 Bytes, about 134 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          29.9 seconds inserted 10000000 Rows, about 334K ops/s
storage size:<PHONE_NUMBER>0 Bytes


Playback is interrupted

During listening, playback is abruptly interrupted with an error.
Error:
AudioPlayerError: aborted
    at connResetException (node:internal/errors:683:14)
    at TLSSocket.socketCloseListener (node:_http_client:407:19)
    at TLSSocket.emit (node:events:377:35)
    at node:net:661:12
    at TCP.done (node:_tls_wrap:577:7) {
  resource: AudioResource {
    playbackDuration: 93900,
    started: true,
    edges: [ [Object], [Object], [Object] ],
    playStream: Encoder {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 5,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      encoder: null,
      _options: [Object],
      _required: 3840,
      _buffer: null,
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: [Function: bound onwrite]
    },
    metadata: Track {
      url: 'youtube url',
      title: 'video title',
      onStart: [Function: onStart],
      onFinish: [Function: onFinish],
      onError: [Function: onError]
    },
    volume: VolumeTransformer {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 5,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      _readInt: [Function (anonymous)],
      _writeInt: [Function (anonymous)],
      _bits: 16,
      _bytes: 2,
      _extremum: 32768,
      volume: 0.5,
      _chunk: null,
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: [Function: bound onwrite]
    },
    encoder: Encoder {
      _readableState: [ReadableState],
      _events: [Object: null prototype],
      _eventsCount: 5,
      _maxListeners: undefined,
      _writableState: [WritableState],
      allowHalfOpen: true,
      encoder: null,
      _options: [Object],
      _required: 3840,
      _buffer: null,
      [Symbol(kCapture)]: false,
      [Symbol(kCallback)]: [Function: bound onwrite]
    },
    audioPlayer: AudioPlayer {
      _events: [Object: null prototype],
      _eventsCount: 2,
      _maxListeners: undefined,
      subscribers: [Array],
      _state: [Object],
      behaviors: [Object],
      debug: [Function (anonymous)],
      [Symbol(kCapture)]: false
    }
  }
}

--------------------------------------------------
Core Dependencies
- @discordjs/voice: 0.3.1
- prism-media: 1.2.9

Opus Libraries
- @discordjs/opus: 0.5.3
- opusscript: not found

Encryption Libraries
- sodium: 3.0.2
- libsodium-wrappers: not found
- tweetnacl: not found

FFmpeg
- version: 4.2.4-1ubuntu0.1
- libopus: yes
--------------------------------------------------

@discordjs/voice version: 0.3.1
Node.js version: 13.0.0-dev.41673b738232f64da2ded3b15be0f798135ae351
Operating system: Ubuntu 20.04.2 LTS

Relevant client options:

partials: none
gateway intents: GUILDS, GUILD_MESSAGES, GUILD_VOICE_STATES

I am fairly sure that this is an error with ytdl-core - it seems to produce this error when playing back the video in real-time as opposed to dumping it to a file. In https://github.com/discordjs/voice/pull/118 I show an example using youtube-dl which does not seem to suffer the same problem.


Provide Grafana Dashboard for Prometheus Metrics

I was playing with the operator and couldn't find a Grafana Dashboard JSON file. As discussed in the https://github.com/aquasecurity/trivy-operator/issues/78, I just noticed this feature is very new and almost 1-week-old. https://github.com/aquasecurity/trivy-operator/pull/96/files by @erikgb
I propose we should provide a new Grafana Dashboard at Configuration section in docs. (screenshot included)
Are there any example dashboards created before? If not, I can get to it.

It would be nice to add a Grafana dashboard, at least to show users an example on how to use the metrics produced by the operator. We have a dashboard for our internal operator, that we want to replace with trivy-operator. @mikaello

@devopstales created three dashboards for their implementation of a trivy operator (trivy-operator); maybe they can be used as a starting point?
Screenshot:

look very nice 🚀

Interesting. Looking forward to seeing the dashboard released soon.

Hello,
Any news about it ? Is someone working on it right now ?

My colleague @pmarkiewka and I created a first version of a trivy op dashboard similar to the devopstales one:

Uploading with Grafana failed, so for now the source can be found in this gist.
We're still struggling with alerting, though. Did anyone of you succeed in alerting for new CVEs?
We tried it, but ran into the issue that PromQL increase() considers creation of new timeseries as reset: prometheus/prometheus#1673. So we would miss the first CVE of each image version ☹
BTW We shared what we learned about trivy op on myCloudogu.

Awesome news @schnatterer !
I will try your dashboard and take a look for alerting for new CVE ASAP

It would be super cool if it would be implemented as a mixin so users can customize it easily and add the library to their existing workflows.

Hello fellaz, I managed to find this one here https://grafana.com/grafana/dashboards/16337-trivy-starboard-operator-vulnerabilities/ .. Is that probably a compatible / comprehensive dashboard with the latest versions of Trivy?
Unfortunately I didn't try it out just yet.

Dashboard available here.

Dashboard available here.

Did you mean to attach a link? "here" is just normal text.

Dashboard available here.

@chen-keinan Did you mean to attach a link? "here" is just normal text.
thanks for catch up , link is updated


[Issue/Suggestion]

Hello,
First things first, Thank you for taking the time to make this amazing mod. It has liven up the battle aspects of  our server and I just want to say thank you. On another note, a player wheres the armor that burns enemies within a certain range every second. I have PVP disabled for our server but that Enchant also affects other players close by. Is there a way to fix this or disable it to affect players when pvp is disabled?

This is an issue for Dungeons Gear, not Dungeons Mobs. Regardless, there is a config option in Dungeons Gear to disable effects targeting players. In the next release, it will be off by default since so many people ask if the option exists (although I hope I don't get an influx of the opposite, "How can we get the effects to target players"?)


fixed spelling and formating in laser cog

As of pups request.

Are you still alive, Kite?

Are you still alive, Kite?

Sorry, i had a very busy week


Vagrantfile has invalid names for Windows

Installing the sandbox on windows, vagrant up results in a fatal error due to the use of : in the filename. Should use a delimiting character that doesn't cause issues in Windows

thank you for bringing this to our attention! we couldn't test this on a Windows machine and didn't realize that would be a problem.

No problem, in the short term, I have just removed the prefix. I have run into some other issues, but I'm not sure if they are relevant to capsule.
Do you all have a preference for the naming scheme for the vagrant box names? It appears - is allowed, the fix could just be to replace : with -; https://docs.microsoft.com/en-us/windows/win32/fileio/naming-a-file

@kamaroyl def. let us know what else you run into. If that's the fix, that's super easy, and we'd accept the pr :).

So, issues thusfar in my windows playground:

Windows virtualization seems to run into issues on update/upgrade, and may require a reinstallation/repair of virtual box :(
Some issue with dpkg; had to manually run sudo dpkg --configure -a  and apt-get install -y linux-headers-uname -r dkms
Silliness with permissions around /var/run/docker.sock
zlib not present in docker container which causes cargo to fail

@kamaroyl thanks for this.

a) Hmm. This may be VB-specific. Is there a known bug for this elsewhere?

b) Regarding dpkg, what error do you have here? I ask b/c we do install headers via https://github.com/capsule-rs/sandbox/blob/master/Dockerfile#L18, but not -r dkms (which I'm not sure if it's related to virtualbox guest plugin which includes dkms, or should?).

c) was the docker.sock perms issue related to something like https://www.digitalocean.com/community/questions/how-to-fix-docker-got-permission-denied-while-trying-to-connect-to-the-docker-daemon-socket? Docker for Windows should probably not have this happen, but this may be worth putting in the README just in case.

d) so adding zlib would be fine, but I've usually only seen it for musl builds of Rust. What was the error exactly? Do other Rust/cargo applications work fine in the environment?

After these steps, did things work? And, to be sure, Docker for Windows is what you're using here, yeah? What Windows and VB versions? If you're down to help on this with me, we can probably even get a github action running that will build the container, run it, and run the test suite on a Windows VM (not necessarily testing the vagrant issue you first mentioned, but we can fix that anyhow). Let me know. We can talk on the discord too!

Hey @zeeshanlakhani
A) seems to be just common on Windows virtual box; I was digging through posts from 2015 - 2020 with similar behaviors
C) It was that exact issue, though, running as root I wouldn't have expected that issue? chmod-ing to 6
D) Looking a bit closer at the issue, it might be something with the version of rustc on the box? (https://github.com/rust-lang/cargo/issues/8258, version installed is rustc 1.43.0 (4fb7144ed 2020-04-20) Looks like the git issue was fixed in June https://github.com/rust-lang/cargo/issues/8258). For whatever reason, rustup is having issues with versions later than 1.43.0; trying to install later versions.
After looking at this with fresh eyes this morning, I'm wondering if most of these errors aren't caused by network instability in VB. Seems like VB gets its network cut out from under it; I'm seeing inconsistent check sum failures on different files when trying to install a later version of rust.
I don't currently have the entire setup working unfortunately, hoping if I can get past this cargo issue I can try and run the testsuite. Windows version could definitely be a problem; I'm on Windows 10 Home Insider Preview Build 20197.rs_prerelease.200818-1420;
VB is 6.1.12 r139181

@kamaroyl cool. For A and C, seemingly we should have a Windows section in the README. Let's discuss what it may look like once you're all up and running.
For D, I can update the rust container to a newer version, if one works. So, let me know if the most recent does: 1.45. You should be able to just take the docker.mk and change where we pull from via this version: https://github.com/capsule-rs/sandbox/blob/master/docker.mk#L14.
VB is a pain, but is most generic sadly.

Okay, so, I blew away the Windows Insider box and rebuilt it; now running Windows Pro 10.0.19041 Build 19041 with virtualization enabled. It's been much smoother this time. I did run into an issue running vagrant up
VagrantUp.log
The container was a bit more work; ran into quite a few seg faults. Ended up having to update a bunch of packages and upgrade rust to stable (1.45.2), but I finally got tests to run 🎉🎉🎉

@kamaroyl dope! It's possible we need a round of updates, and can test that works across distros (and vagrant formatting as you mentioned). Yay to hear. I'll do a PR to update the Rust container version we use first.

@zeeshanlakhani It's kinda wild when you start getting linker segfaults
LinkerSegfault.log

@kamaroyl not surprising. It may be an issue in our build.rs with what modules we can load wrt to the underlying windows host, but I'll check it with Rust 1.45.2 as well. We'll probably need a compilation flag for windows builds specifically. Here's what I'm referring to: https://github.com/capsule-rs/capsule/blob/master/ffi/build.rs#L25.


ENH: Refactor FFT classes for object factory registration

Takes the object factory structure introduced to itk::Forward1DFFTImageFilter in https://github.com/InsightSoftwareConsortium/ITK/pull/2836 and extends it to all FFT classes.
Relying on object factories for generating FFT classes will allow users to flexibly override FFT implementations so that their default backend is the one best for their own system. Developers can develop against base classes such as itk::ComplexToComplexFFTImageFilter, itk::HalfHermitianToRealInverseFFTImageFilter, etc, without needing to know whether the FFT backend is Vnl, FFTW, or something else.
Prior to these changes backend selection was performed with dispatch methods overriding base constructors, which locked in available backends to those included with ITK proper. Moving to the object factory framework will allow remote modules to inject accelerated backend defaults, such as in the case of https://github.com/InsightSoftwareConsortium/ITKVkFFTBackend.
Consuming projects must use include(UseITK.cmake) to properly initialize default object factory registrations. Discussion in https://github.com/InsightSoftwareConsortium/ITK/pull/2836 concluded that ITK tests do not include this line and therefore do not register factories by default, hence filtering cxx tests relying on arbitrary FFT classes have been refactored to include manual object factory registrations.
Python wrapping does include UseITK.cmake and default registers factories, therefore manual registrations are not required in ITK Python testing but may be required for remote module consumption. This will be investigated with changes to igenerator.py in a subsequent PR. Further discussion on object factory architecture and behavior can be found at https://discourse.itk.org/t/proposal-to-improve-itk-factory-registration-cmake-infrastructure/4546/4.

PR Checklist

[x] No API changes were made (or the changes have been approved)
[x] No major design changes were made (or the changes have been approved)
[x] Added test (or behavior not changed)
[x] Updated API documentation (or API not changed)
[x] Added Python wrapping to new files (if any) as described in ITK Software Guide Section 9.5

Refer to the ITK Software Guide for
further development details if necessary.

When this compiles, convert it back into a ready PR. If you want me to review before then, ping me.

After resolving several compile/test issues I'm still seeing a strange failure in Python tests.
CDash shows that PythonGetNameOfClass and three other Python tests are failing with a segfault:

itkTestDriver: Process exception: Segmentation fault

I've traced this back to itk.FFTConvolutionImageFilter.New(), which attempts to instantiate an itk.RealToHalfHermitianFFTImageFilter object in its constructor in order to set a member:
template <typename TInputImage, typename TKernelImage, typename TOutputImage, typename TInternalPrecision>
FFTConvolutionImageFilter<TInputImage, TKernelImage, TOutputImage, TInternalPrecision>::FFTConvolutionImageFilter()
{
  m_SizeGreatestPrimeFactor = FFTFilterType::New()->GetSizeGreatestPrimeFactor();
}

However, FFTFilterType::New() returns a nullptr and the subsequent function call on the nullptr causes the segfault. This implies that the FFT factory is not properly registered before the call executes.
Factories should be registered in Python by default, especially given that force_load() is called at test initialization. On the command line it seems that factories are indeed registered, as instantiating with itk.<FFT class>.New() and then casing to itk.<Vnl FFT class> succeeds. However, calling itk.FFTConvolutionImageFilter[<types>].New() always fails.
Investigation notes:

Adding printouts to itkVnlFFTImageFilterFactories.cxx shows that FFT factories are indeed registered at load time as expected
Trying to replace itk::RealToHalfHermitianForwardFFTImageFilter with any other FFT type in the FFTConvolutionImageFilter constructor fails and returns nullptr. No FFT factories seem to be present (or at least accessible) from the constructor.
FFTConvolutionImageFilter is wrapped with TInternalPrecision = double, which propagates to an attempt to instantiate the forward FFT filter with double type, which is not explicitly wrapped. However, 1) it's not required for classes used internally to be wrapped for the external filter to be used in Python, 2) FFT image filters of double pixel type should be registered alongside float types by default, and 3) changing the default to TInternalPrecision = float does not resolve the issue.
Trying to check factory registrations with itk.ObjectFactoryBase.GetRegisteredFactories() returns an empty tuple, which is unexpected as this should be a singleton against which all factories are registered. After calling itk.imread to register an IO factory itk.ObjectFactoryBase.GetRegisteredFactories() still returns the empty tuple. Not sure exactly what's happening here.
Subclasses of FFTConvolutionImageFilter also segfault during construction (which is not unexpected)

Open to suggestions on how to proceed here. Would like to avoid adding dynamic Python factory registration expansions to this PR as it's already very large, but will reluctantly head that way if that is the required resolution.

In case it is not a red herring, ... in https://github.com/InsightSoftwareConsortium/ITK/pull/2890 there is a short discussion that might explain why factories that are registered don't appear that way to other parts of the code.  It includes

It has to do with ITK python building the ITK libraries static, but having each ITK module as a separate loadable (Shared library) python module. The there are multiple instances of these "global" variables on some system.

Thanks @Leengit , this sounds like the most likely cause right now. I'll have to look more into how the GlobalTimestamp singleton is unified across Python modules and see if that can be applied here.

Some series templating here! :man_dancing:
@tbirdso do these examples still work when built against this branch?
https://itk.org/ITKExamples/src/Filtering/FFT/index.html

Hi @thewtex , yes, the FFT examples still work when built against this branch on Windows. I'll also note that instantiating FFTConvolutionImageFilter in Python works on Windows as well, but not on Linux.
I'm working on building the examples in Linux, in the meantime I've submitted a small path for the inverse FFT example here: https://github.com/InsightSoftwareConsortium/ITKSphinxExamples/pull/312

Verified that Linux FFT examples build. So far the problem seems limited to using object factories in consuming modules in Python on Linux, i.e. using FFT object factories in Convolution filters. The discussion in #2890 in regards to magic statics may be the root of the issue, though I'm not clear at the moment why this would be platform-dependent behavior.

Current hypothesis: registering FFT factories with "magic statics" in itk::ObjectFactoryBase::RegisterInternalFactoryOnce confines registration to the object factory "singleton" in the ITKFFT Python module which is not visible from the ITKConvolution module.
#2890 dealing with magic statics was abandoned (at least for now) due to itkSingletonMacro being required for synchronizing singletons across ITK Python modules. Meanwhile, #2819 recently introduced magic statics with RegisterInternalFactoryOnce for IO object factory thread safety. This is reasonable for IO factories because they are only called within the ITKIO module. However, we can't extend that design pattern here because we expect filters outside of ITKFFT will make use of FFT filters internally.
In an initial test registering itk::VnlRealToHalfHermitianForwardFFTImageFilter with a static HasBeenRegistered variable (per the design pattern prior to #2819) I am able to successfully instantiate an itk::FFTConvolutionImageFilter which instantiates an itk::RealTohalfHermitianForwardFFTImageFilter via the object factory in its constructor in Python on Linux. This addresses the remaining Python issues discussed above; I will add a draft commit along these lines.
As I see it there are two ways to proceed here:

Further investigate global singleton synchronization across ITK modules as is done with TimeStamp::Modified as discussed in #2890. It's not clear to me how much additional effort this will add.
Remove magic statics from FFT factory registration and rely on static booleans to ensure that only one factory is registered. This exposes potential issues with thread safety and race conditions where a factory could be registered multiple times at initialization; however, it's not clear that this will occur in practice. I would favor moving forward with this approach and reconsidering if we see issues during testing.

I'm certainly open to other ideas here as well. Pinging @N-Dekker and @thewtex for discussion of object factories + magic statics.

Marking as ready for review but intentionally leaving the last commit unsquashed for design discussion.

Per discussion with @thewtex we will leave FFT registration as-is for now, but follow up with changes to synchronize itk::ObjectFactoryBase across Python modules as we do with the timestamp singleton. Those changes will open up object factory registration for custom backends across ITK classes going forward.

For future reference, reading a few lines into documentation on C APIs for Python extension modules reveals a likely answer for why FFT factories registered with magic statics were visible across modules in Windows but not in Linux:

When modules are used as shared libraries, however, the symbols defined in one module may not be visible to another module. The details of visibility depend on the operating system; some systems use one global namespace for the Python interpreter and all extension modules (Windows, for example), whereas others require an explicit list of imported symbols at module link time (AIX is one example), or offer a choice of different strategies (most Unices).


about "Test with Visual Studio Mobile Center"

we have been using build in Mobile Center, but didn't get automation of Test with it and I found this task "Test with Visual Studio Mobile Center" in vsts, Can I use this task but with output of Mobile Center build to run the testing?

@jamesatgithub: The "Test with Visual Studio Mobile Center" task allow running tests on mobile center if you using VSTS for build. There is some documentation available on this task here: https://docs.microsoft.com/en-us/mobile-center/test-cloud/vsts-plugin. Is this what you are looking for?
Thanks,
Madhuri


upd(gir1.2-xapp-1.0-deb): 2.4.2 -> 2.6.1

Requires https://github.com/pacstall/pacstall-programs/pull/3756


SSMCache.flush clears all caches on the memcached server instead of the individual cache

When invoking SSMCache.clear(), the expectation is that it will only clear entries from the specific cache named by SSMCache.getName(). Instead, it flushes all entries across all caches on the Memcached server.

Please take a look at documentation: https://github.com/ragnor/simple-spring-memcached/wiki/Getting-Started#settings.
Memcached doesn't support cache zones or cache namespaces, so there's no such operation like clearing only some part of data stored in zone/namespace, only all data stored by memcached instance can be removed


Mesh refinement

Being able to refine the mesh on the left hand side, so that we have roughly 5um/1500 = 0.003 um / cell in the first 5 um.

Fixed with #13


Allow memory configuration to propagate through.

Without this change, you cannot set JAVA_OPTS and override memory settings per app.

This is pointless, since we can just do:
JBP_CONFIG_OPEN_JDK_JRE: '[memory_calculator: {memory_sizes: {stack:512k, metaspace:64m, heap:300m}}]'


Changing default video quality

I want to set custom default video quality on player load. Please help me how this can be done.
Thank you very much.

The video quality we pick on load is based on bandwidth estimates. Our ABR code will calculate a bandwidth estimate based on download rates it sees, but you can change the default estimate we use before we have enough data with the abr.defaultBandwidthEstimate configuration value. This will cause it to start at a different quality; if you want the quality it starts at to be high, you can set a high default bandwidth, and vice-versa.
myPlayer.configure('abr.defaultBandwidthEstimate', myDefaultBandwidth);

Yes please.


Arithmetic

There is still a lot to be done, but it is in a consistent state, is fairly well tested and does not seem unreasonably slow. I think it is ready to merge. I am going on vacation tomorrow until 2016 and I will not touch it again until then.
A few comments:

I have no idea how to write a state-of-the-art arithmetic and computer-algebra library, let alone a proof-producing one. I am completely winging it. I don't even have good references, and am working out tricky recursions and invariants by hand even for problems that may have standard algorithms. Any pointers to books, papers, or relevant code-bases would be much appreciated.

The core of this pull request is a decision procedure for linear inequalities, based on the approach described at https://www.cs.utexas.edu/~moore/publications/non-linear.pdf. It can already handle fairly beastly inequalities over fields, as in https://github.com/dselsam/lean/blob/arith/tests/lean/run/blast_arith_heuristic.lean#L66-L77. Right now it is restricted to linear reasoning, but I plan to add some support for heuristic non-linear reasoning (as described in the paper) over time. Thus it currently treats x * y as a black-box.  For example, it doesn't communicate new equalities to blast even when it finds them, it doesn't try to prove an inequality by contradiction, and it doesn't do case-analysis on inequalities.

The most complicated part of the module so far is the proof-generating field-normalization (https://github.com/dselsam/lean/blob/arith/src/library/blast/arith/normalize.cpp). We wanted to be able to prove a term is equal to its normal form substantially faster than we were able to with the (much more generic) simplifier. The current implementation is faster, but unlike the simplifier, it does not track refl proofs, and as a result the proof terms are much bigger than they need to be. This optimization is doable but does not seem to be high priority.

We also have a non-proof-generating normalization procedure that populates a specialized data structure for representing polynomials. The arithmetic procedure itself works with specialized data structures, and only tries to prove anything in terms of actual Lean expressions if it has found a contradiction.

I am using a light wrapper around Rob's super-fast numeral library. I expect div to be marked reducible in blast, and I have been using inv everywhere instead. The fact that a normalized numeral can be a product (e.g. 3 * inv 5) adds some complication throughout the module though, and I think a better approach might be to define a new, non-reducible function that either means div or even just mul, and for blast to use it to represent normalized numerals.

I have been warned that this idea will be met with overwhelming resistance, but I feel strongly that every application of inv should carry with it a proof that the element is indeed invertible. We can use the same inv whether the inverse is total or partial:

structure has_inv [class] (A : Type) := (invertible : A → Prop) (inv : ∀ a, invertible a → A)
definition invertible {A : Type} [s : has_inv A] : A → Prop := has_inv.invertible
definition inv {A : Type} [s : has_inv A] : ∀ (a : A) {H : invertible a}, A := has_inv.inv

structure group [class] (A : Type) extends monoid A, has_inv A :=
(all_invertible : ∀ a, invertible a)
(mul_left_inv : ∀a, mul (@inv a (all_invertible a)) a = one)

structure division_ring [class] (A : Type) extends ring A, has_inv A, zero_ne_one_class A :=
  (nonzero_invertible : ∀ {a:A}, a ≠ zero → invertible a)
  (mul_inv_cancel : ∀(a) {H : invertible a}, mul a (inv a H) = one)
  (inv_mul_cancel : ∀(a) {H : invertible a}, mul (inv a H) a = one)

This may seem like it will be extremely burdensome, but we can register a tactic hint for the head invertible and a more mature blast should be able to discharge the obligations easily without the user needing to think about it at all.
I think one can make a convincing argument on aesthetics alone, and also on convenience for automation (right now I am tracking all the _ ≠ 0 sub-goals that need to be solved). As Leo pointed out to me, there is an even more pressing reason to store the proofs with the applications: so we can cache them automatically. Imagine a complicated non-zero proof in the middle of a bigger proof, where the conclusion involves the tricky inverse. Then blast will not be able to use the theorem effectively since it will not be able to reprove that the term is non-zero. If we do not embed the proofs in the applications, then users will need to abstract all non-trivial non-zero proofs into stand-alone lemmas.

I did not integrate the new lemmas used by the arithmetic module into the standard library yet. For now I am keeping them in their own file, since it is convenient for them to all be in the same place while implementing the module. The only downside right now is that one needs to import algebra.blast in order to use/test the module. Since this module is still a long way from being usable, and since the standard library will likely be re-factored anyway as we gradually start using blast, this does not seem pressing.

There are a few miscellaneous rough edges, which I will iron out over time. Example: the tracing is still haphazard and does not conform to the budding standard for blast-relevant events.

I thought I have already sent you several references for arithmetic procedures, and have summarized the main ones in the slack channel. Regarding normalization, I never seen this step being described anywhere, it is considered too simple. Most articles will say things such as: "we assume all polynomials are in sum of monomials normal form", ...
I have also sent you links to the z3 code that performs these normalizations. Is the code there too confusing?

I'm still looking through what you've added here, but this seems fantastic! A few preliminary questions.

Poly pot is just a funny name for lower and upper bounds. Note that the acl2 procedure is essentially Fourier Motkin + heuristics for nonlinear arithmetic.

What was the reason to go with this method for linear problems, instead of something based on simplex/Farkas' lemma? Just that it can be extended to nonlinear problems too? (I'm not saying this is a bad reason! Just curious. I've been thinking about implementing a simplex solver for a while.) My experience using ACL2 for nonlinear inequalities was that it was very sensitive to the format of the problem and to which ACL2 books were open. How configurable do you expect your solver to be?

The most complicated part of the module so far is the proof-generating field-normalization

Cool. What is the normal form you reduce to? Can this be made available as a tactic? i.e., given a goal s < t where s and t are terms in a field, normalize both? I'm happy to take a stab at this after the new year.

I have been warned that this idea will be met with overwhelming resistance, but I feel strongly that every application of inv should carry with it a proof that the element is indeed invertible.

I have to think about this one a little longer, but my gut reaction is indeed resistance, and I'm not sure I see the aesthetic argument. I believe you that it would simplify things for the automation, but it feels like this could get quite confusing in manual proofs. By "complicated non-zero proof in the middle of a bigger proof," do you mean a user-supplied proof have H : x ≠ 0, from ...? Why wouldn't this be available as a hypothesis to blast?

What was the reason to go with this method for linear problems, instead of something based on simplex/Farkas' lemma?

I suggested ACL2 for Daniel because it is easier to implement.
Lev's simplex module is still under development, and Daniel would not really learn much by doing it.
It is mainly an integration task. This will be done later. Note that the ACL2 part is a small part of the commit.
It is also useful to have a Fourier-Motzkin (FM) procedure even if we have Simplex.
We can use FM to perform quantifier elimination. In Z3, FM is also use for preprocessing.

Just that it can be extended to nonlinear problems too?

Yes.

How configurable do you expect your solver to be?

Remark: this is one of many solvers that will be constructed.

Cool. What is the normal form you reduce to? Can this be made available as a tactic?

This is possible. However, we didn't implement yet a procedure that converts a blast state object back into a goal.
BTW, all blast related modules are changing too quickly.
We already have tactics such as simp and blast, and many more variants can be constructed, but I don't want to advertise them until they are much more stable.

it feels like this could get quite confusing in manual proofs.

Right now, it is not feasible to move to this approach.
We need a very robust blast/simp before we try it.
The rewrite tactic doesn't really work for inv/div with nested proofs.
Example: the tactic rewrite H1 would fail in the following state:
a b : real, H1 : a = b, H2 : a ≠ 0, H3 : b ≠ 0 |- @inv real a H2 = @inv real b H3

These steps would have to be replaced with simp, or the user would have to manually invert H3, rewrite and reintroduce it.
After we have a robust blast, we need to go over the nat/int/rat/real libraries and simplify all proofs using it.
That being said, the approach has many advantages. Moreover, if everything works as expected, we will have to write very few manual proofs. Anyway, I think this is not the right time to discuss/consider this change.
BTW, there is another issue, this approach may be problematic for the HoTT library since it doesn't have proof irrelevance.

By "complicated non-zero proof in the middle of a bigger proof," do you mean a user-supplied proof have H : x ≠ 0, from ...? Why wouldn't this be available as a hypothesis to blast?

Suppose we have a lemma L whose conclusion contains a/b. Moreover, suppose its proof contains a have b ≠ 0, ..., and the proof is not simple. Now, when blast uses L it doesn't have access to the nested proof for b ≠ 0. The only way to make it available is to create an auxiliary lemma (i.e., factor out the nested proof).

Regarding normalization, I never seen this step being described anywhere, it is considered too simple. Most articles will say things such as: "we assume all polynomials are in sum of monomials normal form", ...

What is considered simple in a paper is not necessarily simple for generating machine-checkable proofs efficiently. Example: basic arithmetic obviously does not need to be explained or justified in a paper, and yet the efficient numeral-normalizer is one of the most complex modules in blast so far. Similarly for the heuristic arithmetic paper, it is simple to implement resolving two linear inequalities, and it just happens to be much more complicated to normalize the terms efficiently with proofs.

I have also sent you links to the z3 code that performs these normalizations. Is the code there too confusing?

It is my current plan to read all the relevant z3 code. We had a misunderstanding though: I thought you specifically suggested I not try to read z3 since it was not designed to be readable. Also, z3 has a much lower standard of proof-generation, and generating proofs is the often the most challenging aspect of implementing these procedures in blast.

Why is this kind of change stuffed inside the arithmetic commit?

It was needed since we were normalizing the simp rules. Note that you made the same change when you implemented this feature: https://github.com/leanprover/lean/commit/b117a10f82ea751c9b932e9e3006d057cc14886e#diff-1bb2394fe3a0f89e80c5e85b2bb6bf34R504

BTW, there is another issue, this approach may be problematic for the HoTT library since it doesn't have proof irrelevance.

In HoTT, all inhabitants of x ≠ 0 are propositionally equal:
https://github.com/leanprover/lean/blob/master/hott/types/pi.hlean#L309
They're not definitionally equal, as in the standard library, though.

After we have a robust blast, we need to go over the nat/int/rat/real libraries and simplify all proofs using it.

One thing which worries me: if we do this and blast doesn't work as well in the HoTT library, then we cannot port files anymore from the standard library to the HoTT library.

@fpvandoorn

In HoTT, all inhabitants of x ≠ 0 are propositionally equal:
https://github.com/leanprover/lean/blob/master/hott/types/pi.hlean#L309
They're not definitionally equal, as in the standard library, though.

This is good. We have a "HoTT compatible" congruence closure procedure that will be able to handle the new inv if the inhabitants of x ≠ 0 are propositionally equal.

After we have a robust blast, we need to go over the nat/int/rat/real libraries and simplify all proofs using it.

One thing which worries me: if we do this and blast doesn't work as well in the HoTT library, then we cannot port files anymore from the standard library to the HoTT library.

I'm also concerned about this issue.
Daniel and I have discussed this issue many times.
For me, the main problem is that the libraries are really hard to maintain.
Whenever we make a change in the elaboration procedure, we have to go over the really painful process of fixing proofs. We have many proofs that rely on higher-order unification and are very fragile.
By replacing them with blast, we avoid all the brittleness associated with higher-order unification.


Fix macOS 14 support

Fix macOS 14 support
Improve code readability
Update to 2024.3

Thank you, but I don’t want to overcomplicate this. This is GUI automation depending on OS version, so being smart with supporting both in the same script is a recipe to get bitten later on. Being “dumb” is more straightforward, clearer, and safer.
A more robust approach is to check the OS version beforehand, then split. This way when pre-Sequoia gets too old, it’s just a matter of deleting that code, not having to rework everything. Plus, it won’t cause a mess if Sequoia makes some change that makes it look like previous OS versions.
Made the changes in 2024.3, just released.

Thanks!


Add LTK

In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:

[x] Filename in the lowercase format
[x] Image format: png
[x] Image size: 256 by 256 px

Please, lowercase file name for next upload, for now, added lowercased image file name here: https://github.com/TrustWallet/tokens/commit/9fa9118b3a151912c39e25d62efdda66a76456dd?short_path=5f89cba#diff-5f89cba93d8b41169bbb188202730cb1


php multiline bug in visual mode

Let I have a little code:
$obj->ping();
$obj->info();
$obj->getSpecList();
$obj->oneRequest("something");

So if I select these lines in visual mode and press <leader>cc, I get
/*$obj->ping();*/
//$obj->info();
//$obj->getSpecList();
/*$*/obj->oneRequest("something");

instead of expected
//$obj->ping();
//$obj->info();
//$obj->getSpecList();
//$obj->oneRequest("something");

An expected behavior is provided only by binding <leader>c<space> 'toggle comment'
The same behavior to unexpected is while calling nested commenting

This appears to have been fixed at some point as it works as expected with the example you give. If this is still a problem for you please comment and I'll re-open this for review.


ParagraphVectors infervector issue after upgrading to M2 version

Hi All...
I have upgraded my DL4J from beta to M2 version. After upgrading, I updated the libraries (so files) from the following links.
https://repo1.maven.org/maven2/org/bytedeco/openblas/0.3.19-1.5.7/openblas-0.3.19-1.5.7-linux-x86.jar
https://repo1.maven.org/maven2/org/nd4j/nd4j-native/1.0.0-M2/nd4j-native-1.0.0-M2-linux-x86_64-compat.jar (as I have older GLIBC version)
But after the upgrade ParagraphVectors inferVector started failing functionally on the same word vector. There is a shape mismatch which indicates the PV is not inferring at all after upgrade.
Older version is inferred vector shape is (1, 300) whereas the new one is inferred the shape (1, 1)
Why it is happening like this ? Any other libs or packages I am missing ?

@kiransarv ParagraphVectors hasn't changed for a while. Could you try with snapshots to see if this is still a problem? I would also appreciate a small reproducer program so I can see the issue. Without a stack trace it's going to take me time to guess what your problem is or where this could be happening.

@agibsonccc Thanks for the info.. Here is the scenario.. I have trained Paragraph Vectors using DL4J in older version.. I am reusing the PV model in both the older version and newer version. When i infer using inferVector Here is the difference in output..
<some_text> :: Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,1],  Stride: [1,1] (New Version)
<som_text> ::   Rank: 2, Offset: 0 Order: c Shape: [1,100],  stride: [1,1] (Older Version)

@kiransarv there might be some issue with the way newer models are handled and vectors. It used to be that old vectors had to be 2d but now most are 1d. Could you do what I asked and give me a reproducer? I can't just take your description and turn that in to code. You're already describing potential issue sources like an old model among other things.

@kiransarv sorry not sure what you mean. I literally need something from you specifically that reproduces this problem. I want actual phsyical java code with a model that shows this issue. I am sorry if I confused you earlier but I am mentioning I need anything specific to you that would help reproduce this. I am not going to spend time on this otherwise. That could be your old model and some code to run it or anything.
I am not going to chase this down just based on an issue description and what you think the problem is. Then I have to debug what you think the problem is + your opinion on the problem. That usually ends up being a waste of time.
Give me a stack trace with real code + your model or something that shows the issue. How you do that is up to you. I don't need anything that's secret to your company or anything. I only want to help fix your issue. How you give that to me is up to you.

Got you.. let me in that case pull the code from repo, run it as a standalone and share the code and data with you

Thanks for the understanding. I know it's a hassle but it will allow me to focus on just solving the problem rather than guessing where it could be.

@agibsonccc figured out the shape issue there is slight change in the setSyn0 method in InMemoryLookupTable between old vs new version which is causing the vector dimension set to 1.
But now I am observing slowness in the inference times after upgrade... Here is the output of both old and M2 version.
Dataset used: https://www.kaggle.com/datasets/therohk/million-headlines?resource=download (select first 10k lines)
Old version (beta) output
Time to train NN embedding model, Docs size :: 10000 :: 135804 ms
Total vocab size :: 2430 :: 100
Avg time for inference, for 500 instances is 0 ms
M2 version output
Time to train NN embedding model, Docs size :: 10000 :: 70719 ms
Total vocab size :: 2430 :: 100
Avg time for inference, for 500 instances is 27 ms
Below is the code to reproduce the issue... The libs(dylib) are copied from the below Url's
https://repo1.maven.org/maven2/org/bytedeco/openblas/0.3.19-1.5.7/openblas-0.3.19-1.5.7-macosx-x86_64.jar
https://repo1.maven.org/maven2/org/nd4j/nd4j-native/1.0.0-M2/nd4j-native-1.0.0-M2-macosx-x86_64.jar
Code to reproduce the issue
import java.io.File;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import org.apache.commons.io.FileUtils;
import org.deeplearning4j.models.paragraphvectors.ParagraphVectors;
import org.deeplearning4j.text.documentiterator.LabelAwareIterator;
import org.deeplearning4j.text.documentiterator.LabelledDocument;
import org.deeplearning4j.text.documentiterator.SimpleLabelAwareIterator;
import org.deeplearning4j.text.tokenization.tokenizerfactory.DefaultTokenizerFactory;
public class ParagraphVectorsM2 {
private static final String[] MAC_OS_LIBRARIES_M2 = {"gcc_s.1", "jniopenblas", "gfortran",
"jniopenblas_nolapack", "openblas", "gomp", "gomp.1", "stdc++", "stdc++.6", "nd4jcpu", "jnind4jcpu"};
private static final String[] MAC_OS_LIBRARIES_OLD = { "gomp", "gomp.1", "stdc++", "stdc++.6", "nd4jcpu", "jnind4jcpu",
		"gcc_s", "gcc_s.1", "jniopenblas", "openblas" };

public static void main(String[] args) throws IOException {
	// TODO Auto-generated method stub
	for (String library : MAC_OS_LIBRARIES_OLD) {
		System.loadLibrary(library);
		System.out.println("Loaded library: " + library);
	}
	System.setProperty("org.bytedeco.javacpp.loadlibraries", "false");

	File file = new File("/Users/kiran.sarvabhotla/Downloads/temp.txt");
	List<String> lines = FileUtils.readLines(file, "UTF-8");
	buildAndInferPV(lines);
}

private static void buildAndInferPV(List<String> lines) throws IOException {
	// TODO Auto-generated method stub
	long start = System.currentTimeMillis();
	List<LabelledDocument> docs = new ArrayList<LabelledDocument>();
	for(int i = 0; i < lines.size(); i++) {
		LabelledDocument labelledDocument = new LabelledDocument();
		labelledDocument.setContent(lines.get(i).toLowerCase());
		labelledDocument.setId(String.valueOf(i));
		docs.add(labelledDocument);
	}

	LabelAwareIterator labelAwareIterator = new SimpleLabelAwareIterator(docs);
	ParagraphVectors vocabulary = new ParagraphVectors.Builder()
			.batchSize(64)
			.epochs(50)
			.trainWordVectors(true)
			.trainSequencesRepresentation(true)
			.allowParallelTokenization(false)
			.minWordFrequency(5)
			.layerSize(100)
			.tokenizerFactory(new DefaultTokenizerFactory())
			.iterate(labelAwareIterator)
			.enableScavenger(true)
			.build();

	// 4. train/create a model
	vocabulary.fit();
	System.out.println("Time to train NN embedding model, Docs size :: " + lines.size() + " :: " + (System.currentTimeMillis() - start) + " ms");

	inferVector(vocabulary, lines);
}

private static void inferVector(ParagraphVectors pv, List<String> lines) {
	System.out.println("Total vocab size :: " + pv.vocab().numWords() + " :: " + pv.getLookupTable().layerSize());
	int count = 0;

	long start = System.currentTimeMillis();
	long time = 0;
	long infertime = 0;

	for(String text : lines) {
		if(count >= 500)
			break;

		try {
			start = System.currentTimeMillis();
			pv.inferVector(text);
			infertime = (System. currentTimeMillis() - start);
			time += infertime;
			//System.out.println(text + " :: " + arr.sumNumber() + " :: " + arr.shapeInfoToString() + " Time :: " + infertime);
		} catch(Exception e) {
			//e.printStackTrace();
			//System.err.println("Unable to infer PV for text :: " + text);
		}

		count++;
	}

	System.out.println("Avg time for inference, for " + count + " instances is " + (time/count) + " ms");
}

}

@kiransarv can you try the M2.1 release? (Just add a .1 on to the M2 version) if that still persists I'll investigate. Thanks!

Thanks for the reply.. If I am not wrong the change from AggregateSkipGram to SkipGramRound is the root cause...
https://github.com/eclipse/deeplearning4j/blob/master/deeplearning4j/deeplearning4j-nlp-parent/deeplearning4j-nlp/src/main/java/org/deeplearning4j/models/embeddings/learning/impl/elements/SkipGram.java#L375

@kiransarv so looking at this....is there any reason you're manually copying libraries and calling System.load(..)? Javacpp already does that for you. Is something broken separately that I should be aware of? None our documentation or examples does this.

@kiransarv I see output like this:
Time to train NN embedding model, Docs size :: 1244185 :: 10889 ms
Total vocab size :: 7 :: 100
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Arr shape Rank: 2, DataType: FLOAT, Offset: 0, Order: c, Shape: [1,100],  Stride: [1,1]
Avg time for inference, for 500 instances is 0 **ms**

Please try to upgrade to M2.1. The vectors are printing out the correct shape. I'm closing this for now. Feel free to come back if you have any issues.


startServer() should fail if no ipaddress available

I'm struggling to understand all of the startup logic, but on the surface this appears to be a bug.  Maybe there's a workaround using the "interfaces" array that can be passed as an option? (I couldn't find an example of how to use this option).
Steps to reproduce

Turn OFF wifi and all internet connections
Run the example server script
Notice that no errors/exceptions are thrown
Even if we turn wifi back on at this point (so that we eventually get an ip address), the server won't respond to any M-SEARCH requests

Expected behavior

SSDPServer.start() should reject the promise if we can't bind to a "real" ip address (<IP_ADDRESS> shouldn't count)
Alternately, the server could implement its own retry logic, but I'm leaning towards just rejecting the promise and leaving this up to the consumer

The module already ignores all internal interfaces (loopback included) but you're correct in that it does not check whether or not it was able to create at least one socket. Rejecting the promise is a good idea.

Published v4.0.0


feat(brand-icons): add two new brand icons

add two new brand icons
Description
Added two new brand icons. minutes and quick-and-easy.
Storybook link: (once netlify has deployed link provide a link to the component)
Design link: design
Screenshot:

Checklist:

[x] The commit messages follow the convention for this project
[ ] I have provided an adequate amount of test coverage
[ ] I have added the functionality to the test app
[x] I have provided a story in storybook to document the changes
[ ] I have provided documentation in the notes section of the story
[ ] I have added any new public feature modules to public-api.ts

Just wondering do we need two PRs? or can we rebase next once the master PR is merged? It's only a minor concern but there is a risk of getting duplicate commits with 2 PRs, might be dependent on @owensgit being back.

Just wondering do we need two PRs? or can we rebase next once the master PR is merged? It's only a minor concern but there is a risk of getting duplicate commits with 2 PRs, might be dependent on @owensgit being back.

I would favour a rebase as well, although I've had some issues with running semantic-release recently, so I would favour just releasing v6 of Canopy. I'm going to close this PR and look into sorting that out soon.