Trying to install cli using the suggested composer command fails

Trying to install cli via the suggested composer method fails:
± |004 ?:9 ✗| → composer global require pantheon-systems/cli
Changed current directory to /Users/gmcinnes/.composer
Using version 0.9.* for pantheon-systems/cli
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for pantheon-systems/cli 0.9.* -> satisfiable by pantheon-systems/cli[0.9.0].
    - pantheon-systems/cli 0.9.0 requires katzgrau/klogger dev-master -> no matching package found.

Potential causes:
 - A typo in the package name
 - The package is not available in a stable-enough version according to your minimum-stability setting
   see <https://groups.google.com/d/topic/composer-dev/_g3ASeIFlrc/discussion> for more details.

Read <http://getcomposer.org/doc/articles/troubleshooting.md> for further common problems.

Installation failed, deleting ./composer.json.

Just got the same message.

Thanks for telling us, @gmcinnes and @kostajh! We're using a package that I have contributed some tweaks to, but has not been updated on Packagist to reflect those changes. I've pointed composer.json to its dev-master, which does not satisfy the default minimum-stability. There are three options to fix, in rough order of best to worst:

Ask the author of Katzgrau/Klogger to update the package on Packagist.
Drop the minimum-stability value to "dev".
Copy the logger file back into CLI.

I'll get on asking about that first one right away.

Go to Terminus root folder where both composer.lock and .json is located, and run "composer install". It will take care of everything.

This is how I updated terminus to version 0.9.1:
$ cd ~/.composer
$ vim composer.json (or using your favorite editor)
{
    "require": {
        "pantheon-systems/cli": "^0.9.1",
        "katzgrau/klogger": "dev-master"
    }
}

$ composer update

I believe I'm seeing the issue occuring in my circleci builds that are failing:
Using version ^0.9.3 for pantheon-systems/cli
./composer.json has been created
Loading composer repositories with package information
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for pantheon-systems/cli ^0.9.3 -> satisfiable by pantheon-systems/cli[0.9.3].
    - pantheon-systems/cli 0.9.3 requires katzgrau/klogger 1.2.*@stable -> no matching package found.

If this is related could this issue be revisited?


[BEAM-6628] Update GCP dependency versions.

This change updates the versions for GCP client artifacts. These
artifacts must be updated together since they share a number of common
dependencies.

Follow this checklist to help us incorporate your contribution quickly and easily:

[ ] Format the pull request title like [BEAM-XXX] Fixes bug in ApproximateQuantiles, where you replace BEAM-XXX with the appropriate JIRA issue, if applicable. This will automatically link the pull request to the issue.
[ ] If this contribution is large, please file an Apache Individual Contributor License Agreement.

It will help us expedite review of your Pull Request if you tag someone (e.g. @username) to look at it.

On Mon, Feb 11, 2019 at 9:07 AM Scott Wegner<EMAIL_ADDRESS>wrote:

@swegner approved this pull request.
Thanks or re-running tests. Everything looks good to me and tests are
green. LGTM
Is this ready to merge?
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/beam/pull/7783#pullrequestreview-202225797,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJ9Ma39uI4QGTVYs8jX4w_Sn3R2Fp-inks5vMaM0gaJpZM4ardH0
.


Add tests for short-lived connection between hosts and containers.

Fixes #448

@paulbellamy would you mind reviewing?

Fixes #546, it wasn't the k8s change.  Sorry Paul.

300 and 320 still fail for me locally. Also, seems like config.sh throws an error when jq fails:  ./config.sh: line 89: [: : integer expression expected
Test ./100_launch_test.sh needs 1 hosts
Test ./105_launch_sans_weave_test.sh needs 1 hosts
Test ./110_shutdown_test.sh needs 1 hosts
Test ./200_clustering_2_test.sh needs 2 hosts
Test ./205_clustering_sans_weave_2_test.sh needs 2 hosts
Test ./300_internet_edge_test.sh needs 1 hosts
Test ./310_container_to_container_edge_test.sh needs 1 hosts
Test ./320_container_edge_cross_host_2_test.sh needs 2 hosts
>>> Running ./100_launch_test.sh on [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
>>> Test ./100_launch_test.sh finished with success after 8.6 secs
>>> Running ./105_launch_sans_weave_test.sh on [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
>>> Test ./105_launch_sans_weave_test.sh finished with success after 5.7 secs
>>> Running ./110_shutdown_test.sh on [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
>>> Test ./110_shutdown_test.sh finished with success after 13.2 secs
>>> Running ./200_clustering_2_test.sh on [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
>>> Test ./200_clustering_2_test.sh finished with success after 37.0 secs
>>> Running ./205_clustering_sans_weave_2_test.sh on [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
>>> Test ./205_clustering_sans_weave_2_test.sh finished with success after 33.2 secs
>>> Running ./300_internet_edge_test.sh on [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
>>> Test ./300_internet_edge_test.sh finished after 128.9 secs with error: exit status 1
Test short lived connections from the Internet
06bd9c52bb661bdd4a11cdd0bf2d2e82085a0c5e7fc68b8c8a6ac856568567e3
Weave Scope is reachable at the following URL(s):
  * http://<IP_ADDRESS>:4040/
  * http://<IP_ADDRESS>:4040/
  * http://<IP_ADDRESS>:4040/
96177e3b3e804793fedbdc97c14aaff6f278866ab05eeb02382f3a8730b9796e
./config.sh: line 89: [: : integer expression expected
./config.sh: line 89: [: : integer expression expected
Failed to find containers nginx The Internet after 60 secs
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
jq: error: null and array cannot have their containment checked
Failed to fing edge The Internet -> nginx after 60 secs
/home/vagrant/src/github.com/weaveworks/weave/test/assert.sh: line 80: 23457 Terminated              do_connections
test #2 "curl -s http://<IP_ADDRESS>:4040/api/topology/containers?system=show | jq -r '[.nodes[] | select(.label_major == "The Internet")] | length'" failed:
	expected "1"
	got "0"
test #3 "curl -s http://<IP_ADDRESS>:4040/api/topology/containers?system=show |  jq -r '.nodes[""].adjacency | contains(["96177e3b3e804793fedbdc97c14aaff6f278866ab05eeb02382f3a8730b9796e"])'" failed:
	expected "true"
	got nothing
2 of 3 tests failed in 126.918s.

>>> Running ./310_container_to_container_edge_test.sh on [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
>>> Test ./310_container_to_container_edge_test.sh finished with success after 13.2 secs
>>> Running ./320_container_edge_cross_host_2_test.sh on [<IP_ADDRESS> <IP_ADDRESS> <IP_ADDRESS>]
>>> Test ./320_container_edge_cross_host_2_test.sh finished after 52.1 secs with error: exit status 1
Test short lived connections between containers on different hosts
548f843c0318a837f3f096416a3e72a9ad9f7c5efd3683ff86d59da841ce23b4
Weave Scope is reachable at the following URL(s):
  * http://<IP_ADDRESS>:4040/
  * http://<IP_ADDRESS>:4040/
  * http://<IP_ADDRESS>:4040/
5ab8401a3a664d0af492c5d3201a2ec580742592cbd0c5a24efd58e7967ecb19
Weave Scope is reachable at the following URL(s):
  * http://<IP_ADDRESS>:4040/
  * http://<IP_ADDRESS>:4040/
2897617a74ae3a0b563df825c88887da75c54adea2f6f2ee38d7fc55a95e9a21
4025e345d40ae19aa627fa53b24bddab70c16aa521d5923b86eb8a0bd89bbb02
Failed to fing edge client -> nginx after 10 secs
Found edge client -> nginx after 4 secs
test #3 "curl -s http://<IP_ADDRESS>:4040/api/topology/containers?system=show |  jq -r '.nodes["4025e345d40ae19aa627fa53b24bddab70c16aa521d5923b86eb8a0bd89bbb02"].adjacency | contains(["2897617a74ae3a0b563df825c88887da75c54adea2f6f2ee38d7fc55a95e9a21"])'" failed:
	expected "true"
	got "false"
1 of 6 tests failed in 50.872s.

>>> Ran 8 tests, 2 failed
>>> Fail ./300_internet_edge_test.sh
>>> Fail ./320_container_edge_cross_host_2_test.sh

Hmmm, thanks.
300 is never going to pass locally (no internet node) - any suggestions on what to do?  A way of detecting its not running on gce, and just skipping it?
320 looks like a legitimate failure, one scope had the edge, and the other didn't.  I'll increase the timeout (10s is a little short).
And I'll try and make the jq stuff a bit less verbose.

re: 300, any way to simulate an internet node? i.e. spoof the remote address of the connection?

re: 300, any way to simulate an internet node? i.e. spoof the remote address of the connection?

if you spoof it at the ip level, it won't connect (need to get packets back).  We could add code in scope to treat a given address as the internet, but I think thats going to far.  The test will run on circle.  Lets just skip it if run locally.  Unless you can think of another way...

I'm not really clear why there's no internet node locally? We're still doing Curl requests against the node from some other random IP (the box running the tests).

I'm not really clear why there's no internet node locally? We're still doing Curl requests against the node from some other random IP (the box running the tests).

The test (in scope) for if a node should be treated as the internet is iff the IP address is not on a local network (according to your local IPs and subnets).  For the vagrant setup, the connection comes from an address on one of the local subnets.

Would be nice if there were a cleaner way to detect GCE than grepping the hostname, but fine.
LGTM, then.


VID-5429 Vid is uploaded for the wrong date

Importing clips is now optimized
Behavior is the same as on all Photos, Galleries apps on Android
Tested with example app

I want to test the app now locally to this branch. I'll remove do not merge when I'm done.


Configuration v1/maintenance windows

Tests for Maintenance Windows
MonitoredEntityFilter
POST Maintenance Windows

@mark-dt can you sign the license/cla as well as commit with your email/username configured so that it shows up as being made by your account instead of "Your Name"


Typo

Hi, at least PNSR was a typo, but note reading from the graph, for Julia seemed less than 10 sec. I suppose should have been five. And if it was off by ten for some reason, ten also assume 2.6. Do not merge if I'm wrong.

FYI: there is from 2019:
An Improved WNNM Algorithm for Image Denoising
https://iopscience.iop.org/article/10.1088/1742-6596/1237/2/022037

The traditional weighted nuclear norm minimization (WNNM) has excellent performance for the removal of non-sparse noise such as Gaussian noise, but attains bad performance for the removing of salt&pepper noise [..]
This paper proposes an improved WNNM image denoising algorithm. [..]
The experimental data demonstrate that the improved WNNM algorithm has better denoising effect than WNNM algorithm.

I was going to point to your package at: https://discourse.julialang.org/t/is-there-julia-library-for-ai-image-denoising/75688
but first read up on it, see if it's "AI denoising". I see it was state-of-the-art at the time, in 2014, and while imported upon in 2019, it's not clear to me it's still best, nor "AI", even though that or neural networks aren't a goal in of itself. I'm now kind of curious how fast algorithm is (since it's supposed to work for video), or the alternative SOTA I point to (i.e. there under my nickname Palli) at the thread.
Also if I should know more about this algorithm I hadn't heard of, even if for some other applications, not just denoising:
https://arxiv.org/abs/1405.6012
https://arxiv.org/abs/1608.04517

In recent years, various applications based on
NNM have been developed, such as image/video denoising
[1, 4], background extraction [5, 6] and face shadow removal
[13, 14].

FYI: If you're still interested in denoising, I believe this is the current SOTA for it (also very intriguing to me for other reasons):
Simple Baselines for Image Restoration
https://arxiv.org/pdf/2204.04676.pdf

In this paper, we propose a simple baseline that exceeds the SOTA methods and is computationally
efficient. To further simplify the baseline, we reveal that the nonlinear
activation functions, e.g. Sigmoid, ReLU, GELU, Softmax, etc. are not
necessary: they could be replaced by multiplication or removed. Thus, we
derive a Nonlinear Activation Free Network, namely NAFNet, from the
baseline. SOTA results are achieved on various challenging benchmarks,
e.g. 33.69 dB PSNR on GoPro (for image deblurring), exceeding the previous SOTA 0.38 dB with only 8.4% of its computational costs; 40.30 dB PSNR on SIDD (for image denoising), exceeding the previous SOTA 0.28
dB with less than half of its computational costs.
Image restoration tasks aim to restore a degraded image (e.g. noisy, blur) to a
clean one. Recently, deep learning based methods[4,35,37,34,5,6,30,7,24] achieve
SOTA results on these tasks, and most of the methods could be viewed as variants of a classical solution, UNet[28].
3.5 Attention
Considering the recent popularity of the transformer in computer vision, its
attention mechanism is an unavoidable topic in the design of the internal structure of the block. [..]
The vanilla self-attention mechanism[32], which
is adopted by [11,3], [..] suffers from the quadratic computational complexity with the size of the feature map. Some image restoration tasks process data at high resolution which makes the vanilla self-attention not
practical. Alternatively, [21,20,34] apply self-attention only in a fix-sized local
window to alleviate the issue of increased computation. While it lacks global information. We do not take the window-based attention

It's not obvious to me that you need global information... noise is local, right, and it should be enough to split the image into many "fix-sized local" (maybe overlapping) windows and denoise them separately.

note reading from the graph, for Julia seemed less than 10 sec. I suppose should have been five. And if it was off by ten for some reason, ten also assume 2.6. Do not merge if I'm wrong.

Ha! I believe the 50s and 26s are the time without multi-threading enabled.

Simple Baselines for Image Restoration

I also noticed this paper and had planned to reproduce it when I get some time. Having a Julia AI-based network as a baseline is a definitely something I want for a long time.
My current focus is still on patch-based analysis, on which I agree with you, that I also believe local patches already give enough information to denoise well.

FWIW, the patch-based WNNM denoiser implemented here actually solves a small WNNM problem per pixel, so to denoise a 512x512 image, you'll have to solve 256^2 (say, stride=2) WNNM problems.
This fact makes it way sloooow to use it in practice.


[BUG] 1.14.4 Console Flooding

What happened?

Console Being Flooded

What should happen?

No Console Flooding, doesn't flood my 1.12.2 server's.

Steps to reproduce

Not sure.

Version

1.14

Mod version 0.0.0
Config file
-https://paste.dimdev.org/ukinutucem.bash

What is flooded in console - https://hastebin.com/cowuhutuwu.css

Additional info

Duplicate of #52


Disabled state

Example has disabled switch section.
But they act like a regular ones.

@alex-melnyk it seems it stopped working after changing state management approach

@vasilich6107 - version?

latest

I pulled from master to test an example

@alex-melnyk I made a PR with fix
Could you check if it is ok?

Great work, thanks! Latest version published.

Thanks for quick response!
Released wrapper for reative_rorms - https://pub.dev/packages/reactive_advanced_switch


Editor not triggering onChange event for Spacebar key input

Do you want to request a feature or report a bug?
Bug
What is the current behavior?
In certain contexts, an instance of the DraftEditor won't recognize spacebar input and trigger an onChange event. If I add the prop keyBindingFn and assign it a function to console.log the event, spacebar keypresses are calling the function properly and emitting the correct event with the correct key for a spacebar keypress. However, the onChange event is not being triggered and DraftEditorLeaf.text property is not being changed upon spacebar input. If the spacebar is pressed in rapid succession several times, and there are alphanumeric characters immediately preceding the current location of the cursor, a space will be inputted into the textbox.
If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem. Not consistently reproducible.
What is the expected behavior?
Pressing spacebar will insert a space into textbox at all times.
Which versions of Draft.js, and which browser / OS are affected by this issue? Did this work in previous versions of Draft.js?
0.11.7, Chrome, Safari, Mac OS

+1 on this issue.


So far, when i was creating sth new or adding classes. Everything was working.
However, now I'm focused on css and it's not loading when i'm doing changes. It seems that is always reloading the same fingerprint css. I thought that i was sth about rails, until I realised that even on changes on my JSX files are cached now and i cannot see changes unless i write new lines there. So i'm uploading the same JS as well from fingerprints.
Do u know a way to disable the cache mode on webpack??? has anyone found this error as well?
Thanks in advance!

@Ruffeng Did you ever precompile your assets?
http://stackoverflow.com/a/32107351/1009332
rake assets:clobber

@Ruffeng get in touch with me if you still have issues.

Hi @justin808 . Sorry for my late reply. I wasn't able to check it proper.
I used to deploy on heroku, so by default I have to precompile my assets to make it work ( I have to recognise that I'm using precompile and not clobber).
Finally I fixed migrating the project on a new one. I have no idea why all my project was like "frozen" in cache. I removed a lot of times the assets but it was still there.
Anyway, thanks Justin for your help :)
Ruffeng


[BUG] egg error when using argument D

Describe the bug
The egg function doesn't work if I use the D argument instead of R.
Code To Reproduce Bug
include <BOSL2/std.scad>
egg(78,d1=50,d2 = 25, D = 100);

Additional context
shapes2d.scad line 1475
D = get_radius(r1=R, d1=D)

should be changed to
R = get_radius(r1=R, d1=D)

am I right? ;)

You're sorta right.  There's a second problem.   I think it works in the PR.


dry-validation doesn't recognize integer in string

README has the following example:
key(:age) { |value| value.int? & value.gt?(18) }
...
errors = schema.call('email' => '', 'age' => '18').messages
...
#   :age => [["age must be greater than 18 (18 was given)", 18]]
...

As you can see 18 declared as string value but validation recognize it as integer value and validate it. Seems like this is don't work now (don't recognize integer value in string and raise error: age must be an integer).
Is it bug in validation or README?

Schema::Form tries to coerce values based on type expectations, so it does coercions prior validation.


Update Python web app tutorial to complete connection settings steps for Flask apps

This pull request includes a change to the articles/app-service/tutorial-python-postgresql-app.md file to add steps on how to configure application settings for deploying a Flask app in the Azure portal. Specifically, this PR adds the steps on how to add the SECRET_KEY required for enabling CSRFProtect when deploying the app.
This pull request would close https://github.com/MicrosoftDocs/azure-docs/issues/123961

@microsoft-github-policy-service agree

@msangapu-msft
Can you review the proposed changes?
Important: When the changes are ready for publication, adding a #sign-off comment is the best way to signal that the PR is ready for the review team to merge.
#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

@msangapu-msft
Can you review this old PR and determine whether it needs to be closed or merged?

I sent email to the content owner today.
@MicrosoftDocs/public-repo-pr-review-team

#sign-off

[like]  Mangesh Sangapu reacted to your message:

From: Tracey Torble @.>
Sent: Friday, December 6, 2024 9:38:15 AM
To: MicrosoftDocs/azure-docs @.>
Cc: Mangesh Sangapu @.>; Mention @.>
Subject: Re: [MicrosoftDocs/azure-docs] Update Python web app tutorial to complete connection settings steps for Flask apps (PR #123962)
I sent email to the content owner today.
@MicrosoftDocs/public-repo-pr-review-teamhttps://github.com/orgs/MicrosoftDocs/teams/public-repo-pr-review-team
—
Reply to this email directly, view it on GitHubhttps://github.com/MicrosoftDocs/azure-docs/pull/123962#issuecomment-2522653094, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AL3FF2G5Z2QXSDRRZ4OWQJL2EFWAPAVCNFSM6AAAAABL377GMGVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMRSGY2TGMBZGQ.
You are receiving this because you were mentioned.Message ID: @.***>

I checked these changes on staging in https://github.com/MicrosoftDocs/azure-docs-pr/pull/291621. Merging now.


Delays

Hello,
Quick question did anyone get this running in pretty much Real Time ? for me if I play a scene of abaout 300 seconds I get a delay of about 100 seconds. Is there a Setting to skip some frames to get rid of the delay or reduce it ?
Thanks for this amazing Project

It's probably hitting a CPU limit from what I've seen. When the sensor isn't moving (and thus the solver isn't having to do much), I can get it to just about keep up. But it quickly gets overwhelmed if it has to do much tracking. The reason it gets "behind" because it is buffering messages in a FIFO (like you are speculating), which doesn't seem like a good idea! Unfortunately, there's no setting to disable this, but I made some code changes to bypass the FIFO and it is able to "keep up" now (it still drops frames when the sensor is moving, but the odometry works fairly well). I am considering opening a PR with this change (and others), but I'm not sure if @Livox-SDK would be open to taking it.

Okay That sounds like what I expected. Do You have a fork which I can maybe try out ? @charlescochran

I ended up leaving this change out of my PR because it's rather messy (I'm just clearing out the lidar message queue instead of popping from it; a proper fix would be get rid of the queue entirely). That being said, you can replicate my quick-fix by applying this diff in PoseEstimation.cpp's process() function:
     if(!_lidarMsgQueue.empty()){
-      // get new lidar msg
-      time_curr_lidar = _lidarMsgQueue.front()->header.stamp.toSec();
-      pcl::fromROSMsg(*_lidarMsgQueue.front(), *laserCloudFullRes);
-      _lidarMsgQueue.pop();
+      // instead of queuing, just use the latest scan and throw away the old ones!
+      time_curr_lidar = _lidarMsgQueue.back()->header.stamp.toSec();
+      pcl::fromROSMsg(*_lidarMsgQueue.back(), *laserCloudFullRes);
+      _lidarMsgQueue = {};
       newfullCloud = true;
     }

That should be all you need to do.


getAllDeals is not returning the custom fields

json mapper in getAllDeals is not returning the custom fields
return CamelCaseHelper::keysToCamelCase($mapper->mapClass($response->body, 'Pipedrive\\Models\\GetDeals'));
Better use this
return CamelCaseHelper::keysToCamelCase($response->body);

Same with others DealsController's methods and other Controllers.


Refactor LayoutSidenav_nav

🚀 Feature Request
Refactor the navigation in the side column to a react sub-component. For example right now the Site and the Help drop down navigation are almost identical except for the links and texts, changes in the appearance of one would not be shown in the other.

can I work on this?  if I understand the question you require another styling for either the Help menu?

Hey @ayobamiseun, I'm glad you're interested in contributing. let me get back to you on November 7th, I need to check that we can accept contributions from outside the EPA without some sort of an agreement.

I just try to run the client on my local. I was asked for a password. or how am I supposed to run it so I can start working on it ASAP?

@ayobamiseun you're one of the first outside our office looking to contribute, thanks in advance for your patience while we iron out the kinks. I'm realizing now there's not a good way to run the client without also running the server.
I'm open to suggestions if you have a ideas on ways we could improve this. This short conversation has already made me realize I need a better way to allow new contributors to get up and running more smoothly.
The easiest way to get up and running is with a dockerized back end.
If you can follow these below commands, from the project root, they should get you up and running.

$ cp server/.env.example server/.env && cp client/.env.example client/.env
comment out the Database configs in server/.env

### Database Configs
# HT_DB_ENGINE=django.db.backends.postgresql_psycopg2
# default: django.db.backends.sqlite3

# HT_DB_NAME=my_database_name
# HT_DB_USER=database_username
# HT_DB_PASSWORD=MyP@ssw0rd
# HT_DB_HOST=<IP_ADDRESS>
# HT_DB_PORT=5432

You will need python on your system as well to use the runhaz.sh shell script
$ ./runhaz -m && ./runhaz -l
bring up the back end services with docker compose
$ docker-compose up server redis celery
start the client,
you should be able to login with the below credentials which where loaded into the db.sqlite3 database with the runhaz.sh shell script commands.

username: "testuser1"
password: "password1"

I wonder if it would just be easier to include the db.sqlite3 database for development purposes. If we did that, then that long list of commands (which I'm realizing may still throw errors since it doesn't mention the virtual environment) would be reduced significantly.
I will need to return to this tomorrow.
Again, thanks for your patience @ayobamiseun

@dpgraham4401  It will be better if we can set up the sqlite3 just for development purposes or develop the client side independently cos I am not comfortable working with python 😃

@dpgraham4401 It will be better if we can set up the sqlite3 just for development purposes or develop the client side independently cos I am not comfortable working with python 😃

I agree, it would be great to have a way to work on parts of the react app that don't depend on the back end independently.
I'll do some research, I suspect we doing something with docker will be the best option.

@ayobamiseun  I've mainlined docker-compose.dev.yaml and updated the docs on how to use it. I need to go through and wordsmith the documentation but let me know if you have any problems.

I am sorry I have not reverted back to you, I have been trying to deploy the project on my local machine I feel I will just have to learn docker At this point.

@ayobamiseun, no problem. Let me know if you'd like to take another crack at this.
If you find the documentation lacking in that area or you're experiencing an error you think could be fixed by us, please open an issue.

docker-compose --env-file configs/.env.dev -f docker-compose.yaml -f docker-compose.dev.yaml up --build
can you like explain what I am using the docker composer for I already have docker installed on my machine.

I apologize in advance if this over explains things, I'm never sure where to start, it's a real problem of mine.
docker-compose if just a way to simplify the process of starting up an application that depends on multiple containerized 'components'. Otherwise, you'd have to do something like open up four different terminals, one for each component, the start each container with a long list of flags. You could do the same thing with bash scripts but that gets kinda messy.
To quote the docker-compose overview

compose is a tool for defining and running multi-container Docker applications.

So, to oversimplify, our docker-compose does a couple things...

builds or pulls docker images for the parts of haztrak (redis, celery, the django RESTful API, and the react front end). So you don't even need python or node.js installed on your computer to run these, docker 'packages' everything that's needed to run haztrak in these containers.
starts the containers and feeds environment variables from the specified --env-file
The .dev.yaml file overrides values in the docker-compose.yaml specific to creating a local development environment (i.e., hot reloading so we don't have to restart the containers every time we make a change). The regular docker-compose.yaml file setups something closer to production environment.

Did you read the documentation on this and, if so, is there something we can add to clarify things?

ERROR: The Compose file './docker-compose.yaml' is invalid because:
services.client.environment.CHOKIDAR_USEPOLLING contains true, which is an invalid type, it should be a string, number, or a null

I got the above error when I tried to run docker-compose up

@ayobamiseun are you still intersted in working on this? should I assign you again?

I am thinking of deleting my old fork cause it has a lot of unmerged changes, what do feel I should do that won't have anything to do with the old changes?

@ayobamiseun sorry just saw this comment, must have slipped through. Just pull the unmerged changes before submitted the PR.


Stop server from creating main on startup

if you don't use main then you need to run /usr/bin/session-open-group-server --delete-room main to stop it from existing

Fixed in #29 / 0.1.10 release.


Enable DB2/z support for modifyDataType tag (Issue#3639)

Impact

[ ] Bug fix (non-breaking change which fixes expected existing functionality)
[ ] Enhancement/New feature (adds functionality without impacting existing logic)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Description
Enabling DB2/z support for modifyDataType tag. Please see details in #3639
Things to worry about

Not sure what was the original reason why it was not allowed. So this original reason might to be considered in this fix too.

Initial/Pre-Review Thoughts
Just impacts db2/z. I've always just relied on the community saying what db2/z can do, so I don't know why it was excluded before either :)
Questions I have:

None

Potential risks:

None

What could make the full review difficult:

Nothing

I converted the pull request to draft, because although the ALTER TABLE works on DB2/zOS, calling REORG TABLE is not supported. The closest command is REORG TABLESPACE. The proper solutions is under analysis.

Thanks for advising us @DTomoo . Please let us know when it's done then we can review it again :+1:

Have you gotten a chance to look into this more, @DTomoo ?

@DTomoo - checking back with you to see if you have any updates on this PR. Thanks!

Hi @DTomoo,
Are you keen to move forward with this PR? If so, could you please resolve the existing merge conflicts and move it back to Open state?
Thanks,
Daniel.


Need to replace DimTuple

I'm trying to update the iree-amd-aie project with the latest version of mlir-aie. That seems to have removed or renamed the DimTuple attr which used to be defined here https://github.com/Xilinx/mlir-aie/blob/c0341aa3d525827a21f25b7423b18f4359a34cf3/include/aie/Dialect/AIE/IR/AIEAttrs.td but is still used in this project here:
https://github.com/Xilinx/mlir-air/blob/3ae51efcb11e9fb5504fa5851b54c0b87ea0f704/mlir/lib/Conversion/AIRToAIEPass.cpp#L2288
https://github.com/Xilinx/mlir-air/blob/3ae51efcb11e9fb5504fa5851b54c0b87ea0f704/mlir/lib/Conversion/AIRToAIESchedulingUtils.cpp#L175

DimTupleAttr has been removed in https://github.com/Xilinx/mlir-air/pull/383 which has been merged.


Not playing well with 'dictionary-it' (2)

Hi, thanks for the wonderful library. The Italian (also Portuguese) languages hang my app that uses nspell. Is there anything new on that front (there is a closed issue on Italian dic). Will that problems be fixed in the future? Is there a workaround?
Thanks,
i.

There’s no news. The Italian dictionary/affix combination results in a huge list of possibilities, and I don’t know Italian so I can’t check if changing the code would still properly spellcheck it!
If you know Portuguese or Italian, we could start off where #9 ended?

Thanks for your quick answer! Unfortunately not. All languages that I speak work very well. Then again, you can find some Italian text on the web and you can see if spelling works as intended ...
I also tried this package:
https://www.npmjs.com/package/typo-js
and it has similar problems with same languages. Maybe vanilla js approach just won't cut it for those dictionaries :(  Will investigate further ...
i.

dictionary-pt-br has the same issue. :disappointed:
It leaks to an insane 1,4GB of RAM at runtime.
It worth checking Hunspell's official algorithm/implementation as it never runs above 50 MBs of RAM while checking against the same dict and process lots of words very fast. I'm currently trying to understand how it works, but sadly can't promise I will come back with a solution soon.

It leaks to an insane 1,4GB of RAM at runtime.

I’d appreciate a fix! PRs welcome!

It worth checking Hunspell's official algorithm/implementation as it never runs above 50 MBs of RAM while checking against the same dict and process lots of words very fast. I'm currently trying to understand how it works, but sadly can't promise I will come back with a solution soon.

We/I can’t, because that would taint this project as being a derivative work of GPL code. nspell is reverse-engineered.

We/I can’t, because that would taint this project as being a derivative work of GPL code. nspell is reverse-engineered.

Switching to GPL wouldn't be a problem for my usage, but I understand your concerns.
However, I found that Myspell (from which Hunspell is based on) is licensed permissively and is probably the base for the comparing algorithm. As I didn't looked into Hunspell's source yet, I will try checking out Myspell's one first, in order to keep any fork/PR compatible with the MIT license (in case I get any useful insight).

It should also be noted that hunspell-spellchecker has the same issue (excessive memory consumption with the use of the IT dictionary).

Thanks Fabio, appreciate that!

I've taken a closer look at this, with the recent performance improvements I can get nspell to stop hanging after a good but manageable amount of seconds, until eventually it throws this error for me:
RangeError: Value undefined out of range for undefined options property undefined

Which according to this SO answer means something ran our of keys available, and according to this v8 engineer's SO answer Maps have an hard-coded 2^24 max number of keys possible.
Basically as I'm understanding it the Italian dictionary is generating so many combination that the engine becomes incapable of handling them.
This should be fixable by using a pool of Map objects internally rather than a single one. It would be useful to measure how many actual keys the Italian dictionary needs though, if instead of 16M it's 16B or something then the whole effort would be pointless.

Alternative idea: the source of the dictionary hasn’t been updated in years. Do any of you Italian-speaking folk know if there are alternative groups making Italian dictionaries?
Or is there a cap we can put on this expression to not generate a bazillion entries?

Thanks for digging into it, @fabiospampinato!

Or is there a cap we can put on this expression to not generate a bazillion entries?

I think yes: I mean, why don't we try to simply read in all contents of the .dic-file, and we already have the flags behind it, so why do we need to resolve these flags immediately? Wouldn't lazy-loading them be easy? I am currently thinking of the following (please tell me if I forget something that prevents this):

The files are loaded, but this "loading" means "Save them to an internal line array"
Once a word is looked up (e.g. "giorno", to stick with the italian example), we could take all possible candidates from the line array (we can be generous with this, because some words may have modifications as a prefix, the goal should only be to not parse all entries) and resolve them until we find a match. If the word is written wrongly, we won't get a match, but the upper cap for this would be once all candidates are resolved.
Then save those resolved words in an internal map (maybe even distinguished by first letters, so that we have, e.g., 26 maps from A to Z containing all resolved words)

This would increase the computational requirements a little bit at the beginning, but this might first reduce the insane times we currently achieve even with English dictionaries during load, and enable loading any DIC file.
Any thoughts? What obvious thing did I overlook?
And thanks again for caring :)

Alternative idea: the source of the dictionary hasn’t been updated in years. Do any of you Italian-speaking folk know if there are alternative groups making Italian dictionaries?

I don't know anything about that.

Or is there a cap we can put on this expression to not generate a bazillion entries?

Doable, but ideally one wouldn't want the spell checker to understand 50% of a language and ignore the remaining 50% or something. That being said I think I got exactly 10 of those errors reported in my console, I don't know if the engine just gave up or if maybe we are only going like 0.00006% over the limit, which would be interesting.
In any case using multiple maps seems like a better option to me, as with a cap on the number of words we'll get nspell not recognizing some words or having a hard limit on the number of custom words that it can handle.

but this might first reduce the insane times we currently achieve even with English dictionaries during load

@nathanlesage with the recent performance optimizations I submitted I'm getting the English dictionary loaded in my ~6yo laptop in ~150ms or so, I think that's pretty good already, if you are running nspell in a worker it doesn't really matter much if it takes 0ms or 150ms to load a dictionary.

Any thoughts? What obvious thing did I overlook?

I'm not very familiar with the details of generating a dictionary currently, I think potentially something like that could be doable, but finding which words to lazily evaluate without parsing the entire thing sounds difficult to me, and it may backfire performance-wise, like performing that computation may pretty quickly take longer than just parsing the entire thing.
IMO further performance optimizations routes might be:

Using character scanning rather than regexes for rules, this may reduce init time by ~15% or so, or not.
Using a bloom filter or similar structure rather than asking the Map if a word is already present in it, as for example ~93% of the times for the french dictionary we are dealing with new words, maybe this could reduce init time by another ~20% or so, but I've never used bloom filters before so maybe I'm talking nonsense.
Pre-parsing dictionaries, this could potentially remove most of the time required for initialization but the problem is things like the french dictionary become a 70MB file from a ~2MB raw dictionary file, so shipping that with the app or downloading that is not really feasible.
Parallelizing initialization so that for example 4 worker threads might work on their own chunk of the dictionary, this could be doable and would reduce init time by an amount proportional to the number of workers and actual cores available, but chunking the dictionary is not trivial because some words can be added as prefix/suffix to other words for example.
Ordering dictionaries by word usage, so like nspell could first load the first 10000 words and if mispelled words are found, requiring some suggestions, the next 10k words could be loaded, and then the next ones etc. until some suggestions are found. As a negative side effect this could reduce the number of suggestions provided, but maybe that would be a good enough trade off for some languages. But then again partially parsing dictionaries like that is not trivial.
Maybe memory usage could be reduced significantly still by not joining words together, like if instead of storing "buongiorno" nspell stored "buon" and "giorno" in an array or something I think v8 could allocate those strings only once in memory for the entire dictionary, saving a good deal of memory for languages with lots of computed words, but then everything internally that currently works on strings would need to be rewritten to work on these kind of disjoined strings, which wouldn't be easy probably.

Folks, I think performance is an XY problem. There are two variables here: a) dictionaries, b) hunspell vs. nspell. Most dictionaries work fine with nspell. Some don’t. As mentioned before, Hunspell does okay on Italian. There is a bug in the code here which causes lots of words, which I don’t believe need to be added (but I don’t speak Italian and hence can’t help here), to be added. The solution is to look at what is going on in the code that shouldn’t be going on. Perhaps it’s depending on a feature that isn’t supported here?

@wooorm are you certain there's a bug generating unnecessary words? 16M+ words sounds like a lot to me but given how lots of words are spelled different in italian depending on their gender or pluralization I can see how a much greater number of words is generated for italian compared to english. I'll skim the generated list of words, maybe I can spot some seemingly problematic ones.

I thought that that was the problem, because (excuse me if I'm off) roughly related languages like Spanish or so don't have this problem.
Dictionaries are written by authors who sometimes aren't that fluent in programming. Some include bugs. I'm under the impression that this one isn't written well 🤷♂ and that maybe hunspell is capping certain ginormous expansions whereas nspell isn't

I think I have some new interesting findings to share:

Generating way fewer words:

I've taken a closer look at the italian dictionary, and it generates about 30 million words, exceeding the ~16M limit significantly.
The number of words generated by the italian dictionary explodes because italian has these silly prefixes that can be attached to words, for example in english one might say "that tree", in italian "that" can be translated as "quello"/"quella" when referring to a noun in the singular form, but it's common to truncate the trailing vowel and attach the remaining characters to the noun, so in italian one would translate that english bit with "quell'albero" rather than "quello albero". Another major issue is that words in italian, but I guess in all languages with a latin alphabet really, can start with an uppercase character if they are at the start of a sentence, that helps making the list of words generated by nspell explode because now in the list you'll have: albero, Albero, quell'albero, quell'Albero, Quell'alberoandQuell'Albero`.

Potential fix number 1: I think it would be totally reasonable to consider those prefixes as their own words, which they are grammatically technically, and instead just having "quell'", "Quell'", "albero" and "Albero" in the dictionary.
Potential fix number 2: Duplicating the number of words just for accounting for the case where "albero" is at the start of a sentence sounds super wasteful to me, there shouldn't be "albero" and "Albero" in the dictionary really, but just "albero", I think we can probably account for this dynamically when computing suggestions and checking for correctness.
Potential fix number 3: "Quell'Albero" is not actually correct spelling because "albero" is not a name like "Fabio" or "Italy", unless somebody explicitly gives something the "Albero" name for some reason, so it shouldn't be present in the final list of words at all.
I think all these potential fixes together could cut down on the number of words generated, and by consequence CPU and RAM required, by up to ~80%.
French also has a similar issue and that's why its dictionary contains millions of words too.

Mind-blowing unbelievably amazing truly outstanding performance:

Computing the list of words dynamically is where most of the CPU time is spent when loading a dictionary, and most of the RAM is spent just storing this huge list of words which contains largely redundant information (i.e. it compresses to a high degree), apparently it's possible to cut down on CPU time almost entirely by pre-parsing dictionaries, and cut down on RAM usage massively by storing the list of words in a fancy ~compressed data structure that doesn't need to be decompressed before being used, more details on how the data structure works here.
With the optimization mentioned above I can get the English dictionary to load in ~2ms in my machine, rather than the current ~150ms, and, unbelievably, the gzipped precomputed list of words weighs less than the gzipped raw dictionary used to generate it in the first place, so it would take less time to download it too.
With the same optimization I can get the French dictionary to load in ~20ms in my machine, rather than ~4s, milliseconds not seconds, and the ~compressed list of words weighs just 1.2MB, which doesn't require nearly as much RAM to keep in memory (currently instead we are at ~240MB for the french dictionary) which goes down to ~800kb when gzipped, and with the potential reduction on the number of words generated these figures would go down too.
Also if for some reason a language legitimately requires 50 million words to be stored I think as a side effect of how the data structure required to generate this ~compressed list of words works we probably won't be hitting the hard-coded 16M keys limit anyway.
Where's the catch?

Generating the ~compressed list of words is expensive, it took half an hour for the french dictionary on my machine, but that must be done only once.
If the way lists of words are generated changes, i.e. if nspell gets better, the ~compressed lists of words should be calculated again.
The ~compressed list of words can't be mutated dynamically, meaning adding and removing words from it must be implemented on top of that, but it should be doable I think.
There aren't many libraries that I could find that could generate the "succinct", as they call them, data structure that we need, none of them widely used, so the available ones might be buggy and difficult to fix.
It might be difficult to extend these libraries to work with arbitrary alphabets (i.e. they might not currently support storing accented words for example, or vietnamese characters or cyrillic characters).

I think first of all we should get those optimizations I mentioned in to cut down massively on the number of words generated, it would be great if @wooorm could take care of that as I'm not nearly as familiar with the library, that would bring down the time required to load even problematic dictionaries like the french or italian ones under 1s, which we can work with. Secondly I, or some other volunteer, will spend some time attempting to achieve that truly "final" mind-blowing optimization. Thirdly we will all share on Twitter our happiness for the outstanding performance achieved, especially in the face of the Rust-all-the-things freaks.
Thoughts?

This sounds amazing! Thank you for digging into it!
If the complete (!) list of computed words (without affixes, if I understand you correctly? Just the words we can search for?) is THAT useable, we could ditch hunspell completely, and just write a small program that generates these compressed lists from hunspell dictionaries. We can leverage Github Actions for that.
Then we would just need mechanisms to load them and do searches in these dictionaries. Then users could download our generated dictionaries and need to load them using nspell.
With regard to the generating program -- if you tell me how you did it, I could volunteer to write such a generator in Rust, so a faster system programming language. We could then run it via push commits, to keep the lists up to date.
This would split up the work and we would be able to implement this faster? @wooorm could then provide guidance with regard to the algorithms.
What do you think?

without affixes, if I understand you correctly?

I think affixes are used during word generation, so affixes should already be taken into account when serializing parsed dictionaries, the affixes themselves are not included in the serialized dictionaries but they weigh next to nothing compared to the actual words generated (just 24kb zipped for the French one, I think the English one is 1kb or something) so they can just be downloaded too with no significant performance implications essentially.

If the complete (!) list of computed words is THAT useable

It's truly amazing, the magic data structure allows for fetching efficiently all possible ways to complete a word too, fetching known possible suffixes essentially, it might allow for fetching known prefixes too, I'm not sure.

we could ditch hunspell completely, and just write a small program that generates these compressed lists from hunspell dictionaries. We can leverage Github Actions for that.

We need something to generate the final list of words (nspell), something to pack that into the magic data structure (wizardry), something that can work directly with that packed data structure (dark wizardry), finally something implemented on top of that that allows for adding and removing words (a future version of nspell maybe, I guess it depends on which path @wooorm wants the project to take).

Then we would just need mechanisms to load them and do searches in these dictionaries.

That's all included in the magic library I tried earlier already.

With regard to the generating program -- if you tell me how you did it, I could volunteer to write such a generator in Rust, so a faster system programming language. We could then run it via push commits, to keep the lists up to date.

All the magic comes from this library: https://github.com/mckoss/dawg, computer science boys.
I wouldn't bother rewriting that in Rust honestly, it takes seconds to generate a packed dictionary for English, we can probably get it to generate packed dictionaries for any languages under 1m with some optimizations both to it and nspell.
If you want to volunteer some time you should probably look into how to make that library work with arbitrary unicode words rather than just ASCII ones. Other than that there doesn't seem to be really much more work necessary I think.

This would split up the work and we would be able to implement this faster? @wooorm could then provide guidance with regard to the algorithms.

I don't currently have too much time to dedicate to this, but I absolutely want to get this done eventually because the resulting performance is incredible. So I don't know, maybe ideally @wooorm could update nspell to generate less words, @nathanlesage could patch the succinct DAWG implementation for our needs, and I could write the wrapper library that ties everything together (meaning patching nspell on writing something else)?
I would probably also want to look into using hunspell directly for words generation as well.
Also some words have some "codes" associated with them in nspell, I'm not too sure what's the deal with them, we probably shouldn't just throw them away, and we can't store them in the DAWG itself either.

Aaaaah a tree (graph) — I've dealt with these during my last project, and as I will probably be using them in my PhD, I could have it count as work time porting that thing to a system language and splitting up the work into two libraries? But we'd probably use the TS DAWG library for nspell to use them as well.
In any way, while the Math behind that looks cool, it's actually not THAT difficult to implement (we could draw on a lot of arXiv papers as the algorithm is already there). This might give us the freedom to also add non-ascii support (ergo increasing the amount of graph starting points from 26 to, idk, 128 or so)
We just need to know how the data structure of the library looks so that we can adapt the output. (Btw the reason for me insisting on a non-js program is simply because it's easier to have a command line program running on a GitHub worker, it could lift work for us to do in the future. And, I don't like utilizing JavaScript for Computational-heavy tasks due to its runtime overhead, but maybe I'm just the stereotypical German in this respect)

@nathanlesage It's not just the graph, it's the combination of the graph and its compact string representation that doesn't really need to be decompressed that gives the amazing performance numbers I reported, if you didn't bother with the succinct string representation and instead required decompressing the dictionary and rebuilding the graph I don't think you will see nearly as much performance improvements, there's a good chance it may backfire even, ~2s of the ~3.3s that nspell currently takes to parse the french dictionary it's just adding words to a JS Map, I really don't think you can just do that 100x faster, you need to work with the ~compressed graph directly.
I think going with TS would be a better choice, mainly because we have a reference implementation already that looks well written well tested and well commented, and we need a JS version at least for working with the string anyway because we need this to work in the browser.
But feel free to reimplement the packing part of it in Rust or whatever language you like, as long as we can get this thing to work I don't care if portions of the succinct DAWG library are written in Rust.

save as tarball/zip (?)
Read the tarball

The plan looks good but we can't afford zipping the graph or any other form of string serialization that requires decompressing before using, the performance gains come from using the string representation directly.

Also an implementation of the DAWG that we can actually use in the browser would allow for storing custom words to add and remove from the dictionary in a memory efficient way potentially, so I don't really see any inherent advantages of using Rust other than potentially saving a few minutes once when generating these graphs.

Ah so you were just talking about the string representation itself - apologies. I'll have a look at that library myself and report back!

Potentially a Rust implementation could be useful in the future to traverse the graph in parallel using SIMD instructions in WebAssembly, but like SIMD instructions aren't here yet, it would probably be difficult to implement that on top of the string representation, and checks for presence in the graph seem instant to me anyway so any optimization in that regard may be pointless.

hunspell's unmunch command can dump all known words from a dictionary, but I think it doesn't work with non-ASCII languages.
Interestingly it seems to generate roughly the same number of words as nspell, so nspell may reimplement enough of it to be able to generate the words list we need.
hunspell's interpretation of the italian dictionary also has the same issue I mentioned with the prefixes, it generates 36 million words, which together make a ~650MB file, which zipped goes down to ~90MB, it'd be interesting to see what the gzipped version of the packed DAWG for this words list is, but that would take hours to compute on my machine, most probably it's going to be unreasonably large, those problematic prefixes need to be handled specially.

It just finished generating the succinct DAWG for the first 6 million words in the italian dictionary, it ~compressed 100MB into 60kb, which screamed "that's buggy for sure" to me, but then again opening the text file containing those 6 million words everything looks kind of the same to me, it's a mess of different prefixes that blow up the combinatorics, plus the DAWG library currently is case insensitive so it deduplicated probably something like 75% of those words anyway.
That's to say that that DAWG library looks really like a piece of magic, and I guess perhaps having lists of words in the hundreds of megabytes might not be that big a deal actually.

Hi everyone, I am following this issue with great interest because I would like to use the spellchecking functionality using only javascript.
I'm the author of bibisco , a desktop app made in Electron.
Currently, I am implementing the spellchecking features using electron-spellchecker which however uses native modules, which from Electron version 9 are deprecated.
I can't use the spellchecker shipped with Electron, because it doesn't allow you to specify the language to use on Mac.
I find the @fabiospampinato's idea of having an exploded vocabulary of terms optimized using DAWG very interesting. The same idea is also implemented by the simple-spellchecker library without DAWG optimization.
However, this solution presents the problem of suggestions; how would they be implemented? In the simple-spellchecker library, a binary search with the damerau-levenshtein distance is used.
Some other notes:

I'm Italian, so I am available in case of linguistic doubts.
The hunspell's unmunch command works well also on UTF-8, but the command execution environment must support UTF-8. I was able to generate Russian and Czech vocabulary.
The hunspell's unmunch command has performance problems with some vocabularies; I was unable to generate Turkish vocabulary. Any suggestions?

However, this solution presents the problem of suggestions; how would they be implemented? In the simple-spellchecker library, a binary search with the damerau-levenshtein distance is used.

You could convert the succinct DAWG into whatever tree structure simple-spellchecker is using, so this issue doesn't really exist as you can map one problem onto the other and at least one of the two is solved in your mind.
In practice though it wouldn't make sense not to use the succinct DAWG directly as that's where the magic is. In the context of the DAWG searching for suffixes means walking down the graph listing all paths to all terminal nodes (fast), searching for prefixes means walking up the graph and listing all paths leading to the staring node (it could be made fast too but requires some preprocessing to kind of invert the graph), searching for spelling suggestions in general could mean exploring the space of strings at reasonable edit distances from the query string and checking for presence in the graph (i.e. walking down the graph in search for a terminal node associated with the edited string in question, which is fast enough but probably slower than binary search), but we could also look for suggestions potentially more efficiently by walking the graph directly as we know already which paths lead to valid words (I think this could potentially be faster than naively computing edit distances and binary search on the whole list).

The hunspell's unmunch command works well also on UTF-8, but the command execution environment must support UTF-8. I was able to generate Russian and Czech vocabulary.

There's an issue on hunspell's repo about implementing another command for dumping all words, I don't think unmunch per se is slow, it extracted half a gigabyte of words in seconds in my machine from the italian dictionary.

The hunspell's unmunch command has performance problems with some vocabularies; I was unable to generate Turkish vocabulary. Any suggestions?

You can attempt to extra the words via nspell or some other somewhat compatible library.

searching for prefixes means walking up the graph and listing all paths leading to the staring node (it could be made fast too but requires some preprocessing to kind of invert the graph)

Actually, not a problem: the search algorithms on acyclic graphs work fast in both directions. I read a few things yesterday night and I think it's really fairly simple to generate and traverse such trees.
But we're still lacking a fast file format for these trees, as I still think it's wasted CPU time to recompile the tree everytime we load the app (plus the additional disk IO) -- even transistors have a limited lifetime and I'd like to efficientiate the hell out of this very, very good idea.
As for the mapping; pretty easy: as far as I know distance algorithms can give you some alternatives, and we can then traverse the tree yielding the necessary words using topological search.
An additional idea, which does not have any priority, but nevertheless: We could add a simple form of machine learning, as one instance will only be run on one computer by one user, so we could save wrongly typed words and their replacements in a hashmap, which can be offloaded and loaded using a simple function call, which can then speed up spell checking even more, if we, e.g. limit suggestions to 10 max or smth like that.

We could add a simple form of machine learning, as one instance will only be run on one computer by one user, so we could save wrongly typed words and their replacements in a hashmap, which can be offloaded and loaded using a simple function call, which can then speed up spell checking even more, if we, e.g. limit suggestions to 10 max or smth like that.

I'm not sure what you are suggesting here, sharing a single spell-checker instance across processes? It would be somewhat interesting but you can't do that in the browser so I personally have no interest in that. Treating commonly spelled/mispelled words specially for faster suggestions? Sounds like a good idea but the performance improvements may only be marginal (i.e. not like making a seconds thing a milliseconds one).
IMHO the next potential steps if somebody would like to strive for spell-checking perfection would be:

Adding another layer of checks in the form of some neural nets trained to understand grammar and potentially higher-level features too (words repeated too many times, plagiarism, good writing in general etc.), which would be too difficult to instead manually code, especially for languages we don't even understand.
Making plugins for common editors and saying "look, it's like Grammarly but you don't need to send us everything you type, what do you say?".
Having a high-accuracy model for detecting the language used, enabling reliable automatic multi-dictionary/multi-grammar checks.
Rebuilding all the dictionaries taking into account the available written corpus of text for each language, which means potentially supporting every language on the planet too.

That's a bit too far ahead of our current state though, let's get the thing to start up in reasonable times first 😄

Adding another layer of checks in the form of some neural nets trained to understand grammar and potentially higher-level features too

You don't need neural nets for that. You can solve this much faster with trees as well.

Having a high-accuracy model for detecting the language used, enabling reliable automatic multi-dictionary/multi-grammar checks.

Nice but I think this overshoots by a wide margin. Plus this will increase the computational burden for many users by far. Neural nets are not end user playtoys, I'm afraid :/

I'm not sure what you are suggesting here, sharing a single spell-checker instance across processes? It would be somewhat interesting but you can't do that in the browser so I personally have no interest in that.

No, just aomething like a per-user cache. Would help make these suggestions the top-ones, and users may thank us for that. Plus, while I understand that you want to focus on your own needs, I would suggest we should keep this library open for many people. Plus, as Notable is Electron-based, you could make use of that yourself ;)
But you are right, we should first focus on making nspell as it is more usable!

You don't need neural nets for that. You can solve this much faster with trees as well.

You don't strictly need neural nets for anything, but how else is someone going to come up with those tree for russian, hebrew and tens of other languages? 🤔

Nice but I think this overshoots by a wide margin. Plus this will increase the computational burden for many users by far. Neural nets are not end user playtoys, I'm afraid :/

I think eventually that would become a viable strategy, if it actually isn't already in some cases, I'm currently using @wooorm franc for document-level language detection and I'd be surprised if one can't get a model of comparable size even just for a handful of languages (perhaps asking the user which languages he/she speaks or something) with much higher accuracy and comparable performance, plus eventually I'd imagine most laptops and smartphones will have some form of accelerators for doing those operations, maybe GPUs are already good enough for this.

Plus, while I understand that you want to focus on your own needs, I would suggest we should keep this library open for many people.

Sure, I didn't mean to imply that some ideas are inherently bad if I won't make use of them.

You don't strictly need neural nets for anything, but how else is someone going to come up with those trees for russian, hebrew and tens of other languages? 🤔

Well, easy. You have a fixed set of characters that are possible, let's take a Japanese example:
こんにちは, こんばんは, and ここ can be represented in this graph:
こ --> ん --> に --> ち --> は --> [EOW]
|     |
|     +--> ば --> ん --> は --> [EOW]
|
+---> こ --> [EOW]

No need for any neural net here. You only need a list of all words ;)

plus eventually I'd imagine most laptops and smartphones will have some form of accelerators for doing those operations, maybe regular GPUs are already good enough for this.

No, the trend actually goes towards having neural nets on remote servers, and using smartphones more and more only as GUIs towards SaaS. There's no neural net (and no model) running locally on almost all computers. But there's a lot of hype as many services advertise using "Artificial Intelligence", but most of it is just hot air without any actual neural net behind the curtains. Don't let yourself be fooled by that ;D

Well, easy. You have a fixed set of characters that are possible, let's take a Japanese example:

We might be referring to two different things, the graph you made looks to me like what the DAWG for those 3 japanese words would look like (if that は character at the end of two different words got deduplicated too), but I don't see that kind of graph telling me anything about grammar, i.e. I really don't think you can make something that understands how sentences work on top of a graph representation of a vocabulary.

No, the trend actually goes towards having neural nets on remote servers, and using smartphones more and more only as GUIs towards SaaS. There's no neural net (and no model) running locally on almost all computers. But there's a lot of hype as many services advertise using "Artificial Intelligence", but most of it is just hot air without any actual neural net behind the curtains. Don't let yourself be fooled by that ;D

iPhones and iPads at the very least, of which there are probably hundreds of millions around, have been shipping with dedicated ML circuitry since a few years now, so if anything some devices are going to have ML accelerators because some of them have them already, whether most Android smartphones will follow suit or not I don't have a crystal ball to say for certain.
To get another data point I just tried a node module for google's cld (Compact Language Detector), node-cld, and it gives me a result using the sample snippet listed in the project's readme in about half a millisecond, I think that's something we can work with  already. And my machine it's a 6yo laptop with, to the best of my knowledge, no dedicated circuitry for the kind of operations these models perform, I don't even have an external GPU, not that that model is probably running in the GPU anyway. And the entire model weighs ~6MB, for 160 languages, that's kind of a lot but let's not forget that to this day I'm still wasting ~200MB storing french words inefficiently in RAM.
So I'm pretty sure that can work already today for some devices without any particular trick, and eventually computers will get faster, will have more ram, probably dedicated ML accelerators too in my opinion, and we don't actually need a single model that can understand 160 languages, for which we don't even have dictionaries for.
But hey, I'm not arguing for you to add language detection on Zettlr, I'm just saying that sounds like the right path to take to me and not a particularly far fetched one 🤷♂

@nathanlesage Did you look into what it would take to have a succinct DAWG implementation that we can use for spell checking?

Hey @fabiospampinato! No unfortunately not, I'm currently drowning in work :/ The only good thing is that I'm currently researching a lot on NLP techniques, among them said graphs, so while I can't tend to an implementation yet, I think this is a good way forward (to quote one of the papers, "Computer Science is basically all about trees").
Setting myself a reminder now for end of January and then I can have a look at it! :)

🤌
To recap, is most of the problem storing those in memory, or generating them?

To recap, is most of the problem storing those in memory, or generating them?

Kind of both, one shouldn't generate millions of words at startup to begin with because that takes multiple seconds already, and one shouldn't store millions of words in memory because that takes hundreds of megabytes (if done naively anyway, but non-naive alternatives are much more computationally expensive).
Potentially the italian dictionary could be patched to remove those numeric words entirely, but it would still generate millions of words. To give a sense of perspective the italian dictionary currently produces a ~700MB file containing ~30 million words, with some post-processing I've shaved that down by 90% and it still contains ~3 million words, it's just not feasible to generate all of them at startup quickly with JS.
There's a reimplementation of Hunspell in Python called Spylls that's really well documented, and from reading some of its documentation I think I learned that Hunspell bypasses both problems kind of by cheating, it generates all the various variations of a stem on demand depending on which word is being spell-checked so it has no major startup or memory issues, and it just stops generating these words early after an hard-coded amounts of milliseconds to keep the process fast so it never spends more than 200ms or whatever on any single word, so it's basically fast by definition.
I'm personally currently pursing another approach, parsing all dictionaries at build time in a way that gives you 0 startup delay, extremely low memory consumption, really fast lookup performance, and you still account for ~all words, but it's pretty complicated and it's unclear how it should work with compound words because as I understand it dictionaries like the one for Finnish encode for potentially billions and billions of potential words that aren't actually all valid in Finnish but the dictionary treats them all as valid, and you can't possibly generate and encode them efficiently.

One ~final comment on this: I've written a custom JS spell checker for a commercial product based on the succinct trie approach mentioned above (using a succinct fsa might work even better), I can't open-source the spell checker itself but if somebody else would like to pursuit the same approach here's some words of encouragement for you:

Italian dictionary:

For the Italian dictionary this new spell checker can now ~parse (it's not really "parsing", it's just deserializing a few simple structures) the dictionary in ~10ms, nspell crashes after processing the dictionary for many seconds. That's essentially a ~100% reduction in startup delay, unless you are using a similar approach whatever language your spell checker is written in this one blows it out of the water, in terms of startup performance at least.
It consumes a grand total of ~9.5MB of memory, nspell crashes after consuming hundreds of megabytes of memory.
It can lookup 7 million words (half hits and half misses) in ~16s, that's ~42 words per millisecond.

English dictionary:

For the English dictionary this new spell checker can ~parse the dictionary in ~5ms, I think that took nspell ~150ms to do in my machine.
It consumes 500KB of memory, nspell used something like ~13MB for it if I recall correctly.
It can lookup ~170k words (half hits and half misses) in ~350ms, that's almost 500 words per millisecond.
It correctly provides suggestions for 85% of the common misspelling cases listed here in ~1.3s (the remaining 15% it's either no suggestions are provided at all or not all the ones that Wikipedia mentions are provided) (and it's kind of impossible to get 100% on that because some examples are in the form of [correct word] -> [other correct word] which is undetectable with regular word-based spell checkers), where I think nspell took something like ~35s to pass ~83% (I don't recall the exact number now) of those tests on my machine. Having words organized in a Trie or an FSA rather than a hash table just allows you to explore nearby words much more efficiently.

I've made an NPM package that is a near complete port of Hunspell, in pure TS. It passes the relevant Hunspell tests. It's named Espells, and you can find the repository here.
With no ill meaning whatsoever, I think this package can be effectively replaced with Espells in any usage. In particular, it handles this Italian dictionary (and others) without issue.  This issue in fact is what drove me to make Espells.
@wooorm If you don't mind my continued forwardness, could I get some free advertising in, say, the dictionary package readmes? I'd love to get the word out.

This looks like a clear license violation, both the python "port", and espells?

This looks like a clear license violation, both the python "port", and espells?

I should be careful with my wording, I think. I use "port" because it's the best way I can think of to describe it. As far as I can tell, and how Spylls is demonstrated, was that it was reverse engineered from Hunspell's documentation and tests. After completing all of the main code, I actually finally took a look at some Hunspell source code and it looks very different, so I feel confident in this.
If Spylls is in violation, so is Espells, but if that happens I'll change the license.

I think both are in violation:

Spylls is explained as a “explanatory rewrite”, this sounds like carefully reading through the code and then implementing it either the same of a more clear way, and in the posts the Spyll author also explicitly mentions reading through the source and what they found there. That means it’s not 100% a new body of work. Which means it has to follow the MPL 1.1/GPL 2.0/LGPL 2.1 with copyright statement of Hunspell and add to it. So the first issue is there, and it’s a big one: spyll uses MIT, which is very permissive, whereas the author of Hunspell chose a strong license, to specifically prohibit such permissiveness. On top of that, it doesn’t acknowledge the copyright that the Hunspell author(s) have over their code
As Espell is more clearly a port, it should at least have the copyright statement of Spylls.

But I think the Spylls violation is a big one.

honestly, if anyone is uncomfortable, I'll just switch the license to the MPL 2.0, like Hunspell. The MIT license I chose was just force of habit.

Every open source maintainer should be uncomfortable.
I understand it happens due to force of habit but it’s still wrong.
People’s hard work is stolen. Their few wishes aren’t honored. Their work is not acknowledged.
That’s the tiny thing a license does: ask for an acknowledgement and a few small wishes.
Hunspell is seriously underfunded: the solution is to fund them.
The goal of nspell is to be an explicit MIT work, at the cost of being far from perfect.

As Espell is more clearly a port, it should at least have the copyright statement of Spylls.

Yeah, meant to do that. Housekeeping with the package, sorry.

People’s hard work is stolen. Their few wishes aren’t honored. Their work is not acknowledged.
That’s the tiny thing a license does: ask for an acknowledgement and a few small wishes.
Hunspell is seriously underfunded: the solution is to fund them.

I understand this, and I will switch to MPL 2.0, as I should've. However, I will be a bit... blunt. Hunspell's source is nigh-unreadable and difficult to contribute to. Commits haven't been made in years. Hunspell is good software, but it's not good for us software. It needs replacement, but very little has come close to parity with it, except Spylls and Espells. Lastly, I will state that I do not wish for my intentions to be read as more extreme than they are, my error was simply an absent-minded mistake.

The goal of nspell is to be an explicit MIT work, at the cost of being far from perfect.

I understand, good luck.

I understand your good intentions and see them separately from the “absent-minded mistake”
Forking and rewriting is easy compared to maintaining a project for 10 years, I remain of the opinion that a proper solution is to fund Hunspell
I don‘t think “not good software” is a good reason for porting without acknowledgement and under
different rules
I would appreciate it if you could perhaps ask Spylls to fix their license
Once the license situation is solved, I’m fine linking from nspell to your alternative!

I would appreciate it if you could perhaps ask Spylls to fix their license

I will make an issue on the repository.

Once the license situation is solved, I’m fine linking from nspell to your alternative!

Awesome!

I don‘t think “not good software” is a good reason for porting without acknowledgement and under different rules

Apologies if it came across like that, I agree with you.

@wooorm (Spylls author here)
Thanks for raising the concern. (And thanks @Monkatraz for bringing the problem to my attention.)
It is an important issue, and it just slipped my mind completely (I thought about it several times while working on the project in private, "dude, don't forget the licensing thing!"... and then, when I finished, I was so excited that I just forgot.)
It is my bad, and I'll be fixing it to MPL (and include an extended notice about Hunspell authors copyright) next week; on the road currently.
No bad intention was here, just underthinking of the important issue (not a really good excuse, but all I have).

Great to hear @zverok! Enjoy your away time!

Ok
I understand there's no interesto to find a solution and then the only option si to CHANGE software.
it's a sin but, well, no alternatives.
So long and thanks for all the fish


Critical 4.4.6 Bug: Module parse failed: Unexpected token (78:10)

Using react-data-grid, which depends on react-resizable, which depends on react-draggable
Since your update to 4.4.6, react-data-grid and react-sizable are now broken, and cannot be compiled into our projects.
Error reported is:
in ./node_modules/_react-draggable<EMAIL_ADDRESS>Module parse failed: Unexpected token (78:10)
You may need an appropriate loader to handle this file type.
| y: position.y,
| prevPropsPosition: {
| ...position
| }
| };
There is another bug report on the same issue, but it it not receiving any attention, so I opened up another with specifics on my issue

Version 4.4.6 introduced multiple ... syntaxes in the build results, which caused incompatibility issues with lower version browsers.

Although browsers that do not support the ... syntax are too old, this change should be a major version upgrade rather than a minor version change.

This was under the latest build of chromium.  It's not a browser issue. In fact, react builds use the node compiler to implement all the latest features on nearly every browser, so again, that isn't the issue. This is simply bad code that breaks core react components that rely on this code.


New release

Is it possible to have a new release published with the missing commits any time soon?

@xclud we need an update as the currently published version has a bug when used with the latest version of Flutter, but the latest version on the main branch fixes that bug. Could you please publish the latest version?

Done.

Thanks!


[WIP] Rewrite + upgrade GatsbyJS version

Todo items

[x] GatsbyJS upgrade
[x] Content structure for speakers + talks
[ ] Content + layout structure for schedules
[x] Layouts for pages + speakers

This PR also experiments with some redesigns. Some feedbacks highly appreciated.

Wrong repo, apologies.


[Feature] Handle URLs in browser

Duplicate of: https://github.com/lbryio/lbry-app/issues/2.

@kauffj, you will not tag this?


Failed to compile ./node_modules/react-router-native/NativeRouter.js

Version
react-router-native 4.2.0
Test Case
I am using create-react-app (react-scripts start) with React Native. Everything works fine until I add react-router-native code. I get the following error:
Failed to compile
./node_modules/react-router-native/NativeRouter.js
Module parse failed: Unexpected token (11:2)
You may need an appropriate loader to handle this file type.
|  */
| const NativeRouter = (props) => (
|   <MemoryRouter {...props}/>
| )
|`

Am I missing something in the setup process? I am using this guide https://github.com/ReactTraining/react-router/tree/master/packages/react-router-native
Steps to reproduce
import React from "react";
import { StyleSheet, Text, View } from "react-native";
+ import { NativeRouter, Route } from "react-router-native";

const page = props => {
  return (
    <View style={styles.container}>
      <Text>Test</Text>
    </View>
  );
};

const styles = StyleSheet.create({
  container: {
    flex: 1,
    backgroundColor: "#fff",
    alignItems: "center",
    justifyContent: "center"
  }
});

export default page;

package.json
{
  "devDependencies": {
    "@commitlint/cli": "^4.2.2",
    "@commitlint/config-angular": "^4.2.1",
    "coveralls": "^3.0.0",
    "gh-pages": "^1.0.0",
    "husky": "^0.14.3",
    "jest-expo": "^22.0.0",
    "lint-staged": "^4.3.0",
    "prettier": "^1.7.4",
    "react-devtools": "^2.5.2",
    "react-native-scripts": "1.6.0",
    "react-native-web": "^0.1.13",
    "react-scripts": "^1.0.16",
    "react-test-renderer": "16.0.0",
    "remote-redux-devtools": "^0.5.12"
  },
  "dependencies": {
    "expo": "^22.0.0",
    "react": "16.0.0",
    "react-dom": "^16.0.0",
    "react-native": "^0.50.0",
+   "react-router-native": "^4.2.0"
  }
}

.babelrc
{
  "plugins": [
    "react-native-web/babel"
  ],
  "presets": [
    "react-native",
    "babel-preset-expo"
  ],
  "env": {
    "development": {
      "plugins": ["transform-react-jsx-source"]
    }
  }
}

Expected Behavior
render
Actual Behavior
error

Something is wrong with your build tooling. This isn't a problem with react router.

So react-router doesn't support create-react-app and create-native-react-app? That's the only tooling I am using.

The error seems to be because react-router-native ships JSX (instead of compiling it into React.createElement calls). I'm not familiar enough with React Native to know if shipping JSX is standard. Are you able to force the build tool to compile certain files/node_modules?

Figured it out. Apparently react-router-native does not play friendly with web. I had to do a solution similar to this: https://github.com/react-everywhere/re-start/blob/master/src/utilities/routing/router.js#L3

Apparently react-router-native does not play friendly with web.

@ethanneff can you elaborate more on this?
Was it just a transpiling issue? If so, you need to make sure react-router-native is getting transpiled to ES5, something like this in your webpack.config.js:
const path = require('path');
const appDirectory = path.resolve(__dirname, '../');

module.exports {
...
   include: [
      path.resolve(appDirectory, 'src')
      path.resolve(appDirectory, 'node_modules/react-router-native')
  ],
...
}


No errors bu nothing

Hello
I'm trying to re-use this library many years after first usage
I got a problem
Function called is :
function getProjectRights(projectID) {
  var service = getService();
  Logger.log(projectID)
  if (service.hasAccess()) {
    Logger.log("access")
    var api = "https://cloudresourcemanager.googleapis.com/v1/projects/"+projectID+":getIamPolicy";
    var headers = {
      "Authorization": " OAuth " + service.getAccessToken()
    };
    var options = {
      "headers": headers,
      "method" : "POST",
      "contentType" : "application/json",
      "muteHttpExceptions": true
    };
    var responseApi = UrlFetchApp.fetch(api, options);
    var responseApi = JSON.parse(responseApi.getContentText());
    var payload = { "bindings" : "",
                   "version" : 1}
    payload.bindings = responseApi.bindings
    Logger.log("----- "+payload)
  }
  return payload;
}

But my code never goes after
if (service.hasAccess()) {
No error seen
How to troubleshoot ?
thanks

Service is just typical function
function getService() {
return OAuth2.createService('gcp')
.setAuthorizationBaseUrl('https://accounts.google.com/o/oauth2/auth')
.setTokenUrl('https://accounts.google.com/o/oauth2/token')
.setClientId(CLIENT_ID)
.setClientSecret(CLIENT_SECRET)
.setCallbackFunction('authCallback')
.setPropertyStore(PropertiesService.getUserProperties())
.setScope('https://www.googleapis.com/auth/cloud-platform', 'https://www.googleapis.com/auth/cloud-platform.read-only')
.setParam('login_hint', Session.getActiveUser().getEmail())
.setParam('access_type', 'offline')
.setParam('approval_prompt', 'force');

The most likely situation is that the service doesn't have access anymore, and so that condition is false. You don't have any else condition, so nothing happens in that case. You likely need to re-approve access to your script, and you can look at the samples for how to do that.

I'm using ScriptApp.getOAuthToken();
this is very more easy to use than this


v2.7.0 Validation error if integer field is filled with string "00"

Initial Checks

[X] I confirm that I'm using Pydantic V2

Description
This seems to be a regression as v.2.6.4 was not affected by this problem.
Basically if you define a model with an integer field and fill it with string "00", you get the following error message:
pydantic_core._pydantic_core.ValidationError: 1 validation error for MyModel
id
  Input should be a valid integer, unable to parse string as an integer [type=int_parsing, input_value='00', input_type=str]
    For further information visit https://errors.pydantic.dev/2.7/v/int_parsing

The problem seems only to happen with multiple zeros (i.e. "00", "000", ...), other numeric strings work properly (e.g. "01", "0").
Example Code
from pydantic import BaseModel

class User(BaseModel):
    id: int = 0

s = User(id="01") # works
print(s)

s = User(id="00") # fails
print(s)

Python, Pydantic & OS Version
pydantic version: 2.7.0
pydantic-core version: 2.18.1

@nico00,
Thanks for reporting this. @samuelcolvin, could you please take a look? I think this is likely related to the int parsing changes you worked on recently...

Yup, good catch, I'll fix over the weekend.
Sorry for the trouble.

should be fixed in https://github.com/pydantic/pydantic-core/pull/1269.

Thanks @samuelcolvin for the quick turnaround!
We'll release this fix in a patch release early next week 👍


"Internal Error: java_class_info should be present for Java classes" using "--eradicate"

Infer is crashing with "Internal Error: java_class_info should be present for Java classes" on any code using "--eradicate"
Infer version 1.0.0 (also tried on 0.17.0, there was no issue, everything worked fine). Operating system is Ubuntu 18.04, kernel version 4.5. The build command is:
infer --html -a checkers --eradicate --quandary -- mvn clean compile
The output is:
Internal Error: java_class_info should be present for Java classes 0% 55.528ms
Error backtrace:in/java/SER05-J-1.java: SER05J1_OuterSer$SER05J1_InnerSer.<init
Raised at Base__String.rindex_from_exn_internal in file "src/string.ml", line 127, characters 7-22/java/NUM01-J-6.java: void NUM01J6.method()
Called from Base__String.rsplit2_exn.rsplit2_exn in file "src/string.ml", line 620, characters 14-55java/OBJ05-J-2.java: OBJ05J2_MutableClass.()
Called from Base__String.rsplit2 in file "src/string.ml", line 633, characters 11-330.0s] src/main/java/dataflow/DIV_BY_ZERO_1.java: DIV_BY_ZERO_1.()
Re-raised at IBase__Die.raise_error.do_raise in file "src/base/Die.ml", line 26, characters 8-56ain/java/NUM14-J-3.java: NUM14J3.()
Called from Nullsafe__ClassLevelAnalysis.analyze_nullsafe_annotations in file "src/nullsafe/ClassLevelAnalysis.ml", line 212, characters 12-50Connection()
Called from Nullsafe__FileLevelAnalysis.analyze_class_and_nested in file "src/nullsafe/FileLevelAnalysis.ml", line 31, characters 20-90
Called from Stdlib__list.fold_left in file "list.ml", line 121, characters 24-34
Called from Base__List0.fold in file "src/list0.ml" (inlined), line 21, characters 22-52
Called from Nullsafe__FileLevelAnalysis.analyze_file in file "src/nullsafe/FileLevelAnalysis.ml", line 58, characters 4-98
Called from Backend__Callbacks.iterate_file_callbacks_and_store_issues.(fun) in file "src/backend/callbacks.ml", line 77, characters 26-46
Called from Stdlib__list.iter in file "list.ml", line 110, characters 12-15
Called from Backend__Ondemand.analyze_procedures.(fun) in file "src/backend/ondemand.ml" (inlined), line 387, characters 6-92
Called from Base__Option.iter in file "src/option.ml" (inlined), line 94, characters 14-17
Called from Backend__Ondemand.analyze_procedures in file "src/backend/ondemand.ml", line 386, characters 2-147
Called from Backend__InferAnalyze.analyze_target.analyze_source_file.(fun) in file "src/backend/InferAnalyze.ml", line 52, characters 10-51
Called from IBase__Logging.task_progress in file "src/base/Logging.ml", line 240, characters 15-19
Called from IBase__ProcessPool.child_loop in file "src/base/ProcessPool.ml", line 356, characters 12-19
Re-raised at IStdlib__IExn.reraise_if in file "src/istd/IExn.ml", line 18, characters 15-63
Called from IBase__ProcessPool.child_loop in file "src/base/ProcessPool.ml", line 358, characters 10-427
Called from IBase__ProcessPool.fork_child in file "src/base/ProcessPool.ml", line 420, characters 6-99
Called from Stdlib__array.init in file "array.ml", line 56, characters 22-27
Called from IBase__ProcessPool.create in file "src/base/ProcessPool.ml", line 445, characters 4-189
Called from Backend__Tasks.Runner.create in file "src/backend/Tasks.ml", line 25, characters 6-286
Called from Backend__InferAnalyze.analyze in file "src/backend/InferAnalyze.ml", line 186, characters 6-130
Called from Integration__Driver.execute_analyze.(fun) in file "src/integration/Driver.ml", line 179, characters 2-34
Called from Backend__GCStats.log_f in file "src/backend/GCStats.ml", line 90, characters 10-14
Called from Integration__Driver.execute_analyze in file "src/integration/Driver.ml", line 176, characters 2-239
Called from IBase__Utils.timeit in file "src/base/Utils.ml", line 451, characters 16-20
Called from IBase__ScubaLogging.execute_with_time_logging in file "src/base/ScubaLogging.ml", line 79, characters 29-44
Called from Dune__exe__Infer.run in file "src/infer.ml", line 21, characters 2-47
Called from IBase__Utils.timeit in file "src/base/Utils.ml", line 451, characters 16-20
Called from IBase__ScubaLogging.execute_with_time_logging in file "src/base/ScubaLogging.ml", line 79, characters 29-44
Called from Dune__exe__Infer.run in file "src/infer.ml", line 25, characters 22-94
Run the command again with --keep-going to try and ignore this error.
Internal Error: Subprocess 5: see backtrace above
Error backtrace:
Raised at Core__Core_unix.improve in file "src/core_unix.ml", line 46, characters 4-43
Called from Core__Core_unix.wait_gen in file "src/core_unix.ml", line 942, characters 4-246
Called from Core__Core_unix.wait in file "src/core_unix.ml" (inlined), line 971, characters 2-34
Called from IBase__ProcessPool.killall.(fun) in file "src/base/ProcessPool.ml", line 189, characters 10-30
Re-raised at IBase__Die.raise_error.do_raise in file "src/base/Die.ml", line 26, characters 8-56
Called from Stdlib__list.iter in file "list.ml", line 110, characters 12-15
Called from Base__List0.iter in file "src/list0.ml" (inlined), line 25, characters 16-35
Called from IBase__ProcessPool.process_updates in file "src/base/ProcessPool.ml" (inlined), line 258, characters 5-1023
Called from IBase__ProcessPool.process_updates in file "src/base/ProcessPool.ml", line 257, characters 2-1023
Called from IBase__Utils.do_finally_swallow_timeout in file "src/base/Utils.ml", line 213, characters 8-12
Re-raised at IStdlib__IExn.reraise_after in file "src/istd/IExn.ml", line 13, characters 2-50
Called from IBase__Utils.do_finally_swallow_timeout in file "src/base/Utils.ml", line 215, characters 6-141
Called from IBase__Utils.try_finally_swallow_timeout in file "src/base/Utils.ml" (inlined), line 223, characters 16-54
Called from IBase__Utils.with_file_lock in file "src/base/Utils.ml", line 262, characters 2-41
Called from IBase__ProcessPool.run in file "src/base/ProcessPool.ml", line 462, characters 2-656
Called from Backend__InferAnalyze.analyze in file "src/backend/InferAnalyze.ml", line 189, characters 24-47
Called from Integration__Driver.execute_analyze.(fun) in file "src/integration/Driver.ml", line 179, characters 2-34
Called from Backend__GCStats.log_f in file "src/backend/GCStats.ml", line 90, characters 10-14
Called from Integration__Driver.execute_analyze in file "src/integration/Driver.ml", line 176, characters 2-239
Called from IBase__Utils.timeit in file "src/base/Utils.ml", line 451, characters 16-20
Called from IBase__ScubaLogging.execute_with_time_logging in file "src/base/ScubaLogging.ml", line 79, characters 29-44
Called from Dune__exe__Infer.run in file "src/infer.ml", line 21, characters 2-47
Called from IBase__Utils.timeit in file "src/base/Utils.ml", line 451, characters 16-20
Called from IBase__ScubaLogging.execute_with_time_logging in file "src/base/ScubaLogging.ml", line 79, characters 29-44
Called from Dune__exe__Infer.run in file "src/infer.ml", line 25, characters 22-94
sources.zip

Thanks for the report, @baloghadamsoftware!


--embed option: don't export any functions

as the --embed option is for embedding within an executable, and unlike dlls, executables don't export any functions or if they do, they are unusable as they are not loadable modules.
when embed option is used, the generated c/c++ file should not export any functions, as the linker generates .def and .lib files which is not needed at all.

Seems reasonable. It should only regard the module init function, right?
PR welcome.

the module init function should be called from within the interpreter, but it shouldn't be exported.
although it doesn't make any difference, but executables won't export functions.