Exit status 137 from mbtiles.go:63 while generating planet

Describe the bug
Generating one of the temporary mbtiles dies with exist status 137
To Reproduce
Just run ./sequentially-generate-planet-mbtiles--unix-amd64-v3.1.0, default settings, whole planet.
Expected behavior
Generation succeeds.
Screenshots
Output (excerpt):
2023/02/09 10:33:19 generating temporary mbtiles: /home/szabolcs.hodossy/data/mbtiles/9426803175.mbtiles from /home/szabolcs.hodossy/data/pbf/slices/1773301992-tmp.osm.pbf; directory output: false
mbtiles file exists, will overwrite (Ctrl-C to abort, rerun with --merge to keep)
Layer place (z0-14)
Layer boundary (z0-14)
Layer poi (z12-14)
Layer poi_detail (z14-14) -> poi
Layer housenumber (z14-14)
Layer waterway (z8-14)
Layer waterway_detail (z12-14) -> waterway
Layer transportation (z4-14)
Layer transportation_name (z8-14)
Layer transportation_name_mid (z12-14) -> transportation_name
Layer transportation_name_detail (z14-14) -> transportation_name
Layer building (z13-14)
Layer water (z6-14)
Layer ocean (z0-14) -> water
Layer water_name (z14-14)
Layer water_name_detail (z14-14) -> water_name
Layer aeroway (z11-14)
Layer aerodrome_label (z10-14)
Layer park (z11-14)
Layer landuse (z4-14)
Layer urban_areas (z4-8) -> landuse
Layer landcover (z0-14)
Layer ice_shelf (z0-9) -> landcover
Layer glacier (z2-9) -> landcover
Layer mountain_peak (z11-14)
Bounding box -180, -85.06, -134.982, 85.06
Reading .shp ocean
Shapefile entity #704 type 5 is invalid. Parts:232. Reason:Geometry has invalid self-intersections. A self-intersection point was found at (-146.579, -15.9312); method: i; operations: i/u; segment IDs {source, multi, ring, segment}: {0, 0, 138, 7}/{0, 0, 138, 9}... failed to correct. Reason: Geometry has invalid self-intersections. A self-intersection point was found at (-146.579, -15.9312); method: i; operations: i/u; segment IDs {source, multi, ring, segment}: {0, 0, 138, 7}/{0, 0, 138, 9}
Shapefile entity #51908 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51909 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51910 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51911 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51912 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51913 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51914 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51915 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51916 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51917 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51918 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51919 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51920 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51921 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51922 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51923 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51924 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51925 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51926 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51927 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51928 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51929 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51930 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51931 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51932 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51933 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51934 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51935 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51936 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51937 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51938 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51939 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51940 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51941 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51942 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51943 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51944 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51945 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51946 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51947 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51948 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51949 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51950 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51951 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51952 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #51953 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52268 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52269 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52270 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52271 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52272 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52273 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52274 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52275 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52276 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52277 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52278 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52279 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52280 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52281 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52282 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52283 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52284 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52285 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52286 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52287 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52288 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52289 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52290 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52291 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52292 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52293 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52294 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52295 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52296 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52297 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52298 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52299 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52300 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52301 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52302 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52303 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52304 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52305 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52306 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52307 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52308 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52309 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52310 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52311 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52312 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52313 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52628 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52629 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52630 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52631 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52632 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52633 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52634 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52635 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52636 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52637 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52638 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52639 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52640 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52641 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52642 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52643 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52644 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52645 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52646 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52647 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52648 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52649 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52650 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52651 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52652 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52653 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52654 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52655 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52656 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52657 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52658 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52659 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52660 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52661 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52662 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52663 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52664 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52665 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52666 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52667 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52668 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52669 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52670 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52671 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52672 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52673 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52988 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52989 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52990 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52991 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52992 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52993 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52994 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52995 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52996 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52997 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52998 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #52999 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53000 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53001 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53002 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53003 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53004 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53005 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53006 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53007 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53008 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53009 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53010 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53011 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53012 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53013 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53014 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53015 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53016 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53017 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53018 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53019 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53020 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53021 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53022 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53023 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53024 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53025 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53026 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53027 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53028 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53029 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53030 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53031 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53032 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Shapefile entity #53033 type 5 is invalid. Parts:1. Reason:Geometry has too few points... corrected
Reading .shp urban_areas
Reading .shp ice_shelf
Reading .shp glacier
Shapefile entity #360 type 5 is invalid. Parts:7. Reason:Geometry has invalid self-intersections. A self-intersection point was found at (-152.169, 78.8039); method: t; operations: u/u; segment IDs {source, multi, ring, segment}: {0, 0, 0, 5}/{0, 0, 0, 87}... corrected
Shapefile entity #1861 type 5 is invalid. Parts:1. Reason:Geometry has invalid self-intersections. A self-intersection point was found at (-155.982, -180); method: c; operations: x/i; segment IDs {source, multi, ring, segment}: {0, 0, -1, 224}/{0, 0, -1, 377}... corrected
Shapefile entity #1875 type 5 is invalid. Parts:54. Reason:Geometry has invalid self-intersections. A self-intersection point was found at (-141.942, 76.9594); method: t; operations: u/u; segment IDs {source, multi, ring, segment}: {0, 0, -1, 2543}/{0, 0, -1, 2505}... corrected
Sorting loaded shapes
Reading .pbf /src/1773301992-tmp.osm.pbf
Block 4007/4222 ways 0 relations 0
Sorting nodes
Block 4217/4222 ways 8000 relations 0
Sorting ways
Problem geometry relation 13121479: Boost.Geometry Centroid calculation exception
Problem geometry relation 9599691: Boost.Geometry Centroid calculation exception
Problem geometry relation 15006398: Boost.Geometry Centroid calculation exception
Problem geometry relation 4562026: Boost.Geometry Centroid calculation exception
Sorting generated geometriesions 8000
Stored 32065249 nodes, 102695 ways, 18264 relations
Shape points: 0, lines: 0, polygons: 7773
Generated points: 96335, lines: 1237744, polygons: 906896
2023/02/09 10:37:27 mbtiles.go:63: exit status 137

Desktop (please complete the following information):
Ubuntu 20.04.5 LTS (GNU/Linux 5.4.0-135-generic x86_64) with 8 GB RAM, 8 cores and 512 GB storage

I looked up a bit, this probably means, that the tilemaker container has been killed due to OOM. I think this project is genius, and we will definitely incorporate it in our operations. The ability to resume is a huge plus. Thanks for creating this! I would like to summarize my experience below, it may help take a step towards the stated goal of the project.

This program aims to be a simple set and forget, one liner which gives anyone - a way to get a full-featured and bang up to date set of vector tiles for the entire planet on small hardware.

So here it is: The auto ram setting decided it can take up to 5950 MB memory, so I have manually lowered that to 4096 MB, but still, I regularly saw memory consumption of the tilemaker process well over 5 GB. Fortunately I managed to add a 8 more GB of RAM to the instance (now total of 16 GB), so I managed to finish the process. I observed a max memory usage of 10.3 GB for an input size of ~190 MB (cut size was determined at 273 MB). That file contained over 43 million tiles, and I think that is reason for the memory requirements. I am wondering if it is possible to take into consideration the number of tiles in a slice, when determining if it requires further slicing.
There were also minor annoyances like when run without a pbf file given, and if there is an error, usually after the first re-run, the planet osm file is removed (considered corrupt for some reason), thus it needs to be downloaded (and possibly sliced) again. So I quickly reverted to a config file, and I downloaded the planet file myself (I got the url from the code). I also noticed that the shape files are always unzipped, seems unnecessary.
A good improvement would be if slicing could be continued from a previously complete state. So for example in my case, after slicing completed for the 5950 MB RAM limit, I had to rerun the whole process, but it would have been much more efficient to just continue with the already present slices.
Also there was an issue, that a journal file has been left in the mbtiles directory, and that caused tile-join to fail, so I had to rerun the whole joining again.
I am closing this issue, as it is not a bug, but I see several improvements here (I have never written a single line in go):

drastically reduce the auto-determined memory limit. I would say 1/4 of detected memory could be OK
or alternatively tile count could be taken into account during the decision if a chunk needs further slicing
do not remove planet.osm.pbf automatically (when no pbf file is given), maybe prompt the user before doing so
improve slicing so it can continue from a previous final state with a new bound (lower than before)
ensure tile-join is only run on *.mbtiles files only, no journal file should be passed to the command

Please let me know what you think about these ideas!


Translate reference/kubernetes-api/service-resources/endpoints-v1.md to chinese

Hi, I'm interested to translate the "reference/kubernetes-api/service-resources/endpoints-v1/" page.
This is a Feature Request:
translate en/docs/reference/kubernetes-api/service-resources/endpoints-v1.md into Chinese.
What would you like to be added:
zh/docs/reference/kubernetes-api/service-resources/endpoints-v1.md
/language zh
/kind feature

/assign


Implement ShouldNotDismissKeyboardWhenMinimumLengthThreholdHasNotBeenReached unit test

Description of Change
Worked around "Focus does not change IsFocused property"
Bugs Fixed

Fixes https://github.com/xamarin/XamarinCommunityToolkit/pull/389#issuecomment-706099734

API Changes
None
Behavioral Changes
None
PR Checklist

[X] Has tests (if omitted, state reason in description)
[X] Has samples (if omitted, state reason in description)
[X] Rebased on top of main at time of PR
[X] Changes adhere to coding standard

CC @jBijsterboschNL

@AndreiMisiukevich , there was another unit test which is testing if the IsFocused is set to false properly. But in that test the IsFocused isn't set to true properly as well. So that test isn't a proper test. Could you have a look at that one as well? And great that you got this unit test working properly! 😀

Looks like the test fails :(

Weird, I didn't change ShouldOnlyExectueCommandOnceWhenTextChangedHasOccurredMultipleTimes test

@AndreiMisiukevich , there was another unit test which is testing if the IsFocused is set to false properly. But in that test the IsFocused isn't set to true properly as well. So that test isn't a proper test. Could you have a look at that one as well? And great that you got this unit test working properly! 😀

Well, we can set "IsFocused" property ourselves inside the behavior (After Unfocus invoked).

Oooh I didn't look close enough. Yeah that one has been failing earlier as well. Seems some timing issue

Oooh I didn't look close enough. Yeah that one has been failing earlier as well. Seems some timing issue

@jfversluis @jBijsterboschNL I found proper fix 👍

@AndreiMisiukevich that seems like a good one! :) great work


Rust backend

Adding Rust backend.

LOC neutral (assuming can park rust.py in ./extra)
Tests mostly pass
Includes examples/export_code_from_onnx.py which creates a zero dependency pure Rust rlib crate
Performance slightly better than Clang (worse compile times, need to compare 2nd run)
Core changes required: renderer needs correct buffer size to build functions

The use case is to take a model from TinyGrad, onnx, pytorch, etc and be able to export a pure Rust crate to implement that model.  This is ideal for my use case were I am dealing with small models in userspace apps and I will be very useful for embedded.  Sure there are other ways to do this they all end up with crappier code, more dependencies, and/or worse performance.
You can test this out fairly easily with ./examples/export_code_from_onnx.py --language rust --crate --save
Most tests pass but not all.  There are a few areas where there are still some differing casting assumptions to hammer out.  Which I will commit to work through.  I'm still working on how to support, variables.  If I don't know the range of a variable that guides a loop the rust compiler doesn't know now to ensure you're not doing a buffer overflow.
Performance is a tiny bit little faster than CLANG (on second run, compile times are worse) when compared using ./test/test_speed_v_torch.py, but line by line there are shocking difference where Rust is faster by 20X or more and some where C is faster by 20X.  Honestly being able to look at how they compile out to different machine code to achieve those differences might justify adding Rust by itself.
There are two core changes which Rust support needs, especially to fit in the CDLL usage that is so useful for testing here.  Well really just one.  When I render a function I need to properly size any input/output buffers.  Rust can infer buffer sizes and check for correctness, but only if those sizes are all known at compile time.  The way you compile for CDLL doesn't allow the sizes to be known so you have to specify.  The knowledge of how big the inputs and outputs are is know at the time of the call of get_runner() so I added that.  However, not all buffers are inputs or outputs and so we also need the buffer size added to the UOpsGraph when a global is added, so I stash a size field in the args tuple for globals.  Originally I hoped only to use that but sometimes inputs/outputs sizes disagrees with UOpsGraph size, and in those cases input/outputs agree.
As a side note as we start looking for optimizations it would see this information being available at render time might be useful for other architectures too.

huh.  So the CI doesn't like it much.  I thought I tested that out better.
Please hold...

Won't merge, for the reasons discussed in Discord.
CLANG is nowhere near its performance limit, would much rather spend lines to make CLANG generate more performant C code.


Merge the meeting subsections into Community

Add copy about merging previously separate meetings.

@Smokyish are we merging the HQ and Dev meetings into Community or just the HQ? AGP7 only mentions the HQ meeting.


Feat: implement Random Projections

This PR implements random projection techniques for dimensionality reduction, as seen in the sklean.random_projection module of scikit-learn

Codecov Report
Attention: 81 lines in your changes are missing coverage. Please review.

Comparison is base (4e40ce6) 36.18% compared to head (1999d0f) 35.77%.

Files
Patch %
Lines

...duction/src/random_projection/sparse/algorithms.rs
0.00%
32 Missing :warning:

...tion/src/random_projection/gaussian/hyperparams.rs
0.00%
28 Missing :warning:

...ction/src/random_projection/gaussian/algorithms.rs
0.00%
17 Missing :warning:

...ms/linfa-reduction/src/random_projection/common.rs
0.00%
4 Missing :warning:

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #332      +/-   ##
==========================================
- Coverage   36.18%   35.77%   -0.41%
==========================================
  Files          92       96       +4
  Lines        6218     6292      +74
==========================================
+ Hits         2250     2251       +1
- Misses       3968     4041      +73

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

I've done a quick review; this looks good to me, but have requested @bytesnake also give it a look as he is probably more familiar with the algorithm side of things.

I've done a quick review; this looks good to me, but have requested @bytesnake also give it a look as he is probably more familiar with the algorithm side of things.

thank you for reviewing @relf @quietlychris

Thank you for the reviews, and @relf for the suggestions, I have implemented them.
Changes:

The rng field for both random projections structs  is no longer optional, and defaults to Xoshiro256Plus with a fixed seed if not provided by the user.
Renamed precision parameter to eps, as increasing this parameter results in a lower dimension embedding and often yields lower classification performance.
Added a check that the projections reduce the dimension of the data, returning an error otherwise. Added tests for this behavior.
Added a test for the function that computes the embedding dimension from a given epsilon, against values from scikit-learn .
Fixed reference issue in docs and other minor issues.

Thanks for your contribution and the changes. Now, gaussian and sparse random projection codes look very alike, I am wondering if you could not refactor even further by using zero-sized types and a unique RandomProjection generic type, something like:
struct Gaussian;
struct Sparse;

pub struct RandomProjectionValidParams<RandomMethod, R: Rng + Clone> {
    pub params: RandomProjectionParamsInner,
    pub rng: Option<R>,
    pub method: std::marker::PhantomData<RandomMethod>,
}

pub struct RandomProjectionParams<RandomMethod, R: Rng + Clone>(
    pub(crate) RandomProjectionValidParams<RandomMethod, R>,
);

pub struct RandomProjection<RandomMethod, F: Float> {
    projection: Array2<F>,
    method: std::marker::PhantomData<RandomMethod>,
}

pub struct GaussianRandomProjection<F: Float> = RandomProjection<Gaussian, F: Float>;
pub struct SparseRandomProjection<F: Float> = RandomProjection<Sparse, F: Float>;

impl<F, Rec, T, R> Fit<Rec, T, ReductionError> for RandomProjectionValidParams<Gausssian, R>
where
    F: Float,
    Rec: Records<Elem = F>,
    StandardNormal: Distribution<F>,
    R: Rng + Clone,
{
    type Object = RandomProjection<Gaussian, F>;

    fn fit(&self, dataset: &linfa::DatasetBase<Rec, T>) -> Result<Self::Object, ReductionError> {...}
}

impl<F, Rec, T, R> Fit<Rec, T, ReductionError> for RandomProjectionValidParams<Sparse, R>
where
    F: Float,
    Rec: Records<Elem = F>,
    StandardNormal: Distribution<F>,
    R: Rng + Clone,
{
    type Object = RandomProjection<Sparse, F>;

    fn fit(&self, dataset: &linfa::DatasetBase<Rec, T>) -> Result<Self::Object, ReductionError> {...}
}

...

What do you think?

I think that's a very good suggestion, it will be easier to maintain than the previous approach using a macro to avoid code duplication. 6b9c2a4 implements a variation of this idea: all the logic has been refactored, and behavior depending on the projection method has been encapsulated in the ProjectionMethod trait. It also makes implementing other projection methods significantly easier.


Remove last semester's results from leaderboard

Purpose: researchers are now unable to view the results of all debates. This means we can't easily track accuracy for the debates we currently care about
Change wanted: Might be something like moving certain JSON files to..

Practice? You'll have to rename the mightiest qorn ones...
New folder, practice-1? or something
Or tagging the debates or whatever was done so the offline debates don't show up on the leaderboard

TO REMOVE Story-(ending numbers)
mightiest-qorn-(0-2)
ambition-(0-5)
pied-piper-of-mars-(0-5)
jaywalker-(0-4)

I'm waiting to rejigger the leaderboard at the moment, until after I get the auto-scheduler set up and we can start running 2 debates per question. That'll give us a way of scoring debaters which is more appropriate for the leaderboard.
As for tracking overall accuracy and properties of the debates, that could make sense to add in the analytics pane later (#73)

But meh, this was a quick change so I just did it now anyway


The "Vehicle Availability" API always returns an empty list

curl --request GET \
     --url 'https://api.autofleet.io/api/v1/availability?lat=43.642566&lng=-79.387057&radius=15000' \
     --header 'accept: application/json' \
     --header 'authorization: Bearer {{authToken}}'

Hey @nusjose thanks for the issue
1.In the app - do you see available vehicles?
2.The locomotion app makes this request to base url  https://client.autofleet.io/  and not api.autofleet.io .
3.try to make the request to client api, and validate you send the same params as the app sends
4.from the headers of the response, please provide the x-trace-id , that way i can debug the issue much quicker

@OmerGery thanks for your reply. After changes you suggested, I was able to get the expected response:
{
    "vehicles": [
        {
            "id": "89cc437f-818d-42da-bea4-3eea5fa8f982",
            "location": {
                "lat": 48.394589,
                "lng": -89.276651,
                "bearing":<PHONE_NUMBER>9893078
            }
        },
        {
            "id": "ba5ea295-c1e4-4026-b9be-cee77609197e",
            "location": {
                "lat": 48.408425,
                "lng": -89.244574,
                "bearing":<PHONE_NUMBER>4670633
            }
        },
        {
            "id": "1a00bdd2-2ad6-4dfa-878c-9993fce511e6",
            "location": {
                "lat": 48.42053,
                "lng": -89.24248,
                "bearing": 20.10269931203976
            }
        }
    ]
}

@otaviolbarbosa thats great


Fix broken qtypes. Set proper qtype when coming in from permalink. Fi…

It turns out that qtypes were completely broken. Changing a qtype in the dropdown box didn't actually change the hash. This was the result of an internal Django change I made awhile back and I didn't notice how it affected the UI.
This set of changes both fixes the fundamental working of qtypes, and handles qtypes coming in from permalinks. Previously these incoming qtypes were just ignored and the default was used.
Note there is no clean way to select a default item for a dropdown in Django 2.x, so this is a horrible hack. In fact ALL of the code in api_get_widget() and search/forms.py is horrible and needs to be rewritten from scratch. That's a project for another day.

I didn’t write that code, but I can upgrade it.


allow foreach on input with 0 length

Description
Fix https://github.com/apache/incubator-mxnet/issues/12470
Checklist
Essentials
Please feel free to remove inapplicable items for your PR.

[ ] Changes are complete (i.e. I finished coding on this PR)
[ ] All changes have test coverage:
[ ] To the my best knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change

Changes
remove check for 0 length

@zheng-da could you help review this? thanks!

@roywei could you add your example code to the test? We usually put tests on control flow operators here: tests/python/unittest/test_contrib_control_flow.py.

@zheng-da added unit test and CI passed


Exp pdcli

https://issues.redhat.com/browse/OSD-8114

@Krishnanunni333  Could you please mention the Jira ticket number in the description.

applied the changes

@Krishnanunni333 don't delete the LICENCE file... don't make any changes to LICENCE file

@Krishnanunni333 could you please make the directory structure similar to https://github.com/openshift-online/ocm-cli
With updated README.md
And makefile
@supreeth7 @MitaliBhalla Could you please help in here to improve App scaffolding

CC: @dofinn

Could you provide an example test in test file


autoPatchelfHook skips position-independent executables

Issue description
autoPatchelfHook determines which executables it should patch via:
isExecutable() {
    readelf -h "$1" 2> /dev/null | grep -q '^ *Type: *EXEC\>'
}

However, PIEs have a type of DYN, as in:
$ readelf -h result/build-tools/28.0.3/zipalign
ELF Header:
  Magic:   7f 45 4c 46 02 01 01 00 00 00 00 00 00 00 00 00
  Class:                             ELF64
  Data:                              2's complement, little endian
  Version:                           1 (current)
  OS/ABI:                            UNIX - System V
  ABI Version:                       0
  Type:                              DYN (Shared object file)
  Machine:                           Advanced Micro Devices X86-64
  Version:                           0x1
  Entry point address:               0x5350
  Start of program headers:          241664 (bytes into file)
  Start of section headers:          198080 (bytes into file)
  Flags:                             0x0
  Size of this header:               64 (bytes)
  Size of program headers:           56 (bytes)
  Number of program headers:         10
  Size of section headers:           64 (bytes)
  Number of section headers:         32
  Section header string table index: 27

Questions:

Should autoPatchelfHook also patch PIEs? An alternative is to just call autoPatchelfFile manually, but I'm having trouble finding out how to run it in the correct phase.
Is there even a reliable way to distinguish between shared libraries and PIEs? I know in general, shared libs don't have an .interp section, but is this a reliable enough assumption?

cc @aszlig

@tadfisher: Actually, executables can also be shared libraries at the same time (eg. libc.so.6 and I'm doing exactly that in my recent project as well), so the only thing that distinguishes them is an .interp section (except if statically linked of course). So I guess it's a better idea to just look for .interp in isExecutable instead of what we currently do. At least if I don't miss something, this should be the most reliable way, after all what we want to do with executables is to patch the .interp section.
If nobody else sends a PR in the meantime, I'm going to fix this next week.
As for autoPatchelfFile it's a bit more difficult, because it needs to be run after the normal patchelf hook, so it might be necessary to add postPhases.


Add Auctioneer chart

We need to add a Helm chart for deploying the Auctioneer. This Helm chart should support all necessary configurations to deploy the Auctioneer in our local Kubernetes environment, such as :

[ ] Connect to the optimistic block gRPC service served by a sequencer node
[ ] Connect to the sequencer node's ABCI server for transaction submission
[ ] Connect to the optimistic execution and bundle gRPC services served by the rollup node

┆Issue Number: ENG-958

This issue is stale because it has been open 45 days with no activity. Remove stale label or this issue
be closed in 7 days.


Add a link to the cluster in the device page

Issue type
[ ] Feature request
[X] Bug report
[ ] Documentation
Environment

Python version: 2.7
NetBox version: v2.2-beta1

Description
We cannot see if a device is in a cluster or not in the device page.
Proposal : add a field with a link to the cluster in the device

Fixed in e2b2815


Auto corrected by following Lint Ruby Performance/StringReplacement

Auto corrected by following Lint Ruby Performance/StringReplacement
Click here to configure it on awesomecode.io

Coverage remained the same at 80.877% when pulling fac03393725d528632daba04a31abe4cf537ace5 on awesomecode-lint-ruby-performance-stringreplacement-48236 into ed40b1d9634eccd994a408834183ce70271b7d5b on master.


Select2TagsField fails if no data sent

Hello,
I Have a problem with the Select2TagsField. I have a non required filled in a form represented by a Select2TagsField, and if I don't send any data for this field, the process_formdata raise an exception because valuelist is used without previous check, is it possible to check the valuelist before using it, this way:
if valuelist:
            if self.save_as_list:
                self.data = [self.coerce(v.strip()) for v in valuelist[0].split(',') if v.strip()]
            else:
                self.data = self.coerce(valuelist[0])

Looks good, why don't you make a pull request?
Maybe on top of #1285.


Redirecting to main page - Authorization code missing

Thanks for this code! Its really helpful
When I try to grant access, it is redirecting to main page. I did not get any authorisation code. Could you please help me on that?
https://account.box.com/api/oauth2/authorize?response_type=code&client_id=XXX&redirect_uri=YYY

Did you figure out? If not describe step by step what you did so far


feat(logging): Swap loggers from zap to log/slog

This PR replaces go-uber/zap with Go standard library's log/slog. The internal/log package within the template folder now includes:

a New(level string) *slog.Logger function to replace the former NewAtLevel function. It still works pretty much the same way but does not return an error through safe defaults.
a NewSpanContextHandler(handler slog.Handler, withSpanID bool) slog.Handler function, that decorates a slog.Handler to add context-provided trace and span IDs as attributes (optional)
a InterceptorLogger(l *slog.Logger) logging.Logger function  which could be used for gRPC services as a logging interceptor.

When adding the span context handler and interceptor logger logic, the tests started failing with an error:
--- FAIL: TestGT_InitNewProject (0.05s)
    --- FAIL: TestGT_InitNewProject/generates_folder_in_target_dir_and_initializes_it_with_go.mod_and_.git (0.01s)
        new_test.go:505:
                Error Trace:    /Users/pinto_ramos_/mygo/github.com/SchwarzIT/go-template/pkg/gotemplate/new_test.go:505
                Error:          Received unexpected error:
                                template: :25: function "Name" not defined
                Test:           TestGT_InitNewProject/generates_folder_in_target_dir_and_initializes_it_with_go.mod_and_.git

After some debugging this error was coming from the text template parser, as it read through the test files, found two open braces and thought it should be handling a template. To address this issue, I've changed pkg/gotemplate not to parse the content with the text template if it is a *_test.go file. This approach can be discussed better in case we'd prefer to have an exclusion list for specific paths, instead.
Closes #311

Hi @zalgonoise, I see that you wrote all test cases manually for the logger. Go provides slogtest, which tests every function of a slog handler. We could use this :)

I also edited the title to fit the conventional commits convention, as we usually use it:

We can discuss getting rid of it, and use GitHub labels instead. This would also make the releases prettier with GitHubs changelog generator.

Hi @zalgonoise, I see that you wrote all test cases manually for the logger. Go provides slogtest, which tests every function of a slog handler. We could use this :)
It checks that all keys and values are printed, that the groups work, and stuff like that.

Hi Marvin :D I also looked into slogtest, at least when adding the span context middleware, but the reason why I scratched for simpler, more straight-forward tests, was because this package seems to be more prepared to test the actual slog.Handler implementations in the context of writing logs to an output.
So, when using the interface as middleware, it felt a bit like I was just testing the standard library's JSON slog.Handler just to look for an attribute! But give me some time to read through the package a bit more, if I can find a good way to use it without checking the other attributes :)

I just found this one minor simplification. Nice job!

@johannes-riecken awesome input! Thanks for noticing, I don't know how I let that slide! :D

Looking into this (and other) PR's CI jobs, it seems that the semgrep one is failing:
example job
Prepare workflow directory
Prepare all required actions
Getting action download info
Error: Unable to resolve action `returntocorp/semgrep-action@v1`, unable to find version `v1`

Looking into semgrep's action on Github, it seems that they've marked it as deprecated. They point to the following guide for Github
The problem that I am having is their requirement to login in order to have the Sarif output that is used for github/codeql-action/upload-sarif. I tried running the semgrep container locally with:
docker run -it -v "${PWD}:/src" returntocorp/semgrep:latest semgrep ci --config 'p/auto'

The command above runs fine (and we can include it in the Makefile as well). However if I add the --sarif flag, it asks to login first. I guess that this is where the Github token comes in, in their guide.
docker run -it -v "${PWD}:/src" returntocorp/semgrep:latest semgrep ci --sarif

run `semgrep login` before using `semgrep ci` or set `--config`

Does anyone have a better idea on how we should approach this change, for semgrep? :)

@MarvinJWendt another update regarding slogtest :D
Unfortunately, we are unable to use it for this particular wrapper / decorator. The problem with slogtest is that it has zero tests using context!!
While that makes it a great suggestion for the experimental repo, we need that context for those tests. A regular (*slog.Logger).Info call uses a context.Background() when it hits the Handle call in the slog.Handler :)

Blocked by #328

I noticed that I left an unresolved comment, it's already as an accepted change :D @johannes-riecken would you like to take another look? :)

GOMAXPROCS seems to be using logfmt. Can we make it use our logger? 🤔


Add possibleMissingProperty to DiagnosticGroup

This new flag allows for disabling missing properties for unknown types.
This behaviour is desired when you have mixed typed and untyped objects.
In this case you still want the compiler to complain about missing
properties for known types but not for the untyped objects.
Example when this flag is disabled using
'--jscomp_error=missingProperty'
'--jscomp_off=possibleMissingProperty':

// Typed variable example
/** @constructor */
var A = function() {};
new A().foo; // This will emit an error about 'foo' undefined on 'A'.

// Untyped variable example
/** @type {?} */
var a = {};
a.foo; // No error is emitted.

This change is

Why?

I want the ability to use the compiler's type checking feature for our NodeJS applications. Using the --check-only flag allows me to do just that without having to execute the compilation phase.
All of the code we write follows the goog.module(...); goog.require(...) module strategy. However, since we rely on third-party libraries, we use NodeJS's module strategy with require(...).
Here is an example of using this:
// In an externs file named globals.js
/**
 * @param {!string} path
 * @return {?}
 */
var require = function(path) {};
// End of globals.js.
// ---
// Database.js:
goog.module('Database');
/** @constructor */
var Database = function() {};
Database.prototype.connect = function() {};
exports = Database; // To satisfy google closure compiler.
module.exports = Database; // To satisfy NodeJs module.
// End of Database.js
// ---
// Application.js:
goog.module('Application');
const Database = goog.require('Database');
const log = require('log4js').getLogger();

var db = new Database();
db.connect();
db.foo(); // Error missing property.
log.info('Done.'); // Error possible missing property.
// End of Application.js

The desired behavior when compiling this code is to complain about 'db.foo' as it is of a 'known type'. However, the actual output of the compilation complains about 'db.foo' and 'log.info'. Hence, the need for this possibleMissingProperty.
With this new flag, I can distinguish between the missingProperty errors set by jscomp_error=missingProperty and possibleMissingProperty warnings (set by jscomp_error=possibleMissingProperty. Those possibleMissingProperty warnings can be fixed eventually in extern files or not care about them at all. (catch those errors during unittests).
This will allow for mixing in third-party libraries with my code. My code follows the 'goog.module' conventions for defining types. Third-party libraries don't and won't. I want the compiler to catch my mistakes and not care about third-party libraries.

@dimvar Is there anything I can help with in testing or verifying to make this pull request go in as smooth as possible?

The compiler only warns for a missing property on unknown, when this property is not defined on any type in the program. That is a very loose check, and it's useful for catching typos in property names. If you want to avoid these warnings, it's easy to do so by defining them on some type in the externs.
/** @type {?} */
SomeRandomType.prototype.info;

I'd rather not add a new flag for this. The compiler has too many flags already.

Sounds good. Thanks for the update.


Android crash strlen+16 signal 11 (SIGSEGV), code 1 (SEGV_MAPERR)

This is same as #2547 but I couldn't reopen it for discussion. Thus, I created a new one.
Same as this one #2746
Our android app based on the iris_tracking_gpu.pbtxt graph. When getting pushed to Play Stored, we receive the following crash on Samsung Galaxy Note5 which runs Android 7.0 (SDK 24).
backtrace:
  #00  pc<PHONE_NUMBER>01b5f8  /system/lib64/libc.so (strlen+16)
  #00  pc<PHONE_NUMBER>aa0afc  /data/app/xyz-1/split_config.arm64_v8a.apk!lib/arm64-v8a/libiris_jni.so (offset 0x1000) (mediapipe::PathToResourceAsFile(std::__ndk1::basic_string<char, std::__ndk1::char_traits<char>, std::__ndk1::allocator<char> > const&))
  #00  pc<PHONE_NUMBER>54468c  /data/app/xyz-1/split_config.arm64_v8a.apk!lib/arm64-v8a/libiris_jni.so (offset 0x1000) (mediapipe::LocalFileContentsCalculator::Open(mediapipe::CalculatorContext*))
  #00  pc<PHONE_NUMBER>b1126c  /data/app/xyz-1/split_config.arm64_v8a.apk!lib/arm64-v8a/libiris_jni.so (offset 0x1000) (mediapipe::CalculatorNode::OpenNode())
  #00  pc<PHONE_NUMBER>afb450  /data/app/xyz-1/split_config.arm64_v8a.apk!lib/arm64-v8a/libiris_jni.so (offset 0x1000) (mediapipe::internal::SchedulerQueue::OpenCalculatorNode(mediapipe::CalculatorNode*))
  #00  pc<PHONE_NUMBER>afb1ec  /data/app/xyz-1/split_config.arm64_v8a.apk!lib/arm64-v8a/libiris_jni.so (offset 0x1000) (mediapipe::internal::SchedulerQueue::RunNextTask())
  #00  pc<PHONE_NUMBER>b31fd0  /data/app/xyz-1/split_config.arm64_v8a.apk!lib/arm64-v8a/libiris_jni.so (offset 0x1000) (mediapipe::ThreadPool::RunWorker())
  #00  pc<PHONE_NUMBER>b31c20  /data/app/xyz-1/split_config.arm64_v8a.apk!lib/arm64-v8a/libiris_jni.so (offset 0x1000) (mediapipe::ThreadPool::WorkerThread::ThreadBody(void*))
  #00  pc<PHONE_NUMBER>068258  /system/lib64/libc.so (__pthread_start(void*)+196)
  #00  pc<PHONE_NUMBER>01dc00  /system/lib64/libc.so (__start_thread+16)

We also used that phone during development and didn't face such crash. It does not always crash because after it crashed, closed the app and reopened it the it worked!
According to https://source.android.com/devices/tech/debug/native-crash#nullpointer, this is just a pure null pointer dereference.
Tracing the source code and all the involved graph files, I suspect the argument path in the following method of file resource_util_default.cc
absl::StatusOr<std::string> PathToResourceAsFile(const std::string& path) {
  return JoinPath(absl::GetFlag(FLAGS_resource_root_dir), path);
}

That path is provided via a side package input in declared in the graph face_landmarks_model_loader.pbtxt
node {
  calculator: "SwitchContainer"
  input_side_packet: "ENABLE:with_attention"
  output_side_packet: "PACKET:model_path"
  options: {
    [mediapipe.SwitchContainerOptions.ext] {
      contained_node: {
        calculator: "ConstantSidePacketCalculator"
        options: {
          [mediapipe.ConstantSidePacketCalculatorOptions.ext]: {
            packet {
              string_value: "mediapipe/modules/face_landmark/face_landmark.tflite"
            }
          }
        }
      }
      contained_node: {
        calculator: "ConstantSidePacketCalculator"
        options: {
          [mediapipe.ConstantSidePacketCalculatorOptions.ext]: {
            packet {
              string_value: "mediapipe/modules/face_landmark/face_landmark_with_attention.tflite"
            }
          }
        }
      }
    }
  }
}

# Loads the file in the specified path into a blob.
node {
  calculator: "LocalFileContentsCalculator"
  input_side_packet: "FILE_PATH:model_path"
  output_side_packet: "CONTENTS:model_blob"
}

Is that possible, under a special condition, the output_side_packet: "PACKET:model_path" if the SwitchContainer emit a null path to the input_side_packet: "FILE_PATH:model_path" of LocalFileContentsCalculator? As I said earlier, it doesn't always happen, on the  same phone that crashed, relaunching the app did not crash

Hi @tri-bao,
Could you share steps to reproduce this issue.

Hi @sureshdagooglecom,
When it happened, my app was uploaded to google Play Store. The app bases on an AAR that was built using the graph iris_tracking_gpu.pbtxt with the following bazel options:
          --compilation_mode=opt \
          --spawn_strategy=local \
          --strip=ALWAYS \
          --host_crosstool_top=@bazel_tools//tools/cpp:toolchain \
          --fat_apk_cpu=armeabi-v7a,arm64-v8a

As I mentioned earlier, this crash does not always occur. After it crashed, I sent the crash log to Play Store as you can see in my origin comment, I killed the app and reopened it, then it doesn't crash any more. Like the person in #2547 It happens once and not again on the same phone.
I wonder if is there any chance at a very special moment that the SwitchContainer emit the model_path side packet out-of-sync and it will be null when the LocalFileContentsCalculator retrieve it.
For my app, since we never use the face_landmark_with_attention.tflite model (mainly due to this issue #2935), to be safer for production app, I decided to replace the whole face_landmarks_model_loader subgraph with a fixed model path, something like following in the face_landmark_gpu.pbtxt
node {
  calculator: "InferenceCalculator"
  input_stream: "TENSORS:input_tensors"

#  input_side_packet: "MODEL:model"

  input_side_packet: "CUSTOM_OP_RESOLVER:op_resolver"
  output_stream: "TENSORS:output_tensors"
  options: {
    [mediapipe.InferenceCalculatorOptions.ext] {
      # Do not remove. Used for generation of XNNPACK/NNAPI graphs.

      model_path: "mediapipe/modules/face_landmark/face_landmark.tflite"
    }
  }
}

Hi @tri-bao ,
It does look like a sync problem, could you please share complete error logs.

hi @sureshdagooglecom ,
I only have the backtrace that I've already posted on the initial comment of this ticket. That is what I caught from Google Play Console. If you mean the whole console log of Android Studio, I don't have it as we have never seen that error during development.

please don't close, bot :)

Hi @tri-bao ,
Replace the SwitchContainer and ConstantPacketCalculator with a simpler file path,
see if the crash can still happen.
Suggest: accept "FILE_PATH" side packet as an input side packet into the graph.

as I mentioned in my previous comment, that was what I did https://github.com/google/mediapipe/issues/3267#issuecomment-1101277893

I also met this issue. Reproduce rate less than 1/100.
pid: 3345, tid: 3621, name: mediapipe/3621  >>> com.blake.aircmd <<<
uid: 10326
x0  0000006fe1b38f40  x1<PHONE_NUMBER>000080  x2<PHONE_NUMBER>000000  x3<PHONE_NUMBER>000000
x4  feff303135322e64  x5  feff303135322e64  x6  feff303135322e64  x7  7f7f7f7f7f7f7f7f
x8<PHONE_NUMBER>000062  x9<PHONE_NUMBER>000000  x10<PHONE_NUMBER>000001  x11<PHONE_NUMBER>000001
x12 0000006eb7048de8  x13<PHONE_NUMBER>003039  x14<PHONE_NUMBER>000000  x15<PHONE_NUMBER>000030
x16 0000006eb72f3700  x17 0000006fcb55dd50  x18 0000006ea5a7a000  x19<PHONE_NUMBER>000000
x20 0000006fe1b38f40  x21 0000006eb7048edb  x22 0000006eb7048ee5  x23<PHONE_NUMBER>000001
x24<PHONE_NUMBER>000881  x25<PHONE_NUMBER>000000  x26 0000006ea5c77ff8  x27<PHONE_NUMBER>0fc000
x28 0000006ea5b7f000  x29 0000006ea5c77c50
lr  0000006eb6e87c20  sp  0000006ea5c779a0  pc  0000006fcb55dd70  pst<PHONE_NUMBER>001000
backtrace:
#00 pc<PHONE_NUMBER>086d70  /apex/com.android.runtime/lib64/bionic/libc.so (syscall+32) (BuildId: 33005d19716b95e4cd346b0d7c61d055)
#01 pc<PHONE_NUMBER>be1c1c  /data/app/~~UXyRPyHfeakBKcnCBG0vbg==/com.blake.aircmd-nCy-jAivafNv7ii9EotlOQ==/base.apk (absl::lts_20210324::synchronization_internal::FutexImpl::WaitUntil(std::__ndk1::atomic, int, absl::lts_20210324::synchronization_internal::KernelTimeout)+108)
#02 pc<PHONE_NUMBER>be1b48  /data/app/~~UXyRPyHfeakBKcnCBG0vbg==/com.blake.aircmd-nCy-jAivafNv7ii9EotlOQ==/base.apk (absl::lts_20210324::synchronization_internal::Waiter::Wait(absl::lts_20210324::synchronization_internal::KernelTimeout)+156)
#03 pc<PHONE_NUMBER>be1a6c  /data/app/~~UXyRPyHfeakBKcnCBG0vbg==/com.blake.aircmd-nCy-jAivafNv7ii9EotlOQ==/base.apk (AbslInternalPerThreadSemWait_lts_20210324+72)
#04 pc<PHONE_NUMBER>be1040  /data/app/~~UXyRPyHfeakBKcnCBG0vbg==/com.blake.aircmd-nCy-jAivafNv7ii9EotlOQ==/base.apk (absl::lts_20210324::CondVar::WaitCommon(absl::lts_20210324::Mutex, absl::lts_20210324::synchronization_internal::KernelTimeout)+172)
#05 pc<PHONE_NUMBER>b7e7e0  /data/app/~~UXyRPyHfeakBKcnCBG0vbg==/com.blake.aircmd-nCy-jAivafNv7ii9EotlOQ==/base.apk (mediapipe::ThreadPool::RunWorker()+164)
#06 pc<PHONE_NUMBER>b7e518  /data/app/~~UXyRPyHfeakBKcnCBG0vbg==/com.blake.aircmd-nCy-jAivafNv7ii9EotlOQ==/base.apk (mediapipe::ThreadPool::WorkerThread::ThreadBody(void*)+796)
#07 pc<PHONE_NUMBER>0efbf4  /apex/com.android.runtime/lib64/bionic/libc.so (__pthread_start(void*)+264) (BuildId: 33005d19716b95e4cd346b0d7c61d055)
#08 pc<PHONE_NUMBER>08c4d0  /apex/com.android.runtime/lib64/bionic/libc.so (__start_thread+68) (BuildId: 33005d19716b95e4cd346b0d7c61d055)

I've also encountered this issue. Can confirm it gets fixed by providing model path directly to InferenceCalculator like:
node {
  calculator: "InferenceCalculator"
  input_stream: "TENSORS:input_tensors"
  output_stream: "TENSORS:output_tensors"
  options: {
    [mediapipe.InferenceCalculatorOptions.ext] {
      model_path: "mediapipe/modules/pose_landmark/pose_landmark_lite.tflite"
    }
  }
}

From that it looks like root cause of the problem is in either SwitchContainer, ConstantSidePacketCalculator or LocalFileContentsCalculator.

Hello @tri-bao,
We are upgrading the MediaPipe Legacy Solutions to new MediaPipe solutions However, the libraries, documentation, and source code for all the MediapPipe Legacy Solutions will continue to be available in our GitHub repository and through library distribution services, such as Maven and NPM.
You can continue to use those legacy solutions in your applications if you choose. Though, we would request you to check new MediaPipe solutions which can help you more easily build and customize ML solutions for your applications. These new solutions will provide a superset of capabilities available in the legacy solutions. Thank you


Contact section

successfully modified the contact section from
https://github.com/user-attachments/assets/fcf2e4a2-e6ce-4551-b18a-11d2bfe70dc4
to
https://github.com/user-attachments/assets/4fd9735d-d97e-401d-bf13-9dab4d9bc810
please label hactoberfest  accepted and gssoc ext

the image should be in contact form only @abhi200446  other guy worked on this issue please work on new issue
try to work on this issue like including all social icons here too

the image should be in contact form only @abhi200446 other guy worked on this issue please work on new issue try to work on this issue like including all social icons here too

please assign this issue @sailaja-adapa

Git will assign automatically


Feature request: Allow cancelling of Firebase Storage put

There may already be a solution to this, but I cannot find a way to cancel a put in progress to firebase storage. If I inadvertently put a large file, is there some way that I can cancel the upload (especially useful for low-bandwidth situations) in progress?
For example:
const meta = {
  contentType: file.type
};
const ref = this.fb.storage().ref().child(path).put(file, meta);
ref.cancel();

Thanks for any guidance,
Hal

i think this page may help you:
https://blog.angular.io/file-uploads-come-to-angularfire-6842352b3b47

@hlesesne is that blog link help you?

Hello, I followed the blog link and implemented the controls pause, resume, and cancel.
I'm having an issue with cancel, it can't cancel when it's paused.
const cancelResult = task.cancel();
this.logger.debug('cancelResult:', cancelResult); // false

the only way to cancel it, would be to resume it first, then cancel it.
so this is my current work-around:
let cancelResult = task.cancel(); // returns false
if (!cancelResult) {
  task.resume();
  cancelResult = task.cancel();
  this.logger.debug('cancelResult:', cancelResult); // true
}

wondering if anyone is having this issue.

@sam-lex I am experiencing the same behavior and I think it's intended.


Create npm package

Creating your first npm package
The goal is to create an NPM package to make the code even more accessible to developers. 🎉

I can do this.

I can do this.

Perfect, all yours!


Enable CI for ACCELQ

Link to GitHub repository

https://github.com/jenkinsci/accelq-ci-connect-plugin.git
When modifying release permission
If you are modifying the release permission of your plugin or component, fill out the following checklist:

### Release permission checklist (for submitters)
- [ ] The usernames of the users added to the "developers" section in the .yml file are the same the users use to log in to [accounts.jenkins.io](https://accounts.jenkins.io/).
- [ ] All users added have logged in to [Artifactory](https://repo.jenkins-ci.org/) and [Jira](https://issues.jenkins.io/) once.
- [ ] I have mentioned an [existing team member](https://github.com/orgs/jenkinsci/teams) of the plugin or component team to approve this request.

When enabling automated releases (cd: true)
Follow the documentation to ensure, your pull request is set up properly. Don't merge it yet.
In case of changes requested by the hosting team, an open PR facilitates future reviews, without derailing work across multiple PRs.
Link to the PR enabling CD in your plugin

https://github.com/jenkinsci/accelq-ci-connect-plugin/pull/17
https://github.com/jenkinsci/accelq-ci-connect-plugin/pull/18
https://github.com/jenkinsci/accelq-ci-connect-plugin/pull/16
### CD checklist (for submitters)
- [x] I have provided a link to the pull request in my plugin, which enables CD according to the documentation.

### Reviewer checklist
- [ ] Check that the `$pluginId Developers` team has `Admin` permissions while granting the access.
- [ ] In the case of plugin adoption, ensure that the Jenkins Jira default assignee is either removed or changed to the new maintainer.
- [ ] If security contacts are changed (this includes add/remove), ping the security officer (currently `@Wadeck`) in this pull request. If an email contact is changed, wait for approval from the security officer.

There are IRC Bot commands for it.

As per above image, it shows get it merged than look for MAVEN_TOKEN and MAVEN_USERNAME then proceed. Isn't that the case?

As per above image, it shows get it merged than look for MAVEN_TOKEN and MAVEN_USERNAME then proceed. Isn't that the case?

That is part of the procedure, but there is more to do than just setting cd to true, as you can see if you read the page you are at and the three bullet points I've outlined above.

Here is the PR: https://github.com/jenkinsci/accelq-ci-connect-plugin/pull/19


ENH: move to cibuildwheel

Is your feature request related to a problem? Please describe.
Now that 1.9 is almost out the door, it is time to start thinking about migrating the multibuild Macpython/scipy-wheels build into cibuildwheel here. The first 80% of the should be "copying what numpy does", the last 80% of the work will need to be scipy-specific.
Describe the solution you'd like.
Have cibuildwheel wheel builds directly from this repo
Describe alternatives you've considered.
we could change the Macpython/scipy-wheels repo to use cibuildwheel instead of doing it here, but I think there is value in having this repo directly produce wheels.
Additional context (e.g. screenshots, GIFs)
No response

My experience with NumPy is that releases have become easier using cibuildwheel.

I thought I already started in https://github.com/scipy/scipy/pull/15925, just got busy dealing with release and stuff.

Ahh, cool. I searched issues but not PRs.

This would be nice to start moving forward indeed. My attempts at building 1.9.0 wheels with Meson failed not because of any problems building wheels with Meson, but because multibuild is too painful to deal with.

Also, I should mention I won't be offended if say someone who currently has a bit of funding takes this over. I spent most of this weekend cramming grant proposals for supercomputer time on Venado (upcoming NVIDIA Grace-Hopper architecture supercomputer).
I suspect I may not get funded until FY23 (October 2022) for OSS contribution work, though I could be wrong, so we may prefer to just have me floating around if it just volunteer time and you really want to drive this forward quickly.

I've been experimenting with cibuildwheel locally. As part of the preliminary setup with CIBW_BEFORE_BUILD I installed openblas using the openblas_support.py script as done here.
However, when I run the build step using python3.8 -m build . I get:

Run-time dependency openblas found: NO (tried pkgconfig and cmake)
../../scipy/meson.build:131:0: ERROR: Dependency "openblas" not found, tried pkgconfig and cmake

@rgommers how does cmake find the openblas setup? The include files are in /usr/local/include. The *.so are in /usr/local/lib, and the cmake config files are in /usr/local/lib/cmake/openblas/.
# cat /usr/local/lib/cmake/openblas/OpenBLASConfig.cmake
SET(OpenBLAS_VERSION "0.3.18")
SET(OpenBLAS_INCLUDE_DIRS /usr/local/include)
SET(OpenBLAS_LIBRARIES /usr/local/lib/libopenblas.so)

The best solution would be for the openblas package to provide a proper pkg-config file. Then the build could add it to the PKG_CONFIG_PATH as is done currently for windows https://github.com/scipy/scipy/blob/69f8322c556e8bf2d832c6457c93f8574a1149a6/.github/workflows/windows.yml#L56
This is one of the goals of repackaging openblas in openblas-libs#86.

CMake detection was indeed fixed in gh-16750

Also, I should mention I won't be offended if say someone who currently has a bit of funding takes this over.

I'll see if we can work this out @tylerjereddy.

I'm happy to work on this, I've already put some familiarisation work into this.

Great, thanks @andyfaff!

closed by #16842


Temporarily skip tests

Disabling tests until https://github.com/dotnet/cli/issues/5172 is resolved.
Tracking work item to re-enable tests: https://github.com/aspnet/Localization/issues/310

FYI @Eilon


fix(xform): support falsy values as valid

That PR allow falsy values to be considered as valid, as it is currently impossible to pass false or null or "" or 0

<EMAIL_ADDRESS>


Fix issue with item invoke and out of sync selection

Description

Set the selected item before we raise selection changed event.
Motivation and Context

Fixes #2315
How Has This Been Tested?

Added new interaction test
Screenshots (if appropriate):

    // To keep the logic the same as RS4, ItemInvoke is before unselect the old item

According to this comment, we should raise ItemInvoked before the call to UnselectPrevItem to keep behavior consistent with what it was before.

Refers to: dev/NavigationView/NavigationView.cpp:1935 in c08d763. [](commit_id = c08d763badbc0154f49139621f1443c59afb13c0, deletion_comment = False)

Right, updated the order of methods calls so we raise the event first and then change the selection states.

/azp run

@chingucoding looks like there were a number of test failures. Please take a look.

@chingucoding looks like there were a number of test failures. Please take a look.

I will take a look, thanks for ping.

Fixed the test issues now.

/azp run


Protecting from import leak by fetching package one directory deeper

There were complaints in the NG that DUB does'nt support libraries whose source files are at the top-level directory.
Reference<EMAIL_ADDRESS>- When cloning repositories, dub does not preserve the
repository's directory name (so e.g. fruit will be cloned to
~/.dub/fruit-1.0.0/).

Somebody has created a Dub package for my library (excluding
certain packages, due to point 1a above), and the other day
someone complained that it doesn't work with Dscanner, because of
the above issue - the module path "ae.dir.module" does not
correspond to the filesystem path "ae-1.0.1/dir/module.d".

I think this is mostly solvable by fetching dependencies one directory deeper :
Reference<EMAIL_ADDRESS>Currently, using ae checkouted in an ae/ directory means having
an "import leak" meaning that you have to import a path outside
the checkouted repositery. That's somehow unhygienic like, er, a
global namespace. But yeah DUB could isolate checkouts one
directory deeper (ae-1.0.0/ae/) to make this a non-problem, this
would support libraries like ae/ and make DScanner work.

What do you think?

This prevent to use the ae library from DUB.
ae is a very useful library with the best command-line parser, best image library in the ecosystem, a HTTP server that supports DFeed, etc...

I think this is a viable workaround. The only thing that should ideally be made sure is that old directory hierarchies get upgraded to the new scheme when the new DUB version is first used.


Feature/input output key value props

Updated custom configuration properties in all the inputs and outputs in order to use a generic implementation.

Coverage decreased (-0.03%) to 51.369% when pulling 544d67f58878e9d3296e609c2e01668987884263 on mtelloz:feature/input-output-key-value-Props into 1ffa459e27bfa0d8d35c75461e31fb62825ca07e on Stratio:master.

Coverage decreased (-0.03%) to 51.369% when pulling 566868a7be40d2304a87d367b76652844fb02484 on mtelloz:feature/input-output-key-value-Props into 1ffa459e27bfa0d8d35c75461e31fb62825ca07e on Stratio:master.


Any way to customize the CSS?

I'd like a slightly different CSS output in *-icons.css (specifically change text-decoration: inherit to text-decoration: none). What would be the easiest way to accomplish this?
For now I've settled with simply overriding it with my own CSS, but I'd like to change one more thing and that is to add the ability to place an icon after an element's contents by giving it a class of "icon--after" and changing the generated CSS from:
.icon-search:before { content: '\e800'; }

To:
.icon-search:before { content: '\e800'; }
.icon-search.icon--after:before { content: normal; }
.icon-search.icon--after:after { content: '\e800'; }

And that would require actually modifying the generated CSS rather than overriding (which would be very tedious every time a new icon is added)
Edit: Now that I think about it I guess all you do is actually download the CSS generated by Fontello? In which case it can't be changed within the Gulp plugin... Perhaps I can write my own Gulp plugin on top of it instead.

I ended up solving it with a simple Gulp Replace;
// Adds support for icons _after_ text using an .after class
gulp.task('icon-after-support', ['get-icon-font'], function () {
    var fontName = 'trumpet';
    var afterClass = 'after';

    var iconFind = /\.icon-(.*?):before {(.*?)}/g;
    var iconReplace = '.icon-$1:before {$2}\n.icon-$1.' + afterClass + ':before {content: normal}\n.icon-$1.' + afterClass + ':after {$2}';

    var baseFind = /\[class\^="icon-"\]:before, \[class\*=" icon-"\]:before/g;
    var baseReplace = '[class^="icon-"]:before, [class*=" icon-"]:before,\n[class^="icon-"]:after, [class*=" icon-"]:after';

    return gulp.src(paths.fontello + 'css/' + fontName + '-icons.css')
        .pipe(replace(baseFind, baseReplace))
        .pipe(replace(iconFind, iconReplace))
        .pipe(gulp.dest(paths.fontello + 'css/'));
});

This way I can do something like:
<button class="icon-search after">Search</button>

And the icon renders after the text.


The Risk of posix_memalign in Ruy which TensorFlow Lite used.

Hi TF Authors,
We recently got a crash as following when we tflite::Interpreter::Invoke the cartoonized model:
2021-03-09 16:07:22.261 18157-18309 W/libc: memalign(64, 411042816) failed: returning null pointer
2021-03-09 15:55:06.148 551-1048 D/AudioMixer: setDRCHandler, mStreamType: 3, device: 2
2021-03-09 15:55:06.150 17799-17959 E/CRASH: 	#00  pc 001b7b00  /data/app/com.magicpal.magicamiti-VBWn90aZTYIGNTtAk2TwYA==/lib/arm/libtensorflowlite.so
2021-03-09 15:55:06.150 17799-17959 E/CRASH: 	#01  pc 0008cf05  /data/app/com.magicpal.magicamiti-VBWn90aZTYIGNTtAk2TwYA==/lib/arm/libtensorflowlite.so
2021-03-09 15:55:06.150 17799-17959 E/CRASH: 	#02  pc 0008cb99  /data/app/com.magicpal.magicamiti-VBWn90aZTYIGNTtAk2TwYA==/lib/arm/libtensorflowlite.so
2021-03-09 15:55:06.150 17799-17959 E/CRASH: 	#03  pc 001b8e85  /data/app/com.magicpal.magicamiti-VBWn90aZTYIGNTtAk2TwYA==/lib/arm/libtensorflowlite.so
2021-03-09 15:55:06.150 17799-17959 E/CRASH: 	#04  pc 001b8bc1  /data/app/com.magicpal.magicamiti-VBWn90aZTYIGNTtAk2TwYA==/lib/arm/libtensorflowlite.so
2021-03-09 15:55:06.150 17799-17959 E/CRASH: memory near r0:
2021-03-09 15:55:06.150 17799-17959 E/CRASH:     96a4513c 00000000 96a45208 00000000 804ff400  .....R........O.
2021-03-09 15:55:06.150 17799-17959 E/CRASH:     96a4514c 00000000 96a451c0 ade93f09 e57af260  .....Q...?..`.z.
2021-03-09 15:55:06.150 17799-17959 E/CRASH:     96a4515c 4f640380 4f6409a0 00000000 00000000  ..dO..dO........
2021-03-09 15:55:06.150 17799-17959 E/CRASH:     96a4516c 804ff400 00000200 00000010 00000010  ..O.............
2021-03-09 15:55:06.150 17799-17959 E/CRASH:     96a4517c ffffffff ffffffff 00000620 ffffffab  ........ .......
2021-03-09 15:55:06.151 17799-17959 E/CRASH:     96a4522c 00000620 00040000 00000620 00000000   ....... .......

……( It’s too much to display, check whole logs on  the end of this log snippet,  tensor_crash.txt)

2021-03-09 15:55:06.167 17799-17959 E/CRASH:     4f640a40 abababab abababab abababab abababab  ................
2021-03-09 15:55:06.167 17799-17959 E/CRASH:     4f640a50 abababab abababab abababab abababab  ................
2021-03-09 15:55:06.167 17799-17959 E/CRASH:     4f640a60 abababab abababab abababab abababab  ................
2021-03-09 15:55:06.167 17799-17959 E/CRASH:     4f640a70 abababab abababab abababab abababab  ................
2021-03-09 15:55:06.167 17799-17959 E/CRASH:     4f640a80 abababab abababab abababab abababab  ................
2021-03-09 15:55:06.193 17799-17823 E/CRASH: other thread is trapped; signum = 11
2021-03-09 15:55:06.194 17799-17823 E/MessageQueue: IdleHandler threw exception
    java.lang.Error: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 00000200
    Build fingerprint: 'Redmi/merlin/merlin:10/QP1A.190711.020/V<IP_ADDRESS>.QJOCNXM:user/release-keys'
    Revision: '0'
    pid: 17799, tid: 17959, name: UnityMain  >>> com.magicpal.magicamiti <<<
        r0 96a4515c  r1 00000010  r2 00000620  r3 00000620
        r4 00000200  r5 00000010  r6 00000010  r7 96a45150
        r8 ffffffab  r9 96a45228  sl 00000620  fp 96a45208
        ip 4f640390  sp 96a45100  lr 4f6409b0  pc adfbeb00  cpsr 00004627

        at libtensorflowlite.001b7b00(Native Method)
        at libtensorflowlite.0008cf05(Native Method)
        at libtensorflowlite.0008cb99(Native Method)
        at libtensorflowlite.001b8e85(Native Method)
        at libtensorflowlite.001b8bc1(Native Method)
2021-03-09 15:55:06.196 551-1048 D/AudioMixer: setDRCHandler, mStreamType: 3, device: 2
2021-03-09 15:55:06.202 551-1048 D/AudioMixer: setDRCHandler, mStreamType: 3, device: 2

    --------- beginning of system

Whole logs is here, tensor_crash.txt.
And we suspect that the memalign called by Ruy cause this crash quietly.
2021-03-09 16:07:22.261 18157-18309 W/libc: memalign(64, 411042816) failed: returning null pointer

411042816 = 392MB is the exact buffer size of the Tensor 154 in model we mentioned. So, we believe that the Interpreter could be crashed when it memalign a big memory and got  a null pointer on Invoke stage.
However, why we reallocate memory after Interpreter::AllocateTensors? And did we ever considered the cases, which the big temporary buffers would be allocated in the Invoke flow? We are confused that why not assert the return of memalign, and only return a null pointer without any processing? It’s seem dangerous and  really unfriendly for developers to debug.
Looking forward to your reply, thx. @renjie-liu @multiverse-tf @wangtz

@talumbau @renjie-liu  Could you take a look at this bug report?

Chatted with the @SunAriesCN offline yesterday, a couple of things:

The model has a very large intermediate layer which has an output of 1515512*1568
Obviously the interpreter should definitely allocate a buffer larger than that one (what happened in the allocate tensor
Ruy can allocate buffer for faster execution, and in this case, it's allocating 384MB which is the size of the output tensor.
It's fine to add an assert in the align_buffer allocation returned pointer.
Meanwhile, we should probably look into use a model with smaller intermediate tensor.

Thanks for your reply, Renjie. It maybe an implicit demand about the big models inference with TFLite, I think. Because of the privacy polities, we would like running some big models under background threads. So, it will be cool, if we have plans for this in TFLite, and we would like to talk about that.

Hi,
Thank you for opening this issue. Since this issue has been open for a long time, the code/debug information for this issue may not be relevant with the current state of the code base.
The TFLite team is constantly improving the framework by fixing bugs and adding new features. We suggest you try the latest TensorFlow version with the latest compatible hardware configuration which could potentially resolve the issue. If you are still facing the issue, please create a new GitHub issue with your latest findings, with all the debugging information which could help us investigate.
Please follow the release notes to stay up to date with the latest developments which are happening in the TFLite space.


High contrast photometric calibration images

Hi,
I turned the VERBOSE option to ON in the CMakeLists.txt, then the rviz can receive the trajectory topic and tracked feature topic. But, as shown in the following figure, the calibrated thermal images are much brighter than the result in the paper, how can I get the enhanced thermal images as shown in the paper? By the way, what's the black line in the calibrated images?
The calibrated image:

The final estimated trajectory is not bad:

hi @stephen9999999 ,
The black lines on the image represent the windows where a persistent landmark should be found. In other words, when you set the number of tiles in the parameters file, then the X library tries to distribute the landmarks more uniformly over the windows.
Regarding the bright images, you can play with the params epsilon_gap and epsilon_base in the yaml file to have a darker image. Refer to the paper Online Photometric Calibration of Automatic Gain Thermal Infrared Cameras
 for more details.
Best,
Vincenzo

Hi,
Thanks for your suggestions.
I have read the paper and tried to adjust the params but still cannot  reproduce the results in the paper. With the PHOTOMETRIC_CALI ON, the contrast of images is lower than the raw images, and the noise is amplified.  Therefore, the tracked feature number is less than the original images.
Anything wrong? Looking forward to your suggestions. Thanks .

Hi,
This #5 should help. It can bring data race problems, but we tested with this piece of code that was missing in this release. Please let me know if it works for you.
Best,
Vincenzo

Hi @stephen9999999, I will close this issue since there is not much activity here, and it should be solved. Please feel free to reopen it if you need to. Thanks.
Best,
Vincenzo

Hi,
I have turned the VERBOSE option to ON ,but the rviz still can't  receive the trajectory topic and tracked feature topic. Can you tell me some details about it ?


Add mobile components to service view

Adding service integrations and service-client-integrations to the service instance view

@JameelB I think this means this can be closed. Is that correct? This has the updated integrations and mobile clients components

@JameelB thanks


Increment version for core releases

Increment package version after release of com.azure azure-core-amqp

Closing in favor of https://github.com/Azure/azure-sdk-for-java/pull/17577


Cache Manifest files

Closes #595.

One test is failing CI (test_duckdb_url_import).
FAILED tests/integration/test_writes/test_writes.py::test_duckdb_url_import - duckdb.duckdb.IOException: IO Error: Failed to download extension "iceberg" at URL "http://extensions.duckdb.org/v0.10.3/linux_amd64_gcc4/iceberg.duckdb_extension.gz"
Extension "iceberg" is an existing extension.
 (ERROR Connection)
= 1 failed, 797 passed, 5 skipped, 2735 deselected, 160 warnings in 261.13s (0:04:21) =
make: *** [Makefile:46: test-integration] Error 1
Error: Process completed with exit code 2.

Seems to be a download error and not due to changes in this PR.
Also, the same test passes locally. Can we re-trigger the tests?

Hmm, it's my first time to see this error. I've merged a PR that bumps duckdb to 1.0.0: #793. Hope that can fix the issue here.

@Fokko @HonahX Thanks for the review!!

@chinmay-bhat @Fokko Sorry for the last-minute feedback, but wouldn't it be better if we explicitly pass
(maxsize=128) for the lru_cache annotation, When a new call comes in, the decorator’s implementation will evict the least recently used of the existing 128 entries to make a place for the new item. that way it would make it clearer for the end user and increase the code visibility, in the near future we can also take that as a user config via pyiceberg.yaml
@lru_cache(maxsize=128)
def _manifests(io: FileIO, manifest_list: str) -> List[ManifestFile]:
    """Return the manifests from the manifest list."""
    file = io.new_input(manifest_list)
    return list(read_manifest_list(file))

I'm not sure why test_duckdb_url_import continues to fail with the same error. I've rebased onto latest main branch, and the test runs locally. I also don't see this issue in other open PRs which have cleared all tests.
Is this a duck db problem? Or do I need to open a new PR (from main branch + my changes) to resolve it?

Being able to configure (and also disable) the cachine would be a very nice
touch
Op ma 10 jun 2024 om 09:52 schreef Chinmay Bhat @.***>

@.**** commented on this pull request.
In pyiceberg/table/snapshots.py
https://github.com/apache/iceberg-python/pull/787#discussion_r1632758206
:

@@ -228,6 +229,13 @@ def eq(self, other: Any) -> bool:
)

@.***_cache
As the default is 128
https://docs.python.org/3/library/functools.html#functools.lru_cache, I
don't think we need to explicitly define maxsize=128.
Also, _manifests() is not a public API, so we would probably not set
the cache size through user config via pyiceberg.yaml. Is there a use-case
where the user would want to set the cache size?
—
Reply to this email directly, view it on GitHub
https://github.com/apache/iceberg-python/pull/787#discussion_r1632758206,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAIU5KBBXVDMDRY62QQEY2LZGVLMNAVCNFSM6AAAAABIUZG46GVHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZDCMBWHEZTSOJSGQ
.
You are receiving this because you were mentioned.Message ID:
@.***>

@chinmay-bhat Looks like the CI is still a bit sad :(

Being able to configure (and also disable) the cachine would be a very nice touch

Hi @Fokko, I'm curious to know why we would want to allow custom sized manifest caches.
I agree user should be able to disable the cache.
Here's my initial implementation for configuring and disabling cache:
<top of file>
MANIFEST_CACHE_SIZE = "manifest-cache-size"

config = Config()
manifest_cache_size = config.get_int(MANIFEST_CACHE_SIZE) if config.get_int(MANIFEST_CACHE_SIZE) else 128
....
@lru_cache(maxsize=manifest_cache_size)
def _manifests(io: FileIO, manifest_list: str) -> List[ManifestFile]:
    """Return the manifests from the manifest list."""
    file = io.new_input(manifest_list)
    return list(read_manifest_list(file))

class Snapshot:
....
    def manifests(self, io: FileIO, use_cache: bool = True) -> List[ManifestFile]:
        """Return the manifests for the given snapshot."""
        if self.manifest_list:
            if use_cache:
                return _manifests(io, self.manifest_list)
            else:
                _manifests.cache_clear()
                return _manifests.__wrapped__(io, self.manifest_list)
        return []

Is this in the right direction?
referencing: how to disable lru_cache and lru_cache doc

Looks like the CI is still a bit sad :(

@Fokko @HonahX I need help resolving this error. Error seems unrelated to the changes in this PR. I've added more info in this comment.

Is this a duck db problem? Or do I need to open a new PR (from main branch + my changes) to resolve it?
I tried this PR locally and did not observe this issue. My testing platforms are

MacOS 14.5 - Apple Silicone
Ubuntu 24.04 - Intel x86/64

It seems this issue can only be reproduced in github action😱:

with lru_cahce, test failed: https://github.com/HonahX/iceberg-python/pull/4
without lru_cache, test passed: https://github.com/HonahX/iceberg-python/pull/5

I created a new PR against my fork, and once the GitHub actions failed, I manually re-tried them.
https://github.com/chinmay-bhat/iceberg-python/pull/1/checks?sha=8c2e79a9c62f98c51eb56ae65369a7bf3f6d49f4
With lru_cache,
Python Integration / integration-test (make test-integration) clears all tests on GitHub and local.
Python CI / lint-and-test (make test-coverage) fails.
Also, with lru_cache, test-integration and test-coverage both clear locally.
My environment is MacOS 14.4.1 - Apple Silicon M2

FYI this is caused by transient networking failures when downloading the duckdb iceberg extension. Using the pre-release duckdb library<EMAIL_ADDRESS>resolves this issue because retry was added in duckdb#13122.
See duckdb#13808 for more info. Might need to wait for duckdb v1.1.0 release

#1149 bumps duckdb to v1.1.0 which has the PR to retry extension installation

@chinmay-bhat do you mind rebasing off main to re-run CI?

rebased off main!


Server error

Its a server related request. It shows "not able to connect, standard network error".
Now on console it is showing


SSD-MobileNet outputs the same image_loc

I've run full evaluation on the COCO 2017 validation set using SSD-MobileNet as follows:
$ ck install package --tags=mlperf,object-detection,model,tf,ssd-mobilenet,non-quantized
$ ln -s `ck locate env --tags=model,tf,ssd-mobilenet,non-quantized`/{frozen_inference_graph.pb,ssd_mobilenet_v1_coco_2018_01_28.pb}
$ export MODEL_DIR=`ck locate env --tags=model,tf,ssd-mobilenet,non-quantized`
$ export DATA_DIR=`ck locate env --tags=dataset,coco,2017,val`
$ export EXTRA_OPS="--accuracy --count 5000 --scenario Offline"
$ ./run_and_time.sh tf ssd-mobilenet cpu

When I examine output/ssd-mobilenet-tf-cpu/TestScenario.Offline.json, I see that all the 21981 bounding boxes detected for the 5000 images have the same image_loc:
[
    {
        "bbox": [
<PHONE_NUMBER>3381348,
<PHONE_NUMBER>7229004,
<PHONE_NUMBER>6589355,
<PHONE_NUMBER>583679
        ],
        "category_id": 61,
        "image_id": 255824,
        "image_loc": "/home/anton/CK_TOOLS/dataset-coco-2017-val/val2017/000000003661.jpg",
        "score": 0.9780274629592896
    },
    {
        "bbox": [
<PHONE_NUMBER>919922,
            0.14655780792236328,
<PHONE_NUMBER>5527344,
<PHONE_NUMBER>6894226
        ],
        "category_id": 47,
        "image_id": 255824,
        "image_loc": "/home/anton/CK_TOOLS/dataset-coco-2017-val/val2017/000000003661.jpg",
        "score": 0.7211153507232666
    },
    {
        "bbox": [
<PHONE_NUMBER>7871094,
            59.856241285800934,
<PHONE_NUMBER>6503906,
<PHONE_NUMBER>725029
        ],
        "category_id": 54,
        "image_id": 417608,
        "image_loc": "/home/anton/CK_TOOLS/dataset-coco-2017-val/val2017/000000003661.jpg",
        "score": 0.9599727392196655
    },
...

In a similarly obtained SingleStream moded, image_loc is something else: /home/anton/CK_TOOLS/dataset-coco-2017-val/val2017/000000186980.jpg.

@jveejay  @psyhtest What is requested here from Nvidia?

for some reason I feel guilty of something :)
I have a hunch that I grab the wrong url when dealing with batchsize != 1. Will take a look.

@DilipSequeira Nothing I can think of :)

Beats me, you must've popped into my head and I thought oh fun I should have @DilipSequeira look at this. LOL. Just kidding, not sure why I did that Dilip. Sorry about that.

Hello, it's a silly bug due to using the index of the previous loop in line 200 of coco.py:
"image_loc": ds.get_item_loc(image_ids[idx]),
I fixed it by changing the get_item_loc function as follows:
def get_item_loc(self, nr):
src = os.path.join(self.data_path, nr)
return src
and changing the wrong line into:
"image_loc": ds.get_item_loc("{:012d}.jpg".format(int(detection[0]))),
In this way i can use the detection[0] value that contains the image id, and i don't need to insert another loop variable.

Can you please issue a PR for this?
On Thu, Aug 15, 2019 at 10:24 AM Emanuele Vitali<EMAIL_ADDRESS>wrote:

Hello, it's a silly bug due to using the index of the previous loop in
line 200 of coco.py:
"image_loc": ds.get_item_loc(image_ids[idx]),
I fixed it by changing the get_item_loc function as follows:
def get_item_loc(self, nr):
src = os.path.join(self.data_path, nr)
return src
and changing the wrong line into:
"image_loc": ds.get_item_loc("{:012d}.jpg".format(int(detection[0]))),
In this way i can use the detection[0] value that contains the image id,
and i don't need to insert another loop variable.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/mlperf/inference/issues/323?email_source=notifications&email_token=ABT6DFAMEJAQXRJZBFLOAJLQEVRJ5A5CNFSM4IKT6X52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4L6L5Y#issuecomment-521659895,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABT6DFAQNMWKCV3R2NLXTK3QEVRJ5ANCNFSM4IKT6X5Q
.

--
Vijay Janapa Reddi, Ph. D. | Associate Professor
33 Oxford St. | Maxwell Dworkin 147 | Cambridge, MA 02138
John A. Paulson School of Engineering and Applied Sciences | Harvard
University
Email<EMAIL_ADDRESS>| Web:
http://scholar.harvard.edu/vijay-janapa-reddi

Not sure, get_item_loc looks like this:
def get_item_loc(self, nr):
src = os.path.join(self.data_path, self.image_list[nr])
return src
It takes the image index from the coco dataset, not a string.
I belief nr is wrong.

done. https://github.com/mlperf/inference/pull/346

It takes the image index from the coco dataset, not a string.

It is explained in the pull request, i had also to modify the get_item_loc function.


Update docker-compose.yml

Add volume for telegraf.weather.conf

Would you please add a config section into telegraf-settings.env.example to provide the required env vars

OPEN_WEATHER_API_KEY
OPEN_WEATHER_CITY_ID


there one error in usemin

if i use set
useminPrepare
concate:generated
uglify':generated
cssminLgenerated
filerev
usemin

and some of page not have css block(),it will report
Running "cssmin:generated" (cssmin) task
Verifying property cssmin.generated exists in config...ERROR

Unable to process task.
Warning: Required config property "cssmin.generated" missing. Use --force to continue.

Verdict?


Fix the reading of mapping api endpoint

Coverage decreased (-0.004%) to 90.159% when pulling 7b85a7ff1bb86ee43bc55c6f64f5c22c3abf42eb on SAP:fix-mapping-api into 48395d88e43c93136acf80fe83da90f0fd3d62ba on cloudfoundry-incubator:master.


Migrate AsyncAPI spec to v2

It looks like you're generating the AsyncAPI file and you don't accept PRs on spec files. In any case, I'd like to share with you how it would look like after migrating it to AsyncAPI 2.0.0.
Hope it helps!

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

@fmvilas this is really helpful, thank you! I'm actually working on a 2.0.0 version of the spec right now and this will get me the rest of the way through. Beyond being 2.0.0 it'll have a fair larger amount of schema and coverage of event types.

This is awesome! Let me know when you have it ready. It will be a great use case for AsyncAPI Hub.


Release Google.Maps.AddressValidation.V1 version 1.0.0-beta03

Changes in this release:
New features

Enable REST transport in C# (commit a6c4606)

Documentation improvements

Document that PREMISE_PROXIMITY is a valid value for an address granularity (commit 174a1c0)

I'm abandoning this release - it looks like the API doesn't work with the REST transport at the moment, at least not for all auth types. I'll remove the tag and release, then revert the change.


[ML] Marked Compute Schedules as Mutable on October Preview API

Description
Fixes several bugs for Compute Instances:

Removes readonly property from Compute Schedules on the 10-01-2022-preview API.
Removes an extra print statement from Compute Instances.
Removes experimental tag from Compute Schedules.

Please add an informative description that covers that changes made by the pull request and link all relevant issues.
If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.

/check-enforcer override

overriding checks since last few updates were just changelog updates, and the CI was already passing before