Ports are not available: unable to list exposed ports

Description
I consistently get this error after a long wait: Error: unable to start the container, Error response from daemon: Ports are not available: unable to list exposed ports: Get "http://unix/forwards/list": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
I’m on MacOS 10.15, have a fresh install of Docker and Nitro, and it doesn't matter which database I select/install during initialization (Postgres or MySQL). As far as I know I don't have any other databases running outside Nitro that could conflict. I also destroyed and re-initialized nitro multiple times, but to no avail.
Full terminal output:
$ nitro init
Checking Nitro…
  … creating network ✓
  … pulling image ✓
  … creating volume ✓
  … creating proxy ✓
Checking network…
  ✓ network ready
Checking proxy…
  ✓ proxy ready
Checking databases…
  … checking mysql-8.0-3306.database.nitro ✗
Error: unable to start the container, Error response from daemon: Ports are not available: unable to list exposed ports: Get "http://unix/forwards/list": context deadline exceeded (Client.Timeout exceeded while awaiting headers)

Steps to reproduce

Run nitro init

Additional info

Nitro version: 2.08
Docker version: 20.10.6

@baukevanderlaan do you have anything running on port 3306? You can try nitro portcheck 3306 to test the port. It might also be running as part of MAMP or another local development tool so its best to have those turned off. However, that error code is a weird combination of HTTP and socket parameters.
I did find this issue which suggests it worked after a reboot? I know it says Docker for Windows but one person mentioned macOS in the comment thread.
Can you try rebooting to see if that solves the issue?

@jasonmccallister Thanks for helping!
nitro portcheck says port 3306 is still available. I then rebooted my computer, uninstalled and reinstalled Docker and still get the same error.

Just wanted to +1 this. I'm running into a similar issue. Would be great if Nitro could tell me which port isn't free.
Error: unable to start the nitro container, Error response from daemon: Ports are not available: port is already allocated.

we are constantly restarting docker. and we are constantly getting this error. constantly having to kill processes to get nitro to play nice.
(HTTP code 500) server error - Ports are not available: port is already allocated.
docker portcheck will often say the exact opposite of what shows in docker dashboard (3000 is available!) no it's not.
yes, we also have valet installed. so what? re booting the machine definitely solves everything. for a while.
what are some tips to workaround all these nitro bugs without stopping and spinning nitro/docker/nitro up 10 times a day?
and yes I know these are not "bugs" per se, but this port business needs work.

Maybe I'm just not smart enough for Nitro development. Does this make sense?

Hi @smockensturm! I'm certain that is not the case! ;)
That error is returned directly from the Docker API and is hard to "catch" (if you will). We are working on improving this flow in the future.
Can you run the following commands and provide the output?

nitro ls
docker container ls

Also, make sure you are not using/running something like MAMP (which would block that database port from being used).

Sure. Things are changing fast though. We have a ton of work to do and I can't be wrestling with this thing every morning for hours and hours. Not speedy.
I've snipped out about a dozen hosts. They're all the same and left the db and proxy stuff. Good luck. Thank you.
Hostname                          Type      Internal Ports         External Ports         Status
food.test                         site      3000,3001,8080         (uses proxy ports)     running
mariadb-10.5-3306.database.nitro  database  3306                   3306                   running
nitro-proxy                       proxy     3000,3001,443,5000,80  3000,3001,443,5000,80  running

CONTAINER ID   IMAGE                        COMMAND                  CREATED        STATUS         PORTS                                                                                NAMES
91071a52eb9d   craftcms/nginx:7.4-dev       "/usr/bin/supervisor…"   3 weeks ago    Up 4 minutes   3000-3001/tcp, 8080/tcp, 9000/tcp, 9003/tcp                                                                  food.test
5c73aa659711   mariadb:10.5                 "docker-entrypoint.s…"   3 weeks ago    Up 4 minutes   <IP_ADDRESS>:3306->3306/tcp                                                                                mariadb-10.5-3306.database.nitro
2be1e274045e   craftcms/nitro-proxy:2.0.7   "/usr/bin/supervisor…"   3 weeks ago    Up 4 minutes   <IP_ADDRESS>:80->80/tcp, <IP_ADDRESS>:443->443/tcp, <IP_ADDRESS>:3000-3001->3000-3001/tcp, <IP_ADDRESS>:5000->5000/tcp   nitro-proxy

Related: why can't we do docker-compose up and down? That might be faster than killing all processes and / or restarting the box to clean up the docker config. idk.

FYI: we are not running MAMP. Killed all the Valet stuff (I think). Shutdown brew mysql service. Lordy, what else?

why can't we do docker-compose up and down?

Nitro is not a wrapper around Docker Compose, it uses the Docker API directly. You can run nitro stop, nitro start, or nitro restart to get the same impact. There is a complete list here: https://craftcms.com/docs/nitro/2.x/commands.html#start

docker portcheck will often say the exact opposite of what shows in docker dashboard (3000 is available!) no it's not.

Just confirming, but I'm assuming you mean nitro portcheck? They way the portcheck command works is by checking the localhost address, not the "public" you would use on the LAN. You can specific the specific address/ip to check by passing --hostname to portcheck. You can see some examples with nitro portcheck --help. So if you grab your LAN IP (e.g. 192.x.x.x) and try nitro portcheck --hostname <ip> 3306 that would give you a better check.
Can you try connecting to localhost:3306 with the username nitro and password nitro to see if you get a valid connection?
It seems that something is running on your machine and using port 3306... Nitro does its best to check for port collisions, but we are definitely going to need to improve the checks in a future update.

I understand.
It seems that something is running on your machine and using port 3306... Nitro does its best to check for port collisions, but we are definitely going to need to improve the checks in a future update.
Pardon my ignorance, but how would I find that out? If you need to improve NItro, fine. But help people find their way please. Lots of folks have other dev stuff running besides yours.

What does this error mean please and what steps can I take to eliminate it?
  … starting mariadb-10.5-3306.database.nitro Error: unable to start container mariadb-10.5-3306.database.nitro: Error response from daemon: Bad response from Docker engine

Pardon my ignorance, but how would I find that out?

The portcheck command is designed to help, but a port can only be used by one program at a time (e.g. mysql, postgres). We allow setting custom ports for databases, but we have no control over what you install and run on your machine, Nitro is there is make getting started and managing Docker development easier. FYI, I have used Docker (with Docker Compose) and if you forget to run docker-compose down when switching projects, you will still get a port collision if trying to allocate the same ports across projects.

But help people find their way please. Lots of folks have other dev stuff running besides yours.

I'm more than willing to help developers be productive, the entire thought behind Nitro was to make a local development environment easier. I've often jumped on Google Hangouts and Discord Screen Share sessions to help people troubleshoot Nitro or their other local development environments so I don't want you to feel as if we are not here to help because we are.

Lots of folks have other dev stuff running besides yours.

The best part of being a developer are the choices we get to make on tools, languages, and etc. We hope Nitro works well for you but there are some basic networking and system limitations we (and other tools) cannot get around (duplicate ports, low system resources, under performing hard drives). If you have other tools that are running and causing those issues, we can only do so much to reduce that friction, if that makes sense?

This yields nothing on my host box btw
lsof -nP -iTCP -sTCP:LISTEN | grep 3306

Can you provide the following?

Which version of Docker and Nitro are you running?
What amount of Memory and CPU do you have allocated to Docker Desktop?

Yep. Definitely not smart enough for Nitro.
I cannot start Nitro yet to get that information, Jason. But I did just do a Nitro self-update if that helps. Docker is 3.5.1 and I have no idea how much memory and CPU I have allocated to Docker Desktop. How do I find that?

Assuming you are on macOS? You can find that information here: https://docs.docker.com/docker-for-mac/#resources

ok. ok. I see what you're getting at. Indeed, increasing resources (and after a little housekeeping) all seems great! Probably should have mentioned we have about 100 containers and 50 databases.
Just kidding. But it's a lot. Thank you for your guidance @jasonmccallister

@smockensturm ok that makes a lot more sense. I think this is whats happening:

The Docker API is telling us the port is in use, because a container is supposed to be running but Docker might be out of system resources (because MySQL is notorious for issues with containers and resources) and is holding that port
The portcheck is checking the network device port, not the Docker API, and is technically "correct" because its free as in not-in-use

Can you provide the resources (memory and CPU) you had assigned to Docker Desktop as well as your host machine memory and CPU? There might be a way for us to examine the system information using the Docker API and host machine and see if that is a likely culprit.

Certainly. It's a 16 GB M1. Docker dashboard says:
Before
CPUs: 4
Memory: 2 GB

After
CPUs: 4
Memory: 8 GB

Putting this here for my future reference, this gets the amount of RAM allocated to Docker Desktop:
package main

import (
	"context"
	"fmt"
	"log"

	"github.com/docker/docker/client"
)

func main() {
	// create the docker client
	docker, err := client.NewClientWithOpts(client.FromEnv)
	if err != nil {
		log.Fatal(err)
	}

	info, err := docker.Info(context.Background())
	if err != nil {
		log.Fatal(err)
	}

	fmt.Println(info.MemTotal /<PHONE_NUMBER>)
}

In my case, this return 6 as I have allocated 6G of RAM to Docker Desktop. We might be able to come up with a command or recommendation based on the memory allocated to the system and Docker Desktop...

Hi, we are closing this issue as we have decided to retire Nitro, so no additional work will occur on this project. You can read the official blog post here https://craftcms.com/blog/retiring-craft-nitro. We appreciate everyones feedback and involvement and we look forward to refocusing our efforts on Cloud!
If you're looking for a new local development environment, we recommend DDEV and have a knowledge base article to help you with the transition: https://craftcms.com/knowledge-base/migrating-from-craft-nitro-to-ddev.


[icu] build failure (wasm32-emscripten)

Host Environment

OS: Linux
Compiler: emsdk 3.1.31

To Reproduce
Steps to reproduce the behavior:
./vcpkg install icu:wasm32-emscripten
Failure logs
CMake Error at scripts/cmake/vcpkg_execute_build_process.cmake:134 (message):
    Command failed: /usr/bin/make V=1 -j 5 -f Makefile all
    Working Directory: /openmw-web/vcpkg/buildtrees/icu/wasm32-emscripten-dbg/
    See logs for more information:
      /openmw-web/vcpkg/buildtrees/icu/build-wasm32-emscripten-dbg-out.log
      /openmw-web/vcpkg/buildtrees/icu/build-wasm32-emscripten-dbg-err.log

Call Stack (most recent call first):
  scripts/cmake/vcpkg_build_make.cmake:149 (vcpkg_execute_build_process)
  scripts/cmake/vcpkg_install_make.cmake:2 (vcpkg_build_make)
  ports/icu/portfile.cmake:137 (vcpkg_install_make)
  scripts/ports.cmake:147 (include)

error: building icu:wasm32-emscripten failed with: BUILD_FAILED
error: Please ensure you're using the latest port files with `git pull` and `vcpkg update`.
Then check for known issues at:
    https://github.com/microsoft/vcpkg/issues?q=is%3Aissue+is%3Aopen+in%3Atitle+icu
You can submit a new issue at:
    https://github.com/microsoft/vcpkg/issues/new?template=report-package-build-failure.md&title=[icu]+Build+error
Include '[icu] Build error' in your bug report title, the following version information in your bug description, and attach any relevant failure logs from above.
    vcpkg-tool version: 2023-01-24-8a88d63f241d391772fbde69af9cab96c3c64c75
    vcpkg-scripts version: 1c48ec7 2023-02-07 (10 days ago)

Cut and paste the appropriate build messages from the console output.
Attach every failure log file mentioned in the console output.

This issue hasn’t been updated in 3 month, if it is still an issue, please reopen this issue.

Hey, I can confirm the issue is still present (I am using the docker image emscripten/emsdk:latest)
I can provide a Dockerfile script to show the issue if needed, but it should be trivial to reproduce

Hey @iTrooz that would be helpful, could you please post your dockerfile

Here it is:
# Tessted as of 2023/12/04
FROM emscripten/emsdk:latest

# Install dependencies for icu
RUN apt update && apt install -y autoconf pkg-config autoconf-archive

# Install vcpkg
RUN git clone https://github.com/microsoft/vcpkg /vcpkg
RUN /vcpkg/bootstrap-vcpkg.sh

# Try to install icu for emscripten (it will fail)
RUN --mount=type=cache,target=/vcpkg/downloads /vcpkg/vcpkg install --triplet=wasm32-emscripten icu || exit 0

ARG UNIQKEY

# Show log files to console
RUN cat /vcpkg/buildtrees/icu/build-wasm32-emscripten-dbg-out.log
RUN cat /vcpkg/buildtrees/icu/build-wasm32-emscripten-dbg-err.log

@JonLiu1993 maybe we shouldn't build icu and use system instead (icu is available under emscripten)

I'm testing a modification which compiles icu data without assembly. This seems to work. Unfortunately, the compilation is driven by icu's pkgdata which never heard of parallel jobs :cry:

Elapsed time to handle icu:wasm32-emscripten: 17 min


logger disabled by default in v1+; add option to enable logger?

I've been looking for a way to config json-server to re-enable the logger from versions < v1.  It's very useful to see a log of processed requests.  Looking at the source code, I think I see the boolean to enable it, but it's currently hardcoded false.  Could there be an option flag "-l", "--log" or some other way to enable it from the command line?  Thank you.

Opened a pull request


whitelist-network.com error!!!!!!!!!!!!!!!!!!!!

I got this problem while trying to connect to whitelist-network.com
You have been redirected because the Polkadot{.js} extension believes that this website could compromise the security of your accounts and your tokens.
https://whitelist-network.com/

This was a scam website which is no longer working.
Mirror
When i see the word "kindly" i wanna close my laptop forever


check_contrastsheet not identifying headers even though they're present

I ran the pipeline through Tower and it keeps kicking back the error of my contrast sheet having no header. I ran just the checkcontrastsheet.py locally on the csv and it's giving the same error. It doesn't happen on Tower when I run it with the test data though. Not sure what to make of this. I'm just including the python I ran locally before because it 100% recapitulates the issue without having to run the whole pipeline.
ejennings@DESKTOP-C17P9CF Downloads % python check_contrastsheet.py contrast_test_share.csv testout
[CRITICAL] The given contrast sheet does not appear to contain a header.
This is the CSV I tested on:
contrast_test_share.csv

Thank you!
--
Emily Jennings, PhD
Senior Scientist I, Bioinformatics
Expansion Therapeutics
www.expansionrx.comhttp://www.expansionrx.com/

From: James Ashmore @.>
Sent: Tuesday, June 6, 2023 8:37:04 AM
To: nf-core/rnasplice @.>
Cc: Emily Jennings @.>; Mention @.>
Subject: Re: [nf-core/rnasplice] check_contrastsheet not identifying headers even though they're present (Issue #63)
Hey @expansion-bioemformaticshttps://github.com/expansion-bioemformatics
The issue has been resolved in the dev branch. It occurred because the contrast sheet checker tried to identify the format of your file automatically. Your particular contrast sheet had a lot of "_" characters, so the sniffer thought the file was delimited by this character instead of the "," character. We have now removed the automatic file formatter checker, and require the user to specify the contrast sheet in CSV file format. This is actually what the documentation tells the user to provide, so it is now more-inline with the documentation. Thanks for bringing this to our attention.
Let me know if this has solved your problem and I will close the issue.
—
Reply to this email directly, view it on GitHubhttps://github.com/nf-core/rnasplice/issues/63#issuecomment-1578687513, or unsubscribehttps://github.com/notifications/unsubscribe-auth/A7O5ZINOENI4Y7A5R5EJ5P3XJ4P7BANCNFSM6AAAAAAY3NRTG4.
You are receiving this because you were mentioned.Message ID: @.***>


Merge internal changes

Changelog:

4908863: Switch to DistributedDataParallelC10d and bump version 0.5.0 -> 0.6.0

no more FP16Trainer, we just have an FP16Optimizer wrapper
most of the distributed code is moved to a new wrapper class called DistributedFairseqModel, which behaves like DistributedDataParallel and a FairseqModel at the same time
Trainer now requires an extra dummy_batch argument at initialization, which we do fwd/bwd on when there's an uneven number of batches per worker. We hide the gradients from these dummy batches by multiplying the loss by 0
Trainer.train_step now takes a list of samples, which will allow cleaner --update-freq

1c56b58: parallelize preprocessing
Misc bug fixes and features

Hey Myle does this include a solution for issue #217?
@myleott
https://github.com/pytorch/fairseq/issues/217
Thanks!

Not yet, we're still iterating on that fix, since depending on how we implement it, it could reduce randomness and give worse results. We should have it ready soon though.


Document how to use Astro components with slots inside MDX files

A recent question in our Discord support channel brought me to the realization that we currently don't document how you can fill slots to be used in Astro components from inside Markdown files.
I think a good location to document this could be here: https://docs.astro.build/en/guides/markdown-content/#using-components-in-markdown (or to a specialized MDX page if we were to add one)
Here's what I would like us to mention:

First of all: Yes, you can define slot content in MDX files. You can wrap an element around your desired slot content and add the slot="your-slot-name" attribute.
By default, the wrapping element will become the outer element of your slot content. If you don't want this, you can use a Fragment instead, which will disappear from the resulting HTML output.
Optionally, we can mention a common issue here as well, which can happen generally everywhere when interleaving Markdown and HTML/JSX: In case you encounter any MDX error messages while developing your page (e.g. Expected a closing tag for '<div>' (2:3-2:20) before the end of 'paragraph'  end-tag-mismatch  mdast-util-mdx-jsx), ensure that your HTML/JSX elements are opened and closed on their own lines and not mixed with text on the same line. MDX is a bit more strict than HTML in that regard.

Draft of code sample:
/* Example: src/pages/using-slots.mdx */

import MyComponent from '../components/MyComponent.astro'

<MyComponent>
  <div slot="main">
    # This is a Markdown heading inside a div!
    And a paragraph with *formatting*.
    Both go into the "main" slot of your component.
  </div>
  <Fragment slot="side">
    If you don't want a wrapping element like a `div` to be
    the outer wrapper of your slot content, you can use a Fragment
    like this to define in which slot this paragraph should go.
  </Fragment>
  This is a regular **Markdown** paragraph going into the default slot.

  And another one.
</MyComponent>

We could optionally also provide a MyComponent.astro sample component that makes use of the slot content, and include that sample component in the frontmatter of the code sample above.

Love this, @hippotastic ! And just to clarify, if we were to include a file name at the top to show where the code sample exists, it could be something like src/pages/posts/my-markdown-post.md ?

Yes, exactly. The code sample is supposed to be a regular markdown file. Your path is great. If we think that readers might assume that this only works in blog scenarios, we could also use src/pages/my-markdown-page.md.

As we are moving away from support for components in Markdown (legacy.astroFlavoredMarkdown), I don't think we should document this? It will be a legacy feature?

Nope, the use case is unchanged for MDX 😄

Thanks @hippotastic ! Shall we update the title  to reflect .mdx then (so we don't confuse Sarah again), and consider this on the stack for .mdx documentation in the docs?

Yes, great point! I updated the title and issue text accordingly to target MDX.

This is likely on hold as Fragment seems to be breaking in MDX currently: https://github.com/withastro/astro/issues/4130

We think this works in MDX like it does in .astro now, so maybe no need to document explicitly? Going to close, but feel free to reopen or even PR if we think there are still specific edge cases that need documenting.


Deprecate UIEvents subclasses in AppUi-React

Is your feature request related to a problem? Please describe.
Many event used in our system (IE. FronstageActivatedEvent in FrameworkFrontstages, ContentLayoutActivatedEvent in ContentLayout) are subclasses of UiEvent defined as publicly available classes, and used.
These are not "body" of events but the actual event emitters to which application register and the event emitted is an interface, so there is no reason to make these public (an application shouldnt have a reason of creating a new FronstageActivatedEvent() emiter in their application, as the only thing this will give them is a UiEvent (emitter) with the FrontstageActivatedEvent type event.
Describe the solution you'd like
Deprecated these emmiter classes. (we should be able to public static readonly onFrontstageActivatedEvent = new BeUiEvent<FrontstageActivatedEventArgs>();

As the FrameworkFronstage interface is exported as public, it would be a violation of support policy to change the interface from a class to a different one, however, these were made public simply because of the system misunderstanding as there is no reason for anyone to recreate an object following this interface (or it to be exported, really), it is simply the type of UiFramework.fronstage, which wouldnt be broken by changing if all the event handler were defined as interfaces themselves as the behavior would remain the same. So when deprecating these classes, the "FrameworkXXX" classes should also be deprecated as they are not publicly meaningful. These interfaces themselves should not be exported but simply used internally as the type of the members of UiFramework and remove all class that forces an implementation method.


feat: jq support

Issue Reference(s):
Fixes https://github.com/tailcallhq/tailcall/issues/1321
/claim #1321

Summary by CodeRabbit

New Features

Enhanced the GraphQL schema to simplify the user query argument structure.
Introduced new evaluation methods across various modules to enhance processing capabilities.
Updated mustache template syntax for improved clarity and consistency in variable references.

Bug Fixes

Adjusted key-value pair transformations to ensure correct string conversion in headers management.

Documentation

Updated internal test cases to align with the new mustache syntax and evaluation methods.

/tip 300$


problem in decode_net.sh

@jpuigcerver
I am running the Laia using my own dataset which is formatted as in Bentham dataset. The training is running good but when I run the decode code on test and validation parts, I get an error:
hyp[line[0]] = line[1:]
IndexError: list index out of range

I checked and It seems that the decode/no_lm/char/va_.txt  decode/no_lm/char/te_.txt contains space lines which gives the error above!
I added this line in compute-errors.py
for line in args.hypothesis:                                                                                │
        line = line.split()                                                                                │
        if len(line)!=0:                                                                                │
           hyp[line[0]] = line[1:]

The code is running but it does not show anything!!!

Hi, sorry for the late reply.

Can you point to the specific file and line that raises the exception?
Can I access your reference text files for validation/test?

Thanks. there is no errors. Actually, the errors is coming fro  my side because i changed a little bit the decode script. So, your code is fine and consistent. Thanks.


About the alternative softmax

https://github.com/tjiiv-cprg/EPro-PnP/blob/209727638ea992d53275a075c95c316182c01efd/EPro-PnP-6DoF/lib/train.py#L163
Why use an alternative of SoftMax instead of original one?
Thank you every much!

This is basically a legacy design choice. In an early experiment we found that this alternative version performed slightly better than standard Softmax, although the difference seems too trivial as validated by some later experiments.
Generally we recommend using the standard Softmax as it is more numerically stable.


STRUCT support in Verdict for scambles

Hi guys,
One of our tables has recently started receiving data in the form of a struct (array / row).
For example:
{city=Jackson, state=WY, zip=83001, county=Teton, msa=null, country=US}

{city=Cheyenne, state=WY, zip=82001, county=Laramie, msa=null, country=US}

{city=Gillette, state=WY, zip=82718, county=Campbell, msa=null, country=US}

I was wondering how Verdict builds its scrambles based on this kind of data? Is this a data structure you actively support? Would each of the internal items be capable of producing fast aggregations?
For example:
SELECT count(distinct(Location.city)) from table
Our scramble performance has dropped significantly but we aren't sure if this correlates?

VerdictDB should just work. One possible reason is that columnar format may not be very efficient for such data types.
If you can load sample data into the cluster, we may be able to test them.

@dongyoungy Can you ask someone to investigate this by comparing different compression formats for our scramble tables? Maybe we can try different formats (e.g., ORC or parquet) with different compression schemes.

I'm unsure as to the internals for it but yes I agree that structs on a columnar are probably not ideal - they seem to be the preferred way in BigQuery (where this data has originated from). We are considering flattening them out as a last resort but we would prefer to get some information on exactly how verdict handles this before we do anything drastic :)

Just an FYI, we are refactoring our tables away from this due to
performance issues with these data structures.
In BigQuery however these are preferred structures (and fairly efficient) -
so might be something you want to look at for that side of things :)
On Tue, 2 Apr. 2019, 02:55 Yongjoo Park<EMAIL_ADDRESS>wrote:

@dongyoungy https://github.com/dongyoungy Can you ask @Beastjoe
https://github.com/Beastjoe to investigate this issue? I see two
related problems:

Performance when the table contains array or struct
Possible performance degradation when samples keep appended

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mozafari/verdictdb/issues/354#issuecomment-478638034,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABBAiqzneUwARuepDX0u-knJF2ktIYb4ks5vciwGgaJpZM4bzEWc
.


Cannot use an access token when 2fa is active

For some reason i cannot use an access token when 2fa on the given account is active.

i can see the token is used when i disable 2fa on that account
Jenkins GitHub Pull Request Builder — repo, repo:status -> Last used within the last day

with 2fa enabled i can still do curl requests on the url ghprb uses for triggering build jobs
curl -I -u my-user-name:my-access-token https://api.github.com/repos/my-org/my-repo/pulls?state=open

But jenkins fails
java.lang.Error: org.kohsuke.github.HttpException: Server returned HTTP response code: 401, message: 'Unauthorized' for URL: https://api.github.com/repos/my-org/my-repo/pulls?state=open

I can run all checks in the configure page of ghprb with success with the given access token

Test basic connection to GitHub
Connect to API
Connected to https://api.github.com as null (null) login: my-user-name
Test Permissions to a Repository
User has access to: Push, Pull

As soon as i deactivate 2fa everything works just fine, which isn't an option at all

Can anyone tell me if i am doing anything wrong or if this is a bug?
Thanks

Hmm it sounds like you're using an OAuth token? Ultimately this will be something in the underlying github-api. See issue here.

Hm, no basicly i am using a normal access token...

Hm, no basicly i am using a normal access token...

Gotcha. I would give this another go by creating an oauth token and using that as part of your credentials. I do think it's an issue that the ghprb plugin told you everything would be ok though, so I'm going to keep this issue open as a bug.

Good idea, will give the oath token a try on monday. Thanks for the hint ;)

By the way, i was adding the token using ghprb's button "create api token" in the configure section. Therefore i was not even thinking about creating a oauth token ;)

Didnt had success with oauth either and since i setup the token by using the configure page, the access token should be the desired way to use this plugin.

I assume the underlying issue is now resolved and we can close this?
If not, please reopen.

Is there a link or help somewhere on exactly how to configure jenkins with 2fa in github? I've seen 3 different how-to sets of instructions and so far none of them work.. most just say "try turning off 2fa or using a different repo" none of which is a viable option.
Instructions like this (https://blog.tentamen.eu/jenkins-and-github-integration-using-webhooks/) which say they will solve the issue do not.. there is no successful code pull for any reason. authentication fails.
Using Oauth,  with a private key and a webhook all at the same time still constantly fails. Any set of instructions that works with original jenkins (not blue ocean) would be great. Please not a link to another blog post saying it can be done or should work, but does someone have actual step by step that actually works?

@mutabletao @nikolaycnpopov - I churned on the same issue for a while.
This got me across the finish line:
CloudBees: GitHub Permissions and API token Scopes for Jenkins


clac: init at 2017-04-16

Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] Linux

[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

It is the same clac I have added an hour ago, right?

Damn. Should've pulled first.


Add support for draft issues to the StateLabel component

This support is adding support for draft issues to the <StateLabel> component.
Screenshots

Before

After

Merge checklist

[ ] Added/updated tests: looks like existing test does not have assertions at the icon level, so I didn't do any changes here.
[x] Added/updated documentation
[x] Tested in Chrome
[x] Tested in Firefox
[x] Tested in Safari
[x] Tested in Edge

I just had a look at the changes in the package-lock.json and I'll need to fix that before we can merge this PR:

I just had a look at the changes in the package-lock.json and I'll need to fix that before we can merge this PR:

Fixed with https://github.com/primer/react/pull/1604/commits/a6e319f23f803314cf8e5b7a001653098692c963
The diff is looking good now:


Diesel print-schema cannot properly generate multiple joinable!

This is my comments schema:
CREATE TABLE users (
  id        SERIAL PRIMARY KEY,
  name      VARCHAR(50)  NOT NULL UNIQUE,
  mail      VARCHAR(100) NOT NULL UNIQUE,
..........
);

CREATE TABLE node (
  id        BIGSERIAL PRIMARY KEY,
  uid       INTEGER      NOT NULL DEFAULT -1 REFERENCES users (id) ON DELETE CASCADE,
........
);

CREATE TABLE comments (
  id      BIGSERIAL PRIMARY KEY,
  nid     BIGINT    NOT NULL REFERENCES node (id) ON DELETE CASCADE DEFERRABLE,
  uid     INTEGER   NOT NULL REFERENCES users (id) ON DELETE CASCADE DEFERRABLE,
  pid     BIGINT    NOT NULL      DEFAULT 0 REFERENCES comments (id) ON DELETE CASCADE DEFERRABLE,
  at      INTEGER REFERENCES users (id),
  likes   INTEGER   NOT NULL      DEFAULT 0,
  body    TEXT      NOT NULL,
  created TIMESTAMP NOT NULL      DEFAULT now(),
  changed TIMESTAMP NOT NULL      DEFAULT now()
);

and the print-schema just generated one joinable~:
joinable!(comments -> node (nid));
// joinable!(comments -> users (uid)); this cannot be auto generated.

I've tried to reproduce this locally using the provided schema and failed to reproduce this.
$ diesel print-schema
table! {
    comments (id) {
        id -> Int8,
        nid -> Int8,
        uid -> Int4,
        pid -> Int8,
        at -> Nullable<Int4>,
        likes -> Int4,
        body -> Text,
        created -> Timestamp,
        changed -> Timestamp,
    }
}
table! {
    node (id) {
        id -> Int8,
        uid -> Int4,
    }
}
table! {
    users (id) {
        id -> Int4,
        name -> Varchar,
        mail -> Varchar,
    }
}
joinable!(comments -> node (nid));
joinable!(node -> users (uid));
allow_tables_to_appear_in_same_query!(
    comments,
    node,
    users,
);
$ diesel --version
diesel 1.3.1

Closed because this does not seem to be a issue.
(Feel free to add more informations how to reproduce this, but without any idea how to reproduce this there is nothing we can do here.)


IGAPP-1005: Removed double separation line in location list view mobile

This pull request belongs to an issue on our bugtracker.
You can find it there by looking for an issue with the key which is mentioned in the title of this pull request.
It starts with the keyword IGAPP.

Tested on chrome, edge and firefox on Windows. The double seperation line is still shown on "middle" window size in firefox. See screenshot below.

(left: chrome; right: firefox)

Can reproduce the issue @Cmd190 described, for 972x720px.

well the purpose of this ticket @steffenkleinle as you created was to remove:
"Double separation line in bottom action sheet"


Incorrect PowerShell code for "Configure online hybrid Mediation Server Settings"

PowerShell code in "Configure online hybrid Mediation Server Settings" section currently says:
Gets-CsOnlineUser -Identity <UserPrincipalName>
This is wrong.
Correct code:
Get-CsOnlineUser -Identity <UserPrincipalName>
note: "Gets" vs "Get"

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 328cfd03-27e5-7756-f004-811f2dc7d2ef
Version Independent ID: 010d16a6-1474-61ae-a55e-47f299705a74
Content: Configure Cloud Connector integration with your Office 365 tenant
Content Source: Skype/SfbServer/skype-for-business-hybrid-solutions/plan-your-phone-system-cloud-pbx-solution/configure-cloud-connector-integration-with-your-office-365-tenant.md
Product: skype-for-business-itpro
GitHub Login: @CarolynRowe
Microsoft Alias: crowe

@shivtorov Hello and thank you for bringing up this issue.  I'll be investigating this and will be getting back to you on this issue, thanks!

@shivtorov Thank you for submitting feedback. We understand that this issue has been resolved.
Please feel free to re-open this issue if there is a specific area of the docs that we can improve or make better. Thank you.


Add an option to just decompress in one triangle for star and acyclic coloring

In practice we only store one triangle of the Hessian of the Lagrangian.
We could an option as one of the following symbols :full, :lower or :upper.

How about we only do this when decompressing into a matrix of type Symmetric? That also gives us the information of lower or upper

It's a good idea!
Unfortunately, Symmetric was never designed for storage; it is mainly used for dispatch in many cases.
We need to ensure that Symmetric is not wrapping a sparse matrix with both triangles stored.
Since we can't guarantee that, we need to specify this clearly in the documentation.
I don't understand why an option uplo='F' was never added in Symmetric or Hermitian...

Why is it bad if a Symmetric wraps a sparse matrix with both triangles filled? It will only consider one anyway.

It depends if you rely on the sparsity pattern of the parent or not.
You need additional checks if you have both triangles.


Add Wayland support for renderdoccmd replay feature

This support has been implemented only for Vulkan at this moment.
Testing environment

RenderDoc build: v1.5, last updated on 09-24-2019
Operating System: Linux
Release Name: NXP i.MX Release Distro, version 4.19-thud
API: Vulkan
Target: i.MX8

Closing due to inactivity on outstanding review issues.


feat: encrypt individual data (towards #111)

[x] Name
[x] Locality
[x] Region
[x] Phone
[x] Email
[x] Support person name
[x] Support person email
[x] Support person phone

Blocked buy https://github.com/accessibility-exchange/platform/issues/565.


Uncaught (in promise) Error     at o (shallowEqual.js:18)

Smooch web widget not loading.
I am getting this when inspecting element.

Failed to load resource: the server responded with a status of 401 ()
shallowEqual.js:18 Uncaught (in promise) Error
    at o (shallowEqual.js:18)

Any feeback please.

looks something wrong with shallowEqual.js

I have found this shallowEqual.js file while googling.
https://gist.github.com/tanat/3d695499ac1a4525ab221583838e6bdb#file-shallowequal-js-L19
it has something different on line 18/19

any feeback on this please. our smooch web-interface is down.

My suspicion is that there is some polyfill or something else in the host page that is messing with Object.prototype.hasOwnProperty.
smooch-bot-example is using a fairly old version of Smooch Web Messenger (3.15.2)
The latest version is 4.12.10, and since version 4 the chat UI loads inside its own iframe, which prevents js env interference such as this. Could you try latest version of Smooch web messenger? Instructions here


[fixed] Cancel requested animation frame on unmount.

Fixes #882 .
Changes proposed:

Be sure to cancel the animation frame that is requested in the ModalPortal open method so that a state change is not potentially requested after the component is unmounted.

In the issue description I mentioned that isMounted might be a potential fix, but then I read that it is an antipattern. The componentWillUnmount method is already clearing a timeout, so canceling the animation seems like the best option.
I can confirm that these changes fix the error message I was seeing in the jest testing environment I am using in a separate project. It is unclear to me how to reproduce this race condition in the testing set up used in this project. Any ideas would be appreciated.
Acceptance Checklist:

[x] The commit message follows the guidelines in CONTRIBUTING.md.
[ ] Documentation (README.md) and examples have been updated as needed.
[ ] If this is a code change, a spec testing the functionality has been added.
[ ] If the commit message has [changed] or [removed], there is an upgrade path above.

Coverage decreased (-0.2%) to 84.474% when pulling ba8b6c26ec37676cc9309e246d252e91d9b50c68 on adriantrunzo:master into fc76b0c9807dcd2483019a3f9005e9a8c1d87438 on reactjs:master.

Good catch! Thanks.

Is it possible to get this released? The project I'm working on is trying to pull in react-modal but finding that our test suite is leaving some warnings sprinkled about that I suspect this might resolve.
Warning: Can't perform a React state update on an unmounted component. This is a no-op, but it indicates a memory leak in your application. To fix, cancel all subscriptions and asynchronous tasks in the componentWillUnmount method.
        in ModalPortal (created by Modal)
        in Modal (created by Modal)
        in Modal (created by ConfirmationModal)
        in ConfirmationModal (created by View)
        in View (created by LoadingContent)
        in div (created by LoadingContent)

Released version 3.14.3.


Release 196 jb2

Description

Types of changes

Checklist

[ ] I have read the CONTRIBUTING doc
[ ] I have checked that unit tests pass locally with my changes
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] I have added necessary documentation (if appropriate)

Further comments

I merged Jen's branch into my branch to fix some bugs, so I closed this PR and created another PR from my branch: https://github.com/bcgov/EDUC-CCOF/pull/455


fs is not defined

hi there, I just installed postcss as well as postcss-style-guide via npm.
i used your configuration for gulp and it came back with the reference error fs is not defined
any thoughts on what I might be doing wrong?

@WhatsNewSaes Did you try to run the example code in README? Of cause, you must require the standard module, fs.
var fs = require('fs')

Ahh i gotcha. It might be good to put that in the gulp documentation as well.
Thanks!


Support schema 1 manifest.

Currently, containerd only supports schema 2 manifest.
For cri-containerd, not supporting schema 1 means:

Users who are still using schema 1 image could not use cri-containerd integration.
Some system container images are still using schema 1 manifest, e.g. pause image.

Regarding to 1), it is fine for our alpha release (this quarter), but it should be fixed before beta (probably the end of the year).
For 2), we need to rebuild our schema 1 system container image.
Ref https://github.com/containerd/containerd/issues/851.

This become the bigest blocker for running node e2e test.
Many images used by node e2e test are using schema 1 image.
Except that, other 2 high priority work for testing:

Implement container log;
ExecSync.

What is ExecSync?

@crosbymichael It's defined in CRI, https://github.com/kubernetes/kubernetes/blob/master/pkg/kubelet/apis/cri/v1alpha1/api.proto#L66.
Basically exec a command in container and return result synchronously.

@crosbymichael I might mislead your guys, the other 2 items are purely on our side.

This is finished in https://github.com/kubernetes-incubator/cri-containerd/pull/81.
Close this one.


fix: component id 0 not being de-selected

Closes #127.
Problem seems to be 0 being considered falsy, so just checking if it's truthy or equal to 0 is the simplest solution. Open to more solutions too.

Maybe we can start the counter at 1?

Maybe we can start the counter at 1?

Yeah, that could work as well.

Maybe we can start the counter at 1?

Yeah, that could work as well.

Edited my comment, the code used to do a pre-increment so essentially started at 1

Should we add a "components can be unselected" test/assertion?

I've just made it start from 1 instead of messing pre-increment. Can change it to that if you want though.
Might be worth adding a test for de-selecting yeah.


Segoe MDL2 Assets are unavailable on win7 or 8

According to Microsoft Segoe MDL2 Assets are only available on windows 10 devices unless users install them manually on other OS
I was testing my app on windows 7 VM and noticed dropdown arrow used in ComboBox is not displayed properly. Had to check ComboBox.xaml file to find out Segoe MDL2 Assets font family was used for DropDownGlyph. Can we replace it so that its compatible with older OS as well?

You can pack mdl assets along with fluent wpf if you need. This one is the
best example :https://stackoverflow.com/a/39912794/12427812
On Sat, 28 Aug, 2021, 2:04 pm SlackingVeteran, @.***>
wrote:

According to Microsoft Segoe MDL2 Assets are only available on windows 10
devices unless users install them manually on other OS
https://docs.microsoft.com/en-us/windows/apps/design/style/segoe-ui-symbol-font
I was testing my app on windows 7 VM and noticed dropdown arrow used in
ComboBox is not displayed properly. Had to check ComboBox.xaml file to find
out Segoe MDL2 Assets font family was used for DropDownGlyph. Can we
replace it so that its compatible with older OS as well?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/sourcechord/FluentWPF/issues/135, or unsubscribe
https://github.com/notifications/unsubscribe-auth/ARKGWVI7ND3ZV3KVVZR3ZZTT7CNPXANCNFSM5C64PQKQ
.

@SlackingVeteran
Thanks for the feedback.
I'd like to remove the dependency for MDL2 Assets font. And, I'll fix it in next release.

Thanks, i’d be greatly appreciated!
Would you mind linking commit to this issue when you do so? I had to modify
few things in fluentWPF and I use it in a unreleased app. I think I cloned
FluentWPF few version back and few things are outdated. If you link the
commit, i could just apply changes manually
On Sun, Sep 19, 2021 at 22:54 minami_SC @.***> wrote:

@SlackingVeteran https://github.com/SlackingVeteran
Thanks for the feedback.
I'd like to remove the dependency for MDL2 Assets font. And, I'll fix it
in next release.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sourcechord/FluentWPF/issues/135#issuecomment-922605470,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABI3ZHRITZI7UXJ4L4F6FKLUC2O67ANCNFSM5C64PQKQ
.

Awesome, thanks!
On Sun, Sep 19, 2021 at 23:13 minami_SC @.***> wrote:

Closed #135 https://github.com/sourcechord/FluentWPF/issues/135 via
86b4641
https://github.com/sourcechord/FluentWPF/commit/86b46413dc9f688200f9bb07513dd88687836b05
.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sourcechord/FluentWPF/issues/135#event-5325726646,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABI3ZHTLTIAFXQLKHHHP753UC2RD3ANCNFSM5C64PQKQ
.


fix: progress issues

This fixes a few crashes I had with the progress bars.

I don't know the context, but I also observed the case where someone didn't see a progress bar moving (only jumping to the end) when downloading repodata.

Thats is indeed not the context of this but that also seems awfully weird!


FFI for All functions for genius_api

All functions mapped FFI
Tested with Linux and Windows

Hi Vampibar,
pull requests should go feat/branch -> develop, then into master after we test and can make sure all platforms work.
Please redo this PR.

Hi Vampibar,
pull requests should go feat/branch -> develop, then into master after we test and can make sure all platforms work.
Please redo this PR.

There is no "develop" branch for this repo. Should I create it?


Pragma Completion

There should be a way to mimick/hack brace completion such that pragma notation ( '{..}' ) is also done.

You mean to automatically insert the ending '.}' as opposed to just '}'?

Yes. In my mind, you would type '{.' and the other '.}' would appear.
Even better would be if only '{' would have to be typed in. I imagine it would require techniques similar to what's done with the doc-comment continuation mechanism.

+1 sounds good to me :)

Ok, the big thing this is going to require is some modification to the language definition. We'll need some sort of 'incomplete pragma' scope that the function can check.

Implemented in commit deec1f36b15e8b1bf1d98e58fdab4efdeab859ce


fix: add support for flask > 2.2

Add this PR just for someone looking for a fix if you are using this lib and Flask > 2.2

@talhakum sorry, update the missing import.


mudblazor.com Trojan.GenericKD Flag on Blink

Bug type
Docs (mudblazor.com)
Component name
No response
What happened?
When I open https://mudblazor.com on a  Blink based Webbrowser I will get a Trojan.GenericKD.48765280 (Engine A) Warning from G Data Antivirus. This happens only on Blink and the behaviour started this morning. The issue does not occure when I use a Gecko based Browser. WebKit was not tested.
The location of the flagged file is e.g.:
C:\Users\Username\AppData\Local\Microsoft\Edge\User Data\Default\ServiceWorker\CacheStorage\Numbersfolder\Numbersfolder\XeXXXXX_0
Expected behavior
Load the page without antivirus warning and file flag.
Reproduction link
https://mudblazor.com
Reproduction steps

have G Data Antivirus installed
use a Browser that uses Blink as engine
surf to mudblazor.com
get Trojan warning on load animation

Relevant log output
No response
Version (bug)
5
Version (working)
No response
What browsers are you seeing the problem on?
Chrome, Microsoft Edge, Other
On what operating system are you experiencing the issue?
Windows
Pull Request

[ ] I would like to do a Pull Request

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Having the same issue in EDGE with BitDefender insalled!

Yeah not much we can do about that, if you google its quite common. Only thing that we can "do" is try contact this software companies like BitDefender and report it as false flag i think.

I submitted the website to BitDefender now as false positive, but anyone can do this really.


Call lwm2m_observe on a resource with pending obs can cause exception?

This is only suspicion from the source, not validated yet:
when a second call lwm2m_observe on the same resource when it already has pending observe call, if the first call lwm2m_observe() failed, the second callback could use the observationP which was actually freed in the first callback?

Did you validate your suspicion ?
Or we could close this one ?

@sbertin-telular do you think the is something to investigate here or we close this one ?

I think it should be investigated before closing.


Largest Palindrome Problem

Hello, please assign me this problem because this is a significant DSA interview question asked in a google coding contest.

Hey, can you assign me this issue. I am quite familiar with palindrome. This would be my first pull request.

Hey, I want to contribute in this issue.


#22 Show tooltip "No projects are monitored." instead of "All builds …

…are good." when there are no projects being monitored
Thought I'd give this bug a try. Is this close to what you had in mind?

Hey @BrandonHuber, thank you for the PR! :+1:
I reviewed it and had some comments. Do you agree with them? Let me know. Thanks.

Thanks for the quick reply! Good comments. I've updated the code based on your feedback.

Sorry for the late reply.
Looks good, except for the pop up message, which is a little casual. Sounds like Come get some! from Duke Nukem. I'll get back to you after consulting my technical writer.

Hi @BrandonHuber, sorry for the late reply, was really busy with work.
I think your initial text, "No project monitored." is short and simple. If you can help push another commit, that will be great. Then i'll just merge everything in and close this PR.
Thank you for your help.

I merged your change to master. I also took the liberty to reuse your initial text as the tooltip, which is simple. Nothing beats simplicity.
I hope the delay does not stop you from contributing. Thanks @BrandonHuber! :+1:

Looks good to me, thanks! I think I like the first version better as well. For a tooltip it's probably more useful just to say what the state is, rather than try to give advice.


Howto record and replay drag&drop operations

Hi Thomas
Sadly I've got a second issue.
How could I record and replay drag&drop operations like I do in the attached video?
Thanks for your help/fix in advance!
DragDropProblem.zip

Drag and drop is something I've looked at in the past.
In comparison with the previous issue (now fixed, see Github), the drag and
drop one is far harder to simulate, as the listening for the events can be
happening in any number of ways.
This is like an enhancement, so if you file it on Github, I'll label it and
it can stay live.
Cheers

From: stbornowski<EMAIL_ADDRESS>Sent: 01 December 2020 20:01
To: tomgallagher/RecordReplay
Cc: Subscribed
Subject: [tomgallagher/RecordReplay] Howto record and replay drag&drop
operations (#5)
Hi Thomas
Sadly I've got a second issue.
How could I record and replay drag&drop operations like I do in the attached
video?
Thanks for your help/fix in advance!
DragDropProblem.zip
<https://github.com/tomgallagher/RecordReplay/files/5625125/DragDropProblem.
zip>

You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/tomgallagher/RecordReplay/issues/5 , or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ADH7WZ4XQ3OTXW2RNRFHIS3SS
VDRLANCNFSM4UJOLSCQ> .
<https://github.com/notifications/beacon/ADH7WZ2ODN7U3YFJ4X4OOULSSVDRLA5CNFS
M4UJOLSC2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4LH3SCKA.gif>
--
This email has been checked for viruses by Avast antivirus software.
https://www.avast.com/antivirus

Hi Thomas
Could you please handle it as an enhancement and realize it in the near future?
It is a key feature for me because there is no possibility to add actions as a workaround for it.
Thanks in advance!
Cheers, Steve

Hi Steve,
It's a big job that cuts right across the whole project.  If you take a look here and here, then that's just the start of it
I can't guarantee any time line for this but I'll have it in my mind
Tom

Hi Tom
Would it be possible to adopt something from an other github project?
Ui.vision works as expected with the function 'dragAndDropToObject'.
Code could be found here:
https://github.com/A9T9/RPA/blob/master/src/common/command_runner.js - Line 405
Thank you, Steve

Yes, I don't think it's impossible.
Simulating the dropping of files is not easy, just doing it with html elements is doable.


Migrate azure-security-attestation to Test Proxy

Description
Migrates azure-security-attestation to Test Proxy.
Updates creation of test resources to output the signing certificates and keys to file instead of requiring them to be updated manually in source code. When rerunning test records with a new resource these existing files simply need to be replaced by what is generated by the resource deployment script.
All SDK Contribution checklist:

[x] The pull request does not introduce [breaking changes]
[x] CHANGELOG is updated for new features, bug fixes or other significant changes.
[x] I have read the contribution guidelines.

General Guidelines and Best Practices

[x] Title of the pull request is clear and informative.
[x] There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

[x] Pull request includes test coverage for the included changes.

API change check
API changes are not detected in this pull request.

/azp run java - attestation - tests

/check-enforcer override

Overriding check enforcer as the failing linting checks are unrelated to this change.


sslSupportMethod not documented

File: docs/reference/pkg/aws/cloudfront/distribution.md
The sslSupportMethod on viewer_certificate is not documented.
A worked example would be really helpful here.  Cloudfront is a complex beast and probably not something most people need to setup or change very often and I found I had to dig around quite a bit online.
My use case is to make a static s3 website with HTTPS support.

Thanks for filing this. It's now documented!


Uglify is too slow on already minified files

Hi,
Uglify is awfully slow when used on files that are already minified ; this issue is referenced there
It takes around 2 minutes to uglify a project using jquery, angular and angular-material, and therefore prevents me to push it on Heroku that has a 60 second startup limit.
So far, the solution is to never use minified version of external libs (like JQuery, Angular and so on); it takes only a few second to uglify then. Thanks to the uglify task, those files are still minified on production.
Maybe playing on the uglify task options could avoid this issue ? Or documentation should tell explicitely to avoid injecting already minified files.

Have you thought about pre-compiling your assets and disabling the grunt hook?
So you build your LESS, JS, templates etc. Deploy with the grunt hook disabled and then when heroku spins up, it takes <1 second?
Some articles for reference
Disabling grunt hook: http://sailsjs.org/documentation/concepts/assets/disabling-grunt
Writing a custom build process: http://ash.zi.vc/sails/2016/02/08/a-better-way-of-deploying-your-sailsjs-application-to-a-production-environment/

this i he best i could get :(
{             mangle: false,             sourceMap: false,             screwIE8: true,             compress: {                 unused: false,                 drop_console: true             }         }

@essuraj thanks for posting those!
@zivc sound advice.
@eric-burel we do more or less exactly what @zivc suggested for our stuff at Treeline.  But that's mainly because we're also using a CDN, and also just because we have just WAY too many front-end assets right now.  Re: documentation: that's a great point.  If anyone has a moment to take a stab at adding a blurb about this somewhere in http://sailsjs.org/documentation/concepts/assets (e.g.  a > block with a couple of sentences and a link back to this issue), I'd appreciate it.

I recommend using non minified versions of the file.
That way is much faster.
I run into the same issue and using non minified version of the files improves the performance of the build.
On the sails build they will end up minified anyway.

As @kvindasAB mentioned, this is the workaround. Switching to gulp would be better :1st_place_medal:

I recommend using non minified versions of the file.
That way is much faster.
I run into the same issue and using non minified version of the files improves the performance of the build.
On the sails build they will end up minified anyway.

@kvindasAB awesome advice
@simonmilz 😭  http://blog.namangoel.com/introduction-to-gulpjs | https://github.com/balderdashy/sails/issues/1461   (btw: @anyonewhohasaminutetohelp this would make a great tutorial on the site now that we have a proper tutorials section.  The goal is extremely short, single page, 2-3 paragraph writeups that describe the current "definitive" way on how do things that aren't exactly core, but that come up all the time like setting up a Sails app to use gulp, etc.  And then linking to other resources like that article for more info)

@mikermcneil thanks for this blog post :+1:

@mikermcneil I'm keeping a list of stuff to add to the Tutorials

Hello Guys,
I have created a small guide on how to deploy pre-built assets.
https://medium.com/@kivind/sailsjs-speeding-up-production-deployment-with-pre-built-assets-17666e9d2232
Hope it helps for the ones still bearing with this issue.

@kvindasAB thanks!

(For the record, this sort of an approach mentioned by @kvindasAB is very similar to what we cover in platzi.com/cursos/javascript-pro/ -- highly recommended)


Decrease brightness of screen

Decrease brightness of screen overall

too bright to see water droplet coming out
may be aversive to mice initially (?)

I will close this issue for now. For documentation's sake, we decided to:

Run screens at full "hardware" brightness. This means all screens should be manually set to the highest brightness setting at all times. Since brightness is controlled via PWM of the backlight, this ensures a continuous brightness signal on the photodiode.
Rendered brightness will be controlled via software by the custom shader introduced in #247 and exposed in the rig schema via the Screen object


Add type in products

Summary
This PR adds column type to the products table. The type column groups a set of targets. For instance products with target 2, 8 and 10 is of type appcat.
Checklist

[x] Categorize the PR by setting a good title and adding one of the labels:
bug, enhancement, documentation, change, breaking, dependency
as they show up in the changelog
[x] Update tests.

Can you link the documentation for this field? Or update the one we have? https://kb.vshn.ch/appuio-cloud/references/architecture/metering-data-flow.html

Thanks for directions, I was very much unsure what has to be updated.

Can you link the docs?

I will create a PR soon in the docs. Meanwhile you can wait approving this PR.

I have no idea what this field is for without the context you have. cc @corvus-ch

Well we need it for reporting dashboards. The goal is internal to team Schedar. Aldebaran also might want to apply some kind of groupings to its products.

Well we need it for reporting dashboards. The goal is internal to team Schedar. Aldebaran also might want to apply some kind of groupings to its products. The value is not mandatory.

That is a dangerous path to go. We must be really cautious about what we add to the billing system and why. Note that there is already some, although implicit, grouping available: the source string of products. If segmenting of the product's source string is done right, it could be used to build grouping.

Well we need it for reporting dashboards. The goal is internal to team Schedar. Aldebaran also might want to apply some kind of groupings to its products. The value is not mandatory.

That is a dangerous path to go. We must be really cautious about what we add to the billing system and why. Note that there is already some, although implicit, grouping available: the source string of products. If segmenting of the product's source string is done right, it could be used to build grouping.

I personally don't see any issues with this new column, it will greatly reduce the complexity of queries and will eliminate the need to update grafana dashboards. I don't see a good alternative but we can have a short call to dive into this topic.

We decided to go with a different approach - changing the source string. Closing this PR


Added custom.ts

Because I accidentally deleted mine and couldn't retrieve it so I spent half an hour to search the web until I found a Taiwanese translation that made it possible for me to fit the pieces back together.

This looks good to me. We need to add a note in the sources to keep the docs in sync. @abchatra


Introduce a gRPC Tagger service

What does this PR do?
This PR introduces a gRPC interface that wraps the Tagger and exposes two RPCs: StreamTags to watch and stream changes to entities in a "push" model, and FetchEntity to pull the current tags for one entity.
Motivation
This is the first part in an epic that aims to centralize the Tagger into a single agent, instead of having one per agent.
Describe your test plan
The changes here can probably be tested more easily when we introduce the other side of this equation, by implementing the client for this. If you still want to give it a go, I think grpcurl can work.

I added some benchmarks to ensure we don't let performance degrade too much. Below are the results. Benchmarks were run with go test -run=NONE -bench=TagStore -count 5 ./pkg/tagger.
$ benchstat old.txt new.txt
name                       old time/op  new time/op  delta
TagStoreThroughput-8        307ms ± 2%   322ms ± 2%  +4.70%  (p=0.008 n=5+5)
TagStore_processTagInfo-8   123µs ±12%   126µs ±12%    ~     (p=0.690 n=5+5)

processTagInfo being run on its own basically shows no change. The overall throughput is a little bit worse (4.7%), but that's expected since there's extra coordination required to notify subscribers.


How do we know if onNotificationOpen comes from a user action or not?

If the user clicks a notifiation, I want to do something. But if the application is already open, I want to handle the situation differently. How can we know?

The notification object (the one you get via onNotificationOpen) has a tap property.
If tap = false => the notification has been processed without a user action (the app was open).
If tap = true => the user has tapped the notification (the app was in the background).

Actually I have an issue with that, the tap property is set to true, even if I did not touch the notification itself when application was in background.
I wanted to do some stuff only when user touches the notification, but if I leave the notification un touched, and run the application after it automatically dismissed, the application goes through "onNotificationOpen" callbak and has tap = true

I have the same problem as @jeangravier. I do not think this issue is closed.


FEATURE: hold back on emoji autocomplete

In chat since "ENTER" submits a message we need to be a bit more protective
of autocomplete dialog.
Hold off for at least 2 chars and avoid autocomplete unless there are 2 non
symbol letters, this catches most commonly typed word emojis

lol, not configured here ... will fix ....


Support multiple css paths

Rationale
I am currently using critical-path-css-rails to generate the critical CSS for a react on rails project using webpack to package our assets. We are using the ExtractTextPlugin to separate the CSS for each page to minimize load. We successfully deployed critical-path-css-rails for one of our pages, but cannot easily apply it to additional pages without a way to handle multiple CSS paths.
Implementation
This is the simplest implementation that occurred to me. It adds a new key, css _paths, which is mutually exclusive from the existing key css_path. Like routes, css_paths can be a list of css files that correspond to the routes; we assume they are in the same respective order. Therefore, both arrays should be the same length.
This change is backwards compatible, so existing users should not experience a breaking change.
The PR also updates the README to address #31.
Tasks

[x] Add new key css_paths to specify separate css path per route
[x] Add error checking to ensure css_paths is used correctly
[x] Clarify error messages
[x] Update Configuration to add new key to config object
[x] Update CssFetcher to pass separate css_path to each Penthouse invocation
[x] Update generator to add css_paths key
[x] Add text to test page that is always visible
[x] Bump version
[x] Squash before merging

@randallreedjr Thanks for the PR! I've been taking a look at this, and I have some feedback. I'll try to post all my feedback by this weekend. One thing that I'd like to see done is the new css_paths config added to the Usage section in the Readme, along with the other config options. That would be a huge help!
I have a few edit suggestions for the tests (which we can always refactor later so not terribly critical), css_fetcher.rb, and config_loader.rb. I'll try to post them as quickly as possible.

Cool, thanks @michael-misshore! I really appreciate your responsiveness. I'll get cracking on your feedback as soon as I can.

@michael-misshore Friendly reminder for feedback :)

@dm-randy Thanks and sorry!. @randallreedjr Your changes gave me some ideas for some refactors. I started working on this before the holiday, and I'll try to have something that can be reviewed and merged into your branch sometime soon.

Oops, both of those accounts are me

@michael-misshore I just pushed an update with an addition to the README. Any feedback from your end?

@randallreedjr README changes look good. Did you see the review I started regarding config_loader_spec.rb? I'd like to hear your feedback (and any changes you make) on it.

@michael-misshore Hm, not sure what's going on, but I don't see any reviews


Bump app build number to  350

Cherry pick is scheduled.

Error trying doing the automated Cherry picking. Please do this manually
+++ Updating remotes...
Fetching upstream
Failed to add the RSA host key for IP address '<IP_ADDRESS>' to the list of known hosts (/app/.ssh/known_hosts).
From github.com:mattermost/mattermost-mobile
 * [new branch]          MM-29317      -> upstream/MM-29317
 * [new branch]          build-349     -> upstream/build-349
   2ccd2a1b2..6c5b2305b  build-pr-5220 -> upstream/build-pr-5220
 * [new branch]          e2e-MM-30430  -> upstream/e2e-MM-30430
   fa58f25a9..72daf965c  master        -> upstream/master
   121656038..ce283324a  unsigned      -> upstream/unsigned
 * [new branch]          validate-opengraph-image -> upstream/validate-opengraph-image
 * [new tag]             v1.41.1       -> v1.41.1
Fetching origin
Failed to add the RSA host key for IP address '<IP_ADDRESS>' to the list of known hosts (/app/.ssh/known_hosts).
+++ Updating remotes done...
+++ Creating local branch automated-cherry-pick-of-mattermost-mobile-#5296-upstream-release-1.42-1617897117
Switched to a new branch 'automated-cherry-pick-of-mattermost-mobile-#5296-upstream-release-1.42-1617897117'
Branch 'automated-cherry-pick-of-mattermost-mobile-#5296-upstream-release-1.42-1617897117' set up to track remote branch 'release-1.42' from 'upstream'.

+++ About to attempt cherry pick of PR #5296 with merge commit 72daf965cf69dde863c85f025952c8554c622486.

error: could not apply 72daf965c... Bump app build number to  350 (#5296)
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'

+++ Conflicts detected:

UU android/app/build.gradle
UU fastlane/Gemfile.lock
UU ios/Mattermost.xcodeproj/project.pbxproj
UU ios/Mattermost/Info.plist
UU ios/MattermostShare/Info.plist
UU ios/NotificationService/Info.plist
Aborting.

+++ Aborting in-progress git cherry-pick.

+++ Returning you to the master branch and cleaning up.


TableView disable scrolling when closing menu

Hi is it possible to disable scrolling of table view in left menu when i am closing it with pan gesture? Because when i have table view inside left menu it looks funny when i scroll left and up slightly.
Cheers

Do you want to disable the pan gesture?

I want to keep pan gesture but i want to disable any movement in menu when it is "panning" away. I dont know how to do it without making scrolling inside menu miserable.

Do you mean this?
SlideMenuOptions.simultaneousGestureRecognizers = false

Have you also tried to keep a specific position in tableview when closing and reopening left menu?

if sender.state == UIGestureRecognizerState.Began {
slideMenuController()?.removeLeftGestures()
slideMenuController()?.removeRightGestures()
print("You Start pan")
    }
    if sender.state == UIGestureRecognizerState.Ended {
        slideMenuController()?.addLeftGestures()
        slideMenuController()?.addRightGestures()
        print("You end pan")
    }

give your table view ui pan gesture, and the code from above do the trick.

@LayChannara  Thanks for help.
SlideMenuOptions.simultaneousGestureRecognizers = false. Its working for me to disable multiple scrolling while scroll the tableview


Relax ipyleaflet pinning

Previously I had somehow got the idea that ipyleaflet 0.17.0 didn't work on colab. I therefore pinned the dependency to >0.17.0. However, perhaps that was a transient thing, because now 0.17.0 seems to work fine on colab. Also, 0.17.0 is actually better than more recent versions because it doesn't have a bug where maps are blank after conversion to html (https://github.com/jupyter-widgets/ipyleaflet/issues/1090). Hence we should allow malariagen_data to be installed with ipyleaflet 0.17.0.

cc @ahernank, doing this to make it easier to install malariagen_data together with ipyleaflet 0.17.0.


Websocket sends tcp rst

Hello!
My program consist of 2 simple websocket clients with ssl, that read some data from remote server. Server send ping every 5 seconds and my program must response with pong in next 5 seconds. I tried both sync and async implementation and have the same result.
So, how its look like. Clients starting read data, ping-pong is working(server drops connection in 15s, if not), but after 5-10 minutes one of the clients receive ping and trying to response with pong, but have boost::system_error "Connection reset by peer" or "Broken pipe" on write.

{"ping":1661506057018}
boost::syserr : dest : {"pong":1661506057018}
Connection reset by peer
wss1

Wireshark said that clients send TCP RST but I cant figure out the reason.

I'm using gcc, c++17 and Boost 1.75.0 on Ubuntu-20.04LTS. Tested also on other Linux computers from other location and internet providers.
Here's sync websocket implementation:
class WebSocket : public std::enable_shared_from_this<WebSocket>
{
protected:
	tcp::resolver resolver_;
	websocket::stream<beast::ssl_stream<beast::tcp_stream>> ws_;
	websocket::response_type res;
	beast::flat_buffer buffer_;

        std::string host_;
	std::string port_;
	std::string realtime_dir_;
	std::vector<std::string> subs_;

	websocket::permessage_deflate opt;

public:
	explicit WebSocket(net::io_context& ioc, ssl::context& ctx)
	:
		resolver_(ioc),
		ws_(net::make_strand(ioc), ctx)
        {}

	void run(const std::string& host, const std::string& realtime_dir, const std::string& port, const std::vector<std::string>& subs)
	{
		// Save these for later
		opt.client_enable = true;

		host_ = host;
		port_ = port;
		realtime_dir_ = realtime_dir;
		subs_ = subs;

		if (!subs.size())
			throw std::runtime_error{"no subscribe topics found"};

		// Look up the domain name
		auto results = resolver_.resolve(host_, port_);

		// Make the connection on the IP address we get from a lookup
		beast::get_lowest_layer(ws_).connect(results);

		// Set SNI Hostname (many hosts need this to handshake successfully)
		if (!SSL_set_tlsext_host_name(
			ws_.next_layer().native_handle(),
			host_.c_str()))
		{
			auto ec = beast::error_code(static_cast<int>(::ERR_get_error()),
				net::error::get_ssl_category());
			return fail(ec, "connect");
		}

		// Perform the SSL handshake
		ws_.next_layer().handshake(ssl::stream_base::client);

		// Set suggested timeout settings for the Basewebsocket
		ws_.set_option(websocket::stream_base::timeout::suggested(beast::role_type::client)); // (?)
		ws_.set_option(opt);

		// Perform the Basewebsocket handshake
		ws_.handshake(res, host_, realtime_dir_);

		if (res.result_int() == 429 || res.result_int() == 503)
			return std::this_thread::sleep_for(std::chrono::seconds (30));

		// Send the message
		std::cout << res << std::flush;

		ws_.write(net::buffer(subs_[0]));

		// Read a message into our buffer
		ws_.read(buffer_);

		for (size_t i = 1; i < subs_.size(); i++)
			write(subs_[i]);
	}

	int read(int (*(parser)) (const std::string& raw))
	{
		std::string dest (4096, 0);
		buffer_.clear();

		try
		{

			size_t bytes_readed = ws_.read(buffer_);

			auto out = gzDecompress (boost::asio::buffer_cast<const char*>(buffer_.data()), bytes_readed, dest.c_str(), 4096);

			dest.resize(out);

			std::cout << '\n' << dest << std::endl;
			if (is_ping(dest))
			{
				dest = generate_pong(dest);
				ws_.write(net::buffer(dest));
				std::cout << '\n' << dest << std::endl;
				dest.resize(4096, 0);
				return 1;
			}
			return (*parser) (dest);
		}
		catch (const boost::system::system_error& e)
		{
			std::cerr << "boost::syserr : dest : " << dest << "\n" << e.what() << std::endl;
			return -1;
		}
		catch (const std::system_error& e)
		{
			std::cerr << "syserr : dest : " << dest << "\n" << e.what() << std::endl;
			return -1;
		}
		catch (const boost::property_tree::ptree_error& e)
		{
			std::cerr << "ptree_err : dest : " << dest << "\n" << e.what() << std::endl;
			return 1;
		}
		catch (...)
		{
			std::cerr << dest << std::endl;
			return -1;
		}
	}
}

And usage:
net::io_context ioc;
ssl::context ctx{ ssl::context::tlsv13_client };

load_root_certificates(ctx);

wss1 = std::make_shared<WebSocket>(ioc, ctx);
wss1->run("host", "/dir", "443", std::vector<std::string>{sub});

wss2 = std::make_shared<WebSocket>(ioc, ctx);
wss2->run("host", "/dir", "443", std::vector<std::string>{sub});

while(true)
{
	int err = wss1->read(&parser1);
	if (err < 0)
	{
		std::cout << "wss1" << std::endl;
		return;
	}

	err = wss2->read(&parser2);
	if (err < 0)
	{
		std::cout << "wss2" << std::endl;
		return;
	}
}

Thanks for any help!

is this a Websocket ping and pong or is it a custom message?

Its custom message in json, that designed by server side

if you are communicating with a "real" websocket server, then sync might not work correctly for you. At a minimum you should be using the async APIs, and always have a call to websocket::stream::async_read or  websocket::stream::async_read_some active.

Can you please explain whats the difference between sync and async read in this case?

So, I've tested it in async read mode and again I have the same issue. Only on wss1, may problem be on server side?

async read will interleave with async write and correctly send websocket ping and pong messages, while the sync versions cannot do that.

Can you post your async code?

Async read:
void WebSocket::read()
{
	buffer_.clear();

	ws_.async_read(buffer_, beast::bind_front_handler(&WebSocket::on_message_read, shared_from_this()));
}

void WebSocket::on_message_read(beast::error_code ec, std::size_t bytes_transferred) {
	if (ec) return fail(ec, "read");

	std::string dest (4096, 0);
	try
	{
		auto out = gzDecompress (boost::asio::buffer_cast<const char*>(buffer_.data()), bytes_transferred, dest.c_str(), 4096);

		dest.resize(out);
		std::cout << '\n' << dest << std::endl;

		if (is_ping(dest))
		{
			dest = generate_pong(dest);
			std::cout << '\n' << dest << std::endl;
			ws_.async_write(net::buffer(dest), beast::bind_front_handler(&WebSocket::on_message_write, shared_from_this()));
			dest.resize(4096, 0);
		}
	}
	catch (const boost::system::system_error& e)
	{
		std::cerr << "boost::syserr : dest : " << dest << "\n" << e.what() << std::endl;
	}
	catch (const std::system_error& e)
	{
		std::cerr << "syserr : dest : " << dest << "\n" << e.what() << std::endl;
	}
	catch (const boost::property_tree::ptree_error& e)
	{
		std::cerr << "ptree_err : dest : " << dest << "\n" << e.what() << std::endl;
	}
	catch (...)
	{
		std::cerr << dest << std::endl;
	}
}

void WebSocket::on_message_write(beast::error_code ec, std::size_t bytes_transferred) {
	if (ec) return fail(ec, "write");
}

Usage:
while (true)
{
	ioc.reset();
	wss1->read();
	wss2->read();
	ioc.run();
}

Also I want to mention again, that this ping and pong messages are not websocket's ping and pong. They are required by server and their payload is json with ping or pong values.

I would need a minimal runnable example.


Databases created for projects don't show up in database navigation tab

I've noticed that database connections that are created in the projects tab don't show up in the database navigation tab.  However you can add database connections in the database navigation tab and assign them to projects.  Maybe I'm misunderstanding something but reciprocity here would make sense to me.
OSX 10.14.6
DBeaver 6.1.4
mysqld  Ver 5.6.43 for osx10.14 on x86_64 (Homebrew)

I believe database navigator only shows connections of currently active project. From projects view you can set the active project.


[BUG] Worker node key chosen is null or undefined after 6 retries at

Duplicate issue

[X] I've searched for any related issues and avoided creating a duplicate issue.

Description
Error: Worker node key chosen is null or undefined after 6 retries at L.execute (/usr/src/app/node_modules/poolifier/lib/index.js:1:14589) at exports.DynamicThreadPool.chooseWorkerNode (/usr/src/app/node_modules/poolifier/lib/index.js:1:36288) at /usr/src/app/node_modules/poolifier/lib/index.js:1:32596 at new Promise () at exports.DynamicThreadPool.execute (/usr/src/app/node_modules/poolifier/lib/index.js:1:32044) at HttpModeTask.process (/usr/src/app/dist/lib/Task/task.service.js:450:18) at HttpModeTask.execute (/usr/src/app/dist/lib/Task/task.service.js:264:32) at process.processTicksAndRejections (node:internal/process/task_queues:95:5) at async Object.processJob [as fn] (/usr/src/app/dist/utils/agenda/jobs/task.js:22:24) { [stack]: 'Error: Worker node key chosen is null or undefined after 6 retries\n' + ' at L.execute (/usr/src/app/node_modules/poolifier/lib/index.js:1:14589)\n' + ' at exports.DynamicThreadPool.chooseWorkerNode (/usr/src/app/node_modules/poolifier/lib/index.js:1:36288)\n' + ' at /usr/src/app/node_modules/poolifier/lib/index.js:1:32596\n' + ' at new Promise ()\n' + ' at exports.DynamicThreadPool.execute (/usr/src/app/node_modules/poolifier/lib/index.js:1:32044)\n' + ' at HttpModeTask.process (/usr/src/app/dist/lib/Task/task.service.js:450:18)\n' + ' at HttpModeTask.execute (/usr/src/app/dist/lib/Task/task.service.js:264:32)\n' + ' at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' + ' at async Object.processJob [as fn] (/usr/src/app/dist/utils/agenda/jobs/task.js:22:24)', [message]: 'Worker node key chosen is null or undefined after 6 retries' }
Expected behavior
no error
Actual behavior
throws error
Steps to reproduce
It occurs randomly while using DynamicThreadPool
poolifier version
3.0.9
Node.js version
20.10.0
System
No response
Attachments
No response

Could you please please test version 3.0.13?

More information is needed:

pool configuration
task function
stack trace with --enable-source-maps

Basically, the complete poolifier usage to reproduce it. And ideally a small standalone reproduction case.

will try to create a small project replicating this issue, but I've observed till now it's mostly random.

Could you please test version 3.0.13 or superior?

Yes, 3.0.14 had the same issue / error. Currently on 3.1.3, will update to the latest version. I'll let you know if issue arise.

Basically, the complete poolifier usage to reproduce it. And ideally a small standalone reproduction case.

will try to create a small project replicating this issue, but I've observed till now it's mostly random.

It's not random, there's conditions triggering it and the list is quite short:

No worker nodes are ready
Worker choice strategy algo is buggy

That's why the pool options used was asked.

Example Interface file -
import { DynamicThreadPool, availableParallelism } from "poolifier";
const MAX_POOL_SIZE = availableParallelism() > 1 ? availableParallelism() - 1 : 1;

export const poolProcessHttp = new DynamicThreadPool<HttpWP, Promise<HttpWR>>(
    0,
    MAX_POOL_SIZE,
    PROCESS_HTTP_WORKER_PATH,
    {
        workerOptions: {
            name: "Http Worker",
        },
        errorHandler: (e: Error) => {
            logger.error("error in http worker %o", e);
            return e;
        },
        onlineHandler: () => {
            logger.info("HTTP Worker is online");
        },
    }
);

Example Worker file -
import { ThreadWorker } from "poolifier";
import { z } from "zod";
import { processTask } from "../../helper";
import { MakeHttpJsonCallResponse } from "../../../types/utils";

const WorkerPayloadSchema = z.object({
    url: z.string(),
    method: z.union([
        z.literal("get"),
        z.literal("post"),
        z.literal("put"),
        z.literal("patch"),
        z.literal("delete"),
    ]),
    body: z.record(z.string().or(z.number()), z.unknown()).optional(),
    headers: z.record(z.string(), z.string()).optional(),
    requestTimeoutSec: z.number().min(1).optional(),
});

export type WorkerPayload = z.infer<typeof WorkerPayloadSchema>;

export type WorkerResponse = MakeHttpJsonCallResponse;

export class ProcessHttpWorker extends ThreadWorker<WorkerPayload, Promise<WorkerResponse>> {
    constructor() {
        super(async (data) => await this.execute(data), {
            maxInactiveTime: 1000 * 60 * 2,
            killBehavior: "HARD",
        });
    }

    private async execute(data: WorkerPayload | undefined): Promise<WorkerResponse> {
        const valid = await WorkerPayloadSchema.safeParseAsync(data);

        if (!valid.success) {
            return {
                error: true,
                errors: {
                    message: valid.error.message,
                    cause: valid.error.cause,
                },
            };
        }

        const resp = await processTask(valid.data);
        if (resp.error) {
            return resp;
        }

        return {
            error: false,
            response: {
                body: resp.response.body,
                headers: resp.response.headers,
                statusCode: resp.response.statusCode,
                statusMessage: resp.response.statusMessage,
            },
        };
    }
}

export default new ProcessHttpWorker();

I have a scheduler(agenda) implemented to invoke this worker function, the concurrency for the job-processor / scheduler has similar limit i.e. MAX_POOL_SIZE = availableParallelism() > 1 ? availableParallelism() - 1 : 1

Example Interface file -
import { DynamicThreadPool, availableParallelism } from "poolifier";
const MAX_POOL_SIZE = availableParallelism() > 1 ? availableParallelism() - 1 : 1;

poolifier availableParallelism() always return a values higher or equal to 1.

export const poolProcessHttp = new DynamicThreadPool<HttpWP, Promise>(
0,

That's why the bug is happening. Until poolifier code is able to properly cope with no workers ready at tasks submission, you can set the min to 1 to avoid the issue.

MAX_POOL_SIZE,
PROCESS_HTTP_WORKER_PATH,

This throws error, can't set Dynamic Pool Min and Max value same, I had encountered this issue when I tried running on a 2 core / 2 thread CPU device. After that I set the min value to 0.
import { DynamicThreadPool, availableParallelism } from "poolifier";
const MAX_POOL_SIZE = availableParallelism() > 1 ? availableParallelism() - 1 : 1;

export const poolProcessHttp = new DynamicThreadPool<HttpWP, Promise<HttpWR>>(
    1,
    1,
    PROCESS_HTTP_WORKER_PATH,
    {
        workerOptions: {
            name: "Http Worker",
        },
        errorHandler: (e: Error) => {
            logger.error("error in http worker %o", e);
            return e;
        },
        onlineHandler: () => {
            logger.info("HTTP Worker is online");
        },
    }
);

A dynamic pool with min = max is a fixed pool, hence the error.
Why you do not set max to availableParallelism() (should be 2 and != from min)?

Will replace max value to availableParallelism() - 1 -> availableParallelism(),  and min value to 1. Thanks for the guidance 😊

Will replace max value to availableParallelism() - 1 -> availableParallelism(), and min value to 1. Thanks for the guidance 😊

It's a workaround, poolifier must handle properly min = 0.

I'm not totally convinced by the fix, so please reopen if it still occurs.

Error: Worker node key chosen is null or undefined after 16 retries
at X.executeStrategy (/usr/src/app/node_modules/poolifier/lib/index.cjs:1:15851)
at X.execute (/usr/src/app/node_modules/poolifier/lib/index.cjs:1:15656)
at exports.DynamicThreadPool.chooseWorkerNode (/usr/src/app/node_modules/poolifier/lib/index.cjs:1:38730)
at /usr/src/app/node_modules/poolifier/lib/index.cjs:1:35268
at new Promise ()
at exports.DynamicThreadPool.execute (/usr/src/app/node_modules/poolifier/lib/index.cjs:1:34716)
at HttpModeTask.process (/usr/src/app/dist/lib/Task/task.service.js:452:18)
at HttpModeTask.execute (/usr/src/app/dist/lib/Task/task.service.js:266:32)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.processJob [as fn] (/usr/src/app/dist/utils/agenda/jobs/task.js:22:24) {
[stack]: 'Error: Worker node key chosen is null or undefined after 16 retries\n' +
'    at X.executeStrategy (/usr/src/app/node_modules/poolifier/lib/index.cjs:1:15851)\n' +
'    at X.execute (/usr/src/app/node_modules/poolifier/lib/index.cjs:1:15656)\n' +
'    at exports.DynamicThreadPool.chooseWorkerNode (/usr/src/app/node_modules/poolifier/lib/index.cjs:1:38730)\n' +
'    at /usr/src/app/node_modules/poolifier/lib/index.cjs:1:35268\n' +
'    at new Promise ()\n' +
'    at exports.DynamicThreadPool.execute (/usr/src/app/node_modules/poolifier/lib/index.cjs:1:34716)\n' +
'    at HttpModeTask.process (/usr/src/app/dist/lib/Task/task.service.js:452:18)\n' +
'    at HttpModeTask.execute (/usr/src/app/dist/lib/Task/task.service.js:266:32)\n' +
'    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)\n' +
'    at async Object.processJob [as fn] (/usr/src/app/dist/utils/agenda/jobs/task.js:22:24)',
[message]: 'Worker node key chosen is null or undefined after 16 retries'
}

No worker has crashed before? And fail to restart?
Because I do not see how it can happen with one minimum worker in the pool.

Today I have made some modifications in the worker interface -
import { DynamicThreadPool, availableParallelism } from "poolifier";
const MAX_POOL_SIZE = availableParallelism();

export const poolProcessHttp = new DynamicThreadPool<HttpWP, Promise<HttpWR>>(
    1,
    MAX_POOL_SIZE,
    PROCESS_HTTP_WORKER_PATH,
    {
        // start
        enableTasksQueue: true,
        tasksQueueOptions: {
            concurrency: 8, // as per provided examples
        },
        // end ^ these task queue options
        workerOptions: {
            name: "Http Worker",
        },
        errorHandler: (e: Error) => {
            logger.error("error in http worker %o", e);
            return e;
        },
        onlineHandler: () => {
            logger.info("HTTP Worker is online");
        },
    }
);

This error occurs when the workers are unable to perform a HTTP request (server down/unreachable). I have already handled HTTP request timeout + maxInactiveTime: 1000 * 60 * 2 and killBehavior: "HARD" for worker options, so it should not block any workers.

Even if all workers are stuck in limbo, a worker shall be chosen unless they all crashed at the same time. Their queues will grow until an out of memory is triggered leading to a crash. But all of them crashing ...
You should also ensure the HTTP client used is properly close in the killHandler on the worker side.

I'm using got lib for making request

export async function makeHttpJsonCall<
    RequestBody = Record<string | number, unknown> | undefined,
    ResponseBody = Record<string | number, unknown> | undefined,
>(payload: MakeHttpJsonCallPayload<RequestBody>): Promise<MakeHttpJsonCallResponse<ResponseBody>> {
    const options: OptionsOfJSONResponseBody = {
        method: payload.method,
        responseType: "json",
        timeout: {
            request:
                1000 *
                (typeof payload.requestTimeoutSec === "number" ? payload.requestTimeoutSec : 120), // 2 mins
        },
    };
    options.headers = { "trigger-from": "wms" };
    if ("headers" in payload) {
        options.headers = { ...options.headers, ...payload.headers };
    }

    if ("body" in payload && payload.body && typeof payload.body === "object") {
        options.json = payload.body;
    }
    // console.time("api time");
    try {
        const response = await got<ResponseBody>(new URL(payload.url), options);
        // console.timeEnd("api time");
        response?.destroy();
        return {
            error: false,
            response: {
                body: response.body,
                headers: response.headers,
                statusCode: response.statusCode,
                statusMessage: response.statusMessage,
            },
        };
    } catch (err) {
        const errorPayload: {
            message: string;
            statusCode?: number;
            body?: unknown;
            headers?: unknown;
            statusMessage?: string;
            cause: unknown;
        } = {
            message: "something went wrong",
            cause: {},
        };

        if (err instanceof HTTPError) {
            logger.warn("[Http Error] unable to make request -> %s", err.message);
            errorPayload.message = err.message;
            errorPayload.cause = err.cause;
            errorPayload.statusCode = err.response.statusCode;
            errorPayload.body = err.response.body;
            errorPayload.headers = err.response.headers;
            errorPayload.statusMessage = err.response.statusMessage;
            return { error: true, errors: errorPayload };
        }

        if (err instanceof RequestError) {
            logger.warn("[Request Error] unable to make request -> %s", err.message);
            errorPayload.message = err.message;
            errorPayload.cause = err.cause;
            errorPayload.statusCode = err.response?.statusCode;
            errorPayload.body = err.response?.body;
            return { error: true, errors: errorPayload };
        }
        // console.timeEnd("api time");
        const { message, stack, cause } = err as Error;
        logger.error("[Error] unable to make request -> %s", message);
        return {
            error: true,
            errors: {
                message,
                stack,
                cause,
            },
        };
    }
}

If got only require a destroy() call on the response to cleanup the resources used to perform a request, that should be fine. Otherwise you have a memory leak on the worker side.

I would also pre instantiate in a Map got instances with a size = concurrency. And reuse them to process requests to avoid the got instantiation overhead on the worker side. Unless got is already smart enough to cache instances at got invocation.

This might be the issue -
By default, Got will retry on failure. To disable this option, set options.retry to 0.

Since it's not a poolifier bug, I reclose it.
The discord is more appropriate for support request than GitHub issues.

Issue is not fixed. I now have a test to reproduce the issue.


Remove image-size logic from cellpose_segmentation task

We have this block of code in cellpose_segmentation.py:
    # Extract image size from FOV-ROI indices
    # Note: this works at level=0, where FOVs should all be of the exact same
    #       size (in pixels)
    FOV_ROI_table = ad.read_zarr(f"{zarrurl}tables/FOV_ROI_table")
    list_FOV_indices_level0 = convert_ROI_table_to_indices(
        FOV_ROI_table,
        level=0,
        full_res_pxl_sizes_zyx=full_res_pxl_sizes_zyx,
    )
    ref_img_size = None
    for indices in list_FOV_indices_level0:
        img_size = (indices[3] - indices[2], indices[5] - indices[4])
        if ref_img_size is None:
            ref_img_size = img_size
        else:
            if img_size != ref_img_size:
                raise Exception(
                    "ERROR: inconsistent image sizes in "
                    f"{list_FOV_indices_level0=}"
                )
    img_size_y, img_size_x = img_size[:]

This was probably a leftover from previous implementation, and the image size information is not used in the rest of the task.

Will be closed by https://github.com/fractal-analytics-platform/fractal-tasks-core/pull/306/commits/6b345ca1a5e303e71bbd0bb3b8caf89e39632ae0, as part of #306


Please release 4.0.0.beta3

Hi Ben, thanks for all your work on 4.0! :heart:
4.0.0.beta2 was released January 16, 2015 (https://rubygems.org/gems/paper_trail).  I've been using master in production without any issues so far.  Please release beta3.  Thanks!

Was trying to release an rc1 after resolving one or two more open issues, hopefully by the end of the week. I've been super busy traveling, but I haven't forgotten about this, will try to see what I can do tonight.

Thanks Ben, you're awesome!


Support for Hash Arguments

According to the Handlebars documentation:

Handlebars helpers can also receive an optional sequence of key-value pairs as their final parameter (referred to as hash arguments in the documentation):
{{{link "See more..." href=story.url class="story"}}}
The keys in hash arguments must each be simple identifiers, and the values are Handlebars expressions. This means that values can be simple identifiers, paths, or Strings.

In the javascript version, this come across in the last parameter passed to the helper function.
I see several possible ways of handling this:

If hash args are present, append a value onto the arguments array inside the HandlebarsHelper delegate. This stays closer to the functionality that the Javascript version uses, though it means we've got to do custom checking and casting every time we want to use it.
Pass hash args in an object to the HandlebarsHelper delegate. This feels more idiomatic but we run into a collision with the HandlebarsBlockHelper delegate, which already has an options parameter with the unfortunately-named class of HelperOptions (which should more appropriately be named BlockHelperOptions, IMO)

What about passing it as a dictionary as the last arguments value?

Right, that's what I'm getting at with Option 1 above. But we'd probably want to wrap it in another class to match Handlebars, which gives you an object like this { "hash": { ... }, "data": { ... } }

I've been noodling around in the parser in an attempt to implement hash arguments, so I can take a stab at fully implementing this if you're interested.

Go for it!

Will do!

We got a lot closer to this via @amaclean's PR#93. In fact, it may just work - but there's no test coverage to verify. At most, it looks like we would just need an improved expression visitor that can bind a HashExpression to the arguments array of a helper expression.

I did some basic tests and confirmed #93 does indeed cover the helper scenario as well as the partial scenario. If you supply hash arguments to the end of a helper statement, as specified in the handlebarsjs docs, the last element in the args array will be a Dictionary<string, object> of the hash arguments.


shinn2020 example colab notebook broken

Hi,
It seems that the code for visualizing the Shin2020 model is broken.
The following error is given when trying to run the visualize cell:
ArgumentTypeError: Invalid argument type: solution=<ddm.solution.Solution object at 0x7f4ed67f5810> is not of type Generic(<class 'ddm.solution.Solution'>) in OverlayChain.apply

Thanks for the bug report.
Looks like this was caused by a recent change.  While I work on fixing it, as a solution, replace the line in the Jupyter notebook which starts "!pip" with the following:
!pip -q install<EMAIL_ADDRESS>Make sure you do this before you run the first cell.  (You may have to click on "Show code" first.)

Should be working again, even without the workaround!

thanks!


Add build123d.py birdhouse example

This PR is to add a birdhouse example for build123d, which is a new python-based CodeCAD package.

Thanks @jdegenstein, I've merged this but I noticed that build123 isn't in the main readme?