Version packages (alpha)

This PR was opened by the Changesets release GitHub action. When you're ready to do a release, you can merge this and publish to npm yourself or setup this action to publish automatically. If you're not ready to do a release yet, that's fine, whenever you add more changesets to main, this PR will be updated.
⚠⚠⚠⚠⚠⚠
main is currently in pre mode so this branch has prereleases rather than normal releases. If you want to exit prereleases, run changeset pre exit on main.
⚠⚠⚠⚠⚠⚠
Releases
<EMAIL_ADDRESS>Patch Changes

c628b63: Swap out performance ponyfill

<EMAIL_ADDRESS>Patch Changes

Updated dependencies [c628b63]

<EMAIL_ADDRESS>

Will release after https://github.com/thefrontside/interactors/pull/75


Merging changes from SBOLInputDialog into SBOLDesigner

Within iBioSIm, gui/modelEditor/sbol/SBOLInputDialog.java, there is a branch of this class.  Merge this branch back into SBOLDesigner.

@tramyn Do you mind walking me through what changes you made sometime?  SBOLInputDialog has 500 more lines than RootInputDialog, so it might be easier for me to just modify the original.

@michael13162 Yes. Let's do this in person when we're both in lab. Let me know when.


[BUG] In German some Words on the left are broken

Please confirm these before moving forward

[X] I have searched for my issue and have not found a work-in-progress/duplicate/resolved issue.
[X] I have tested that this issue has not been fixed in the latest (beta or stable) release.
[X] I have checked the FAQ section for solutions.
[X] This issue is about a bug (if it is not, please use the correct template).

UniGetUI Version
3.1.2 beta4
Windows version, edition, and architecture
Windows 11 Pro 22635.4367
Describe your issue
In German some Words on the left are broken (Einstellungen = Settings).

Steps to reproduce the issue
No response
UniGetUI Log
-

Package Managers Logs
-

Relevant information
No response
Screenshots and videos
No response

Did this happen in 3.1.1?

No, only on 3.1.2 beta 2 and up. The column seems to short.

+1


ip4defrag: fix maximum fragment offset security check

The maximum acceptable value for the fragment offset is expressed in
units of 8 bytes (8189), so the comparison must be done directly on the
header value, before doing the multiplication.

Sorry, I'm going to resubmit with a different author address to match the corporate CLA agreement we have on file.


Why did I download the source code and display the following error：from . import inference  ImportError: cannot import name 'inference'

you should run this command: python -m nmt.nmt --xxxxxxxx, instead python nmt/nmt.py

I also try  this: python -m nmt.nmt --xxxxxxxx
--src=vi --tgt=en
--vocab_prefix=/tmp/nmt_data/vocab
--train_prefix=/tmp/nmt_data/train
--dev_prefix=/tmp/nmt_data/tst2012
--test_prefix=/tmp/nmt_data/tst2013
--out_dir=/tmp/nmt_model
--num_train_steps=12000
--steps_per_stats=100
--num_layers=2
--num_units=128
--dropout=0.2
--metrics=bleu
But it gives Error as:
**    from . import inference
ImportError: attempted relative import with no known parent package
**

From what location within the directory structure are you running this command? I encountered the same issue when I ran it one level too deep (in the 'nmt' directory). Try running it from the same directory where you find the README.md. That solved the problem for me.


(SUGGESTION) - Saving fastflags when needed.

Acknowledgement of preliminary instructions

[X] I have read the preliminary instructions, and I am certain that my problem has not already been addressed.

What problem did you encounter?
I was playing a high quality game which i was interested but my fastflags removes most of the details to make it a realistic game... I would like to save the fastflags so i can change back when im done playing.

Sounds like you want a "loadout" type thing?

Sounds like you want a "loadout" type thing?

Indeed indeed.

Duplicate of #1098.
Also, there might be a better solution to this than a profile system.


Orchestrator doesn't receive cluster changes through discovery

Chart version: 0.1.12
Operator image version: v0.1.12
From operator log:
I1004 07:40:16.312659       1 sync.go:36] Syncing cluster: silly
I1004 07:40:17.513492       1 orc_reconciliation.go:40] Orchestrator reconciliation for cluster 'silly' started...
E1004 07:40:17.530144       1 orc_reconciliation.go:48] Fail to get cluster from orchestrator: [orc]: status: 500 msg: at error, json unmarshal error: unexpected end of JSON input, details: <nil>. Now tries to register nodes.
I1004 07:40:17.701285       1 utils.go:172] Endpoints for service 'silly-mysql-master' were up-to-date.
E1004 07:40:17.707812       1 controller.go:225] mysqlclusterController controller: Re-queuing item "mysql-operator/silly" due to error processing: sync error: Operation cannot be fulfilled on mysqlclusters.mysql.presslabs.org "silly": the object has been modified; please apply your changes to the latest version and try again

From helper container in mysql pod:
I1004 07:53:37.332505      12 apphelper.go:54] Configured orchestrator user...
I1004 07:53:37.334482      12 apphelper.go:60] Configured replication user...
I1004 07:53:37.336214      12 apphelper.go:66] Configured metrics exporter user...
I1004 07:53:37.336254      12 util.go:70] Orc request on: http://mysql-operator-orchestrator.mysql-operator/api/master/silly.mysql-operator
I1004 07:53:37.350705      12 util.go:53] Unmarshal data is:
E1004 07:53:37.350759      12 util.go:197] Failed to connect to orc for finding master, err: [orc]: status: 500 msg: at error, json unmarshal error: unexpected end of JSON input, details: <nil>. Fallback to determine master by its id.
I1004 07:53:37.350811      12 apphelper.go:72] Configured topology...
I1004 07:53:37.376783      12 apphelper.go:77] Flag setup as complete...

From orchestrator log:
[martini] Started GET /api/master/silly.mysql-operator for <IP_ADDRESS>
2018-10-04 07:53:37 ERROR Unable to determine cluster name
[martini] Completed 500 Internal Server Error in 7.107462ms

After I add the cluster to Orchestrator manually the operator log output doesn't change much:
I1004 07:56:43.034502       1 orc_reconciliation.go:40] Orchestrator reconciliation for cluster 'silly' started...
W1004 07:56:44.192475       1 orc_reconciliation.go:158] [getRecoveriesToAck]: Cluster is not ready for ack.
I1004 07:56:45.377957       1 utils.go:172] Endpoints for service 'silly-mysql-master' were up-to-date.
E1004 07:56:45.384547       1 controller.go:225] mysqlclusterController controller: Re-queuing item "mysql-operator/silly" due to error processing: sync error: Operation cannot be fulfilled on mysqlclusters.mysql.presslabs.org "silly": the object has been modified; please apply your changes to the latest version and try again

The last line is repeating in the log all over again.
MysqlCluster manifest:
apiVersion: v1
kind: Secret
metadata:
  name: silly
  namespace: mysql-operator
type: Opaque
data:
  ROOT_PASSWORD: dGVzdHRlc3Q=

---

apiVersion: mysql.presslabs.org/v1alpha1
kind: MysqlCluster
metadata:
  name: silly
  namespace: mysql-operator
spec:
  replicas: 3
  secretName: silly

Hi @jkrcma, The operator registers the cluster when the first node is ready, then the orchestrator is in charge of discovering new nodes. Please give me more details about your cluster, more logs of the operator and cluster state. We can chat further on the gitter channel to debug this issue together.
In the next version of mysql-operator, which is in progress right now, the operator will register/unregister nodes into orchestrator, this will solve this issue.

We figure it out what is the cause for this issue. The operator does not update the status for MysqlCluster so nodes will not be registered in orchestrator. This is because status is declared as subresource and in k8s >= 1.11 is not updated.
The next version of the operator, that is in progress, will work on 1.11. Fixed by #102


EACCES: permission denied, mkdir in local node path

persisted volume on local node path /home/rancher/wp only creates directory wp but can not write to it.

Is it caused by security context like https://github.com/rancher/local-path-provisioner/issues/7 ?

Can you share the yaml file to reproduce it?

sorry it was fixed and needs nothing to be done. just had to chown root chmod 775 the directory.


POST Request interception hanging

I'm trying to process a basic forms authentication flow with some interception and getting the navigation timeout within the POST request. It seems to be quite strange as it is rather basic functionality and e.g. #616 suggests that this had worked. Anyway, see reproduced issue targeting www.webscantest.com.
The code below works, but uncommenting two commented lines cause it to stop working.
        public static async Task Main(string[] args)
        {
            await new BrowserFetcher().DownloadAsync(BrowserFetcher.DefaultRevision);
            using (var browser = await Puppeteer.LaunchAsync(new LaunchOptions {Headless = true}))
            {
                using (var page = await browser.NewPageAsync())
                {
//                    await page.SetRequestInterceptionAsync(true);
                    page.Request += async (sender, e) =>
                    {
                        await Console.Out.WriteLineAsync($"{e.Request.Method} {e.Request.Url} {e.Request.PostData}");
//                        await e.Request.ContinueAsync();
                    };

                    page.Response += async (sender, e) =>
                    {
                        await Console.Out.WriteLineAsync($"{e.Response.Status:d} {e.Response.StatusText}");
                    };

                    await page.GoToAsync("http://www.webscantest.com/login.php");
                    await page.GetContentAsync();
                    await page.TypeAsync("input[name=login]", "admin");
                    await page.TypeAsync("input[name=passwd]", "admin");
                    await Task.WhenAll(page.ClickAsync("input[type=submit]"), page.WaitForNavigationAsync());

                    await browser.CloseAsync();
                    Console.WriteLine("Done!");
                }
            }
        }

Expected behavior:
Ability to continue the POST request.
Actual behavior:
NavigationTimeout
Versions

Which version of PuppeteerSharp are you using?
1.9
Which .NET runtime and version are you targeting? E.g. .NET framework 4.6.1 or .NET Core 2.0.
.NET Core 2.1
.NET framework 4.7.1

@arabcewicz thank you for the detailed information. I'll take a look at it.

@arabcewicz I was able to reproduce it. The problem is that it also hangs on Puppeteer. There is an issue with the DevTools.
Puppeteer Issue https://github.com/GoogleChrome/puppeteer/issues/3471

@kblok Thanks, good to know that has been recognized already. So, we have to wait...

Closed due to inactivity. Feel free to reopen it if needed.

@kblok , I think this should stay open until the issue is not resolved. Moreover, I'd suggest putting some warning banner on the project page informing that such basic stuff doesn't work :(.
I'm working on some poc project that needs this. Is there a way to get to know when it will be resolved? Should I look for some other tool? (it's actually the last thing I would like to do...)
Unfortunately, I can't reopen the ticket.

For Puppeteer-Sharp to determine what's "basic" is quite simple. We are a port of Puppeteer. If it's on Puppeteer we should support it. If not, we could implement it, but that's not "basic".
Even more, it's something missing in Chromium so there is nothing to do about that.
As we have an open issue on Puppeteer (https://github.com/GoogleChrome/puppeteer/issues/3471), there is no need to have one issue here, because we will implement it as soon as they find a solution.
I know it's not fun being unable to do something and not getting a solution. But there isn't much we can do about this.


Snippets don't auto indent

Inserting a snippet on anything but col 1 doesn't preserve indentation beyond first line.
Example:
Snippet code:

From col 1:

From col 5:

I can confirm from a quick test in VSCode that it should be auto-indenting the subsequent lines.
The code looks to take the first lines character position into consideration, but then after that just fits the rest of the snippet in with no formatting.
https://github.com/onivim/oni/blob/38566d92489ebce97d82a899a1fda8467eff786c/browser/src/Services/Snippets/SnippetSession.ts#L146-L158
We'd instead want to use the start of the prefix position as the starting point for the rest of the lines.

@CrossR we use a package called detectIndent somewhere else for setting lines i think in the BufferManager file so perhaps that could be reused or maybe it isn't actually working correctly as it's I think used in the setLines function and maybe should already be indenting insertions correctly?

Added a PR for this now (#2394 ), using detect-indent.
It looks like setLines is just a wrapper for the neovim API function to set lines, it doesn't apply any other logic which is why no indents were being applied. The first lines indent is maintained by using the prefix, and the rest are left untouched.


rest endpoint to expose shared feed entries + sharing between members of a commafeed instance

Hello Athou,
I'm implementing an rest endpoint to access a "shared" category publicly(without credentials) for each user but I wonder how I could permit subscription between members in the same commafeed instance (i know that you make sure that it is not possible for now). You might have already think about that and so, i'd like to know your opinion.

Would you accept a pull request for the "shared entries" public endpoint?
Would you accept a pull request for a feature which would permit subscriptions on this endpoint between users of the same instance of commafeed?

Explicit or automatic subscription inside a same instance? Should each user subscribe to other members to see their shared entries or should a pre-defined category expose members' shared feeds entries?
Real/virtual subscriptions? Should subscriptions to others members of the instance need to be created the normal way?Or would it be prefereable to create a different kind of subscriptions (ex : based on view)?

As those features might need substantial work on my part and as i'd like them to be part of your future releases, I'd really appreciate your opinion on those before going to far. For now, i'm mainly interested in how you want commafeed to evolve but i might come back with more technical questions.
Thank you :)

Why not showing your changes right now even before making a pull request?


⚠ ClamAV has degraded performance

In 5e154cd, ClamAV (https://spamassassin.apache.org/) experienced degraded performance:

HTTP code: 200
Response time: 69 ms

Resolved: ClamAV performance has improved in 312e847 after 543 days, 17 hours, 10 minutes.


Taverna retired in 02/2020

Just wanted to let you know that Taverna has been retired as Apache Incubator Project since February 2020 by community voting.

@b2m Thanks for the heads up!

Unfortunate, it was a great project but understandable at the same time. Thanks for letting us know.


Ruff ignore format

Improved and supplemented version (I hope it's not boring yet)
only 3 rules:

Pyflakes (F) -
pycodestyle-warnings (W)
pyupgrade (UP)

Gah! ruff autofixes are proliferating faster than Tribbles on the OG NCC-1701 class USS Enterprise.  Explicit is better than implicit. Thus, I strongly prefer class MuhClass(object): to class MuhClass:. The latter reads super-weird to me. You're always subclassing something – so just come out with it already. There's no tangible benefit to "hiding" the fact that you're subclassing the root object superclass.
The only ruff autofixes I mildly like here are the ones removing extraneous trailing whitespace. I totally agree with those, actually. Extraneous trailing whitespace irrationally bugs me so much. I even have a vim plugin that explicitly highlights extraneous trailing whitespace in red across the current buffer and tags all such whitespace as erroneous in the quickfix list. Annnnnnnnnyway. Boring vim stuff. :smile:
If you'd like to reduce this to just the following, I'll probably merge that immediately without punching myself in the head too much:

Remove extraneous trailing whitespace.
Reorder imports lexicographically (i.e., alphabetically).

B-b-but... Is There Any Point, Really?
Honestly, all of this is bit more work than is really desirable. Most whitespace is already clean; likewise, most imports are already ordered lexicographically. It's not like any of these things are actual errors, anyway. I disagree with ruff so strongly on most things that there's not much left for ruff to do at the end of the day.

ruff: you are a bad doggo.

@leycec You're not auto deleting trailing whitespace? This is the vim snippet I stole from the internet, and it hasn't let me down. On file save, strips all trailing whitespace. Never think about it again.
" Strip trailing whitespace from file!
fun! <SID>StripTrailingWhiteSpaces()
  let l = line(".")
  let c = col(".")
  %s/\s\+$//e
  call cursor(l, c)
endfun

augroup vimrc
autocmd!
autocmd BufWritePre * :call <SID>StripTrailingWhiteSpaces()
augroup END

(There's also alternative ways)


Unwind removing use of streamingCellThreshold to downgrade excelTable

Discussed w/Lee yesterday - will revisit how to make tree grid exports more performant. In the meantime, we do need the streamingCellThreshold to downgrade large exports (realized the limitation in performance was not style-related, though the change to cache groupedColors prevents the apache error of exceeding 64k styles)

Just restoring the status quo API, as we are in the midst of several investigations about performance.
Taking the more efficient handling of styling, which is a win.


--botname doesn't seem to delete messages [weird output of # of messages to be deleted]

Hello! I'm running slack-cleaner v.0.4.0 on Python 3.6, via PowerShell. I'm trying to delete one particular bot's messages from one particular channel:
PS C:\1\insta_env> slack-cleaner --token xxxx --message --channel internal-updates --botname notion
Running slack-cleaner v0.4.0
Deleting messages from channel internal-updates
0 message(s) will be cleaned.
Now you can re-run this program with `--perform` to actually perform the task.
I guess it's a character issue, but still, even after initiating the delete via --perform with this output,
0 message(s) cleaned.
All the messages remain. None are deleted. Have I missed a crucial step here? This is the bot whom I'm trying to wipe.
Here's what the app looks like:

re ansi escape codes in powershell -> #17

there is a new option --info to get a list of all users and so on. Please, check if you have the right name.

maybe related to #36 such that not --botname implies --bot

stale


Azure Storage Account connections via proxy fail in an Azure environment.

Do you already have an issue opened with F5 support?
No.
Description
Storage Account connections via proxy fail in an Azure environment.
Environment information

Cloud Failover Extension Version:  2.1.0
BIG-IP version: <IP_ADDRESS>
Cloud provider: Azure

Severity Level
Severity: <3>
Log Detailed

It has already been confirmed that connections without proxy settings (direct) work correctly.

Failure logs when using proxies.
/var/log/restnoded/restnoded.log

Thu, 02 May 2024 20:29:50 GMT - finest: socket 425 opened
Thu, 02 May 2024 20:29:50 GMT - fine: [f5-cloud-failover] HTTP Request - POST /declare
Thu, 02 May 2024 20:29:50 GMT - fine: [f5-cloud-failover] Successfully validated declaration
Thu, 02 May 2024 20:29:50 GMT - info: [f5-cloud-failover] Global logLevel set to 'silly'
Thu, 02 May 2024 20:29:50 GMT - finest: [f5-cloud-failover] Modifying existing data group f5-cloud-failover-state with body {"name":"f5-cloud-failover-state","type":"string","records":[{"name":"state","data":"eyJjb25maWciOnsiY2xhc3MiOiJDbG91ZF9GYWlsb3ZlciIsImVudmlyb25tZW50IjoiYXp1cmUiLCJjb250cm9scyI6eyJjbGFzcyI6IkNvbnRyb2xzIiwibG9nTGV2ZWwiOiJzaWxseSJ9LCJyZXRyeUZhaWxvdmVyIjp7ImVuYWJsZWQiOnRydWUsImludGVydmFsIjoyfSwiZXh0ZXJuYWxTdG9yYWdlIjp7InNjb3BpbmdUYWdzIjp7ImY1X2Nsb3VkX2ZhaWxvdmVyX2xhYmVsIjoiQklHSVBfRkFJTE9WRVJfU1RPUkFHRSJ9fSwiZmFpbG92ZXJBZGRyZXNzZXMiOnsic2NvcGluZ1RhZ3MiOnsiZjVfY2xvdWRfZmFpbG92ZXJfbGFiZWwiOiJCSUdJUF9GQUlMT1ZFUl9BRERSRVNTMiJ9LCJyZXF1aXJlU2NvcGluZ1RhZ3MiOmZhbHNlfSwic2NoZW1hVmVyc2lvbiI6IjIuMS4wIn19"}]}
Thu, 02 May 2024 20:29:51 GMT - finest: socket 426 opened
Thu, 02 May 2024 20:29:54 GMT - info: [f5-cloud-failover] Successfully wrote Failover trigger scripts to filesystem
Thu, 02 May 2024 20:29:54 GMT - finest: [f5-cloud-failover] Device initialization complete
Thu, 02 May 2024 20:29:55 GMT - finest: [f5-cloud-failover] Fetched proxy settings:
Thu, 02 May 2024 20:29:55 GMT - finest: [f5-cloud-failover] {"protocol":"http","host":"x.x.x.x","port":"3128","username":"","password":"********"}
Thu, 02 May 2024 20:29:55 GMT - finest: [f5-cloud-failover] Subscriptions:  {"0":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"}
Thu, 02 May 2024 20:29:55 GMT - finest: [f5-cloud-failover] Listing Storage Accounts
Thu, 02 May 2024 20:29:56 GMT - finest: [f5-cloud-failover] Storage Account Information:  f5cfe12345
Thu, 02 May 2024 20:29:56 GMT - finest: socket 426 closed
Thu, 02 May 2024 20:29:57 GMT - finest: [f5-cloud-failover] Status: "HTTP request failed: 502 \"<!DOCTYPE html PUBLIC \\\"-//W3C//DTD HTML 4.01//EN\\\" \\\"http://www.w3.org/TR/html4/strict.dtd\\\">\\n\\n<meta type=\\\"copyright\\\" content=\\\"Copyright (C) 1996-2020 The Squid Software Foundation and contributors\\\">\\n<meta http-equiv=\\\"Content-Type\\\" content=\\\"text/html; charset=utf-8\\\">\\n<title>ERROR: The requested URL could not be retrieved</title>\\n\\n\\n
\\n
ERROR
\\n
The requested URL could not be retrieved
\\n
\\n
\\n\\n
\\n
The following error was encountered while trying to retrieve the URL: https://f5cfe12345.blob.core.windows.net/?
\\n\\n
\\n
Read Error
\\n
\\n\\n
The system returned: [No Error]
\\n\\n
An error condition occurred while reading data from the network. Please retry your request.
\\n\\n
Your cache administrator is <a href=\\\"mailto:webmaster?subject=CacheErrorInfo%20-%20ERR_READ_ERROR&body=CacheHost%3A%20ip-10-0-1-12%0D%0AErrPage%3A%20ERR_READ_ERROR%0D%0AErr%3A%20%5Bnone%5D%0D%0ATimeStamp%3A%20Thu,%2002%20May%202024%2020%3A29%3A57%20GMT%0D%0A%0D%0AClientIP%3A%2<IP_ADDRESS>%0D%0AServerIP%3A%20f5cfe12345.blob.core.windows.net%0D%0A%0D%0AHTTP%20Request%3A%0D%0AGET%20%2F%3Fcomp%3Dlist%20HTTP%2F1.1%0AAccept%3A%20application%2Fjson,%20text%2Fplain,%20*%2F*%0D%0AAuthorization%3A%20Bearer%20XXXXXXXXXXXXXXXXXXXXXXXXXXXX%0D%0Ax-ms-version%3A%202017-11-09%0D%0AUser-Agent%3A%20axios%2F0.21.4%0D%0AConnection%3A%20keep-alive%0D%0AHost%3A%20f5cfe12345.blob.core.windows.net%0D%0A%0D%0A%0D%0A\\\">webmaster.
\\n
\\n
\\n\\n
\\n
\\n
Generated Thu, 02 May 2024 20:29:57 GMT by ip-10-0-1-12 (squid/5.7)
\\n\\n
\\n\\n\"" Retries left: 60

Access Loged at Proxy Server(Squid).

"2024/05/02 20:01:50.724" <IP_ADDRESS> 1825 <IP_ADDRESS> 2474 443 application/json "GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Storage/storageAccounts?api-version=2023-01-01 HTTP/1.1" 200 "-" "axios/0.21.4" "TCP_REFRESH_MODIFIED:HIER_DIRECT"
"2024/05/02 20:01:51.537" <IP_ADDRESS> 1687 <IP_ADDRESS> 5535 443 text/html "GET https://f5cfe12345.blob.core.windows.net/?comp=list HTTP/1.1" 502 "-" "axios/0.21.4" "TCP_MISS:HIER_DIRECT"
"2024/05/02 20:01:53.345" <IP_ADDRESS> 1687 <IP_ADDRESS> 5535 443 text/html "GET https://f5cfe12345.blob.core.windows.net/?comp=list HTTP/1.1" 502 "-" "axios/0.21.4" "TCP_MISS:HIER_DIRECT"
"2024/05/02 20:01:55.148" <IP_ADDRESS> 1687 <IP_ADDRESS> 5535 443 text/html "GET https://f5cfe12345.blob.core.windows.net/?comp=list HTTP/1.1" 502 "-" "axios/0.21.4" "TCP_MISS:HIER_DIRECT"
"2024/05/02 20:01:56.950" <IP_ADDRESS> 1687 <IP_ADDRESS> 5535 443 text/html "GET https://f5cfe12345.blob.core.windows.net/?comp=list HTTP/1.1" 502 "-" "axios/0.21.4" "TCP_MISS:HIER_DIRECT"
"2024/05/02 20:01:58.756" <IP_ADDRESS> 1687 <IP_ADDRESS> 5535 443 text/html "GET https://f5cfe12345.blob.core.windows.net/?comp=list HTTP/1.1" 502 "-" "axios/0.21.4" "TCP_MISS:HIER_DIRECT"

Hi @yamashin55, can you provide the following:

Your squid.conf
RBAC roles assigned to identity used by the BIG-IP instances

It looks like you have a password set in the proxy db variables, but no username. Is that correct or is it redacted?
Also, how did you deploy CFE? Did you use the ARM templates?
Thanks

@mikeshimkus, Thank you for replay.

Below is /etc/squid/squid.conf

ubuntu@ip-10-0-1-12:~$ grep -v '^\s*#' /etc/squid/squid.conf |grep -v '^\s*$'
acl localnet src <IP_ADDRESS>-<IP_ADDRESS>  # RFC 1122 "this" network (LAN)
acl localnet src <IP_ADDRESS>/8             # RFC 1918 local private network (LAN)
acl localnet src <IP_ADDRESS>/10          # RFC 6598 shared address space (CGN)
acl localnet src <IP_ADDRESS>/16         # RFC 3927 link-local (directly plugged) machines
acl localnet src <IP_ADDRESS>/12          # RFC 1918 local private network (LAN)
acl localnet src <IP_ADDRESS>/16         # RFC 1918 local private network (LAN)
acl localnet src fc00::/7               # RFC 4193 local private network range
acl localnet src fe80::/10              # RFC 4291 link-local (directly plugged) machines
acl allallow src <IP_ADDRESS>/0
acl CONNECT method CONNECT
acl SSL_ports port 443
acl Safe_ports port 80          # http
acl Safe_ports port 21          # ftp
acl Safe_ports port 443         # https
acl Safe_ports port 70          # gopher
acl Safe_ports port 210         # wais
acl Safe_ports port 1025-65535  # unregistered ports
acl Safe_ports port 280         # http-mgmt
acl Safe_ports port 488         # gss-http
acl Safe_ports port 591         # filemaker
acl Safe_ports port 777         # multiling http
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access allow localhost manager
http_access deny manager
include /etc/squid/conf.d/*.conf
http_access allow localhost
http_access allow allallow
http_access deny all
http_port 3128
logformat mycombined   "%{%Y/%m/%d %H:%M:%S}tl.%03tu" %>a %>st %<a %<st %>rP %mt "%rm %>ru HTTP/%rv" %>Hs "%{Referer}>h" "%{User-Agent}>h" "%Ss:%Sh"
access_log daemon:/var/log/squid/access.log mycombined
coredump_dir /var/spool/squid
refresh_pattern ^ftp:           1440    20%     10080
refresh_pattern ^gopher:        1440    0%      1440
refresh_pattern -i (/cgi-bin/|\?) 0     0%      0
refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims
refresh_pattern \/Release(|\.gpg)$ 0 0% 0 refresh-ims
refresh_pattern \/InRelease$ 0 0% 0 refresh-ims
refresh_pattern \/(Translation-.*)(|\.bz2|\.gz|\.xz)$ 0 0% 0 refresh-ims
refresh_pattern .               0       20%     4320
ubuntu@ip-10-0-1-12:~$

Below is RBAC roles

{
    "id": "/subscriptions/XXXXXXXXXX/providers/Microsoft.Authorization/roleDefinitions/XXXXXXXXXXX",
    "properties": {
        "roleName": "F5-BIGIP-FailOver-Role",
        "description": "F5 Networks BIG-IP",
        "assignableScopes": [
            "/subscriptions/XXXXXXXXXX/resourceGroups/f5-rsg-vm",
            "/subscriptions/XXXXXXXXXX/resourceGroups/f5-rsg-vnet"
        ],
        "permissions": [
            {
                "actions": [
                    "Microsoft.Network/*/join/action",
                    "Microsoft.Network/networkInterfaces/write",
                    "Microsoft.Network/publicIPAddresses/write",
                    "Microsoft.Network/routeTables/*/read",
                    "Microsoft.Network/routeTables/*/write",
                    "Microsoft.Storage/storageAccounts/read",
                    "Microsoft.Storage/storageAccounts/blobServices/containers/read",
                    "Microsoft.Storage/storageAccounts/blobServices/containers/write",
                    "Microsoft.Authorization/*/read",
                    "Microsoft.Compute/locations/*/read",
                    "Microsoft.Compute/virtualMachines/*/read",
                    "Microsoft.Compute/virtualMachineScaleSets/*/read",
                    "Microsoft.Compute/virtualMachineScaleSets/networkInterfaces/read",
                    "Microsoft.Network/networkInterfaces/read",
                    "Microsoft.Network/publicIPAddresses/read",
                    "Microsoft.Resources/subscriptions/resourceGroups/read",
                    "Microsoft.Compute/virtualMachines/extensions/*"
                ],
                "notActions": [],
                "dataActions": [
                    "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/read",
                    "Microsoft.Storage/storageAccounts/blobServices/containers/blobs/write"
                ],
                "notDataActions": []
            }
        ]
    }
}

It looks like you have a password set in the proxy db variables, but no username. Is that correct or is it redacted?

This proxy server has no authentication.
I have not set a proxy ID and password on my BIG-IP. I have not been changed from the default.
I used fllowing command:
tmsh modify sys db proxy.host value 54.85.XX.XX
tmsh modify sys db proxy.port value 3128
tmsh save sys config

Also, how did you deploy CFE? Did you use the ARM templates?

No. I didn't use ARM templates. I deployed the BIG-IP manually.
CFE Configuration is :
{
    "class": "Cloud_Failover",
    "environment": "azure",
    "controls": {
      "class": "Controls",
      "logLevel": "silly"
    },
    "retryFailover": {
        "enabled": true,
        "interval": 2
     },
    "externalStorage":{
        "scopingTags": {
            "f5_cloud_failover_label": "BIGIP_FAILOVER_ADDRESS2"
        }
    },
    "failoverAddresses": {
        "scopingTags": {
            "f5_cloud_failover_label": "BIGIP_FAILOVER_STORAGE"
        }
    }
}

*Below are the logs when no proxy is used. (Direct access is successful.)
Thu, 02 May 2024 20:21:18 GMT - fine: [f5-cloud-failover] HTTP Request - POST /declare
Thu, 02 May 2024 20:21:18 GMT - fine: [f5-cloud-failover] Successfully validated declaration
Thu, 02 May 2024 20:21:18 GMT - info: [f5-cloud-failover] Global logLevel set to 'silly'
Thu, 02 May 2024 20:21:18 GMT - finest: [f5-cloud-failover] Creating new data group f5-cloud-failover-state with body {"name":"f5-cloud-failover-state","type":"string","records":[{"name":"state","data":"eyJjb25maWciOnsiY2xhc3MiOiJDbG91ZF9GYWlsb3ZlciIsImVudmlyb25tZW50IjoiYXp1cmUiLCJjb250cm9scyI6eyJjbGFzcyI6IkNvbnRyb2xzIiwibG9nTGV2ZWwiOiJzaWxseSJ9LCJyZXRyeUZhaWxvdmVyIjp7ImVuYWJsZWQiOnRydWUsImludGVydmFsIjoyfSwiZXh0ZXJuYWxTdG9yYWdlIjp7InNjb3BpbmdUYWdzIjp7ImY1X2Nsb3VkX2ZhaWxvdmVyX2xhYmVsIjoiQklHSVBfRkFJTE9WRVJfU1RPUkFHRSJ9fSwiZmFpbG92ZXJBZGRyZXNzZXMiOnsic2NvcGluZ1RhZ3MiOnsiZjVfY2xvdWRfZmFpbG92ZXJfbGFiZWwiOiJCSUdJUF9GQUlMT1ZFUl9BRERSRVNTMiJ9LCJyZXF1aXJlU2NvcGluZ1RhZ3MiOmZhbHNlfSwic2NoZW1hVmVyc2lvbiI6IjIuMS4wIn19"}]}
Thu, 02 May 2024 20:21:21 GMT - info: [f5-cloud-failover] Successfully wrote Failover trigger scripts to filesystem
Thu, 02 May 2024 20:21:21 GMT - finest: [f5-cloud-failover] Device initialization complete
Thu, 02 May 2024 20:21:21 GMT - finest: [f5-cloud-failover] Fetched proxy settings:
Thu, 02 May 2024 20:21:21 GMT - finest: [f5-cloud-failover] {"protocol":"http","host":"","port":"8080","username":"","password":"********"}
Thu, 02 May 2024 20:21:21 GMT - finest: [f5-cloud-failover] Subscriptions:  {"0":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"}
Thu, 02 May 2024 20:21:21 GMT - finest: [f5-cloud-failover] Listing Storage Accounts
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] Storage Account Information:  f5cfe12345
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] Container {"Name":"f5cloudfailover"} was found...
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] Container f5cloudfailover already exists, continuing...
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] Cloud Provider initialization complete
Thu, 02 May 2024 20:21:22 GMT - fine: [f5-cloud-failover] Performing failover - initialization
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] Device initialization complete
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] Fetched proxy settings:
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] {"protocol":"http","host":"","port":"8080","username":"","password":"********"}
Thu, 02 May 2024 20:21:22 GMT - fine: [f5-cloud-failover] config:
Thu, 02 May 2024 20:21:22 GMT - fine: [f5-cloud-failover] {"class":"Cloud_Failover","environment":"azure","controls":{"class":"Controls","logLevel":"silly"},"retryFailover":{"enabled":true,"interval":2},"externalStorage":{"scopingTags":{"f5_cloud_failover_label":"BIGIP_FAILOVER_STORAGE"}},"failoverAddresses":{"scopingTags":{"f5_cloud_failover_label":"BIGIP_FAILOVER_ADDRESS2"},"requireScopingTags":false},"schemaVersion":"2.1.0"}
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] proxySettings:
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] {"protocol":"http","host":"","port":"8080","username":"","password":"********"}
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] Telemetry submitted successfully
Thu, 02 May 2024 20:21:22 GMT - finest: [f5-cloud-failover] Telemetry payload: {"customerId":"d3563489-ec7f-4117-830b-b5ed2ef02816","failover":{"event":false,"success":true},"product":{"version":"2.1.0","locale":"en-US","installDate":"2024-05-02T20:21:22.065Z","installationId":"","environment":"azure","region":"japaneast"},"featureFlags":{"ipFailover":false,"routeFailover":false},"operation":{"clientRequestId":"e7855baa-7140-41e1-ab6f-f4903a70cf43","userAgent":"f5-cloud-failover/2.1.0","result":"SUCCESS","resultSummary":"Configuration Successful"}}
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Device initialization complete
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Fetched proxy settings:
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] {"protocol":"http","host":"","port":"8080","username":"","password":"********"}
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Subscriptions:  {"0":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"}
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Listing Storage Accounts
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Storage Account Information:  f5cfe12345
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Container {"Name":"f5cloudfailover"} was found...
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Container f5cloudfailover already exists, continuing...
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Cloud Provider initialization complete
Thu, 02 May 2024 20:21:23 GMT - finest: [f5-cloud-failover] Failover initialization complete

I thought that CONNECT Method is normally used for HTTPS communication in case of Explicit Proxy.
CONNECT abc.com

Use proxy Web access from BigiP.

[admin@10-0-1-15:Standby:In Sync] ~ # curl --proxy http://<IP_ADDRESS>:3128 https://httpbin.org/ip
{
  "origin": "<IP_ADDRESS>"
}
[admin@10-0-1-15:Standby:In Sync] ~ #

Squid Access Logs

"2024/05/03 01:34:20.325" <IP_ADDRESS> 987 <IP_ADDRESS> 5838 443 - "CONNECT httpbin.org:443 HTTP/1.1" 200 "-" "curl/7.47.1" "TCP_TUNNEL:HIER_DIRECT"

However, when accessing from CFE, it appears that the GET method is being used.
GET https://management.azure.com/...
GET https://f5cfe12345.blob.core.windows.net/?comp....
"2024/05/02 20:01:50.724" <IP_ADDRESS> 1825 <IP_ADDRESS> 2474 443 application/json "GET https://management.azure.com/subscriptions/XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX/providers/Microsoft.Storage/storageAccounts?api-version=2023-01-01 HTTP/1.1" 200 "-" "axios/0.21.4" "TCP_REFRESH_MODIFIED:HIER_DIRECT"
"2024/05/02 20:01:51.537" <IP_ADDRESS> 1687 <IP_ADDRESS> 5535 443 text/html "GET https://f5cfe12345.blob.core.windows.net/?comp=list HTTP/1.1" 502 "-" "axios/0.21.4" "TCP_MISS:HIER_DIRECT"

However, the access log of Azure Resource Manager(management.azure.com) shows a normal 200 response.
The Storage Account (f5cfe12345.blob.core.windows.net) access log is a 502 response.

@yamashin55 I created internal issue EC-510 for this.
Can you also share the JSON config of the storage account (click on the account overview in the Azure portal and then the JSON View link). Do you have any ACLs or NSG rules applied to the storage account that would deny traffic from the proxy IP/vNET but allow it from the BIG-IP instance IPs?
Regarding the method, the curl command is using the proxy db settings directly, while CFE passes them to the Axios client as proxy options. Regardless of that difference, this was successfully tested with identical db var and squid configuration, so I suspect something blocking access from the proxy server specifically.

@mikeshimkus,
Thank you for comment about the difference of method.

JSON config of the storage account.

{
    "sku": {
        "name": "Standard_LRS",
        "tier": "Standard"
    },
    "kind": "StorageV2",
    "id": "/subscriptions/d3563489-ec7f-4117-830b-xxxxxxxxxxxx/resourceGroups/f5-rsg-vm/providers/Microsoft.Storage/storageAccounts/f5cfe12345",
    "name": "f5cfe12345",
    "type": "Microsoft.Storage/storageAccounts",
    "location": "japaneast",
    "tags": {
        "f5_cloud_failover_label": "BIGIP_FAILOVER_STORAGE"
    },
    "properties": {
        "dnsEndpointType": "Standard",
        "defaultToOAuthAuthentication": false,
        "publicNetworkAccess": "Enabled",
        "keyCreationTime": {
            "key1": "2024-05-02T18:56:52.9341396Z",
            "key2": "2024-05-02T18:56:52.9341396Z"
        },
        "allowCrossTenantReplication": false,
        "privateEndpointConnections": [],
        "minimumTlsVersion": "TLS1_2",
        "allowBlobPublicAccess": false,
        "allowSharedKeyAccess": true,
        "networkAcls": {
            "resourceAccessRules": [],
            "bypass": "AzureServices",
            "virtualNetworkRules": [
                {
                    "id": "/subscriptions/d3563489-ec7f-4117-830b-xxxxxxxxxxxx/resourceGroups/f5-rsg-vnet/providers/Microsoft.Network/virtualNetworks/<IP_ADDRESS>_16/subnets/<IP_ADDRESS>_24",
                    "action": "Allow",
                    "state": "Succeeded"
                }
            ],
            "ipRules": [
                {
                    "value": "<IP_ADDRESS>",
                    "action": "Allow"
                }
            ],
            "defaultAction": "Deny"
        },
        "supportsHttpsTrafficOnly": true,
        "encryption": {
            "requireInfrastructureEncryption": false,
            "services": {
                "file": {
                    "keyType": "Account",
                    "enabled": true,
                    "lastEnabledTime": "2024-05-02T18:56:52.9497731Z"
                },
                "blob": {
                    "keyType": "Account",
                    "enabled": true,
                    "lastEnabledTime": "2024-05-02T18:56:52.9497731Z"
                }
            },
            "keySource": "Microsoft.Storage"
        },
        "accessTier": "Hot",
        "provisioningState": "Succeeded",
        "creationTime": "2024-05-02T18:56:52.8560186Z",
        "primaryEndpoints": {
            "dfs": "https://f5cfe12345.dfs.core.windows.net/",
            "web": "https://f5cfe12345.z11.web.core.windows.net/",
            "blob": "https://f5cfe12345.blob.core.windows.net/",
            "queue": "https://f5cfe12345.queue.core.windows.net/",
            "table": "https://f5cfe12345.table.core.windows.net/",
            "file": "https://f5cfe12345.file.core.windows.net/"
        },
        "primaryLocation": "japaneast",
        "statusOfPrimary": "available"
    }
}

Do you have any ACLs or NSG rules applied to the storage account that would deny traffic from the proxy IP/vNET but allow it from the BIG-IP instance IPs?

I changed the "Public network access" setting to "Enabled from selected virtual networks and IP addresses".

Azure portal -> StorageAccout -> Networking -> Firewalls and virtual networks, for Selected networks.

And at Firewall rules section, I added the global address of squid proxy server.
Other than that, no specific ACLs have been changed.

Thanks. It might be helpful to configure the storage account like the ARM templates do just for testing, for example: https://github.com/F5Networks/f5-azure-arm-templates-v2/blob/9efd07d357ef01e35e5db0a95a7ac6debca15a57/examples/modules/bigip-standalone/bigip.json#L424
I have placed this issue in the queue and will update here with the outcome.

I tested using StorageAccount of ARM standalone BigIP Template with the same result. This 502 error symptom did not improve.
I found a similar problem was raised with axios. issues-5078
I have no programming knowledge and don't know...

@yamashin55 We just released https://github.com/F5Networks/f5-cloud-failover-extension/releases/tag/v2.1.1 with a fix for this issue.

@mikeshimkus  It's working fine with the new version(v2.1.1)! Thank you for your help!
I couldn't find the rpm package file, so I replaced the following two files with v2.1.1 and checked the reproducibility in my lab env. It works fine.

src/nodejs/providers/azure/cloud.js
src/nodejs/util.js

Could you upload the rpm file when you have time. I think I have solved the problem.


create_tabix_file output mixed up.

Hi,
I recently ran the create_tabix_file function on a dataframe that looks like:

This is a modified version of the megalodon per_read_modified_base_calls.tsv file (I added the motif column to satisfy requirements of the create_tabix_file function for recognizing Megalodon output). I was able to run the create_tabix_file, however I'm running into an issue where the function puts the read_id under the chromosome column as well as shows nothing for the strand, statistic, or read_name columns. See below:

This does not resemble the file I'd expect from your example. I understand that you are starting from Nanopolish while I'm starting from Megalodon but that shouldn't cause this effect. Any help would be greatly appreciated! Thanks!
sessionInfo():
R version 4.1.1 (2021-08-10)
Platform: x86_64-apple-darwin17.0 (64-bit)
Running under: macOS Catalina 10.15.7
Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/4.1/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] parallel  stats4    stats     graphics  grDevices utils     datasets  methods   base
other attached packages:
[1] Homo.sapiens_1.3.1                      TxDb.Hsapiens.UCSC.hg19.knownGene_3.2.2
[3] org.Hs.eg.db_3.13.0                     GO.db_3.13.0
[5] OrganismDbi_1.34.0                      GenomicFeatures_1.44.2
[7] GenomicRanges_1.44.0                    GenomeInfoDb_1.28.4
[9] AnnotationDbi_1.54.1                    IRanges_2.26.0
[11] S4Vectors_0.30.0                        Biobase_2.52.0
[13] BiocGenerics_0.38.0                     NanoMethViz_1.2.0
[15] ggplot2_3.3.5
loaded via a namespace (and not attached):
[1] colorspace_2.0-2            bsseq_1.28.0                rjson_0.2.20                ellipsis_0.3.2
[5] XVector_0.32.0              fs_1.5.0                    rstudioapi_0.13             scico_1.2.0
[9] bit64_4.0.5                 fansi_0.5.0                 xml2_1.3.2                  R.methodsS3_1.8.1
[13] sparseMatrixStats_1.4.2     cachem_1.0.6                knitr_1.34                  Rsamtools_2.8.0
[17] dbplyr_2.1.1                png_0.1-7                   R.oo_1.24.0                 graph_1.70.0
[21] HDF5Array_1.20.0            BiocManager_1.30.16         readr_2.0.1                 compiler_4.1.1
[25] httr_1.4.2                  assertthat_0.2.1            Matrix_1.3-4                fastmap_1.1.0
[29] limma_3.48.3                cli_3.0.1                   BiocSingular_1.8.1          htmltools_0.5.2
[33] prettyunits_1.1.1           tools_4.1.1                 rsvd_1.0.5                  gtable_0.3.0
[37] glue_1.4.2                  GenomeInfoDbData_1.2.6      dplyr_1.0.7                 rappdirs_0.3.3
[41] Rcpp_1.0.7                  vctrs_0.3.8                 Biostrings_2.60.2           rhdf5filters_1.4.0
[45] rtracklayer_1.52.1          DelayedMatrixStats_1.14.3   xfun_0.25                   stringr_1.4.0
[49] beachmat_2.8.1              lifecycle_1.0.0             irlba_2.3.3                 restfulr_0.0.13
[53] gtools_3.9.2                XML_3.99-0.7                zlibbioc_1.38.0             scales_1.1.1
[57] vroom_1.5.4                 BSgenome_1.60.0             hms_1.1.0                   MatrixGenerics_1.4.3
[61] RBGL_1.68.0                 SummarizedExperiment_1.22.0 rhdf5_2.36.0                curl_4.3.2
[65] yaml_2.2.1                  memoise_2.0.0               biomaRt_2.48.3              stringi_1.7.4
[69] RSQLite_2.2.8               BiocIO_1.2.0                ScaledMatrix_1.0.0          permute_0.9-5
[73] filelock_1.0.2              BiocParallel_1.26.2         cpp11_0.3.1                 rlang_0.4.11
[77] pkgconfig_2.0.3             matrixStats_0.60.1          bitops_1.0-7                evaluate_0.14
[81] lattice_0.20-44             purrr_0.3.4                 Rhdf5lib_1.14.2             GenomicAlignments_1.28.0
[85] patchwork_1.1.1             bit_4.0.4                   tidyselect_1.1.1            magrittr_2.0.1
[89] R6_2.5.1                    generics_0.1.0              DelayedArray_0.18.0         DBI_1.1.1
[93] pillar_1.6.2                withr_2.4.2                 KEGGREST_1.32.0             RCurl_1.98-1.4
[97] tibble_3.1.4                crayon_1.4.1                utf8_1.2.2                  BiocFileCache_2.0.0
[101] tzdb_0.1.2                  rmarkdown_2.10              progress_1.2.2              locfit_1.5-9.4
[105] grid_4.1.1                  data.table_1.14.0           blob_1.2.2                  forcats_0.5.1
[109] digest_0.6.27               tidyr_1.1.3                 R.utils_2.10.1              munsell_0.5.0

Thanks for the bug report, could you try to install the developmental version using the following?
remotes::install_github("shians/NanoMethViz")

I've updated megalodon importing a few times since the release version.

Hi Shians, unfortunately it appears that the problem still persists. The position has shifted to the strand column and there is only an asterisk in the position column. The statistic and read_name columns are still empty. I did get a warning though that I need to look further into. Warning and table output shown below:
[E::get_intv] Failed to parse TBX_GENERIC, was wrong -p [type] used? The offending line was: "Examp_data	fff8c78c-60c4-46ff-b807-9654b3c5126d		*		169899"

Also I'm not sure if it matters but I'm using a per_read_modified_base_calls file from a Megalodon run I ended early. I'm going to try using one that has completely gone through the whole Megalodon process. I'll keep you updated.

Hi Shians, so it appears that the eighth column I added to satisfy the earlier version's column requirements for Megalodon data ended up causing the new issue in the development version. After I took away that eighth column (the motif column). Everything worked great! Sorry for all the trouble and thanks for your time.


For Localhost Enterprise Deployments, store the in-cluster pachd hostname as issuer instead of 'localhost'

The Change:
To make the identity-server accessible + usable for pods other than pachd that are deployed in the same k8s cluster (in this case Dash), we must set the Issuer (a.k.a Identity-Server / Dex) as having the in-cluster URL so that the other in-cluster clients can run through the OIDC flows (Client establishment + Token endpoint).
Since the browser gets redirected to the auth callback endpoints, we overwrite those endpoints' Host to be localhost:30658
Dependencies:

For Minikube deployments, one may need to update the networking config so that pods can connect to themeselves via service,
i.e. pachd pod can reach itself by using its service DNS host as pachd:1658.
with:
minikube ssh -- sudo ip link set docker0 promisc on

Dex's Library needs to be updated so that it doesn't enforce that the it's Callback URL (localhost:30650/...) doesn't equal its Issuer URL: (pachd:1658/...)

@acohen4 can we add the REACT_APP_RUNTIME_ISSUER_URI to the values.schema.json? We don't need to set it for local deployments, but hub generates some code based on that file to set the console config.


Select inside table cell overlaps TableHead component inside of Table component

If you place a dropdown select component inside of a TableCell as part of a table, and create the Table component with a stickyHeader, when scrolling through the table, the Select component will overlap the the column title in the corresponding TableHeader TableCell

[ ] The issue is present in the latest release.
[ ] I have searched the issues of this repository and believe that this is not a duplicate.

Current Behavior 😯

Expected Behavior 🤔

Should not over lap.
Steps to Reproduce 🕹

Create Table.
Create column titles.
Create table body.
in one of the table cells place a select component with menu items.
set stickyHeader on Table component.
set height of the table and create an autoflow so you can scroll without losing the header.
see that the select component over laps like so:

Steps:

Context 🔦

Your Environment 🌎

Tech
Version

Material-UI
v4.?.?

React

Browser

TypeScript

etc.

@becherers7 Duplicate of #17330


refactor: get git info

将 git 相关处理放入 project-utils
git 返回 group/project 等信息
重构 git 信息获取的实现

Codecov Report

Merging #765 (93f15de) into release/0.17.0 (95b82d5) will not change coverage.
The diff coverage is n/a.

@@               Coverage Diff               @@
##           release/0.17.0     #765   +/-   ##
===============================================
  Coverage           57.14%   57.14%
===============================================
  Files                   1        1
  Lines                  70       70
  Branches               18       18
===============================================
  Hits                   40       40
  Misses                 30       30

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 95b82d5...93f15de. Read the comment docs.


add secs to hold block field

Add simple support for read-only units on number controls.

@bgoldowsky I added you to this one, because Joe is out for a few days. If things go well I'll have a bigger PR for you later today, so this is kind of a warm up. :)
But if you don't have time since you are leaving tomorrow, let me know.


Workaround tox gevent install bug

Our Jenkins builds are failing with the following error.  This workaround should fix it.
Command /var/lib/jenkins/workspace/unit-test/.tox/py27/bin/python2.7 -c "import setuptools, tokenize;__file__='/var/lib/jenkins/workspace/unit-test/.tox/py27/build/gevent/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-8oXIho-record/install-record.txt --single-version-externally-managed --compile --install-headers /var/lib/jenkins/workspace/unit-test/.tox/py27/include/site/python2.7 failed with error code 1 in /var/lib/jenkins/workspace/unit-test/.tox/py27/build/gevent Traceback (most recent call last):   File ".tox/py27/bin/pip", line 11, in <module>     sys.exit(main())   File "/var/lib/jenkins/workspace/unit-test/.tox/py27/local/lib/python2.7/site-packages/pip/__init__.py", line 185, in main     return command.main(cmd_args)   File "/var/lib/jenkins/workspace/unit-test/.tox/py27/local/lib/python2.7/site-packages/pip/basecommand.py", line 161, in main     text = '\n'.join(complete_log) UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 33: ordinal not in range(128)

As it turns out our Jenkins server was running out of memory; so there's no immediate requirement for this change.


I want to see the transfer identifier

User Story
As Chris I want to see the transfer identifier so that I can identify the transfer of my partner.
Acceptance criteria
Tasks

Closing, it is already possible to identify transfers.


fix: Remove all client side validation for OLM, allow nonspecific lif…

Removes client side validation by refactoring LifecycleAction into a non-abstract class and allowing for nonspecific lifecycle actions.

@JesseLovelace are you still addressing the clirr issue?

/cherry-pick 2.1.x

/cherry-pick 2.1.x


Could not load BlazorMobile.Common

Hei,
After updating to the latest version of BlazorMobile i cant start the server project anymore. Here is the exception:
System.IO.FileLoadException: 'Could not load file or assembly 'BlazorMobile.Common, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null'. The located assembly's manifest definition does not match the assembly reference.
I also downloaded the latest BlazorMobile.Sample from repo and its also getting the same exception.
Any ideas what could be causing this?

Damn, you are the second to tell me this issue but i never had it personally.
Can you try to delete obj and bin folders of your solution ?
Can you try to delete BlazorMobile from your NuGet cache ?
On wich platform are you trying to deploy ?

Deleting obj, bin and clearing nuget cache did not help.
Current development machine is Windows 10, build 19041.264
Dotnet version 3.1.300

The version shown does not always reflect the dll version, as i version it mainly from the NuGet package versioning.
Do you have any other BlazorMobile.Common.dll present in Nuget cache from an other BlazorMobile package ? If yes, does replacing the expected one fix the issue ?
Do you have this issue while using BlazorMobile for Mobile or using ElectronNET for Desktop ?
Do you have a minimal repro step to have this bug ?

The version shown does not always reflect the dll version, as i version it mainly from the NuGet package versioning.
Do you have any other BlazorMobile.Common.dll present in Nuget cache from an other BlazorMobile package ? If yes, does replacing the expected one fix the issue ?
Do you have this issue while using BlazorMobile for Mobile or using ElectronNET for Desktop ?
Do you have a minimal repro step to have this bug ?

I totally cleared out my nuget cache folder so there cant be any other BlazorMobile.Common.dll's left from other versions.
I get same exception while using the sample project from BlazorMobile repo.
When i try to run BlazorMobile.Sample.Blazor.Server project in Visual Studio 2019 Preview (Version 16.7.0 Preview 1.0).
Here's the full exception trace

I will test this on my side. If there is a problem, in my opinion it's something related to Nuget dependencies. I have seen that sometime BlazorMobile.Common is not always generated at first NuGet build task. So i imagine that there is some misleading in the Nuget config maybe.
I will take a look shortly today.

Ironically i don't have the same error while testing the Server project.
On my side it's:
FileNotFoundException: Could not load file or assembly 'BlazorMobile.Common, Version=<IP_ADDRESS>, Culture=neutral, PublicKeyToken=null'. Le fichier spécifié est introuvable.

BlazorMobile projects are working except the one using the .NET Core stack.
I'm investigating.

Hi @soend .
First, thanks for your bug report, as this functionnality was working before.
I don't know if this is because of the migration of BlazorMobile.Web package to be packed with dotnet pack instead of nuget pack but this weird error, only occuring on .NET Core project disappeared on my side when i have migrated BlazorMobile.Common with dotnet pack too.
I don't know if it's related to the tooling directly, or because dotnet pack alter the dll version too when publishing, but it seemed to fix the error for me.
I think i will try to update the way Nuget and dll are generating at build.
But i need some time and some test.
I will come back on this very soon.

A fix is ready for this issue, at least i don't have error at start anymore on .NET Core / Server project.
I will ship a new version tomorrow, thanks for your patience !

Hi @soend !
I don't know if you still use BlazorMobile, but a fix has been released for this issue:
https://github.com/Daddoon/BlazorMobile/blob/master/MIGRATION.md#blazormobile-327-preview4202108-to-328
Packages has been published on Nuget.org. If you don't see it, they will surely appear in one hour.


Server segfaults on parse_url

Bug Report
I attempted to reach the /authenticate endpoint by typing the URL in Google Chrome and before I got the prompt, the server crashed with the following.
[2023-11-04 08:12:16.669] [P:2566900] [error] [T:2566985] parse_url: curl_url_get(CURLUPART_QUERY) error: 16
Segmentation fault (core dumped)

Below is what I typed in the browser.
http://localhost:9000/irods-http-api/0.1.0/authenticate

This was caused by my config not having appropriate OIDC settings defined. The implementation currently replies on a GET request as part of the OIDC flow.
If we want to allow users to authenticate using Basic auth in a browser, we'll need to adjust how the HTTP API handles OIDC. Perhaps an op param can be introduced or a separate endpoint.

So we definitely need to put bumpers on that use case (missing config items).
A separate endpoint might be easier / separate the concerns.

Supporting authentication using the following syntax is not a requirement of the HTTP API.
http://<username>:<password>@localhost:9000/irods-http-api/0.1.0/authenticate

For that reason, this issue has been resolved since attempting to do what's described in this issue no longer results in the server crashing.

Need to add tests for this.


Deprecate the ToolButton node

ToolButton is a Button that has flat enabled by default. That's it.
I don't think such a thing needs a node of its own, and would probably be for the better if it was marked as deprecated and then removed in 4.0.
But wait! It's heavily used in the editor itself!
Indeed, and it saves a total of one single line. Sure, the editor has lots of buttons, so those single lines pile up, but here's a better solution, making Button take a argument when created, like Label does:
Label *new_label = memnew(Label("Placing text here avoids needing a extra line."));

I love how single copyright header is longer than whole declaration and implementation of this class XD
Constructor argument makes sense, but is flat really that important option to be there? (I mean outside editor ofc)

Making ToolButton an editor-only feature makes even more sense IMHO.

Should we simply make the editor theme have flat by default for all buttons, and drop ToolButton + the manual set_flat()s? Or are there good use cases for both flat and non-flat buttons in the editor?

Should we simply make the editor theme have flat by default for all buttons

That would make sense, but for now the "flat" button property is not overridable in the theme. It should be made an overridable constant.

There are a lot of non-flat buttons (mainly the "Cancel" and "OK" in dialogs) that would look odd if flat, so I don't think setting all buttons to one option would work.

Indeed, if we move that to a theme constant, it can also be overriden later on for specific nodes.

For the record, the editor theme redesign I'm currently working on doesn't need ToolButton, so I already removed it there.

Closed by #39690.


fix validation for radio on emptyState

Description
Fix validation to display selection

Fixes # (THEEDGE-3792)
Type of change
What is it?

[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Documentation update
[ ] Tests update

Checklist:

[ ] I have commented my code, particularly in hard-to-understand areas
[ ] I have made corresponding changes to the documentation
[ ] I have added tests that prove my fix is effective or that my feature works
[ ] New and existing unit tests pass locally with my changes
[ ] I run npm run lint:js:fix to check that my code is properly formatted

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (4f68c8d) 25.43% compared to head (4633488) 25.43%.

:exclamation: Current head 4633488 differs from pull request most recent head 44ee07a. Consider uploading reports for the commit 44ee07a to get more accurate results

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #797   +/-   ##
=======================================
  Coverage   25.43%   25.43%
=======================================
  Files         138      138
  Lines        3834     3834
  Branches     1379     1379
=======================================
  Hits          975      975
  Misses       2560     2560
  Partials      299      299

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Move configuration out of app.config to be able to have it nicely structured

Some configuration could also be stored in database


support for completions endpoint

Is your feature request related to a problem? Please describe.
The recent -instruct models are instruction tuned rather than dialogue tuned and should be very useful for most use cases of this library.
class UserDetail(BaseModel):
    name: str
    age: int

user: UserDetail = openai.Completion.create(
    model="gpt-3.5-turbo-instruct",
    response_model=UserDetail,
    messages=[
        {"role": "user", "content": "Extract Jason is 25 years old"},
    ]
)

This should work.
Describe the solution you'd like
Patch should also patch the openai.Completion.create method.

Theres no function calling API so we'd have to define our own output prompts so it'll look more like
class UserDetail(BaseModel):
    name: str
    age: int

user: UserDetail = openai.Completion.create(
    model="gpt-3.5-turbo-instruct",
    response_model=UserDetail,
    prompt="Extract Jason is 10 years old"
)


Add basic example of login with Firebase in Java branch #dev-firebase-login

Along with example of firebase login, do it using both custom UI as well as Firebase UI. And make sure to add examples of login using email id and phone numbers also.


Can't use werbdriver.io: Error: Failed to initilialise launcher service unknown: Error: Couldn't initialise "wdio-vscode-service".

I've tried to init a webdriver io project for testing my vscode extension, but Im not able to get it running with 'npm run wdio':

I have node version 18.4.0
package.json
{
  "name": "@uadebugger/integration-test",
  "type": "module",
  "devDependencies": {
    "@wdio/cli": "^8.16.22",
    "@wdio/cucumber-framework": "^8.16.22",
    "@wdio/local-runner": "^8.16.22",
    "@wdio/spec-reporter": "^8.16.22",
    "ts-node": "^10.9.1",
    "typescript": "^5.2.2",
    "wdio-vscode-service": "^5.2.1"
  },
  "scripts": {
    "wdio": "wdio run ./wdio.conf.ts"
  }
}

wdio.conf.ts:
import type { Options } from '@wdio/types'
export const config: Options.Testrunner = {
    //
    // ====================
    // Runner Configuration
    // ====================
    // WebdriverIO supports running e2e tests as well as unit and component tests.
    runner: 'local',
    autoCompileOpts: {
        autoCompile: true,
        tsNodeOpts: {
            project: './tsconfig.json',
            transpileOnly: true
        }
    },

    //
    // ==================
    // Specify Test Files
    // ==================
    // Define which test specs should run. The pattern is relative to the directory
    // of the configuration file being run.
    //
    // The specs are defined as an array of spec files (optionally using wildcards
    // that will be expanded). The test for each spec file will be run in a separate
    // worker process. In order to have a group of spec files run in the same worker
    // process simply enclose them in an array within the specs array.
    //
    // If you are calling `wdio` from an NPM script (see https://docs.npmjs.com/cli/run-script),
    // then the current working directory is where your `package.json` resides, so `wdio`
    // will be called from there.
    //
    specs: [
        // ToDo: define location for spec files here
        "./test/features/*.feature"
    ],
    // Patterns to exclude.
    exclude: [
        // 'path/to/excluded/files'
    ],
    //
    // ============
    // Capabilities
    // ============
    // Define your capabilities here. WebdriverIO can run multiple capabilities at the same
    // time. Depending on the number of capabilities, WebdriverIO launches several test
    // sessions. Within your capabilities you can overwrite the spec and exclude options in
    // order to group specific specs to a specific capability.
    //
    // First, you can define how many instances should be started at the same time. Let's
    // say you have 3 different capabilities (Chrome, Firefox, and Safari) and you have
    // set maxInstances to 1; wdio will spawn 3 processes. Therefore, if you have 10 spec
    // files and you set maxInstances to 10, all spec files will get tested at the same time
    // and 30 processes will get spawned. The property handles how many capabilities
    // from the same test should run tests.
    //
    maxInstances: 10,
    //
    // If you have trouble getting all important capabilities together, check out the
    // Sauce Labs platform configurator - a great tool to configure your capabilities:
    // https://saucelabs.com/platform/platform-configurator
    //
    capabilities: [{
        browserName: 'vscode',
        browserVersion: 'stable', // also possible: "insiders" or a specific version e.g. "1.80.0"
        'wdio:vscodeOptions': {
            // points to directory where extension package.json is located
            extensionPath: "./UAVSCodeDebuggerExtension",
            // optional VS Code settings
            userSettings: {
                "editor.fontSize": 14
            }
        }
    }],

    //
    // ===================
    // Test Configurations
    // ===================
    // Define all options that are relevant for the WebdriverIO instance here
    //
    // Level of logging verbosity: trace | debug | info | warn | error | silent
    logLevel: 'info',
    //
    // Set specific log levels per logger
    // loggers:
    // - webdriver, webdriverio
    // - @wdio/browserstack-service, @wdio/devtools-service, @wdio/sauce-service
    // - @wdio/mocha-framework, @wdio/jasmine-framework
    // - @wdio/local-runner
    // - @wdio/sumologic-reporter
    // - @wdio/cli, @wdio/config, @wdio/utils
    // Level of logging verbosity: trace | debug | info | warn | error | silent
    // logLevels: {
    //     webdriver: 'info',
    //     '@wdio/appium-service': 'info'
    // },
    //
    // If you only want to run your tests until a specific amount of tests have failed use
    // bail (default is 0 - don't bail, run all tests).
    bail: 0,
    //
    // Set a base URL in order to shorten url command calls. If your `url` parameter starts
    // with `/`, the base url gets prepended, not including the path portion of your baseUrl.
    // If your `url` parameter starts without a scheme or `/` (like `some/path`), the base url
    // gets prepended directly.
    baseUrl: '',
    //
    // Default timeout for all waitFor* commands.
    waitforTimeout: 10000,
    //
    // Default timeout in milliseconds for request
    // if browser driver or grid doesn't send response
    connectionRetryTimeout: 120000,
    //
    // Default request retries count
    connectionRetryCount: 3,
    //
    // Test runner services
    // Services take over a specific job you don't want to take care of. They enhance
    // your test setup with almost no effort. Unlike plugins, they don't add new
    // commands. Instead, they hook themselves up into the test process.
    services: ['vscode'],

    // Framework you want to run your specs with.
    // The following are supported: Mocha, Jasmine, and Cucumber
    // see also: https://webdriver.io/docs/frameworks
    //
    // Make sure you have the wdio adapter package for the specific framework installed
    // before running any tests.
    framework: 'cucumber',
    //
    // The number of times to retry the entire specfile when it fails as a whole
    // specFileRetries: 1,
    //
    // Delay in seconds between the spec file retry attempts
    // specFileRetriesDelay: 0,
    //
    // Whether or not retried spec files should be retried immediately or deferred to the end of the queue
    // specFileRetriesDeferred: false,
    //
    // Test reporter for stdout.
    // The only one supported by default is 'dot'
    // see also: https://webdriver.io/docs/dot-reporter
    reporters: ['spec'],

    //
    // If you are using Cucumber you need to specify the location of your step definitions.
    cucumberOpts: {
        // <string[]> (file/dir) require files before executing features
        require: ['./test/steps/**/*.ts'],
        // <boolean> show full backtrace for errors
        backtrace: false,
        // <string[]> ("extension:module") require files with the given EXTENSION after requiring MODULE (repeatable)
        requireModule: [],
        // <boolean> invoke formatters without executing steps
        dryRun: false,
        // <boolean> abort the run on first failure
        failFast: false,
        // <boolean> hide step definition snippets for pending steps
        snippets: true,
        // <boolean> hide source uris
        source: true,
        // <boolean> fail if there are any undefined or pending steps
        strict: false,
        // <string> (expression) only execute the features or scenarios with tags matching the expression
        tagExpression: '',
        // <number> timeout for step definitions
        timeout: 60000,
        // <boolean> Enable this config to treat undefined definitions as warnings.
        ignoreUndefinedDefinitions: false
    },

    //
    // =====
    // Hooks
    // =====
    // WebdriverIO provides several hooks you can use to interfere with the test process in order to enhance
    // it and to build services around it. You can either apply a single function or an array of
    // methods to it. If one of them returns with a promise, WebdriverIO will wait until that promise got
    // resolved to continue.
    /**
     * Gets executed once before all workers get launched.
     * @param {object} config wdio configuration object
     * @param {Array.<Object>} capabilities list of capabilities details
     */
    // onPrepare: function (config, capabilities) {
    // },
    /**
     * Gets executed before a worker process is spawned and can be used to initialise specific service
     * for that worker as well as modify runtime environments in an async fashion.
     * @param  {string} cid      capability id (e.g 0-0)
     * @param  {object} caps     object containing capabilities for session that will be spawn in the worker
     * @param  {object} specs    specs to be run in the worker process
     * @param  {object} args     object that will be merged with the main configuration once worker is initialized
     * @param  {object} execArgv list of string arguments passed to the worker process
     */
    // onWorkerStart: function (cid, caps, specs, args, execArgv) {
    // },
    /**
     * Gets executed just after a worker process has exited.
     * @param  {string} cid      capability id (e.g 0-0)
     * @param  {number} exitCode 0 - success, 1 - fail
     * @param  {object} specs    specs to be run in the worker process
     * @param  {number} retries  number of retries used
     */
    // onWorkerEnd: function (cid, exitCode, specs, retries) {
    // },
    /**
     * Gets executed just before initialising the webdriver session and test framework. It allows you
     * to manipulate configurations depending on the capability or spec.
     * @param {object} config wdio configuration object
     * @param {Array.<Object>} capabilities list of capabilities details
     * @param {Array.<String>} specs List of spec file paths that are to be run
     * @param {string} cid worker id (e.g. 0-0)
     */
    // beforeSession: function (config, capabilities, specs, cid) {
    // },
    /**
     * Gets executed before test execution begins. At this point you can access to all global
     * variables like `browser`. It is the perfect place to define custom commands.
     * @param {Array.<Object>} capabilities list of capabilities details
     * @param {Array.<String>} specs        List of spec file paths that are to be run
     * @param {object}         browser      instance of created browser/device session
     */
    // before: function (capabilities, specs) {
    // },
    /**
     * Runs before a WebdriverIO command gets executed.
     * @param {string} commandName hook command name
     * @param {Array} args arguments that command would receive
     */
    // beforeCommand: function (commandName, args) {
    // },
    /**
     * Cucumber Hooks
     *
     * Runs before a Cucumber Feature.
     * @param {string}                   uri      path to feature file
     * @param {GherkinDocument.IFeature} feature  Cucumber feature object
     */
    // beforeFeature: function (uri, feature) {
    // },
    /**
     *
     * Runs before a Cucumber Scenario.
     * @param {ITestCaseHookParameter} world    world object containing information on pickle and test step
     * @param {object}                 context  Cucumber World object
     */
    // beforeScenario: function (world, context) {
    // },
    /**
     *
     * Runs before a Cucumber Step.
     * @param {Pickle.IPickleStep} step     step data
     * @param {IPickle}            scenario scenario pickle
     * @param {object}             context  Cucumber World object
     */
    // beforeStep: function (step, scenario, context) {
    // },
    /**
     *
     * Runs after a Cucumber Step.
     * @param {Pickle.IPickleStep} step             step data
     * @param {IPickle}            scenario         scenario pickle
     * @param {object}             result           results object containing scenario results
     * @param {boolean}            result.passed    true if scenario has passed
     * @param {string}             result.error     error stack if scenario failed
     * @param {number}             result.duration  duration of scenario in milliseconds
     * @param {object}             context          Cucumber World object
     */
    // afterStep: function (step, scenario, result, context) {
    // },
    /**
     *
     * Runs after a Cucumber Scenario.
     * @param {ITestCaseHookParameter} world            world object containing information on pickle and test step
     * @param {object}                 result           results object containing scenario results
     * @param {boolean}                result.passed    true if scenario has passed
     * @param {string}                 result.error     error stack if scenario failed
     * @param {number}                 result.duration  duration of scenario in milliseconds
     * @param {object}                 context          Cucumber World object
     */
    // afterScenario: function (world, result, context) {
    // },
    /**
     *
     * Runs after a Cucumber Feature.
     * @param {string}                   uri      path to feature file
     * @param {GherkinDocument.IFeature} feature  Cucumber feature object
     */
    // afterFeature: function (uri, feature) {
    // },

    /**
     * Runs after a WebdriverIO command gets executed
     * @param {string} commandName hook command name
     * @param {Array} args arguments that command would receive
     * @param {number} result 0 - command success, 1 - command error
     * @param {object} error error object if any
     */
    // afterCommand: function (commandName, args, result, error) {
    // },
    /**
     * Gets executed after all tests are done. You still have access to all global variables from
     * the test.
     * @param {number} result 0 - test pass, 1 - test fail
     * @param {Array.<Object>} capabilities list of capabilities details
     * @param {Array.<String>} specs List of spec file paths that ran
     */
    // after: function (result, capabilities, specs) {
    // },
    /**
     * Gets executed right after terminating the webdriver session.
     * @param {object} config wdio configuration object
     * @param {Array.<Object>} capabilities list of capabilities details
     * @param {Array.<String>} specs List of spec file paths that ran
     */
    // afterSession: function (config, capabilities, specs) {
    // },
    /**
     * Gets executed after all workers got shut down and the process is about to exit. An error
     * thrown in the onComplete hook will result in the test run failing.
     * @param {object} exitCode 0 - success, 1 - fail
     * @param {object} config wdio configuration object
     * @param {Array.<Object>} capabilities list of capabilities details
     * @param {<Object>} results object containing test results
     */
    // onComplete: function(exitCode, config, capabilities, results) {
    // },
    /**
    * Gets executed when a refresh happens.
    * @param {string} oldSessionId session ID of the old session
    * @param {string} newSessionId session ID of the new session
    */
    // onReload: function(oldSessionId, newSessionId) {
    // }
}

tsconfig.json:
{
    "compilerOptions": {
        "moduleResolution": "node",
        "module": "ESNext",
        "target": "es2022",
        "types": [
            "node",
            "@wdio/globals/types",
            "expect-webdriverio",
            "wdio-devtools-service",
            "wdio-vscode-service",
        ],
        "skipLibCheck": true,
        "noEmit": true,
        "allowImportingTsExtensions": true,
        "resolveJsonModule": true,
        "isolatedModules": true,
        "strict": true,
        "noUnusedLocals": true,
        "noUnusedParameters": true,
        "noFallthroughCasesInSwitch": true
    },
    "include": ["test"]
}

Same happens also when I freshly initialize a webdriver project and run the 'npm run wdio' command.


Schedule automatic data retrieval from Strava

Currently, the data displayed on the dashboard are manually updated by periodically running theR/scrape_strava.R script, then saving the newly acquired data to strava_dashboard/data/. I would love to automate this process so that the app retrieves data on a set schedule.
Some potentially helpful resources:

Deploying Data for Content On Connect
Basic Builds :: How to update data in a Shiny App on RStudio Connect (notes that content is out of date and points to the above resource)
Using Shiny with Scheduled and Streaming Data (notes that content is out of date and points to the above resource)

Steps:

store data in external service (e.g. GoogleSheets, S3)
modify application to pull data from external service (rather than from within the GitHub repo, as it's currently doing)
modify data scraping/wrangling script to upload data to external service
automate data scraping script (e.g. on set schedule) e.g. using GHA, local machine, etc.

Thanks @brenshanny <3

And always grateful for Notes from a data witch: https://blog.djnavarro.net/posts/2022-03-17_using-aws-s3-in-r/

Also see R for the Rest of Us: https://rfortherestofus.com/2023/05/github-actions

Text for landing page (once updates have been made)

The data displayed are periodically updated by manually running R/scrape_strava.qmd, which cleans and saves the newly acquired data to an AWS S3 Bucket. Data are read directly into the application from this S3 Bucket. A mid-term future goal is to automate new data retrieval and app redeployment using GitHub Actions!


Global and device kernels are unsound

Motivation
Launching NVPTX global kernels is unsafe - they are unsafe fn, and this requires the Rust program that launches to use an unsafe block. For most examples below, this program has undefined behavior because the unsafe code it contains is incorrect.
However, must of the kernels below are never correct, so it would be very helpful for the compiler to reject them, or to at least warn about their issues.
Examples
These are some examples of code that's accepted today. Most of these examples are always UB.
Launching these global kernels
#[no_mangle]
pub unsafe extern "ptx-kernel" fn foo(a: &mut f32) {} // UB: multiple &mut

pub struct Bar<'a>(&'a mut f32);
#[no_mangle]
pub unsafe extern "ptx-kernel" fn bar<'a>(a: Bar<'a>) {}  // UB: multiple &mut

is always undefined behavior: these kernels are spawned in multiple threads of execution, each containing a copy of the same&mut T to the same data. On the other hand:
#[no_mangle]
pub unsafe extern "ptx-kernel" fn foo(a: &mut f32) {}

#[no_mangle]
pub unsafe extern "ptx-kernel" fn bar(mut a: f32) {
    foo(&mut a); // OK - global kernels can be called from other kernels
}

global kernels that are called from other kernels are executed in the same thread of execution. Device kernels as well:
fn device(a: &mut i32) { a += 1 } // OK

#[no_mangle]
pub unsafe extern "ptx-kernel" fn global() {
   let mut a = 0;
   device(&mut a); // OK: each a is local to each thread of execution
}

We don't support static and dynamic shared arrays in kernels yet, but NVPTX does, and we'd like to support them at some point. These arrays are shared across all threads of execution without any synchronization:
fn device(a: &mut [i32; 32]) {
    a[0] += 1;  // UB: data-race
}

#[no_mangle]
pub unsafe extern "ptx-kernel" fn global() {
   let mut a = UnsyncShared::<[0_i32; 32]>::new(); // OK: create unsynchronized shared memory array
   device(&mut a); // UB: multiple &mut to same object
}

Note that there are two issues with these. When a device function creates them, these are shared across all execution threads of that device function. That is, taking a &mut T to the whole array creates many copies, one on each execution thread, of the same &mut T to the exact same data. This is already undefined behavior, and can be used to introduce data-races.
We might want to support synchronized (e.g. atomic) versions of the shared memory arrays as well. While they might avoid the data-race, taking a &mut T to the array still creates multiple &mut T to the same data, which is undefined behavior. That is, just adding synchronization does not solve the problem (this is also not desirable for performance).
We'd like to accept this code:
fn device(a: &mut i32) {  *a += 1; }

#[no_mangle]
pub unsafe extern "ptx-kernel" fn global() {
   let mut a = UnsyncShared::<[0_i32; 32]>::new(); // OK: create unsynchronized shared memory array
   device(&mut a[nvptx::_thread_idx_x()])); // UB
}

but note that IndexMut::index_mut(&mut self) would create multiple &mut T to the shared array, one on each thread, which results in UB as well. The following example should work, but is not very nice:
fn device(a: &mut i32) {  *a += 1; }

#[no_mangle]
pub unsafe extern "ptx-kernel" fn global() {
   let mut a = UnsyncShared::<[0_i32; 32]>::new(); // OK: create unsynchronized shared memory array
   let p = &mut a as *mut _ as *mut i32; // OK: &mut T as *mut T does not create a &mut T
   device(unsafe { &mut * p.add(nvptx::_thread_idx_x()) }); // OK
}

Questions
What general approaches do we have to make these examples sound?

trivial: reject global kernels (abi ptx-kernel) that are not unsafe fn

Should we also pursue an approach that lints on "improper global/device kernel arguments" ? E.g.

global and device kernel arguments require: Sync - probably as a too hard constraint, since it does not allow raw pointers, also we technically only require Sync for mutable references to shared memory. Mutable references that do not point to shared memory are fine.
launching a global kernel requires the types passed to the kernel to be: SendGPU or similar (DeviceCopy as @bheisler put it below), since these arguments need to be sendable from the Host to the Device, and Copyable to the multiple execution threads of the device.

It might get tricky to propagate these lints through generic code, e.g., when calling Index::index as a device function. Also, Sync prevents raw pointers. A simple wrapper solves this, but we might want to allow raw pointers for convenience here.
What do we do about shared memory device arrays? Taking a &mut to them is always undefined behavior, which makes them extremely easy to use incorrectly, and very hard and unergonomic to use correctly.
Are there any other ways of tackling this problem?

cc @rkruppe @nikomatsakis

In RustaCUDA I have a marker trait (DeviceCopy) for types which are safe to copy to the GPU, along with a custom derive macro that checks that DeviceCopy must be implemented on all fields of a type. It's implemented by default on all of the primitive numeric types, but not on references (because that would be unsound) or raw pointers (because they could be pointing to host memory). Instead, the user has to use some other function to get a DevicePointer, which contains the *mut T pointing to device memory and does implement DeviceCopy.
All of the device allocation structures require that their contents implement DeviceCopy. I haven't implemented kernel launching yet, but I was planning to have the same restriction for parameters.
Perhaps a similar approach could be taken here, at least for kernel parameters. This doesn't help limit unsafety within a kernel, though.
I chose not to have DeviceCopy be a subtrait of Copy. It's quite likely that users will want to implement DeviceCopy for large and complex structures, where it would be inefficient to pass-by-copy.

The function parameters to a kernel can be anything that nvcc successfully compiles, and which can be copied as bits; see the docs for cuLaunchKernel which basically passes kernel parameters via a void **. So, for unsafe "ptx-kernel" function signatures to be sound really only requires their parameters to be Copy, imo.

@gnzlbg Also re: your UnsyncShared example, shared memory is essentially just a pointer. If we want individual threads to take mutable references of independent parts of shared memory, what we probably want is something in the same spirit as the existing split_ APIs on slices (https://doc.rust-lang.org/std/primitive.slice.html), except that the shared memory-related APIs would take a shared reference to the shared memory and return either shared or mutable references to independent parts of the shared memory.
An example:
fn device(a: &mut i32) {  *a += 1; }

pub unsafe extern "ptx-kernel" fn kernel() {
  let shm_ptr: *mut i32 = /* somehow, get shared memory... */;
  let shm = SharedMem::from_raw(shm_ptr);
  // fn split_tid_x<'a>(&'a self) -> &'a T { ... }
  // fn split_tid_x_mut<'a>(&'a self) -> &'a mut T { ... }
  // internally the impl of split_tid does some pointer weakening coercions like in your last example
  let x: &mut i32 = shm.split_tid_x_mut();
  device(x);
}

what we probably want is something in the same spirit as the existing split_ APIs on slices

Yeah, we probably want something like this, but people often like to do more "complicated" things like this - like touching multiple disjoint non-necessarily-contiguous elements of an array from each of the threads.
Enabling those use cases while at the same time rejecting code that has undefined behavior is the tricky part.

That makes sense. Sounds to me then there are at least a couple of separate issues all under the "soundness" umbrella:

Currently, global kernel functions can have an unsound signature, e.g. your example: pub unsafe extern "ptx-kernel" fn foo(a: &mut f32) {}. @bheisler proposes the params to be DeviceCopy which doesn't derive Copy. An alternative resolution I proposed requires the params of a extern "ptx-kernel" function to be Copy + 'static. In any case, constraining the params of an extern "ptx-kernel" function ought to be easily (?) implementable in rustc today.
Currently, the sound usage of a device function like fn device(a: &mut i32) { a += 1 } depends on having a sound mutable reference in the first place. However Rust does not seem to know about sound SIMT semantics for (mutably) referencing disjoint regions of memory (either global or shared) from parallel threads, using special PTX rank registers like threadIdx.x. In other words the issue is soundly obtaining a mutable reference of thread-disjoint memory regions, while also enabling more complicated/higher-level use cases.


Error when dismissing Announcement

Seems that we get a missing field error from Post Gres
PG::UndefinedColumn: ERROR:  column starburst_announcement_views.user does not exist
LINE 1: ...ews".* FROM "starburst_announcement_views"  WHERE "starburst...
^
: SELECT  "starburst_announcement_views".* FROM "starburst_announcement_views"  WHERE "starburst_announcement_views"."user" = 7 AND "starburst_announcement_views"."announcement" = 12 LIMIT 1
Completed 500 Internal Server Error in 19ms
ActiveRecord::StatementInvalid (PG::UndefinedColumn: ERROR:  column starburst_announcement_views.user does not exist
LINE 1: ...ews".* FROM "starburst_announcement_views"  WHERE "starburst...
I assume its the controller needs updating as teh field name is user_id not user?

Fixed in a26007757b69c095e400acb20e0b30229559513d
Will release next version as a gem in a bit

Pushed as v1.0.3 - on rubygems so feel free to update the gem in your gemfile. Thank you for reporting this.


ValidationError should raise a 422

Traceback reported by @zonia3000
xonya@pjem:~/code/fractal-server (main)*$ poetry run fractalctl set-db
Run alembic.config, with argv=['-c', '/home/xonya/code/fractal-server/fractal_server/alembic.ini', 'upgrade', 'head']
WARNING:root:FRACTAL_TASKS_DIR="Tasks" is not an absolute path; converting it to "/home/xonya/code/fractal-server/Tasks"
2024-04-10 16:13:38,931 - fractal_server.app.db - WARNING - SQLite is supported (for version >=3.37) but discouraged in production. Given its partial support for ForeignKey constraints, database consistency cannot be guaranteed.
INFO  [alembic.runtime.migration] Context impl SQLiteImpl.
INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
INFO  [alembic.runtime.migration] Running upgrade  -> 50a13d6138fd, Initial schema
INFO  [alembic.runtime.migration] Running upgrade 50a13d6138fd -> 4c308bcaea2b, Add Task.args_schema and Task.args_schema_version
INFO  [alembic.runtime.migration] Running upgrade 4c308bcaea2b -> f384e1c0cf5d, Drop Task.default_args columns
INFO  [alembic.runtime.migration] Running upgrade f384e1c0cf5d -> 70e77f1c38b0, Add ApplyWorkflow.first_task_index and ApplyWorkflow.last_task_index
INFO  [alembic.runtime.migration] Running upgrade 70e77f1c38b0 -> a7f4d6137b53, Add ApplyWorkflow.workflow_dump
INFO  [alembic.runtime.migration] Running upgrade a7f4d6137b53 -> 8f79bd162e35, Add Task.docs_info and Task.docs_link
INFO  [alembic.runtime.migration] Running upgrade 8f79bd162e35 -> 99ea79d9e5d2, Add Dataset.history
INFO  [alembic.runtime.migration] Running upgrade 99ea79d9e5d2 -> 84bf0fffde30, Add dumps to ApplyWorkflow
INFO  [alembic.runtime.migration] Running upgrade 84bf0fffde30 -> e75cac726012, Make ApplyWorkflow.start_timestamp not nullable
INFO  [alembic.runtime.migration] Running upgrade e75cac726012 -> d4fe3708d309, Make ApplyWorkflow.workflow_dump non-nullable
INFO  [alembic.runtime.migration] Running upgrade d4fe3708d309 -> 71eefd1dd202, add slurm_accounts
INFO  [alembic.runtime.migration] Running upgrade 71eefd1dd202 -> 97f444d47249, add ApplyWorkflow.project_dump
INFO  [alembic.runtime.migration] Running upgrade 97f444d47249 -> efa89c30e0a4, add Project.timestamp_created
INFO  [alembic.runtime.migration] Running upgrade efa89c30e0a4 -> 4cedeb448a53, WorkflowTask foreign keys not nullables
INFO  [alembic.runtime.migration] Running upgrade 4cedeb448a53 -> 9fd26a2b0de4, add Workflow.timestamp_created and Dataset.timestamp_created
INFO  [alembic.runtime.migration] Running upgrade 9fd26a2b0de4 -> d71e732236cd, v2
xonya@pjem:~/code/fractal-server (main)*$ poetry run fractalctl start --port 8000
WARNING:root:FRACTAL_TASKS_DIR="Tasks" is not an absolute path; converting it to "/home/xonya/code/fractal-server/Tasks"
INFO:     Started server process [26112]
INFO:     Waiting for application startup.
2024-04-10 16:13:43,265 - fractal_server.app.db - WARNING - SQLite is supported (for version >=3.37) but discouraged in production. Given its partial support for ForeignKey constraints, database consistency cannot be guaranteed.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://<IP_ADDRESS>:8000 (Press CTRL+C to quit)
INFO:     <IP_ADDRESS>:41476 - "GET /auth/current-user/ HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:41484 - "GET /api/alive/ HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:41476 - "GET /auth/current-user/ HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:41496 - "GET /api/v1/task/?args_schema=false HTTP/1.1" 200 OK
INFO:     <IP_ADDRESS>:41508 - "POST /api/v1/task/collect/pip/ HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/home/xonya/code/fractal-server/fractal_server/app/routes/api/v1/task_collection.py", line 111, in collect_tasks_pip
    venv_path = create_package_dir_pip(task_pkg=task_pkg)
  File "/home/xonya/code/fractal-server/fractal_server/tasks/endpoint_operations.py", line 160, in create_package_dir_pip
    venv_path.mkdir(exist_ok=False, parents=True)
  File "/usr/lib/python3.10/pathlib.py", line 1175, in mkdir
    self._accessor.mkdir(self, mode)

FileExistsError: [Errno 17] File exists: '/home/xonya/code/fractal-server/Tasks/.fractal/fractal-tasks-core0.14.3'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/uvicorn/protocols/http/h11_impl.py", line 412, in run_asgi
    result = await app(  # type: ignore[func-returns-value]
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in __call__
    return await self.app(scope, receive, send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/fastapi/applications.py", line 1054, in __call__
    await super().__call__(scope, receive, send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/applications.py", line 123, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 186, in __call__
    raise exc
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/middleware/errors.py", line 164, in __call__
    await self.app(scope, receive, _send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
    await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 758, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 778, in app
    await route.handle(scope, receive, send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 299, in handle
    await self.app(scope, receive, send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 79, in app
    await wrap_app_handling_exceptions(app, request)(scope, receive, send)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app
    raise exc
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
    await app(scope, receive, sender)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/starlette/routing.py", line 74, in app
    response = await func(request)
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/fastapi/routing.py", line 299, in app
    raise e
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/fastapi/routing.py", line 294, in app
    raw_response = await run_endpoint_function(
  File "/home/xonya/.cache/pypoetry/virtualenvs/fractal-server-b9daiYIL-py3.10/lib/python3.10/site-packages/fastapi/routing.py", line 191, in run_endpoint_function
    return await dependant.call(**values)

  File "/home/xonya/code/fractal-server/fractal_server/app/routes/api/v1/task_collection.py", line 115, in collect_tasks_pip
    task_collect_status = get_collection_data(venv_path)
  File "/home/xonya/code/fractal-server/fractal_server/tasks/v1/get_collection_data.py", line 14, in get_collection_data
    return TaskCollectStatusV1(**data)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 12 validation errors for TaskCollectStatusV1
task_list -> 0 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 1 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 2 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 3 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 4 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 5 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 6 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 7 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 8 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 9 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 10 -> is_v2_compatible
  field required (type=value_error.missing)
task_list -> 11 -> is_v2_compatible
  field required (type=value_error.missing)

How to reproduce within a test:

Create the appropriate folder, and create a collection.json file with some arbitrary content.
Trigger task collection

Ref https://github.com/fractal-analytics-platform/fractal-server/blob/2119ad731b6826bba2f9382ad69c70db52004ad4/fractal_server/app/routes/api/v2/task_collection.py#L133, where we should add another except option
(I suggest we fix this in both V1 and V2 endpoints)