Proposed Changes #2

Real-Time Preview

https://rpaladin.github.io/

Requests for Lubos

Delete master branch
Rename gh-pages branch to master
Fix Website link(s) indexing for Google searches (most are outdated)
Docs: https://developers.google.com/search/docs/advanced/crawling/ask-google-to-recrawl
The new 404 webpage works well for my domain user domain (rpaladin.github.io).
The armory_web repository however, is not hosted on the armory.github.io domain but rather a custom domain, which may require the manually stating of it on Github.
Docs: https://docs.github.com/en/pages/getting-started-with-github-pages/creating-a-custom-404-page-for-your-github-pages-site

Questions for Lubos

Should the Fund & Download pages be moved under one dropdown category, or kept as separate buttons?
Should the navbar-text (near navbar-icon) be kept named as Armory3D, or changed to Armory 3D Engine (like the api), Armory 3D, or even Armory
Which color is best for the navbar download button do you think. Blue, red, etc.

README.md

Improved information, grammar and markdown

Global

Migrated FeaturedCover.jpg from header.html (now contributor must add per-webpage)
This is to prevent it from showing in the new 404 webpage and taking up 75% of the error screen :P

style.css

Implemented new ro-navbar-light class for new navbar dropdowns

404.html

Implemented new 404 webpage

build folder

Renamed build folder to templates for clearer clarification

make.js

Tweak for new 404 webpage + cleanup grammar

header.html

Fix proper CSS type statements
Implemented homepage switcher for both large-screen & mobile-screen + cleanup mobile navbar links
Tweaked color, mode, timeout values, margin, etc.` for both large-screen & mobile-screen
Implemented new categorized dropdown menus for links
Note: large-screen dropdown menus open on hover, while mobile-screen dropdown menus open on click
Migrated a few buttons to sub-menus

footer.html

Removed redundant links
Implemented several new links

index.html

Removed www. from ArmoryPaint + ArmoryLab links to avoid getting security alert message

community.html

Removed redundant links
Implemented Artstation link
Changed priority-order for some links
Implemented www. extension to all links that are not asset-related sources
Improved markdown, grammar

fund.html

Removed www. from ArmoryPaint links to avoid getting unsecure security alert message

notes.html

Removed redundant "Footer Content" comment

Special Thanks

I'd like to especially thank @MoritzBrueckner for lending his eyes and time.

That's awesome! I really like the hover effect for the menu entries!

Should the Fund & Download pages be moved under one dropdown category, or kept as separate buttons?

I'm not Lubos, but I would like to propose yet another solution: the download page has not much content, and it could work to move the content from the fund page onto the download page and thus combine both pages (and maybe even redesign the combined page so that the user directly sees both the download embed and the possibility to donate). But I guess the menu is already much clearer than before so a separate fund page also looks ok to me.
A few little things that I noticed:

Upon reloading the page or changing pages, the website layout flickers and looks wrong for a tiny amount of time. It's not a big problem, but maybe there is a simple solution for this?
Click to see the flickering image

What do you think about adding a hover effect to the download button in the navbar? Currently the only thing that changes is the mouse cursor.
Can the clickable area for navbar entries be enlarged so that it spans the entire height of the menu bar (or at least a larger vertical area)? It's easy to overshoot the text with the mouse and then you have to readjust it in order to be able to click or hover the entries.

Upon reloading the page or changing pages, the website layout flickers and looks wrong for a tiny amount of time. It's not a big problem, but maybe there is a simple solution for this?

Yes I noticed that too. It's due to the dropdown menus being initially unhidden on load.
The ArmorPaint also reproduces the issue for its Website Switcher dropdown menu.
Most noticeable at the Cloud webpage.
Watch the top-left while the webpage is refreshed repeatedly by me.
https://user-images.githubusercontent.com/69180012/159083012-e2712984-859c-420e-a652-6fe38be6eeea.mp4
I think I can fix the issue though. I'll work on that.

What do you think about adding a hover effect to the download button in the navbar? Currently the only thing that changes is the mouse cursor.

I thought of implementing it. But than I noticed that the Unity3D website has no hover effect for their download (Get started) button which lead me to believe it was unneeded.

Can the clickable area for navbar entries be enlarged so that it spans the entire height of the menu bar (or at least a larger vertical area)? It's easy to overshoot the text with the mouse and then you have to readjust it in order to be able to click or hover the entries.

Could you share me an screen of what you're describing? I'm having some trouble visualizing which area you're describing.

Finally, thanks for the input! Much appreciated.

Could you share me an screen of what you're describing? I'm having some trouble visualizing which area you're describing.

This is the current hitbox for the menu entries:

I'm wondering if the following would be better:

It's basically Fitts' law: if your mouse cursor is somewhere over the page and you move it vertically to go to the menu bar, it is likely that you will overshoot the target if your mouse movement is sufficiently fast, because the target height is relatively small. With a larger hitbox you have more room for potential "mis-target" errors.
I'm not sure though how common and desired it is this way, maybe somewhere in between (but close to the second screenshot) is already sufficient.

Real-Time Preview

https://rpaladin.github.io/

README.md

Improved grammar

Global

Fixed loading glitches

CSS

Removed unused proposed styles

header.html

[global] Improved formatting
[global] Fixed button highlighting bug
[global] Renamed DOCUMENTATION categories titles to LEARN
[global] Implemented Tutorials link under LEARN navbar categories
[global] Gave LEARN category links higher priority over SHOWCASE category links
[global] Partially-reverted proposed styles due to potential & unavoidable complications
[global] Implemented dark theme for scroll-bars
[large-screen] Implemented unique download icon for download button
[mobile-screen] Implemented old style download button
[global] Added dividers for all dropdown menus
[global] Added border(s) for all dropdown menus
[global] Corrected border alignment for dropdown menus (except menu switcher menu due to issues)
[mobile-screen] Improved mobile-screen -> menu icon

footer.html

Migrated scripts to header.html

Sounds fine to me as well!

I like the red one, so it also matches with the itch.io download widget. What do you think?

Hm, these two subjects actually contradict each other a bit, as having a unique download button somewhat ruins the affect of a dropdown button that can be seen with the other navbar buttons.
Should we perhaps remove the idea of a unique button for Download and/or Fund and replace it with an average navbar button?

Should be ok as is with Armory3D, with first content paragraph then being Armory 3D Engine.

Makes sense. Thanks for the reply!

Hm, these two subjects actually contradict each other a bit

Do they? To me it sounds as if one question was about merging the funding page into the download page and the other one was about the button that leads to the download page. So we could have a red download button that leads to the download page where there is also a section on funding.

To me it sounds as if one question was about merging the funding page into the download page and the other one was about the button that leads to the download page.

Ah maybe I misunderstood.. I was under the impression you wanted a dropdown menu to replace the Download button, which would also include the Fund button under the dropdown:

Perhaps you/Lubos was referring to literal combining of HTML source for both download.html & fund.html ?

Updates

Migrated fund.html content to download.html.
Changed download button color(s) to red.
Changed footer Kha link to Kode as it's the parent of Haxe, Kha, Krom, etc.
Major typo code cleanup. A few by me, several in notes.html by @Naxela ❤ (we all mess up sometimes)
P.S. Aways check that you're not pasting code typos! And please also remember to add alt attributes for img tags.
Removed some outdated HTML or replaced with current code rules.

New Questions for Lubos

Would it be alright to migrate the UIKit CSS files to the Github repository to reduce extra HTTP requests?
Or is there perhaps some unknown reason they're residing in external CloudFlare servers.

Or if you're still hesitant to the idea of that, perhaps you'd prefer still using the official UIKit CDN links listed at their download page, but more up-to-date than the ones currently used for the website:
UIKit v1: https://getuikit.com/docs/installation#download
UIKit v2: https://cdnjs.com/libraries/uikit

Is the current website using UIKit v1 or UIKit v2?
I believe it's using v2, but wanted to double-check with you.

Final

If this PR is believed ready to merge, could you consider implementing 2 (if not all 4) of the original Lubos requests stated in my original PR post. Thank you!!

@rpaladin
Hi,
Thanks for the code clean-up. Do you have any specific typos that gets repeated in the notes.html? - Since I'm using a script to auto-generate the notes, and just to prevent that the typos become redundant it's just as well to have it fixed in the auto-script once I'm at it

@Naxela

Do you have any specific typos that gets repeated in the notes.html?

https://github.com/armory3d/armory_web/pull/15/commits/39c1a1acefd4ca95e201900a21c6850ba70d88da#diff-edf678be2313d5f4b8196db3c3589e20b5609825f3ec0178f2e1f80e32b78bf1
templates/notes.html (OLD)
Accidental closing </a> tag with no opening tag.
 Thanks!
I have already fixed it for your previous notes PR so no need to go back and fix those commits, but for future notes commits that would be great!

A Humbly Shared Tip

Use a HTML validator in the future to help track down potential typos/code-deprecations/errors:
https://validator.w3.org/
NOTE: The service will always raise some errors because of the UIKit framework classes - this is unavoidable but in the long term it doesn't hurt to double-check one's work.

Would it be alright to migrate the UIKit CSS files to the Github repository to reduce extra HTTP requests?

Yes! I think the reason is that it was just done that way by Rovacado who greatly upgraded the armory website a while ago.

Migrated external UIKit links/file to local & relative-paths.
Humble request again, requests for Lubos: https://github.com/armory3d/armory_web/pull/15#user-content-placeholder

Happy to merge when ready. Will also check on the 404 webpage once in.

Hi, sorry the recent spam.. there was a lot of confusion and some fixed conflicts relating to the master -> main branch switch on my end. I also accidentally created a new PR, #18 but closed it. Please ignore it :-)
Despite no visible commit activity above, I have successfully merged/commited @Naxela awesome changes. Commit diff for this PR should show everything is up to date and resolved.

No prob at all! There appears to be a merge conflict with build/notes.html and notes.html files, should I check on that?

@rpaladin
Is it possible to just overwrite the notes.html with the fixes you've made (Since your version is the more W3C correct version)?

@rpaladin Is it possible to just overwrite the notes.html with the fixes you've made (Since your version is the more W3C correct version)?

Yes it definitely is. If @luboslenco agrees to force merge this PR, @Naxela your latest 2022.5 changes will be kept, as I already migrated manually all of the changes to my modified, alternate version. The only remaining conflicts are the fixes and shouldn't break anything, as can be seen in the preview link I shared above.
By the way, I forgot to thank you for still maintaining the notes. Thanks for those!

Thanks! Merged at https://github.com/armory3d/armory_web/commit/6628571e326a369fa737e5c643d2e1312f54d9e4 via command line, let me know if I missed something.


GKE Cluster replace before delete doesn't work with some config options

I have changed some of the cluster settings that caused failure during pulumi apply. The issue was that pulumi attempted to do replace before delete for a cluster that was explicitly specifying services and pods subnetworks. The update fails with:
Diagnostics:
  gcp:container:Cluster (default):
    error: Plan apply failed: Error waiting for creating GKE cluster: Retry budget exhausted (5 attempts): Services range "services" in network "default", subnetwork "europe-west1-b" is already used by another cluster.

As you can see the replacement cluster will always fail as it's trying to use the subnetwork already used by the existing cluster.
It seems that for some cluster setups it needs to always to delete before replace.

I forgot to add that the change I was making was to add enableKubernetesAlpha = true to configuration. When I applied the change, I noticed that a new alpha cluster was created but in a faulty state in the GCP console and the old cluster was still up and running.
When I tried to run pulumi up again it didn't notice that there is already a replacement cluster running and it created another alpha cluster instance.

I think the answer to this is adding a deleteBeforeReplace option when a replacement is needed. I don't think there is a universal mechanism to infer this automatically.


Check that installation procedures all work in a clean environment

pip installation from official release now automatically tested. Conda still pending until package available.


documentation

modes
commands
link types
ops
order
networks
render layers
targets
all nodes that differ from how they work in fundsp (turbofish)

modes

edit: (press e or esc in any other mode) interact with entities / execute commands

draw: (press d in any other mode) draw new circles

connect: (press c in any other mode) connect entities
target: (hold t in connect mode) target an entity from another

return-terminated:
:e
:w
:info
:lt
:ht
:set
n
r
x
y
z
h
s
l
a
v
o | rot | rotation
op
ord | order
arr | array
tar | targets
:tsel
immediate:
d
c
et
er
en
eh
es
el
ea
eo
ev
ee
Et
Er
En
Eh
Es
El
Ea
Eo
Ev
ht
hf
o
l
II
IC
ID
ni
no
ii
in
ira
ix
iy
iz
ih
is
il
ial
iv
iro
ior
iop
iar
it
iL
iO
sa
sc
sh
vv
vc
vb
vw
va
quartz

you africans, please listen to me as africans
and you non-africans, listen to me with open mind

remember playfulness? what happened?

order:
every circle has an order (0 or higher). things in order 0 do nothing.
each frame, every circle with a positive order gets processed (does whatever its op defines)
this processing happens.. you guessed it, in order
we process things breadth-first. so when a circle is processed, all its inputs must have their data ready (they processed in this frame) to make sure that's the case, their order has to be lower than that of the circle reading their data.... that's probably more confusing than no docs

ops can have the std math constants in them https://doc.rust-lang.org/std/f32/consts/index.html

process is an op that will take its targets and process them in the order they appear in that targets array. it doesn't care about those targets' order. even if they're at order 0, in face it's preferable they are at 0 so you don't cause unexpected things.
so for every frame "process" itself is processed, it processes all of its targets in order.
combining that with the "reorder" and "rise" ops can allow you to perform things like that in a triggered way.
combining that with the ability to store any number of targets (and repeated targets with the "repeat" op) allows for complex logic-time looping :smiling_imp:
aaaaaa fuck i'm terrible at explaining things

circle anatomy
first, define your terms:

circle: an object that you create in draw mode
hole: a connection object. these always come in (black hole - white hole) pairs
entity: i'll use that to refer to any object (circle or hole)

now let's get to the biz:
in addition to the shared properties that both circles and holes have (position, color, radius, vertices) a circle holds other things:

a number (just a float)
an op string: defining what that circle does
for example: "sum", "screenshot", "lowpass()" (will get to those parentheses in a minute)
an array of numbers: for different uses (don't worry it's empty by default, thus allocating no memory)
an array of target entities that this circle controls in some way (empty by default too)
an audio network (defined by the op string)...

the amount of policy changes that i've encountered in the past 4 months alone is laughable

cargo r --features inspector will build with the inspector enabled

targets:
a circle has an array of "targets" those are just entity id's. so it's like a "pointer" to another circles or hole. think of it as a one-way wireless connection.
some ops make a circle do things to its targets. like del_targets, spin_target, distro, reorder, open_target, close_target (those two work on hole targets)
(they allow some things that aren't easy through normal processing. since circles read their input when they process, while targets are written to when the controller circle is processed instead)

hmm.. ditch the tables and use lists for the ops?

-

inputs:
notes:
etc

*

inputs:
notes:

now slowly describe each node.. thank you amy, you've done good work here


Raven java.lang.OutOfMemoryError

09-17 14:32:05.328   13737-9374/? E/AndroidRuntime﹕ FATAL EXCEPTION: raven-pool-1532-thread-1
Process: io.pack.readlog, PID: 13737
java.lang.OutOfMemoryError
at java.util.zip.Deflater.createStream(Native Method)
at java.util.zip.Deflater.(Deflater.java:191)
at java.util.zip.Deflater.(Deflater.java:165)
at java.util.zip.DeflaterOutputStream.(DeflaterOutputStream.java:54)
at net.kencochrane.raven.marshaller.json.JsonMarshaller.marshall(JsonMarshaller.java:107)
at net.kencochrane.raven.connection.HttpConnection.doSend(HttpConnection.java:122)
at net.kencochrane.raven.connection.AbstractConnection.send(AbstractConnection.java:78)
at net.kencochrane.raven.connection.AsyncConnection$EventSubmitter.run(AsyncConnection.java:137)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1112)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:587)
at java.lang.Thread.run(Thread.java:818)

I'm not entirely sure what we can do here. We definitely use some memory to send events, but we're likely not the actual cause of the OOM. In PHP we're able to define some crud variable to effectively reserve memory space for ourselves.

@mrgren - In the (unlikely) case that you're OOMing due to a large volume of messages being queued to be sent to the Sentry server, you can set the raven.async.queuesize option to put an upper bound on the number of messages that will be kept in memory, with older ones being discarded.
In the (again, unlikely) case that this is actually due to Raven, this probably means one (or more) of a few things:

you are trying to send too much data to Sentry (messages are too large, and there is a reasonable probability that they would be rejected by the Sentry server for being too large),
you are trying to send too many events to Sentry (errors are being generated at a higher rate than they can be accepted by the server.)


in dynamic environment consul-agent should be init

I modified this value in my puppet repo after check consul-agent was finished with clean exit or somethig of these signals SIGHUP, SIGINT, SIGTERM or SIGPIPE
I need consul is always working in live "nodes"
I also read this documentation: https://www.freedesktop.org/software/systemd/man/systemd.service.html

I think it might make better sense that this be configurable for systemd.

I think this is totally fine, we are already doing ensure=>running in most cases, so puppet is going to be doing this anyway just at a slower pace.


Remove unnecessary labels for yolo classes.txt file

When I save my labels file with yolo format (txt) it keeps saving other tags.
How can I stop this behavior? I tried to find it in the app but I can't figure out how.
Also I tried creating the classes.txt only with my tags but it was overridden.
 The file saves the labels defined in /labelimg/data/predefined_classes.txt.
The solution is to modify your class file and add new tags.

I tried editing the predefined_classes.txt but the other tags keeping coming up

Why isn't this empty by default? Now I have to manually adjust classes to all of my 500 images haha

Thank you. The solution to clear the predefined classes worked for me.

Your Welcome!
On Tue, 10 Jan, 2023, 8:52 pm Avv2, @.***> wrote:

Thank you. The solution to clear the predefined classes worked for me.
—
Reply to this email directly, view it on GitHub
https://github.com/heartexlabs/labelImg/issues/624#issuecomment-1377436162,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AQIVW6LD2H3EPXGWKRK5YUTWRV5D3ANCNFSM4PS26TKA
.
You are receiving this because you commented.Message ID:
@.***>

Why isn't this empty by default? Now I have to manually adjust classes to all of my 500 images haha

You can use a python script to fix it


clang-format Pathing Issue

I was able to successfully (and easily!) install this lsp, however I can't seem to get clang-format pathed correctly.  I'm tried leaving "executable" as clang-format because it's on the PATH, but that didn't work.  Also tried absolute pathing as mentioned in setup using both / and \\ as directory separators to the clang-format file (like this -> "executable": "C:/Users/Ed/AppData/Roaming/npm/clang-format"), also with no luck.  Any guidance on the correct pathing formatting to get clang-format to work on a Windows install? Do I need to change this option somewhere other than vscode's settings.json, such as in package.json or config.ts?

It appears that clang-format was working correctly by setting "executable" to "clang-format" without any pathing, however, some global options in vsCode were preventing some options in clang-format from displaying as expected.  Here's the final answer I used, assuming clang-format is on the global PATH:  "executable": "clang-format"


Clarification of Micro URI requirements

Hi there 👋
Wanted a bit of help clarifying some of the language around Micro URIs:

Do all the fields of a Micro URI need to contain valid (nonzero) bits and be the exact length specified for each type? I'm struggling to reconcile the two following requirements:
a. "All fields of micro UUri MUST be populated"
b. "Wildcard is is represented in a long or short URI only when the segment is blank."

Does this mean that it is not possible to have wildcards due to requirement a? Or is requirement a combined with b saying something different?

It seems required to serialize to Micro form when transporting "over the wire". This means between uDevices? If so, I'm unsure how we can send Topics, RpcRequest, or RpcResponse between uDevices. The requirements I'm trying to make sense together:

"MUST use the Miro Uri serialization format "on the wire""
The formats of the Micro formats don't seem to include necessary pieces:

RPC seem to require at a minimum the UResource name and method in addition to id: "MUST return UStatus::Failed if the UUri.resource.name != "rpc" || UUri.resource.instance != "response" || UUri.resource.id != 0"
Topics seem to require UResource message / fragment: "MUST contain UE_NAME, UE_VERSION, RESOURCE, and MESSAGE"

Maybe I've missed how this is accomplished. Could you point me in the right direction?

As a follow-up, wanted to mention I saw under the section about mapping UURIs into SOME/IP that there was a mention of how to pack the METHOD_ID relevant for RPC:

And I didn't see this field mentioned in the section over here about Micro URI form:

Further, I saw there was some mention of topics over under the section on uSubscrtiption for SOME/IP:

which seemed to imply that RESOURCE_ID field would be repurposed for holding an id of this message assigned in the protobuf file.
Is that the case?

Let me answer the various questions:

Do all the fields of a Micro URI need to contain valid (nonzero) bits and be the exact length specified for each type?
Yes.

Does this mean that it is not possible to have wildcards due to requirement a?
Yes

It seems required to serialize to Micro form when transporting "over the wire". This means between uDevices?
Wire I mean transport so not only device-2-device

And I didn't see this field mentioned in the section over here about Micro URI form:
This is a bug, I'll fix using this PR.

Yup, this clears it up. Thx :heart:


More than one mask_token ([MASK]) is not supported

More than one mask_token ([MASK]) is not supported.

Hi there,
We use GitHub issues as a place to track bugs and other development-related issues.
Please see the link below to our dedicated support line:
Help Center :  Click Here
Ticket ID: WT240
Note: Click on the live chat icon at the bottom corner of the page to start a conversation.

Not Found
The requested URL was not found on this server.
Additionally, a 404 Not Found error was encountered while trying to use an ErrorDocument to handle the request.


Update documentation of NIF Converter

Please update the documentation of the nif converter:
Implementation notes:
This endpoint allows to convert plain text, a document in any format supported by e-Internalisation(link) or in the RDF formats supported by FREME into a NIF document with the RDF serialisation format specified by the accept header. Further it can convert all data formats supported by Apache Tika to NIF, you can find a list of formats here: https://tika.apache.org/1.13/formats.html
To convert plaintext to NIF, one RDF format to another, or to use e-Internationalization, you specify the input format in the Content-Type header, output format in the Accept header and send the data to convert in the document body.
To use Apache Tika, you specify a multipart request and send the file in the document body. The format of the input file will be determined by the filename. Further you need to set query string parameter informat=TIKAFile. The output format is specified by the Accept header.
Example cURL to convert plaintext to Turtle
Please add an example
Example cURL to convert HTML to RDF-XML, using e-Internationalization
Please add an example, using e-Internationalization
Example cURL to convert PDF to Turtle
curl -X POST -F<EMAIL_ADDRESS>"http://api.freme-project.eu/current/toolbox/nif-converter?informat=TIKAFile" -H "Accept: text/turtle"

Parameters
Please add the informat parameter. Is it possible to implement it with two values - "TIKAFile" when you want to use Tika and blank when you do not want to use it? I hope the blank value of informat does not override the Content-Type header.

The description is added. To be done : add informt parameter. Check newlines in curl examples and provide the example html file.

The linked HTML and PDF files
http://api-dev.freme-project.eu/doc/api-doc/someText.html
http://api-dev.freme-project.eu/doc/api-doc/source_pdf.pdf
are not on the server, it seems.
2016-08-18 17:46 GMT+02:00 bgrusdt<EMAIL_ADDRESS>
The description is added. To be done : add informt parameter. Check
newlines in curl examples and provide the example html file.
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/freme-project/freme-project.github.io/issues/284#issuecomment-240766256,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABH5AserkMVQmeodX26Yb8XS2VDNDa-Dks5qhH5rgaJpZM4JnQHb
.

test.pdf
I tried the request with the attached PDF

curl -X POST -F<EMAIL_ADDRESS>"http://api.freme-project.eu/current/toolbox/nif-converter?informat=TIKAFile" -H "Accept: text/turtle"

I get an exception

{
"exception": "org.springframework.http.converter.HttpMessageNotReadableException",
"path": "/toolbox/nif-converter",
"message": "Required request body content is missing: org.springframework.web.method.HandlerMethod$HandlerMethodParameter@cc41d2a9",
"error": "Bad Request",
"status": 400,
"timestamp":<PHONE_NUMBER>385
}

Is this an error in the implementation or is my request wrong?

@fsasaki please use the dev API.
curl -X POST -F<EMAIL_ADDRESS>"http://api-dev.freme-project.eu/current/toolbox/nif-converter?informat=TIKAFile" -H "Accept: text/turtle"

I changed the examples such that it is not necessary to download any file, the html is inside the curl command now and the user is asked to use his or her own pdf file.
I added the informat parameter to the documentation.  When TIKAFile is not chosen, informat is not taken into account.
@jnehring  Shall we add the pdf file and the inputFile parameter to the documentation because otherwise the TIKAFile parameter cannot be used, or am I wrong?

@jnehring Shall we add the pdf file and the inputFile parameter to the documentation because otherwise the TIKAFile parameter cannot be used, or am I wrong?

If it works out, then fine, please add the parameter to the documentation. Note that in this case the request needs to be a multipart request. But when it is too difficult to have an API request that is either multipart of normal POST request, then do not invest too much work in exposing this functionality in the interactive API documentation.

I've added the functionality to the documentation. Both requests are possible now, but when you made a multipart request, the website has to be loaded again before another normal POST request can be made. So far I haven't figured out how this can be changed. @jnehring  do you think it is ok like this or shall I check how to solve this?

Thanks! Sorry that I mixed up freme dev and freme live - @jnehring 's request at https://github.com/freme-project/freme-project.github.io/issues/284#issuecomment-241330861 works fine.

Maybe you can enable the multipart request when you add a Content-Type header "multipart/form-data" ?

This issue is solved. you can ignore my last comment


[docker] use latest debian base images

Motivation
Upgrade debian base images everywhere with the new release.
Have you read the [Contributing Guidelines on pull requests]
Yes
Test Plan
CI
Related PRs
8 dependabot prs that will autoclose when this lands.
If targeting a release branch, please fill the below out as well

Justification and breaking nature (who does it affect? validators, full nodes, tooling, operators, AOS, etc.)
Comprehensive test results that demonstrate the fix working and not breaking existing workflows.
Why we must have it for V1 launch.
What workarounds and alternative we have if we do not push the PR.

/land

:broken_heart: Test Failed - ci-test

/land


[22.11] cilium-cli: apply patch for CVE-2023-28114

Description of changes
https://github.com/advisories/GHSA-6f27-3p6c-p5jc

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[x] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 224191 run on x86_64-linux 1

1 package built:

cilium-cli

Changelogs are big and I was not sure if everything was fine to backport. Since the patch applies cleanly I preferred to go this way.


Suggestion: Add data syncing support between systems

I have a suggested feature for the enterprise edition.
Add a data syncing tool like SqlWorkbench has.
Right now we use that tool for it's syncing capabilities to bring our development database in sync with our production database or to selectively sync some records and not others.
Here is the screenshot of how it works:

SqlWorkbench is open source java program, so take a peak :-)

Yes, it is one of planned features (although I can't find it in FR list so this will be the one)

The schema compare feature is already available in EE

@uslss does schema compare only do table schemas or does it also do data in the table and synchronize the data rows?

@uslss Hi, can you please re-check this work item that was closed. You said "schema compare feature is already available in EE", but this work item is talking about data comparisons, not schema comparisons.
I think this was closed in error.


Issue with docs.zowe.org/stable/user-guide/start-zowe-zos/

Description

On the intro to the page, we discuss 3 services:

ZWESLSTC as Zowe main started task,
ZWESISTC as Zowe cross memory server
and ZWESASTC as Zowe cross memory auxiliary server.

However, when reading through the doc, the order you encounter each is the first, then the last, and then middle.  The order as the services appear in the bulleted list should match the order in the rest of the document.
Pages to Update

https://docs.zowe.org/stable/user-guide/start-zowe-zos/#starting-and-stopping-the-cross-memory-server-zwesistc-on-zos
Screenshots

Expected behavior

Additional context

See https://github.com/zowe/docs-site/pull/3542


Retrieve high, low close price over last hour

I'm looking to query the high, low and closing price for each hour. So far, the only way I see to accomplish this is by calling .get_product_historic_rates() every hour. However, it is discouraged by the documentation to poll the historic rates frequently. Can someone explain why that is and how I might continue?

Per the API doc the rate limit for public endpoints by IP is 10 requests per second, up to 15 requests per second in bursts. If you are getting the historical rates once an hour then I don't think that is an issue.
With that said, the alternative is to use the websocket feed.


Context menu crash

When I right click on any element in the navigation pane (bar on the left), Windows Explorer crashes. After disabling NitroShellExt with ShellExView it works fine again.

I came to comment this same thing.
Basically my Windows Explorer started crashing if I tried to right-click on the Quick Access bar (left side with pinned items).
I did try and deselect the option in NitroShare to disable this (so that NitroShare didn't show up in the menu anymore), but the crashes kept happening.
At the beginning I wasn't even sure if it's NitroShare that was causing this but by narrowing down the problem, I figured out that it was exactly that.
Steps taken:
Downloaded NirSoft ShellExView (just like @onixiya1337 ), I looked at what programs have context menu entries. I uninstalled MPC-HC (Media Player Classic - Home Cinema, no longer in developement since 2017 btw), Backup & Sync (by Google), Google Drive amongst other things.
After uninstalling and disabling stuff one by and and countless reboots, I tried disabling NitroShare's context menu and voilà: the Explorer crashes seized to exist.
So NitroShare is causing these Windows Explorer crashes when it comes to the left side of the Explorer window and disabling the context menu setting within the app doesn't help in this regard (although it does hide the icon).
Disabling this with NirSoft's ShellExView does fix the problem so I can keep using NitroShare without having crashes.


FFI error from snabb config set with "next-hop {}"

Supplying next-hop {} (which I think is valid according to the schema?) as an argument for /softwire-config/internal-interface triggers the FFI error below:
$ sudo ./snabb config set -s snabb-softwire-v1 22218 "/softwire-config/internal-interface" "next-hop {}"
lib/yang/data.lua:405: cannot convert 'nil' to 'unsigned int'
stack traceback:
        core/main.lua:149: in function <core/main.lua:147>
        [C]: in function 'finish'
        lib/yang/data.lua:405: in function 'parser'
        program/config/common.lua:77: in function 'parse_command_line'
        program/config/set/set.lua:7: in function 'run'
        program/config/config.lua:23: in function 'run'
        core/main.lua:67: in function <core/main.lua:43>
        [C]: in function 'xpcall'
        core/main.lua:211: in main chunk
        [C]: at 0x004536b0
        [C]: in function 'pcall'
        core/startup.lua:3: in main chunk
        [C]: in function 'require'
        [string "require "core.startup""]:1: in main chunk

My brief look into this, I actually think what you're doing is incorrect based on the schema. My understanding is this (if it were to succeed) would set the contents of internal-interface to next-hop {}. The internal-interface has mandatory fields such as ip and mac. I think if you wish to set next-hop to empty what you'd do is:
$ snabb config set lwaftr-name "/softwire-config/internal-interface/next-hop" "{}"

My suspicion as one of the mandatory fields is assumed to be there in the code and so a cast of "nil" to an unsigned int is occurring. I think we probably should check for this on mandatory fields and display a better error.

Yes, better error handling around missing mandatory settings with better error messages sounds really good.

Fixed with PR #730


Document supported list of kubernetes feature.

API - V1

[x] Secrets
[x] Namespaces
[x] Rolling updates
[x] Configmaps
[ ] ResourceQuouta  #4454
[x] PV HostPath
[ ] PV GlusterFS #4348
[ ] PV NFS #4423
[ ] PV EBS #4444

API - V1beta

[x] Deployments
[x] DeploymentRollback
[x] Jobs
[x] Daemonsets
[x] replicasets
[ ] HorizontalPodScaling (Depend on Heapster #4451 )

Addons

[ ] Dashboard #4452
[ ] Heapster #4451

@galal-hussein thanks for the list, very helpful!

@deniseschannon ^^ :)

@alena1108 You are welcome

@galal-hussein Thanks for the list !
Looking at the list for K8s features it seems emptyDir volume support is not being tracked.

@girishshilamkar I must have forgot to add it, it was tested and it worked perfectly.

@galal-hussein Good to know. Created an issue to add gitRepo support 4976 could you please add it to the list. Thanks !

@girishshilamkar added, thanks!

AFAIU, io.rancher.container.create_agent io.rancher.container.agent.role labels are Rancher specific and they won't work for Kubernetes objects, is that correct ?

These labels will work for containers created in k8s running on Rancher.

@deniseschannon Thanks for the update. Let me try it in one of the templates then.

@galal-hussein can you test HorizontalPodScaling as the Heapster is fixed already? I don't believe we have a separate ticket opened for it.

@galal-hussein @alena1108 maybe we should also add Deis workflow to the list

@alena1108 Heapster still has some issues that prevents it to collect node information, please check its issue https://github.com/rancher/rancher/issues/4451, i tested the same setup on AWS k8s and it worked without throwing the same error.
@prachidamle Deis added under Addons.


whereis R/config

This dependency was not found:

R/config in ./src/layouts/index.tsx

To install it, you can run: npm install --save R/config
run:npm install --save R/config also hung up

https://github.com/YaoApp/xgen/blob/main/tsconfig.json
R is path alias.


The grouping of content on this page doesnt seem right

E.g. PO_FX_COMPONENT_V1 has little to do with CM_PARTIAL_xyz structures.
But then this is missing : PoFx related methods. e.g. PoFxActivateComponent.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 589fc5cd-8d96-b640-d9ae-e0638b32fa2f
Version Independent ID: 37d302b0-35ea-6356-b302-9a469bf29ffb
Content: wudfwdm.h header - Windows driver
Content Source: wdk-ddi-src/content/wudfwdm/index.md
Product: windows-hardware
GitHub Login: @windows-driver-content
Microsoft Alias: windowsdriverdev

Thanks @pankajbharti sorry for the delay.  We removed this header from the docs a month or so ago after feedback for Shyamal and Wei


Option to disable tqdm (crashes on Matlab/Windows10)

The Matlab-Python integration works beautifully for most applications now, however there are some glitches left, especially in the way the standard streams are handled on Windows. As known unsupported features, Mathworks lists py.input and py.raw_input: https://fr.mathworks.com/help/matlab/matlab_external/limitations-to-python-support.html
But I just found out that there are also other things that do not work correctly, like the tqdm progress bars.
Minimal example to reproduce the bug, suggested by @larsoner :
clear all; close all; home
tqdm = py.importlib.import_module('tqdm');
cell(py.list(tqdm.tqdm(py.str('10 11 12').split())))

It works on WSL(Win10)/Ubuntu18 + Matlab 2018b + Python 3.6 + tqdm 4.51.0:
ans =
  1×3 cell array
    {1×2 py.str}    {1×2 py.str}    {1×2 py.str}

It fails on Windows10 + Matlab 2019a/2020a/2020b + Python 3.8 + tqdm 4.51.0:
>> cell(py.list(tqdm.tqdm(py.str('10 11 12').split())))
Error using utils>__getattr__ (line 169)
Python Error: AttributeError: 'NoneType' object has no attribute 'write'
Error in std>fp_write (line 329)
Error in std>print_status (line 336)
Error in std>display (line 1520)
Error in std>refresh (line 1386)
Error in std>__init__ (line 1092)

This is most likely due to a bug in MATLAB on Windows, therefore you could decide that it is 2x not your problem :)
But since there are tons of forum posts about broken tqdm progress bars in various non-tty contexts, I think it might be a relevant question even for non-matlab/non-windows users.
How complicated for you would it be to add a global flag to disable tqdm in MNE-Python?
@larsoner suggested something like MNE_TQDM='off' on gitter.
Note that I'm using CPython instead of Anaconda because the integration with Matlab is much smoother. Matlab+CPython really works out-of-the-box now. FYI: the notes I'm preparing as I move forward:
https://neuroimage.usc.edu/brainstorm/MnePython
As a side request: it would be great to add an option to skip the validation of the licenses when downloading, and not via argv. I see that there is an option to accept the license with a flag --accept-brainstorm-license on the command line, but this means calling python via a system call, which is not exactly what we want here (the python environment configured in Matlab is not necessarily the one available from the system path of the process)
And same goes for anything that uses input(), actually.
Thanks!
Francois

MNE_TQDM='off'

This would be easy enough to add and support. Looking at the traceback another option would be to quickly test it with something like fp.write('') in a try/except and just not print anything if it fails. I can make a quick PR for this and you can test it out.

this means calling python via a system call, which is not exactly what we want here (the python environment configured in Matlab is not necessarily the one available from the system path of the process)

Technically you can set sys.argv to whatever you want -- it's just a list so you can put the accept string in there programatically. But I agree it's not the cleanest interface and I wouldn't want to force actual end users to do this :)
We can add an accept=False default kwarg that means "prompt for answer", and people have to do accept=True to accept it. Would that work for you? And I mean this not just as a user of the API, but also as a representative of Brainstorm who can speak to how we are allowed to distribute the Brainstorm datasets -- IIRC we put in this accept-input protocol to force people to read and accept the Brainstorm license.
I was wondering if there was some way to detect if Python is being run within MATLAB, but it looks like another try/except pattern will probably work:
>> py.input()
Python Error: EOFError: EOF when reading a line

In other words, anywhere in MNE-Python we use input we should wrap it in a try/except giving people an informative error. In the case of the datasets, we can tell them to use the accept keyword argument. In cases for overwriting files (I think this is the only other place?), we can tell them to pass overwrite=True.
I'll probably just wrap all of these changes in a PR, I think it should be easy enough to test. Then if there are more snags in your testing @ftadel we can iterate until it's smooth!

This would be easy enough to add and support. Looking at the traceback another option would be to quickly test it with something like fp.write('') in a try/except and just not print anything if it fails.

Given everything I've read on tqdm yesterday, I'm sure there will be other use cases where it will be interesting to completely turn it off :)

We can add an accept=False default kwarg that means "prompt for answer", and people have to do accept=True to accept it.
In other words, anywhere in MNE-Python we use input we should wrap it in a try/except giving people an informative error.

For solving both problem with a very simple fix, maybe you could simply add a try/except around the input() call, and if it fails, consider it's worth answer = 'y'?
This is a bug, and hopefully it will be fixed in Matlab at some point. Maybe this doesn't need to be handled too carefully, these are very rare use cases anyway, maybe I'm the only person who will ever download the MNE datasets from Matlab using a Python call. And if this fails, it means that the user is working in Matlab, and have probably already accepted the Brainstorm license in other ways...

I was wondering if there was some way to detect if Python is being run within MATLAB, but it looks like another try/except pattern will probably work: py.input()

Considering that this problem will (hopefully) disappear soon, it might be better to find another way.
Is there a way from Python to access its parent process?

Given everything I've read on tqdm yesterday, I'm sure there will be other use cases where it will be interesting to completely turn it off :)

Fair enough, I'll just implement 'off' env var support then.

For solving both problem with a very simple fix, maybe you could simply add a try/except around the input() call, and if it fails, consider it's worth answer = 'y'?

If you (as a Brainstorm person) are okay with us bypassing the license message when input is unavailable then I'm fine with implementing this.

Is there a way from Python to access its parent process?

This works but requires psutil:
>> py.psutil.Process(py.os.getppid()).name

ans =

  Python str with no properties.

    matlab

Can you see if it works on your machine?
I actually think the try/except is more general, though, since it should work on any platform where input is unavailable for any reason. And we can catch just EOFError so we can be fairly precise with our check. So I'd prefer to go that route rather than rely on psutil (and add it as a an optional requirement).

Fair enough, I'll just implement 'off' env var support then.

Great, thanks!

If you (as a Brainstorm person) are okay with us bypassing the license message when input is unavailable then I'm fine with implementing this.

Sure, go for it.

>> py.psutil.Process(py.os.getppid()).name

Works from a Python console, but not from Matlab:
>> py.psutil.Process(py.os.getppid())
Error using __init__>_init (line 365)
Python Error: NoSuchProcess: psutil.NoSuchProcess no process found with pid 4740
Error in __init__ (line 324)

Ok, let's drop this idea.

I actually think the try/except is more general, though, since it should work on any platform where input is unavailable for any reason. And we can catch just EOFError so we can be fairly precise with our check. So I'd prefer to go that route rather than rely on psutil (and add it as a an optional requirement).

I thought you were talking about a more formal way of identifying if the Python is called from a Matlab context, for more general purposes than we discussed here. But that's not needed anywhere so far. The try/except around input() sounds good.
I'll let you know if I find other functions that require some extra care for Matlab.

nice !
that's great to see how mne-python core algorithms can be made available
to the matlab world !


Extension versions 5.0.0

The page talks about settings only applicable for extensions versions starting at 5.0.0, but I could not find any information about this version. Is it unreleased? I could only find references about versions 3 and 4. And is this talking about the FUNCTIONS_EXTENSION_VERSION setting or Microsoft.Azure.Functions.ExtensionBundle?
I had been looking at the "Java" version of the page but even selecting "C#" doesn't really make the page that much clearer.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: fff7ae2f-4f02-7ed9-c2ed-a2715e407975
Version Independent ID: 542016c1-3c25-f6af-cc83-0c5b6218e0ce
Content: Azure Blob storage trigger and bindings for Azure Functions
Content Source: articles/azure-functions/functions-bindings-storage-blob.md
Service: azure-functions
GitHub Login: @ggailey777
Microsoft Alias: glenga

With a bit of reasearch I was able to find that Extension Bundle v3 does contain Storage Extension v5: https://github.com/Azure/azure-functions-extension-bundles/blob/v3.x/src/Microsoft.Azure.Functions.ExtensionBundle/extensions.json
I think changing the note at the end of the article where it says "This section doesn't apply to extension versions before 5.0.0. For those earlier versions, there aren't any function app-wide configuration settings for blobs." to include the information which bundle this is included in.

@therealppa Thanks for your feedback and it totally makes sense. We will review the docs and update as appropriate.


Usage in the admin?

Is it possible to give an example of how to use this in Django admin to profile actions?
I have tried:
from debug_toolbar_line_profiler import profile_additional
...

    @profile_additional()
    def my_custom_action(self, request, queryset):
        ...

But no profiling takes place.

Did you ever find a good way of doing this?


Switch to unittest.mock from pretend.stub

Use https://docs.python.org/3/library/unittest.mock.html#auto-speccing

Also too much effort, might write a small stub class later to remove the pretend dependency.


Stacked area graph shading is hard to see if a series contains negative values

Checklist

[ x] I updated the card to the latest version available
[x ] I cleared the cache of my browser
[ ] I verified that I'm really running the lastest version in my browser console — not done, as I’m running through HA Companion App. However, I installed the addon via HACS for the first time earlier today, so doubtful that I could be running an older version
[ x] I checked if there is another issue opened with the same problem

Describe the bug
With stacked area graphs, negative values cause the graph to be hard to read.
When all values are positive, the graph appears to render the first series, then the second series above it (on y axis) and so on.
However, if the value of the first series is negative then things get strange with respect to the area shading. In my case, the first series is total grid electricity consumption, which can be negative if my solar panels are pushing more power out than my house is consuming.
Specifically, the first series renders below the y=0 line (correctly) but the shading of the area extends upwards to the y=0 line. This means that subsequent series’ areas are rendered in a place that is already shaded by the first series’ negative values, causing areas where there is shading for multiple series that overlap. This leads to portions of the second/third/etc. series that are hard to read because they’re obscured by the first series’ shading.
Version of the card
Version: v1.10.0
To Reproduce
This is a simplified example to show the scenario, using data generators so you don’t need any specific sensor values. The screenshots attached reflect this example.

type: custom:apexcharts-card
header:
  show: true
  title: ApexCharts-Card
  show_states: true
  colorize_states: true
all_series_config:
  type: area
  curve: stepline
  stroke_width: 2
stacked: true
series:
  - entity: sensor.bedroom_geyser_consumption
    name: f(𝑥) = 1.75𝑥² - 250
    data_generator: |
      const now = new Date();
      const data = [];
      for(let i = 0; i <= 24; i++) {
        data.push([now.getTime() - i * 1000 * 60 * 60, Math.pow(1.75 * i, 2) - 250])
      }
      return data.reverse()
  - entity: sensor.kitchen_geyser_consumption
    name: f(𝑥) = 20𝑥 - 50
    data_generator: |
      const now = new Date();
      const data = [];
      for(let i = 0; i <= 24; i++) {
        data.push([now.getTime() - i * 1000 * 60 * 60, i * 20 - 50])
      }
      return data.reverse();
  - entity: sensor.bedroom_geyser_consumption
    name: f(𝑥) = 50
    data_generator: |
      const now = new Date();
      const data = [];
      for(let i = 0; i <= 24; i++) {
        data.push([now.getTime() - i * 1000 * 60 * 60, 50])
      }
      return data.reverse();

This renders three simple mathematical formulae and stacks them. I added stepline to make it easier to see the issue. Note how to the right side of the graph, the shadings for the areas overlap, and become harder to see as more areas overlap.
Screenshots
Note the area from around 4pm where the overlapping shading occurs. If you look very carefully you can still see the lines from the other series, but they’re very hard to see because of the orange series’ shading.

Expected behavior
If the value of a series is negative, all values in the stacked graph should still be readable. One possibility might be to have the negative value be excluded from shading, ie have it have a shaded height of 0 for all points where y < 0
Smartphone (please complete the following information):

Device: iPhone 12
OS: iOS 15.1.1
Home Assistant Companion App
Version 2021.12.1

I know I hate bug reports, so first and foremost - thank you for your hard work in integrating with Apex Charts - this is a really, really useful addon!!

FYI, here’s a more complex example of the same problem with real data. This one requires specific sensor values to reproduce though, which is why I added the simplified example.


Prepare release 1.6.0

I realized there was some unfinished work lying around, which deserved a release.
After an approving review, I will add another commit with the generated files (i.e., docco and the minified code) and then upload the release to NPM.
@stephenkao I saw your response to #4268 and wanted to send you a formal review request, but GitHub's Reviewers form field would not let me do it. I hope a mention works for you.

@Rayraz could you take a look?

Thanks @paulfalgout! 🙏

Ah, sorry, @jgonggrijp!  I've been a little busy the last couple of weeks, but I'm open to check out future pull requests in the future.  Glad there was a happy ending here :)


x/tools/imports: slow due to unnecessary vendor scans

What version of Go are you using (go version)?
go version go1.9 darwin/amd64
Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (go env)?
GOARCH="amd64"
GOOS="darwin"

What did you do?
Ran goimports -v main.go on a very small file missing an import.
What did you expect to see?
Expected the goimports command to run considerably faster, since on a
properly configured IDE this is ran on each file save.
What did you see instead?
Running goimports took much longer than expected. Running it several times
produces similar results, so it's not "cold".
goimports -v main.go  1.09s user 17.68s system 383% cpu 4.895 total
goimports -v main.go  1.06s user 17.12s system 445% cpu 4.081 total
goimports -v main.go  1.07s user 17.19s system 414% cpu 4.406 total

Debugging
A one line change was made to the x/tools/imports package to output each
directory being scanned:
diff --git a/imports/fix.go b/imports/fix.go
index e2460849..4eb5818b 100644
--- a/imports/fix.go
+++ b/imports/fix.go
@@ -548,6 +548,8 @@ func scanGoDirs(goRoot bool) {
                }
                testHookScanDir(srcDir)
                walkFn := func(path string, typ os.FileMode) error {
+                       log.Println(">>>", path)
+
                        dir := filepath.Dir(path)
                        if typ.IsRegular() {
                                if dir == srcDir {

It turns out that most of the directories being scanned were part of external
vendor directories, which could never be imported by the current package.
All of the output was captured into a log.txt file.
With my $GOPATH, these were the findings:
$ cat log.txt | grep ">>>" | wc -l
  353744
$ cat log.txt | grep ">>>" | grep "vendor" |  wc -l
  316463

In summary, there were 316k directories scanned that could have been ignored.
With a hacky local change to ignore external vendors during the scan, we saw a
significant performance improvement:
$GOPATH/src/golang.org/x/tools/cmd/goimports/goimports -v main.go   0.07s user
0.34s
system 428% cpu 0.097 total

cc @prashantv

After digging in more I found that the function canUse gets called after all the candidates have been assembled (including hundreds of thousands of packages that can't be used).
Ideally, the recursive exploration of candidates under $GOPATH should short-circuit when encountering vendor and internal that are known not to be usable in the future. Recursing into them is a big waste of i/o and causes the bad performance numbers shown above.

@glibsm, are you going to send a change?

@bradfitz Yes, I plan to do a CL this weekend. It will be my first, so I have to get all the setup done. I've had a local patch working to great effect for the past few weeks.

I think this might be worth doing, but it's only going to get less important as modules takes over and vendor directories become less common.

I'm going to close this issue, as ever since moving to modules this has been a non-issue.


generate  Paperclip urls with data provided via Elastic search

instructions how to render Paperclip urls without ActiveRecord calls
Based on this article: http://www.eq8.eu/blogs/26-rendering-paperclip-url-via-elasticsearch-without-making-calls-to-activerecord-db
...closed the old PR and reopening it in order to retriger CLA github hook

@karmi any feedback on this PR ?

Hi Tomas, apologies for the lack of feedback on this patch as well!
I've been thinking about it, and while I understand the general motivation of what you're trying to demonstrate, I'm worried that the focus of this example is not clear enough.
First, I'm worried that people see "paperclip" in the example name and get an idea that it will show how to handle attachments, only to be frustrated later, because it doesn't :)
Second, if all we're trying to demonstrate here is that it's frequently best to work with the data loaded from Elasticsearch, the activerecord_associations.rb should be the place to do that. I've expanded this example in the attached commit (a086ded). Can you look at it and ping me back if you think it's more or less what you had in mind? Maybe we can either further expand the "associaton" example, or work on the "paperclip" example to cover more features in a clear way.

hi @karmi ,
sorry I didn't get back sooner,
basically my aim  is to demonstrate how to  render the paperclip (or Carrierwave doesn't really matter) image url without touching relational database.
My only motivation for this PR was that I wrote that blog post where I demonstrate how to do this and I told myself: I may as well just create PR here :)
back to your points:
1 yes, fully agree. It's  hard to explain that this is just for "view" only  in one file. Therefore for sake not to cause confusion I'm ok to close this PR
2 pretty much yes, I understand you are trying to explain general principle so that people understand how to approach this type of data  where as my PR was just Copy - Paste - Edit solution for the lazy ones :)
so I'm ok to close this PR


workflow

Codecov Report

Merging #7 (9b943b9) into main (fffb3c7) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main       #7   +/-   ##
=======================================
  Coverage   33.33%   33.33%
=======================================
  Files           1        1
  Lines          12       12
=======================================
  Hits            4        4
  Misses          8        8

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update fffb3c7...9b943b9. Read the comment docs.


Update alert-policies.md

Added new roles which have access to DLP alerts

@JohanFreelancer9 - kindly do the copy editor review for this PR. Thanks

@shrey920 PR merged. Thanks for adding these new roles.


.Net: Log the details of each action for Azure OpenAI connectors

Context
The newly migrated AzureOpenAI SDK v2 OpenAI connectors have started logging action (method call) details:
    // OpenAIAudioToTextService
    public Task<IReadOnlyList<TextContent>> GetTextContentsAsync(
        AudioContent content,
        PromptExecutionSettings? executionSettings = null,
        Kernel? kernel = null,
        CancellationToken cancellationToken = default)
    {
        this._client.LogActionDetails();
        return this._client.GetTextFromAudioContentsAsync(content, executionSettings, cancellationToken);
    }

    // ClientCore
    /// <summary>
    /// Logs OpenAI action details.
    /// </summary>
    /// <param name="callerMemberName">Caller member name. Populated automatically by runtime.</param>
    internal void LogActionDetails([CallerMemberName] string? callerMemberName = default)
    {
        if (this.Logger.IsEnabled(LogLevel.Information))
        {
            this.Logger.LogInformation("Action: {Action}. OpenAI Model ID: {ModelId}.", callerMemberName, this.ModelId);
        }
    }

ToDo
Consider logging the action details for Azure OpenAI connectors as well.

Agreed to roll back the logging in new OpenAI connectors and add it later if needed.


Don’t flash a message while fitness data is loading

Fixes #542

LGTM

@HansMuller also brought us a progress indicator which we could use during this load as well if you want.

I tried flashing the progress indicator but it didn't look good rendering for a fraction of a second, I think it's better to just show the data

Another option is to show the message or progress indicator after a few milliseconds' delay. (Try this on a flaky network.)


NFS server file system bug

If you find a similar existing issue, please comment on that issue instead of creating a new one.
If you are submitting a feature request, please start a discussion instead of creating an issue.
Describe the bug
When using nfs-server as a file system boot disk of the nfs-server instance is shared while attached disk remains unmounted, which leads to smaller then expected shared volume size, also in this configuration additional disk contains cent os and has 20gb file system. In some cases it could be other way around when attached disk gets mounted as root and is shared and boot disk remains unmounted.
Steps to reproduce
Steps to reproduce the behavior:

Create hpc cluster project with nfs-server as file system

Expected behavior
Boot disk mounted as root additional disk mounted as /exports/data
Actual behavior
Boot disk mounted as root additional disk is not mounted at all OR Additional disk mounted as root, boot disk is not mounted at all
Version (ghpc --version)
Blueprint
If applicable, attach or paste the blueprint YAML used to produce the bug.

Expanded Blueprint
If applicable, please attach or paste the expanded blueprint. The expanded blueprint can be obtained by running ghpc expand your-blueprint.yaml.
Disregard if the bug occurs when running ghpc expand ... as well.

Output and logs

Screenshots
Nfs server on the left, controller node on the right

Execution environment

OS: [macOS, ubuntu, ...]
Shell (To find this, run ps -p $$): [bash, zsh, ...]
go version:

Additional context
Add any other context about the problem here.

Hi @maxveliaminov , could you please share your blueprint (exclude any sensitive information)?

Hi @mr0re1 here is one,
blueprint_name: palm-model

vars:
  project_id: <PROJECT_ID>
  deployment_name: <PROJECT_ID>
  region: <REGION>
  zone: <ZONE>
  machine_type: <MACHINE_TYPE>
  node_count_dynamic_max: <NODE_COUNT_DYNAMIC_MAX>
  slurm_cluster_name: palm1
  disable_public_ips: true
  enable_shielded_vm: true

deployment_groups:
  - group: primary
    modules:
      - id: network1
        source: modules/network/vpc
        kind: terraform
      - id: appsfs
        source: community/modules/file-system/nfs-server
        kind: terraform
        use:
          - network1
        settings:
          machine_type: n2-standard-2
          auto_delete_disk: true
          local_mounts: ['/apps']
      - id: spack
        source: community/modules/scripts/spack-install
        settings:
          install_dir: /apps/spack
          spack_url: https://github.com/spack/spack
          spack_ref: v0.19.1
          log_file: /apps/spack.log
          spack_cache_url:
            - mirror_name: <SPACK_CACHE_NAME>
              mirror_url: <SPACK_CACHE_URL>
          configs:
            - type: file
              scope: defaults
              content: |
                modules:
                  default:
                    tcl:
                      hash_length: 0
                      all:
                        conflict:
                          - '{name}'
                      projections:
                        all: '{name}/{version}-{compiler.name}-{compiler.version}'
          compilers:
            - gcc<EMAIL_ADDRESS>target=x86_64
          environments:
            - name: palm
              content: |
                spack:
                  definitions:
                  - compilers:
                    -<EMAIL_ADDRESS>                  - mpis:
                    -<EMAIL_ADDRESS>                  - python:
                    -<EMAIL_ADDRESS>                  - python_packages:
                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                  - packages:
                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                  - mpi_packages:
                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                    -<EMAIL_ADDRESS>                  specs:
                  - matrix:
                    - - $packages
                    - - $%compilers
                  - matrix:
                    - - $python
                    - - $%compilers
                  - matrix:
                    - - $python_packages
                    - - $%compilers
                    - - $^python
                  - matrix:
                    - - $mpis
                    - - $%compilers
                  - matrix:
                    - - $mpi_packages
                    - - $%compilers
                    - - $^mpis

      - id: spack_startup
        source: modules/scripts/startup-script
        kind: terraform
        use:
          - network1
        settings:
          runners:
            - $(appsfs.mount_runner)
            - $(spack.install_spack_deps_runner)
            - $(spack.install_spack_runner)
            - type: data
              destination: /apps/palm/palm-install.yaml
              content: |
              123
            - type: data
              destination: /apps/spack/activate-palm-env.sh
              content: |
              456
            - type: data
              destination: /apps/palm/palm-install.sh
              content: |
              789
            - type: shell
              content: sudo chmod -R 777 /apps
              destination: chmod-apps-dir.sh
            - type: shell
              content: 'shutdown -h now'
              destination: shutdown.sh

      - id: spack_builder
        source: modules/compute/vm-instance
        kind: terraform
        use:
          - network1
          - appsfs
          - spack_startup
        settings:
          name_prefix: spack-builder
      - id: homefs
        source: community/modules/file-system/nfs-server
        kind: terraform
        use:
          - network1
        settings:
          machine_type: n2-standard-2
          auto_delete_disk: true
          local_mounts: ['/home']
      - id: debug_node_group
        source: community/modules/compute/schedmd-slurm-gcp-v5-node-group
        use:
          - network1
          - homefs
          - appsfs
        settings:
          node_count_dynamic_max: <DEBUG_MAX_NODE_COUNT>

      - source: community/modules/compute/schedmd-slurm-gcp-v5-partition
        kind: terraform
        id: debug_partition
        use:
          - network1
          - homefs
          - appsfs
          - debug_node_group
        settings:
          is_default: true
          enable_shielded_vm: null
          machine_type: null
          node_count_dynamic_max: null
          partition_name: debug

      - id: compute_node_group
        source: community/modules/compute/schedmd-slurm-gcp-v5-node-group
        use:
          - network1
          - homefs
          - appsfs

      - source: community/modules/compute/schedmd-slurm-gcp-v5-partition
        kind: terraform
        id: compute_partition
        use:
          - network1
          - homefs
          - appsfs
          - compute_node_group
        settings:
          enable_shielded_vm: null
          machine_type: null
          node_count_dynamic_max: null
          partition_name: compute

      - source: community/modules/scheduler/schedmd-slurm-gcp-v5-controller
        kind: terraform
        id: slurm_controller
        use:
          - network1
          - debug_partition
          - compute_partition
          - homefs
          - appsfs
        settings:
          machine_type: n2-standard-8

      - source: community/modules/scheduler/schedmd-slurm-gcp-v5-login
        kind: terraform
        id: slurm_login
        use:
          - network1
          - homefs
          - appsfs
          - slurm_controller
        settings:
          machine_type: n2-standard-8
          disable_login_public_ips: true

@maxveliaminov , we found a root cause, working on the fix, expect it to be fixed in develop early next week.

@maxveliaminov, #1406  contains a fix for this problem. Could you please build ghpc from develop branch and confirm if it fixes your problem? Please re-open the issue if needed.


build_docker_aspnetcore failing

https://source.cloud.google.com/results/invocations/933ae910-1f14-451d-a5bc-5a20210c563e/targets/grpc%2Fcore%2Fpull_request%2Flinux%2Fgrpc_interop_tocloud/log
https://source.cloud.google.com/results/invocations/8076f4e0-7dd5-482d-b8d5-e866f6785f2e/targets/grpc%2Fcore%2Fpull_request%2Flinux%2Fgrpc_interop_toprod/log
Downloading install script: https://dot.net/v1/dotnet-install.sh => /var/local/git/grpc-dotnet/build/obj/dotnet-install.sh
dotnet_install: Warning: Unable to locate libunwind. Probable prerequisite missing; please install libunwind.
dotnet-install: Downloading link: https://dotnetcli.azureedge.net/dotnet/Sdk/3.0.100-preview4-010737/dotnet-sdk-3.0.100-preview4-010737-linux-x64.tar.gz
dotnet-install: Extracting zip from https://dotnetcli.azureedge.net/dotnet/Sdk/3.0.100-preview4-010737/dotnet-sdk-3.0.100-preview4-010737-linux-x64.tar.gz
dotnet-install: Adding to current process PATH: `/root/.dotnet`. Note: This change will be visible only when sourcing script.
dotnet-install: Installation finished successfully.
Downloading https://packages.grpc.io/archive/2019/02/ac4c291773245d942d8542f3be4315e90d0bf62f-ff71de50-7499-4ccb-b404-04821dc2b11e/csharp/Grpc.Core.1.19.0-dev201902121041.nupkg
Downloading https://packages.grpc.io/archive/2019/02/ac4c291773245d942d8542f3be4315e90d0bf62f-ff71de50-7499-4ccb-b404-04821dc2b11e/csharp/Grpc.Core.Api.1.19.0-dev201902121041.nupkg
Downloading https://packages.grpc.io/archive/2019/02/ac4c291773245d942d8542f3be4315e90d0bf62f-ff71de50-7499-4ccb-b404-04821dc2b11e/csharp/Grpc.Tools.1.19.0-dev201902121041.nupkg
A compatible installed dotnet SDK for global.json version: [3.0.100-preview4-010737] from [/var/local/git/grpc-dotnet/global.json] was not found
Please install the [3.0.100-preview4-010737] SDK or update [/var/local/git/grpc-dotnet/global.json] with an installed dotnet SDK:
  3.0.100-preview3-010431 [/usr/share/dotnet/sdk]
2019-03-18 13:54:18,541 FAILED: build_docker_aspnetcore [ret=145, pid=9480, time=297.7sec]

Fixed by https://github.com/grpc/grpc/pull/18421

https://source.cloud.google.com/results/invocations/6765c773-a735-454f-b767-13881faff79c/details
A compatible installed dotnet SDK for global.json version: [3.0.100-preview4-011108] from [/var/local/git/grpc-dotnet/global.json] was not found
Please install the [3.0.100-preview4-011108] SDK or update [/var/local/git/grpc-dotnet/global.json] with an installed dotnet SDK:
  3.0.100-preview3-010431 [/usr/share/dotnet/sdk]

Fixed by https://github.com/grpc/grpc/pull/18714


Declare a fuchsia_asset_builder sdk_atom.

cc @alainv @pylaligand are you still working on this PR?

ping @pylaligand PTAL

Can someone help merge this? Thanks!

Taking another approach.


fix #1103 by not nesting the TypeaheadMatch object into an object

fix #1103 by not nesting the TypeaheadMatch object into an object

Current coverage is 85.88% (diff: 85.71%)

Merging #1105 into development will not change coverage

@@           development      #1105   diff @@
=============================================
  Files               35         35
  Lines             1275       1275
  Methods            227        227
  Messages             0          0
  Branches           212        212
=============================================
  Hits              1095       1095
  Misses             180        180
  Partials             0          0

Powered by Codecov. Last update b80c0b4...8fe433c

Thanks :+1:

you can use it already with
npm install ng2-bootstrap@beta --save


Add more sources to gene_burden

We want to study the inclusion of more resources that report relevant burden analyses results.
Context
The gene_burden source currently collates the results of 2 analyses: REGENERON and AZ PheWAS Portal. However, more research is being published on the topic.
Although we do not intend to collect all results that are being published, the idea is to identify key resources or research that:
a) are findings of high quality
b) the integration and maintenance can be done automatically and nearly painless (no changes to the schema, no excessive parsing,...)
This way Open Targets could become a go-to resource for the most relevant collapsing studies and perhaps make the case for the creation of a new catalogue.
Actions
Interesting ExWAS results to investigate

Association of 10 genes with schizophrenia. https://www.nature.com/articles/s41586-022-04556-w.pdf

Data is available in a nice format at the Schizophrenia Exome Sequencing Meta-analysis (SCHEMA) consortium website: https://schema.broadinstitute.org
There are 3 similar initiatives from which we could also pull data from:

Autism Sequencing Consortium (ASC): https://asc.broadinstitute.org
The Epilepsy collaboration: https://epi25.broadinstitute.org
The bipolar exome sequencing project (BipEx): https://bipex.broadinstitute.org

4 novel associations with CKD. https://jasn.asnjournals.org/content/30/6/1109
OTAR project with associations between 27 genes and 38 metabolites. https://www.cell.com/ajhg/fulltext/S0002-9297(22)00157-4?s=09#sectitle0025

Measurement of metabolites have already been added to EFO (https://github.com/EBISPOT/efo/issues/1257)

Burden results on metabolites using Nightingale's NMR on the INTERVAL cohort https://journals.plos.org/plosgenetics/article?id=10.1371/journal.pgen.1008605

Some notes and questions on the new data:
Schizophrenia Consortium
Manuscript: https://www.nature.com/articles/s41586-022-04556-w.pdf

Whole exomes of 24,248 schizophrenia cases and 97,322 controls
7 continental populations
10 gene associations that implicate ultra rare coding variants (MAC ≤ 5 - MAF ≤ 0,005% given a sample size of 121,570)

Class 1 variants: PTVs and missense variants with a MPC pathogenicity score > 3
Class 2 variants: missense variants with a MPC between 2 and 3

Burden tests were performed:

separately for Class I and Class II variants
together using Stouffer's weighted Z-score method

Significance threshold set at P < 2.14 × 10−6
The Q-value is the P-value adjusted for the False Discovery Rate.
Last updated: Sept. 2020

Questions

Do we want to report the Q number?
What should we do with the ancestries?
Cohort is the aggregation of 11 collections of schizophrenia patients.

Data:
 # Sample row
 gene_id                   | ENSG00000006283
 group                     | meta
 Case PTV                  | 10
 Ctrl PTV                  | 13
 Case mis3                 | 8
 Ctrl mis3                 | 4
 Case mis2                 | 55
 Ctrl mis2                 | 134
 P ca/co (Class 1)         | 3.1600e-06
 P ca/co (Class 2)         | 3.1700e-02
 P ca/co (comb)            | 4.5700e-07
 De novo PTV               | NA
 De novo mis3              | NA
 De novo mis2              | 1
 P de novo                 | 2.7900e-01
 P meta                    | 4.5700e-07
 Q meta                    | 1.5400e-03
 OR (PTV)                  | 3.0900e+00
 OR (Class I)              | 4.2500e+00
 OR (Class II)             | 1.6800e+00
 OR (PTV) lower bound      | 1.2100e+00
 OR (PTV) upper bound      | 7.6300e+00
 OR (Class I) lower bound  | 2.0700e+00
 OR (Class I) upper bound  | 8.7800e+00
 OR (Class II) lower bound | 1.2100e+00
 OR (Class II) upper bound | 2.3100e+00

# Translated to the schema

{
    "datasourceId": "gene_burden",
    "datatypeId": "genetic_association",
    "diseaseFromSourceMappedId": "MONDO_0005090",
    "pValueMantissa": 4.57,
    "pValueExponent": -7,
    "oddsRatio": ?,
    "oddsRatioConfidenceIntervalLower": ?,
    "oddsRatioConfidenceIntervalUpper": ?,
    "resourceScore": 4.57e-7,
    "ancestry": ?,
    "ancestryId": ?,
    "literature": [
        "35396579"
    ],
    "projectId": "SCHEMA consortium",
    "cohortId": null,
    "studyId": null,
    "studySampleSize": 121570,
    "studyCases": 24248,
    "studyCasesWithQualifyingVariants": 73,
    "statisticalMethod": ?,
    "statisticalMethodOverview": "Burden test carried out with PTV and missense variants with a MAF smaller than 0.005%."
}

Autism Sequencing Consortium data
Manuscript: https://www.biorxiv.org/content/10.1101/484113v3

De novo variants called in family-based data consisting of 6,430 probands with ASD and 2,179 unaffected siblings as well as rare variants called in 5,556 ASD cases and 8,809 ancestry-matched controls
Collapsing method is by using TADA, a Bayesian model
Last updated: ?

Questions
The data looks quite different. No OR, no P value... How should we go around it?
{
  "gene_id": "ENSG00000000938",
  "group": "All",
  "xcase_dn_ptv": 0, <--- cases with de novo PTVs
  "xcont_dn_ptv": 0,
  "xcase_dn_misb": 0, <--- cases with de novo missense variants with MPC between 1 and 2
  "xcont_dn_misb": 0,
  "xcase_dn_misa": 0, <--- cases with de novo missense variants with MPC > 2
  "xcont_dn_misa": 0,
  "xcase_dbs_ptv": 0, <--- cases with PTVs in the iPSYCH cohort
  "xcont_dbs_ptv": 0,
  "xcase_swe_ptv": 0, <--- cases with PTVs in the Swedish cohort
  "xcont_swe_ptv": 0,
  "xcase_tut": 0, <--- Protein-truncating variants transmitted to probands
  "xcont_tut": 2, <--- Protein-truncating variants not transmitted to probands
  "qval": 0.91586
}

4.2. Bipolar Exome sequencing project
Manuscript: https://www.medrxiv.org/content/10.1101/2<IP_ADDRESS>52930v2 - still a preprint

Exome sequencing of 14,210 bipolar cases with different subtypes and 14,422 controls
Qualifying variants are 2 types of ultra rare coding variants: PTVs and damaging missense variants (MAC ≤ 5 - MAF < 0,015% given a sample size of 28,632)

They are also only interested in variants not observed in a large control repository (gnomAD non-neuro).

Collapsing method is by using Fisher's test in each gene to test for case-control enrichment
No test hits the GWAS significance threshold (P < 2.14 × 10−6)
They also perform a metaanalysis with the Bipolar and the Schizophrenia cohorts, given their strong overlap in common variation, revealing one exome-wide significant gene, AKAP11 (P = 2.83 × 10-9), and one suggestive gene, ATP9A (P = 5.36 × 10-6).
Last updated: ?

Questions

Since the data in the portal won't show any significant results, we should be using the meta analysis from the preprint. Do we want to do that?

{
  "gene_id": "ENSG00000000003",
  "group": "Bipolar Disorder without Psychosis",
  "n_cases": 3812,
  "n_controls": 14422,
  "damaging_missense_case_count": 0,
  "damaging_missense_control_count": 1,
  "damaging_missense_fisher_gnom_non_psych_pval": 1,
  "damaging_missense_fisher_gnom_non_psych_OR": 0,
  "damaging_missense_fisher_gnom_non_psych_case_count": 0,
  "damaging_missense_fisher_gnom_non_psych_case_no_count": 3812,
  "damaging_missense_fisher_gnom_non_psych_control_count": 0,
  "damaging_missense_fisher_gnom_non_psych_control_no_count": 14422,
  "ptv_case_count": null,
  "ptv_control_count": null,
  "ptv_fisher_gnom_non_psych_pval": null,
  "ptv_fisher_gnom_non_psych_OR": null,
  "ptv_fisher_gnom_non_psych_case_count": null,
  "ptv_fisher_gnom_non_psych_case_no_count": null,
  "ptv_fisher_gnom_non_psych_control_count": null,
  "ptv_fisher_gnom_non_psych_control_no_count": null
}

Epilepsy collaboration
Aimed at characterizing the contribution of rare genetic variation to a spectrum of epilepsy syndromes to identify individual risk genes.
Manuscript: https://www.cell.com/ajhg/fulltext/S0002-9297(19)30207-1

Gene burden results of 9,170 cases and 8,364 controls broken down by three primary epilepsy types:

developmental and epileptic encephalopathy (DEE, N=1,021)
genetic generalized epilepsy (GGE, N=3,108)
non-acquired focal epilepsy (NAFE, N=3,597)

Qualifying variants are 3 types of ultra-rare deleterious variants: PTVs, missense variants with MPC≥2, and inframe indels not present in the DiscovEHR database.
Results show 2 genes trespassing the GWAS statistical significance: ENSG00000050030, ENSG00000144285
European ancestry
Last updated: 2018

Questions

Do we want to split the evidence by type of qualifying variants? For the following evidence, we could report the aggregation of the PTVs and missense, or report them one by one.

{
  "gene_id": "ENSG00000004139",
  "group": "DEE",
  "xcase_lof": 0,
  "xctrl_lof": 0,
  "pval_lof": 1,
  "xcase_mpc": 0,
  "xctrl_mpc": 2,
  "pval_mpc": 1,
  "xcase_infrIndel": 0,
  "xctrl_infrIndel": 0,
  "pval_infrIndel": null,
  "pval": 1
}

Notes about the OTAR associations
Manuscript: https://www.cell.com/ajhg/fulltext/S0002-9297(22)00157-4?s=09#sectitle0025
Blogpost: https://blog.opentargets.org/using-blood-metabolites-to-find-new-drug-targets/

WGS or WES of blood samples from healthy individuals from the INTERVAL cohort (English population)
40 novel associations implicating rare coding variants (27 genes and 38 metabolites), of which 28 (15 genes and 28 metabolites) were replicated.

Of these, 16 gene-metabolite associations were completely novel.
28 of these associations were also replicated using a non-overlapping sample of participants with WGS data from the INTERVAL study.

3 types of collapsing analyses:

"CODING": all coding region variants plus untranslated regions (UTRs) and essential splice sites
"MLOF": LoF and missense variants
"LOF": LoF variants predicted as high confidence by LOFTEE (loss-of-function transcript effect estimator) <-- this is coming from VEP

Data is available in the Supplementary Table 5, in the "Gene-based results" sheet.

Questions

Do we want to report the metabolic pathway?
The INTERVAL study is carried on English people. We would expect to have different ancestries covered, right?
The analyses is done on WES data and further replicated on WGS. We want to capture all associations statistically significant in WES, regardless they have been replicated, right?
If I understand correctly, the "Gene based results" and the "Strongest gene metabolite" tables differ in that the latter reports variant level summary stats.
Do we want to capture the direction of effect?

# Data in the Gene based results table
{
  "approach": "MLOF",
  "Gene_name": "ACADS",
  "P-value": 2,16E-12,
  "Test": "burden",
  "Biochemical": "butyrylcarnitine",
  "SuperPathway": "Lipid",
  "Pathway": "Fatty Acid Metabolism (also BCAA Metabolism)"
}

# Data in the strongest gene/metabolite table
{
  "Gene_Symbol": "ACADS",
  "Best_metabolite": "butyrylcarnitine",
  "Number of Metabolites": 2,
  "Additional Metabolites": "ethylmalonate",
  "SuperPathway": "Lipid",
  "Pathway": "Fatty Acid Metabolism (also BCAA Metabolism)",
  "N_driver_variant": 10,
  "Driver_variant_GRCh38": "12:120738329:G:A;12:120738416:TG:T;rs199717731;rs375272147;rs28941773;rs368469075;rs371934737;rs28940875;rs537072819;rs767774362",
  "Driver_variant_GRCh37": "12:121176132:G:A;12:121176219:TG:T;rs199717731;rs375272147;rs28941773;rs368469075;rs371934737;rs28940875;rs537072819;rs767774362",
  "N_informative": 3881,
  "EAF": "0.000128833;0.000130582;0.000128866;0.000128866;0.000257666;0.000257666;0.000128833;0.000128833;0.000257666;0.000129099",
  "Beta_SE": "2.0877 (1.0008);1.8332 (1.0008);2.9025 (1.0008);1.6642 (1.0008);2.6497 (0.7078);1.7352 (0.7078);1.967 (1.0008);1.1825 (1.0008);1.5426 (0.7078);1.6843 (1.0008)",
  "P-value": "3.69727e-02;6.69987e-02;3.72962e-03;9.63434e-02;1.81255e-04;1.42202e-02;4.93612e-02;2.37377e-01;2.92875e-02;9.23806e-02",
  "Best_RVT": "ACADS.w2_burden_naive",
  "Best_RVT_p-value (full model)": 2.3e-9,
  "Best_RVT_p-value (driver variants model)": 2,80567E-12,
  "Consequence": "missense_variant;frameshift_variant;missense_variant;synonymous_variant;missense_variant;missense_variant;missense_variant;missense_variant;missense_variant;stop_gained",
  "Gene Overlap Shin et al": "IEM(Shin)",
  "reported in mGWAS": "yes",
  "Mendelian Disease": "ACYL-CoA DEHYDROGENASE, SHORT-CHAIN, DEFICIENCY OF; ACADSD {OMIM:https:\\/\\/www.omim.org\\/entry\\/201470}",
  "Inheritance": "AR",
  "direction of effect": "pos",
  " consistent direction of effect": "yes",
  "biochemical explanation": "increased butyrylcarnitine level consistent with elevated butyrylcarnitine in patients with acyl-CoA dehydrogenase"
}

# Proposed schema

{
    "datasourceId": "gene_burden",
    "datatypeId": "genetic_association",
    "diseaseFromSourceMappedId": "EFO_0020010",
    "pValueMantissa": 2.16,
    "pValueExponent": -12,
    "resourceScore": 2,16E-12,
    "literature": [
        "35568032"
    ],
    "projectId": "OTAR022",
    "cohortId": "INTERVAL",
    "ancestry": null,
    "ancestryId": null,
    "studyId": null,
    "studySampleSize": 3881,
    "studyCases": null,
    "studyCasesWithQualifyingVariants": null,
    "statisticalMethod": "MLOF",
    "statisticalMethodOverview": "Burden test carried out with LoF and missense variants with a MAF smaller than 0.1%."
}

Genebass made a new release of its portal last week. In it their analysis was expanded from 280k UK Biobank samples to 450k, matching the sample size of REGENERON and AZ PheWAS Portal.
https://www.medrxiv.org/content/10.1101/2<IP_ADDRESS>59117v4
Since the portal is a reference for the community, we want these gene-based analyses to be part of the gene_burden source. The results are available as a Hail-processable MatrixTable at gs://ukbb-exome-public/500k/results/results.mt
Methods
We have been able to process the data and generate 8808 evidence strings out of it. The p value of statistical significance has been set to 6.7e-07, based on the methodology explained in their preprint.

Stats

Evidence count
8808

Associations count
7238

Targets count
2645

Traits count (EFO)
715

Traits count (Trait)
2358

They have done 4 different burden tests based on 4 sets of variants:

'pLoF': 'Burden test carried out with rare pLOF variants.',
'missense|LC': 'Burden test carried out with rare missense variants including low-confidence pLOF and in-frame indels.',
'synonymous': 'Burden test carried out with rare synonymous variants.',
'pLoF|missense|LC': 'Burden test carried out with pLOF or missense variants.',

Trait mapping
These traits had to be mapped to EFO and for that purpose we did a multi step strategy to come up with the better suggestion so that the manual mapping process became easier.

common_label. Suggestion is extracted from a previous mapping already present in OTP. The label of the trait is exactly the same.2.
ontoma. High quality suggestion from ontoma.
zooma. Medium and high quality suggestion from zooma.
burden_association. Suggestion is extracted from a known gene burden association for the target the trait is associated with. This causes a big explosion.

Here is the working sheet: https://docs.google.com/spreadsheets/d/1SjYXtfqUGJAipXly3RnD3tb0aEPd55BSdDwDGlxdZSM/edit#gid=581323175
This has been the most time consuming step (thanks to @d0choa, @DSuveges). The mappings tagged as valid in the spreadsheet have been imported to our disease curation table.

For 22.06 release we will be integrating all the ExWAS burden results described in the original post except for point # 4, the analysis on metabolites using Nightingale's NMR. This is expected to be in the next release and will also require effort in the mapping of the metabolomics traits.
So here is the extract on how the gene_burden evidence is split:
+--------------------+-----+
|           projectId|count|
+--------------------+-----+
|AstraZeneca PheWA...|17785|
|                null|    3| --> These are the CKD associations (point # 2)
|             OTAR022|   21|
|            Genebass| 8808|
|   SCHEMA consortium|    9|
|           REGENERON| 7755|
|Autism Sequencing...|  102|
| Epi25 collaborative|    2|
+--------------------+-----+

@ireneisdoomed I have some questions...

Does the new evidence bring in any new associations? (If so, from which study?)
Where the papers reported more associations than we imported, why were some of the associations dropped? e.g. Schizophrenia paper, 10 reported in the paper but 9 evidence strings; CKD paper, 4 reported but 3 evidence strings; Bomba et al. found 40 associations but we import 21...

@HelenaCornu I have some answers...

We have 1 465 new disease/target pairs that not only gene_burden is the only source supporting them, but that these come exclusively from the new burden analyses. Here's a breakdown of the contribution per study:

+--------------------+-----+
|           projectId|count|
+--------------------+-----+
|             OTAR022|   14|
|            Genebass| 1392|
|   SCHEMA consortium|    3|
|Autism Sequencing...|   55|
| Epi25 collaborative|    1|
+--------------------+-----+

Reasons why their reported associations and what we include are not the same differ:

Schizophrenia. I missed including the TRIO association. This will be available in the next release.
CKD. Only PKD1, PKD2, and COL4A5 show study-wide significance, the other reported 2 genes show suggestive signals. So we only cover the first 3.
Bomba et al. The data is based on Table 1 of their publication. 2 reasons:

Not all of the 27 associated genes result from burden tests, as SKAT is a regression-based approach that differs from the burden family analyses. We are not integrating SKAT results.
4 metabolite measurements were not in EFO and therefore we had to request them. Our disease index is based on the second to last EFO version, so the new terms are not yet part of the Platform but will be there for the next release.

One important note is that unfortunately, we shouldn't mention the Cameron-Christie et al publication (CKD) for now. The reported disease is specifically a congenital subtype of kidney disease. This term was not yet in EFO so we'd requested it. This is the same situation as above, this new term is not yet part of the disease index and therefore the evidence from this paper won't make it for this release.
Tomorrow I'll give you a nice example that you can mention.

We are expecting this work to be integrated in 22.06. I will close it for now and we'll bear it in mind for the 22.09 release - so that we integrate results from the Nightingale's NMR measurements (and perhaps more).

@HelenaCornu I think it would be nice to cite the examples mentioned in the publication.
We have examples of known associations, all captured in the Platform:

First, we recapitulate many known associations from previous studies, including associations between PCSK9 and LDL cholesterol (pLoF burden p = 3.5 x 10-132), COL1A1 and bone density (pLoF burden p = 2 x 10-9) (Mann et al., 2001), KLF1 and several red blood cell traits (pLoF burden < 2 x 10-12) (Perkins et al., 2016), and LRP5 (Wnt coreceptor) and bone density and osteoporosis phenotypes (pLoF burden < 5 x 10-7) (Baron and Rawadi, 2007).

And an example of a novel one, between SCRIB and white matter microstructure measurement,  which they claim to capture essentially thanks to the increment in statistical power of the burden test

We highlight novel biological signals identified in the exome dataset, enabled by the Genebass browser. In particular, we find an association between predicted loss-of-function of SCRIB and white matter integrity of tapetum. Notably, this association is not significant at any single pLoF variant, but when aggregated into a SKAT-O or burden group test.


Issue 39 support othellogpt in SAELens

Description
Adding support for OthelloGPT. Adding logic to train on sequences with a start and end position offset similar to PR 294. Adding tests for the added logic and to benchmark the OthelloGPT SAETrainingRunner.
Closes Issue 39
Fixes # (issue)
Type of change
Please delete options that are not relevant.

[x] New feature (non-breaking change which adds functionality)

Checklist:

[x] I have commented my code, particularly in hard-to-understand areas
[x] I have made corresponding changes to the documentation
[x] My changes generate no new warnings
[x] I have added tests that prove my fix is effective or that my feature works
[x] New and existing unit tests pass locally with my changes
[x] I have not rewritten tests relating to key interfaces which would affect backward compatibility

You have tested formatting, typing and unit tests (acceptance tests not currently in use)

[x] I have run make check-ci to check format and linting. (you can run make format to format code if needed.)

Thanks @decandido and @zhenningdavidliu!
This has some overlap with https://github.com/jbloomAus/SAELens/pull/294. Specifically, I'd like to use the seqpos slicing instead of the start and end pos offsets. Other than that this looks good I think!
Requests:

[] Please rebase or merge with https://github.com/jbloomAus/SAELens/pull/294 (which will also make sure the seqpos slice ends up in the config)
[] Optional: Do you want to make a short tutorial on training / using the OthelloGPT SAE?

Once that's done I'll accept :)

Requests:

[] Please rebase or merge with Support seqpos slicing #294 (which will also make sure the seqpos slice ends up in the config)
[] Optional: Do you want to make a short tutorial on training / using the OthelloGPT SAE?

Thanks @jbloomAus for reviewing our PR! We've rebased on #294 and removed the start and end pos offsets.
We also added a short script to train SAEs on othelloGPT (scripts/training_a_sparse_autoencoder_othelloGPT.py). Let us know if that's what you were looking for!

Hey, thanks for all your awesome work here! Could I possibly bump on this PR? Just cause it would also be super useful for the final bits of the ARENA material to run smoothly (-:

Thanks! (I'm actually also realizing that the context length issue wasn't a part of this PR, it was an earlier one, and that was the main thing that was making things a bit clunky, so less important thank I thought - but would still be great!)


Update Calculation for Users Est Account Value

Fixes #1121 .
Changes:

Remove steemit user est account value calculations and add our own using busy what busy v1 had: https://github.com/busyorg/busy/blob/v1/src/user/UserTransfers.js#L68

We should never see NaN, now it appear when loading the wallet:

@bonustrack should be updated now, lmk if you still see the NAN


Added Movie details scrapper Closes #452

Hi, thank you for your interest in the project! Just a reminder, this repository is focused specifically on the Python Voice Assistant. Please make sure that any issues or PRs are relevant to this project's scope. Unrelated features or suggestions should be directed elsewhere. Thanks for understanding!


Pull in updates

This PR just pulls in updates that have been reviewed elsewhere

Fix caching for write events
separate change and render events

So to clarify, you made the changes on WEBHMI and now we need to port them to LUX?

I made the callback updates to improve the debounce and invalid cache. Works without the omjson updates, but could be improved once we have those working.


Alexa slot synonym fix

Description:
In the new Alexa Skills Kit you can also create synonyms for slot type values, which can be used in place of the base value in utterances. With this commit, synonyms will be replaced with their associated slot value in the intent request sent to the Alexa API endpoint, but only if there are not multiple synonym matches. Otherwise, the value of the synonym that was spoken will be used.
Related issue: fixes #9075
Pull request in home-assistant.github.io with documentation: home-assistant/home-assistant.github.io#3995

Awesome! Thanks 🌮


Make it possible to use the empty string as the custom referer

At the moment this fails due to the falsiness of ''.

Thanks @fblundun

Why can't user set to null?