FOODON Imports fail to load

When loading FOODON using Protege or programmatically using Owlready, most imports fail to load:
  INFO  09:28:46  Loading ontology from https://raw.githubusercontent.com/FoodOntology/foodon/v0.5.1/foodon.owl
   INFO  09:28:46  URL connection input stream is compressed using gzip
   INFO  09:28:46  Imported ontology document http://purl.obolibrary.org/obo/bfo/2.0/bfo.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:47  URL connection input stream is compressed using gzip
   INFO  09:28:47  Finished loading imported ontology at http://purl.obolibrary.org/obo/bfo/2.0/bfo.owl
   INFO  09:28:47  Imported  document http://purl.obolibrary.org/obo/foodon/imports/bfo_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:47  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/bfo_import.owl
   INFO  09:28:47  Imported onontologytology document http://purl.obolibrary.org/obo/foodon/imports/chebi_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:47  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/chebi_import.owl
   INFO  09:28:47  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/deprecation_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:47  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/deprecation_import.owl
   INFO  09:28:47  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/envo_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:48  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/envo_import.owl
   INFO  09:28:48  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/foodon_product_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:48  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/foodon_product_import.owl
   INFO  09:28:48  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/gaz_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:48  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/gaz_import.owl
   INFO  09:28:48  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/geem_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:48  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/geem_import.owl
   INFO  09:28:48  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/general_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:49  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/general_import.owl
   INFO  09:28:49  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/metadata_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:49  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/metadata_import.owl
   INFO  09:28:49  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/ncbitaxon_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:49  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/ncbitaxon_import.owl
   INFO  09:28:49  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/product_type_import.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:49  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/product_type_import.owl
   INFO  09:28:49  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/robot_animals.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:49  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/robot_animals.owl
   INFO  09:28:49  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/robot_dietary_supplement.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:50  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/robot_dietary_supplement.owl
   INFO  09:28:50  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/robot_pasta.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:50  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/robot_pasta.owl
   INFO  09:28:50  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/robot_plant_parts.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:50  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/robot_plant_parts.owl
   INFO  09:28:50  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/robot_process.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:50  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/robot_process.owl
   INFO  09:28:50  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/robot_seafood.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:51  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/robot_seafood.owl
   INFO  09:28:51  Imported ontology document http://purl.obolibrary.org/obo/foodon/imports/robot_wine.owl was not resolved to any documents defined in the ontology catalog.
   INFO  09:28:51  Failed to load imported ontology at http://purl.obolibrary.org/obo/foodon/imports/robot_wine.owl

I see by this problem that there are obviously tools that won't check for local imports by default.  Going forward we have switched to having the root /foodon.owl file include all imports, but I see I need to make a new release (needed anyways) to make this simpler version available.  Will do so shortly.

Not sure if this is the same issue, but Lexmapr seems to struggle equally resulting in 0 terms in foodon:
Fetching and parsing http://purl.obolibrary.org/obo/foodon.owl ...
It has 0 import files ...
Metadata: {
    "resource": "http://purl.obolibrary.org/obo/foodon.owl",
    "title": "Food Ontology",
    "description": "FoodOn is a consortium-driven project to build a comprehensive and easily accessible global farm-to-fork ontology about food, that accurately and consistently describes foods commonly known in cultures from around the world. See https://foodon.org for more details.",
    "versionIRI": "http://purl.obolibrary.org/obo/foodon/releases/2023-05-03/foodon.owl",
    "prefix": "FOODON",
    "license": "https://creativecommons.org/licenses/by/3.0/",
    "date": "2023-06-14",
    "type": "ontology",
    "status": "release"
}
Doing term hierarchy query starting at: http://purl.obolibrary.org/obo/BFO_0000001
Doing terms: 0
´´´

I'm going to close this as resolved now that FoodOn's latest release no longer includes import files.
For LexMapr functionality, if there are remaining issues there, please open a separate issue.


chore(main): release 5.1.1

:robot: I have created a release beep boop
5.1.1 (2024-01-23)
Documentation

fix documentation latex formulas (#383) (#385) (582568b)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/rickstaa/stable-learning-control/releases/tag/v5.1.1 :sunflower:


CNAME support

Hi,
Thanks for a great tool! I wrote very similar one and I would like to send PR for some things that I don't see like CNAME support. Are you planning to add this? Is it ok if I work on it?

That would be great, that's on the list of planned features.
How would you implement that?
I would like to have SSL support for custom domains, but need to check how to do that.
For the command, original idea was scotty --domain example.com and scotty -d example.com, but feel free to recommend anything you thing would work better.
Also, share your lib if it's open source :)

Tool I worked on is at https://github.com/foundcenter/spajs. The idea is the same, to do quick setup and deployment of static websites/spa to aws.
Tool has two commands:

init setup bucket, cf distribution, ssl... with CNAME support. Also has support for different environments, all configs are saved under .spajs/ in current dir - so you can easily target: staging, testing, production - whatever env.
deploy take files from certain dir and sync with s3 bucket. This is implemented using couple of gulp tasks (I had to come up with something fast :) ) I've added also resource minification and versioning and deploy (maybe out of scope for this tool) using sync (awspublish module) - not just copy. The interesting part is caching since that is going to be crucial for performance.

Take a look tell me what you think :)
Now back to the issue, the flow would go like this:

If --domain is specified we would first check if there is a valid certificate on ACM, if there isn't that we would prompt the user if he want's to request one. The info what to do next would be displayed to the user like: Verification email has been sent to<EMAIL_ADDRESS>Please verify certificate request and re-run scotty. To get already verified SSL certificate from ACM take a look at: https://github.com/foundcenter/spajs/blob/f6ed94127cd75307c9c367ef4e0f99fde3f29120/src/initializer.ts#L187 The catch is that in order for certificate to be valid for CF distribution it needs to be requested in us-east-1.
Once certificate is setup scotty would needs to create bucket with name corresponding to domain and setup CNAME and custom certificate on CF. To set CF certificate take a look at: https://github.com/foundcenter/spajs/blob/f6ed94127cd75307c9c367ef4e0f99fde3f29120/src/initializer.ts#L174

Would this work?

I started for the dns pointing part with the following: https://github.com/ckaatz-here/scottyjs/commit/9a3c89eb977e8d8d52e51267f2cd3ba968844337 but it is in progress

Spajs looks nice :)
I'll try it a bit later today.
I think resource compression is out of scope for tool like this, it should be handled by some other tool before deployment, ie. webpack or gulp.
Flow looks good. In v2 we can optimize everything a bit more, but this sounds great for v1.
I need to write the plan for v2 somewhere, but we'll wait a bit more just to see if people are using Scotty before we do some of the improvements.
I'll close this issue and wait for PR :)
Thanks!

@ckaatz-here just saw your comment, looks like a good start. I have some small comments, but I'll wait for PR because they are not affecting the concept.
@amirilovic do you want to help @ckaatz-here, or? Not sure what's the best way to proceed, I didn't expect so many PRs for scotty haha :)

@ckaatz-here I see that you are focusing on r53, did you plan to add ssl support, do you need any help regarding this?

@amirilovic I did not intend to add ssl support but would love to get help with that part.

@ckaatz-here Great, I'll work on that.


Use lsp_types

Hey, I stumbled across your project while looking for some lsp client implementation for testing my server. Do you consider using the lsp_types crate instead of writing your own protocol implementation?

Good idea. Apologies. I was a bit busy these few weeks. I'll try to get this done this weekend.

Actually, I found a testing suite in the lib I was using. So no need to hurry. This is just a suggestion in case you didn't know about the library. Maybe you could contribute there, if there is something missing


Not able to select an option on iOS

I did some debugging and found out that the selected value is passed differently to the valueChange function on iOS.
Mind that I use the widget autocomplete set to false.
For instance, if you type "Ro" and see the option "Roy van der Valk" appearing and click on it then valueChange is normally triggered with value "Roy van der Valk" while on iOS it only contains the search string "Ro".
@JelteMX Have you got any idea why this is happening?

I found out that it has something to do with touch vs. click. When you press a bit longer on the option it seems to work.
I think maybe it has something to do with that there is a 300ms second delay after touching before the click event is fired. Dojo created a dojoClick property which says it makes it fire instantly (see: http://dojotoolkit.org/reference-guide/1.10/releasenotes/1.9.html#id15).
Could that be something that is happening?


Separate binary executables to different folders

As of 10/5/21, there are 30 binary executables generated in the bin directory with a mix of useful binaries, debug binaries, and automatic testing binaries, manual testing binaries, and irrelevant binaries. They can be put in separate directories so the functionality of each directory is clearer.

I've submitted a PR #2872 to remove the obviously obsolete test drivers. The remaining test drivers in the test_apps directory fall into two categories:

Test drivers in the smokeTests subdirectory are invoked by the python scripts contained therein during our CI automated tests. These drivers would be good candidates for memory analysis with valgrind. The python scripts demonstrate invocation syntax.
The remainder of the test drivers might be more appropriately named 'debug' drivers; they are used to troubleshoot problems in specific components and are constantly modified to meet a specific troubleshooting task. Some of these could, however, be fodder for future test drivers.

Assigning this back over to @shaomeng to create a 'debug' (or other) directory if desired.


Possible bug in eval.py & question about visualization

Hi Huy,
Thanks for sharing your code! I believe there may be a bug in the evaluation code: in the provided CLI command for OVSSC on the NYU dataset, there's a flag --merge_nyu_classes that isn't included in the list of arguments under utils.py::config_parser(). Further, the NYU dataset comes with a scc_split.pkl file, but the exp_dict accepts a vool_split.pkl by default.
I also wanted to confirm whether the example Habitat Matterport and ARKitScenes pickle files are included in the 269GB dataset? Under the visualization section, it's mentioned that those files should be provided under a folder named scene_files, but I don't seem to see it in the repository.

Thank you so much for your interest in our codebase and reporting all these issues!

I believe there may be a bug in the evaluation code: in the provided CLI command for OVSSC on the NYU dataset, there's a flag --merge_nyu_classes that isn't included in the list of arguments under utils.py::config_parser()

I've removed the line from the README, it was used in my internal codebase to merge the full 894 classes to 11 classes. I didn't include it because it's quite messy, but if you're interested, shoot me an email, I'm happy to send what I have over.

Further, the NYU dataset comes with a scc_split.pkl file, but the exp_dict accepts a vool_split.pkl by default

Fixed!

Under the visualization section, it's mentioned that those files should be provided under a folder named scene_files, but I don't seem to see it in the repository.

I've just pushed it! I didn't realized they were ignored by the .gitignore.


Host via Vercel

Host website on Vercel

Hey 👋, I haven't made any website for this repository yet. Would make one later on :D

Hey 👋, I haven't made any website for this repository yet. Would make one later on :D

Oh, I thought this was a website xD. If you host a website, I suggest Netlify for HTML, CSS, JS


Auto restart

Add feature to notifier, so it will restart every 2 hours to ensure stability of the software.

This shouldn't be necessary. Horizon is stable.

I request this feature because horizon closed by itself after few days of constantly running

Please be my guest in providing your log file so the issue can actually be solved rather than an auto-restart being thrown on it to hide the issue.

Will send log next time when this incident happen because I stopped using notifier and trade bot for awhile because of UUC incident.

Alright. What version were you on when the issue last occurred?

Closing due to irrelevancy. If you encounter the issue again, open a new Issue providing the log file.


Missing main and ignore entries in bower.json

bower jquery.xdomainrequest#~1.0.2 invalid-meta jquery.xdomainrequest is missing "main" entry in bower.json
bower jquery.xdomainrequest#~1.0.2 invalid-meta jquery.xdomainrequest is missing "ignore" entry in bower.json

I'm not familiar with how the bower.json should be set up.  Could you submit a pull request if you know how it should be done?  Thanks.


test releases against UDS-CLI

Releases should test integration with UDS-CLI (more to be assessed later) - this should be implemented by next release.

Does this not flip the dependency on its head? Might just be me but I have not built a project where the implementer tests downstream dependents. Could this not also be achieved through getting to a point where the exported SDK is stable and what we export is a conscious decision.


Added TPCH content

The tutorial for a standalone data lake has been updated to use TPCH sample data. This change adds the required sample files and updates the readme.

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.


margin text token?

I was wondering. how can i manage margin text (not spacers) via token?

What do you mean by margin text, can you give an example?


Growing Tree Algorithm

Hello!
I have made some changes on Maze.js file to add the new GrowingTree.js file.
For now, it is just behaving as the Recursive Backtracker. (.pop()) for selecting the next cell).
Do not merge yet.
I wanted also to know how can I do testing. I couldn't test with node local.js.
Regards!

Hi, looks good so far.
The only change that will have to be made to the tests is to add "growingtree" to the array of algorithms in main.test.ts.
The tests use Deno's built-in testing framework. I wouldn't worry about testing it yourself, though, as the tests are all very basic at the moment. However, if you want to run them, then you will have to download Deno and run deno test from inside the maze_generator folder.

Thanks for the info!
I'm trying to use the Deno extension so then the main.test.ts is not marked with errors.
I didn't see any .vscode/settings.json would you recomend to add .vscode/ it to the .gitignore?

Yes, sure: go ahead.

At last I did not updat the .gitignore... Not really needed.
Well, I added the algorithm to the tests and most important I changed the way to select and remove the Cell.
It just left to add a function to select the behavior, maybe from the maze settings can be sent another flag that only works with this algorithm.
I'll wait your response to know if you think this is a good implementation 😊

Nevermind, I came with this solution.
I think now we can discuss on which other selection methods can be added. (newest, oldest, middle and random are already working)
I also added the behavior flag (only works with Growing Tree) to choose the selection method in the maze settings.
And finally updated the README to show this new flag and its defaul value.

Thanks for doing this. I have tried it out and it works well. There are just a few small changes  I think you should make (tell me if you disagree):

Please can you change the default behaviour to "random". This will help distinguish between the recursive backtracker / depth-first search algorithm and the growing tree algorithm.
Rename behavior to something a little more descriptive (perhaps cellSelectionMethod). This doesn't matter too much as everything is still unstable so I might change the API a little anyway.
Throw an error (rather than defaulting when the behavior value is not valid.

I think the only thing left to do is add the option for combinations of selection methods, as you mentioned. I think the best way to do this would be to allow behavior to be an object with weights for each of the values.
For example:
{
  random: 0.5,
  oldest: 0.25,
  newest: 0.25
}

or
{
  middle: 10,
  newest: 40,
}

The first one would each time have a 50% chance of randomly selecting the cell, a 25% chance of selecting the oldest cell and a 25% chance of selecting the newest cell.
For the second one it would be 20% for middle and 40% for newest.
Also, I'm

Excellent! I'm on it.
For the "behavior" object the second example I assume that is %20 and %80 respectively, right?
Should I need to throw an error also when some property of the object is invalid or just ignore?

Whoops, that's what I meant.

Should I need to throw an error also when some property of the object is invalid or just ignore?

I think maybe throw an error if there are no valid properties and if there are some invalid ones as well as some valid ones, just ignore the invalid ones?

I think now is working after making the suggested changes.

[x] Changed flag name.
[x] Added options where random is the default and you can add more options with any type of weights (It takes the sum of all and calculates the probability).
[x] Added throw error when a cellSelectionMethod is invalid (ignoring when there is at least one valid).
[x] Updated README regarding these changes (fell free to modify it)

Please test it and tell me if you think is correct.

I was checking the code and there are two things that remain:

The throw "message" syntax, I think is wrong (correct me if not) but I think should be throw Error("message").
Throw an error if some weight is less or equal than 0.

It works well! I have a few minor changes I'd like to make as well but I think it will probably be easier if I just merge this now and make the changes myself, as they are all very small. I will also make the changes you suggested as well.
Thank you! This has been great. I'll add a hacktoberfest-accepted tag as well, in case you are doing that.

Fixes #2


Minor fixes for cadence-aws project

Fix statsd prefix for different services(so that it is possible to use '*' to select all operations for frontend/matching/history
Fix docker build issue #377

Coverage increased (+0.006%) to 66.339% when pulling ca3f9111ef8d9808618ab2761b96b04959c43a57 on longquanzheng:master into 949009edb5baf67d769887095cce449c5e84eed8 on uber:master.


Fix #2000.

Do not review.

@daejunpark please review. This should fix some of the issues found in kool and simple related to extra parenthesis.
There were several issues in the disambiguation filters, but the main change here was to modify the way we generate klabels automatically. More precisely, we removed the quote ' in front of the labels.
This should fix at least #2000 and #1935.


Updated DumpJob to create empty folders for znodes without any data.

Hi,
I've been using Guano to make backup copies of a ZooKeeper hierearchy and found that empty znodes where missed from the directory structure created. I'd like to persist these so have added a little extra code to create empty folders.
Please consider my change for including in Guano?
Kind regards
Steve

Hi, I've made the changes you suggested. Hopefully it's good to go?

Yup looks great; thanks for the contribution 😄


updated README.md

This is referenced to issue #13

Please review this PR and kindly notify me if any changes are requested

I added the images in a separate folder and also added screen shot when we execute python gui_main.py

now I added relative path for the images. Can you please review it now @djmgit


Cannot load metadata of NFT item when use deployNftCollection with not empty commonContent

I have a problem with commonContent. I cannot load metadata of NFT item when use deployNftCollection with NOT empty commonContent, but it work when commonContent = ''.
Can you guy help me explain about this case? Many thanks :)
Here is my code:
Deploy NFT Collection
...
const commonContent = await sdk.storage.uploadFile(
  Buffer.from(JSON.stringify(content), "utf-8")
);

const collection = await sdk.deployNftCollection(
  {
    collectionContent: content,
    commonContent: commonContent, // Issue here
  },
  {
    adminAddress: adminAddress,
  }
);
...

Deploy NFT Item:
...
const collection = sdk.openNftCollection(Address.parse(COLLECTION));

const content = Buffer.from(
  JSON.stringify({
    name: "Test NFT 1",
    description: "This is a test NFT 1",
    image: uploadedImageURL,
    "attributes":[{"trait_type":"Awesomeness","value":"Super cool"}],
  }), "utf-8"
);

const metadataURL = await sdk.storage.uploadFile(content);

const { nextItemIndex: index } = await collection.getData();
await collection.sendMint(sender, {
  index: index,
  owner: sdk.sender?.address!,
  individualContent: metadataURL,
});
...

hi ser any solution?

my solution is:
leave commonContent empty and add option param onchainContent: true when deployNftCollection
const collection = await sdk.deployNftCollection(   {     collectionContent: content,     commonContent: commonContent, // Issue here   },   {     adminAddress: adminAddress,     onchainContent: true   } );


fixing typo

Hi @4c74356b41, I'm your friendly neighborhood Azure Pull Request Bot (You can call me AZPRBOT). Thanks for your contribution!

This seems like a small (but important) contribution, so no contribution license agreement is required at this point. Real humans will now evaluate your PR.

TTYL, AZPRBOT;

@4c74356b41 : Thanks for your contribution to the Azure documentation! The author, @azdocprs, has been notified to review your proposed change.

@chipalost Hi Chipalo Please review this proposed change. Thanks.

Just a ping ;)

@4c74356b41 - it looks like there's now a merge conflict on this PR - can you fix the merge conflict and then we can accept your change?
Thanks, Dominic

#please-close
@4c74356b41 @chipalost - rather than you fixing the merge conflict - I've made the change directly.


[AMHACK] Cannot find the Manage Alert Types option in API Publisher

Description:
Cannot find a  MANAGE ALERT TYPES option in API publisher as specified in the document APIM 2.1.0
Suggested Labels:
Bug
Suggested Assignees:
Affected Product Version:
APIM WUM Updated - wso2am-<IP_ADDRESS>0585460470
OS, DB, other environment details and versions:
Standalone
DB - H2
Browser - Chrome Version 61.0.3163.100 (Official Build) (64-bit)
JDK - jdk1.8.0_121
Steps to reproduce:

Run the APIM 2.1.0 Wum updated pack (wso2am-<IP_ADDRESS>0585460470)
Log in to the API publisher using https://localhost:9443/publisher/    as a user who has the permission to publish a API
Search for the option Manage Alert Types in API publisher UI as specified in step 2 in [1]
Cannot find a option as a Manage Alert Types [2]

[1].https://docs.wso2.com/display/AM210/Subscribing+for+Alerts#e5983da0f69c4b5ca304dd809a002f57
[2].publisher-ui.png
Related Issues:

This issue was resolved with the correct configurations under [1]
[1]. https://docs.wso2.com/display/AM210/Configuring+Alerts


Add peak task user memory usage to query stats

This information is useful for debugging.

Approved by @dain at https://github.com/prestodb/presto/pull/12168
cc @sopel39

Merged, thanks!


Cannot open terminal through ssh connection

Type: Bug

Connect to remote ssh
open a folder
click terminal + new terminal
nothing reponds

VS Code version: Code - Insiders 1.90.0-insider (26c4a07b47a2bd34480a5e4cba2c2384c590b039, 2024-05-10T18:25:42.483Z)
OS version: Windows_NT x64 10.0.22621
Modes:
Remote OS version: Linux x64 5.15.0-105-generic

System Info

Item
Value

CPUs
12th Gen Intel(R) Core(TM) i9-12900H (20 x 2918)

GPU Status
2d_canvas: enabledcanvas_oop_rasterization: enabled_ondirect_rendering_display_compositor: disabled_off_okgpu_compositing: enabledmultiple_raster_threads: enabled_onopengl: enabled_onrasterization: enabledraw_draw: disabled_off_okskia_graphite: disabled_offvideo_decode: enabledvideo_encode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabledwebgpu: enabled

Load (avg)
undefined

Memory (System)
31.71GB (12.20GB free)

Process Argv
--crash-reporter-id 62fdc735-8859-448b-b429-b08e2b716358

Screen Reader
no

VM
0%

Item
Value

Remote
SSH: <IP_ADDRESS>

OS
Linux x64 5.15.0-105-generic

CPUs
AMD Ryzen 5 5600X 6-Core Processor (12 x 3713)

Memory (System)
46.97GB (40.75GB free)

VM
0%

Extensions (26)

Extension
Author (truncated)
Version

vsc-python-indent
Kev
1.18.0

jupyter-keymap
ms-
1.1.2

remote-containers
ms-
0.365.0

remote-ssh
ms-
0.112.2024050815

remote-ssh-edit
ms-
0.86.0

remote-wsl
ms-
0.88.2

remote-explorer
ms-
0.5.2024031109

open-in-browser
tec
2.0.0

pdf
tom
1.2.2

msft-todo-unofficial
tyl
0.0.18

prettier-vscode
esb
10.4.0

chatgpt-copilot
fei
4.4.1

copilot
Git
1.190.0

copilot-chat
Git
0.16.2024051001

ipython
Hoa
2024.3.0

vscode-docker
ms-
1.29.1

black-formatter
ms-
2024.2.0

debugpy
ms-
2024.6.0

python
ms-
2024.7.11311010

vscode-pylance
ms-
2024.5.100

jupyter
ms-
2024.4.2024042601

jupyter-keymap
ms-
1.1.2

jupyter-renderers
ms-
1.0.17

tensorboard
ms-
2023.10.1002992421

vscode-jupyter-cell-tags
ms-
0.1.9

vscode-jupyter-slideshow
ms-
0.1.6

A/B Experiments
vsliv368cf:30146710
vspor879:30202332
vspor708:30202333
vspor363:30204092
tftest:31042121
vswsl492:30256197
vscod805:30301674
vsaa593cf:30376535
py29gd2263:31024238
vscaat:30438846
c4g48928:30535728
vscrp:30624060
962ge761:30841072
pythongtdpath:30726887
welcomedialog:30812478
pythonidxpt:30768918
pythonnoceb:30776497
asynctok:30898717
dsvsc013:30777762
dsvsc014:30777825
dsvsc015:30821418
pythontestfixt:30866404
pythonregdiag2:30926734
pyreplss1:30879911
pythonmypyd1:30859725
pythoncet0:30859736
h48ei257:31000450
pythontbext0:30879054
accentitlementst:30870582
dsvsc016:30879898
dsvsc017:30880771
dsvsc018:30880772
cppperfnew:30980852
pythonait:30973460
showvideot:31016890
chatpanelt:31014475
bdiig495:31013172
a69g1124:31018687
dvdeprecation:31040973
pythonprt:31036556
dwnewjupytercf:31039676

Can you share your settings.json file and also do a trace log while creating a terminal? https://github.com/microsoft/vscode/wiki/Terminal-Issues#enabling-trace-logging
/label info-needed terminal

settings.json
{
"liveServer.settings.AdvanceCustomBrowserCmdLine": "Chrome",
"editor.inlineSuggest.enabled": true,
"security.workspace.trust.untrustedFiles": "open",
"files.autoSave": "afterDelay",
"workbench.editorAssociations": {
"*.pdf": "default"
},
"remote.SSH.remotePlatform": {
"<IP_ADDRESS>": "linux",
"<IP_ADDRESS>": "linux",
"<IP_ADDRESS>": "linux",
"<IP_ADDRESS>": "linux"
},
"jupyter.askForKernelRestart": false,
"git.autofetch": true,
"open-in-browser.default": "Chrome",
"liveServer.settings.donotShowInfoMsg": true
}

I am not sure if this is the correct terminal log:
2024-05-13 14:53:59.272 [info] Latency measurements for local backend
window<->ptyhost (message port): 0.40ms
window<->ptyhostservice<->ptyhost: 1.00ms
ptyhostservice<->ptyhost: 0.42ms []
2024-05-13 14:54:06.113 [info] Latency measurements for ssh-remote+<IP_ADDRESS> backend
window<->ptyhostservice<->ptyhost: 457.10ms
ptyhostservice<->ptyhost: 0.58ms []
2024-05-13 14:54:06.113 [info] Latency measurements for ssh-remote+<IP_ADDRESS> backend
window<->ptyhostservice<->ptyhost: 456.70ms
ptyhostservice<->ptyhost: 0.51ms []
2024-05-13 14:54:06.113 [info] Latency measurements for ssh-remote+<IP_ADDRESS> backend
window<->ptyhostservice<->ptyhost: 456.70ms
ptyhostservice<->ptyhost: 0.47ms []
2024-05-13 14:54:06.113 [info] Latency measurements for ssh-remote+<IP_ADDRESS> backend
window<->ptyhostservice<->ptyhost: 456.70ms
ptyhostservice<->ptyhost: 0.45ms []
2024-05-13 14:55:13.813 [trace] [6d23137] terminalInstance#ctor (instanceId: 9) [{}]
2024-05-13 14:55:13.847 [debug] [6d23137] Unbinding from mouse events. []
2024-05-13 14:55:13.865 [trace] [6d23137] Webgl was loaded []
2024-05-13 14:55:14.183 [debug] [6d23137] terminalInstance added capability [1]
2024-05-13 14:55:14.184 [trace] [6d23137] Spawning remote agent process [{"terminalId":31}]
2024-05-13 14:55:14.231 [debug] [6d23137] Terminal process ready (shellProcessId: 147043) []
2024-05-13 14:55:14.237 [debug] [6d23137] terminalInstance added capability [3]
2024-05-13 14:55:14.364 [debug] [6d23137] parsing data "]633;P;ContinuationPrompt=> " [[27,93,54,51,51,59,80,59,67,111,110,116,105,110,117,97,116,105,111,110,80,114,111,109,112,116,61,62,32,7]]
2024-05-13 14:55:14.365 [debug] [6d23137] terminalInstance added capability [2]
2024-05-13 14:55:14.610 [debug] [6d23137] Terminal process exit ["instanceId",9,"code",0,"processState",4]
2024-05-13 14:55:14.610 [trace] [6d23137] terminalInstance#dispose (instanceId: 9) []

thanks for that info, here's the meaning of process state:4
https://github.com/microsoft/vscode/blob/89b7b824ff5db9106fd3b624c6241c3149361111/src/vs/workbench/contrib/terminal/common/terminal.ts#L316-L318

Similar issue here, but with zsh crashing on open
Process state 6, exit code 1. Opens fine using plain ssh connection and the executable in error:

does disabling shell integration fix it for you, terminal.integrated.shellIntegration.enabled?

It seems that adding terminal.integrated.shellIntegration.enabled in the settings.json works for me. Thank you so much. However, I cannot open the second terminal once I connect to the remote server. I have to delete all terminals, and then everything works fine.

/duplicate #212678


os/exec: undocumented breaking change in exec.Command behavior on Windows between Go versions 1.16.8 and 1.17.1

What version of Go are you using (go version)?

$ go version
go version go1.17.1 windows/amd64

Does this issue reproduce with the latest release?
Yes.
What operating system and processor architecture are you using (go env)?
go env Output
$ go env
go env
set GO111MODULE=
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\mbushkov\AppData\Local\go-build
set GOENV=C:\Users\mbushkov\AppData\Roaming\go\env
set GOEXE=.exe
set GOEXPERIMENT=
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=C:\Users\mbushkov\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=C:\Users\mbushkov\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=C:\Program Files\Go
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=C:\Program Files\Go\pkg\tool\windows_amd64
set GOVCS=
set GOVERSION=go1.17.1
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\mbushkov\AppData\Local\Temp\2\go-build3022852068=/tmp/go-build -gno-record-gcc-switches

What did you do?
On Windows, when using exec.Command and relying on inheriting handles and using inherited handles in the child process, code that worked when compiled Go 1.16.8 no longer works when compiled with Go 1.17.1.
The culprit seems to be the AdditionalInheritedHandles attribute introduced in the context of  https://github.com/golang/go/issues/44011, and particularly the following breaking change: https://github.com/golang/go/commit/2d760816ff30bea82f54682f3049cfb6c6027da7#diff-ec673c10a75fe2d2faa9c788e03823294b263c68cc3de50f4a0865a53e28f3a3R340. In Go 1.16, on Windows, child processes inherit all handles marked as inherited. In Go 1.17 the handles to be inherited have to be explicitly added to the AdditionalInheritedHandles list.
Sample program:
package main

import (
	"fmt"
	"log"
	"os"
	"os/exec"
	"strconv"
	"strings"
	"syscall"
)

func main() {
	var in_handle *os.File = nil

	for _, e := range os.Environ() {
		pair := strings.SplitN(e, "=", 2)
		if pair[0] == "INHERITED_FD" {
			fd, err := strconv.Atoi(pair[1])
			if err != nil {
				log.Fatalf("Can't parse INHERITED_FD")
			}
			in_handle = os.NewFile(uintptr(fd), "file")
		}
	}

	if in_handle == nil {
		self_exe := os.Args[0]
		fmt.Printf("Haven't seen INHERITED_FD: assuming being a parent process, starting child with %s\n", self_exe)

		pr, pw, err := os.Pipe()
		if err != nil {
			log.Fatalf("Can't create pipe")
		}

		cmd := exec.Command(self_exe)

		fd := pr.Fd()
		syscall.SetHandleInformation(syscall.Handle(fd), syscall.HANDLE_FLAG_INHERIT, 1)
		cmd.SysProcAttr = &syscall.SysProcAttr{}
		// NOTE: The code fails without this line on 17.1:
		// cmd.SysProcAttr.AdditionalInheritedHandles = append(cmd.SysProcAttr.AdditionalInheritedHandles, syscall.Handle(fd))

		cmd.Env = append(os.Environ(), fmt.Sprintf("%s=%d", "INHERITED_FD", fd))

		cmd.Stdout = os.Stdout
		cmd.Stderr = os.Stderr
		if err := cmd.Start(); err != nil {
			log.Fatalf("Couldn't start: %v", err)
		}

		n, err := pw.Write([]byte("blah"))
		if err != nil {
			log.Fatalf("Couldn't write to pipe")
		}
		fmt.Printf("Written %d bytes\n", n)
		pw.Close()

		if err := cmd.Wait(); err != nil {
			log.Fatalf("Waiting for process failed: %v", err)
		}
	} else {
		fmt.Printf("Found INHERITED_FD (%v), assuming being a child process\n", in_handle.Fd())
		data := make([]byte, 4)
		n, err := in_handle.Read(data)
		if err != nil {
			log.Fatalf("Can't read data from inherited fd")
		}
		fmt.Printf("Read %d bytes: %v\n", n, string(data))
	}
}

Running this on different versions of Go produces different results:
$ go1.16.8 build inherited_attrs.go

$ inherited_attrs.exe
Haven't seen INHERITED_FD: assuming being a parent process, starting child with inherited_attrs.exe
Written 4 bytes
Found INHERITED_FD (312), assuming being a child process
Read 4 bytes: blah

$ go1.17.1 build inherited_attrs.go

$ inherited_attrs.exe
Haven't seen INHERITED_FD: assuming being a parent process, starting child with inherited_attrs.exe
Written 4 bytes
Found INHERITED_FD (320), assuming being a child process
2021/09/14 22:03:08 Can't read data from inherited fd
2021/09/14 22:03:08 Waiting for process failed: exit status 1

What did you expect to see?
I expected one of the following:

Either that the output of the code compiled with Go 1.16.8 and Go 1.17.1 to be the same
Or that the breaking change in exec.Command's behavior on Windows to be clearly marked as such in release notes.

What did you see instead?
Same code compiled with Go 1.16.8 and Go 1.17.1 exhibits different behavior.
The underlying breaking change in Go's behavior is not clearly marked as such in Go 1.17 release notes. The current document states:

The Windows version of SysProcAttr has two new fields. AdditionalInheritedHandles is a list of additional handles to be inherited by the new child process. ParentProcess permits specifying the parent process of the new process.

This is misleading and seems to actually be incorrect, as only the handles list in AdditionalInheritedHandles are going to be inherited by the new child process.

/cc @zx2c4 @alexbrainman

Is this a duplicate of #48040?

I can submit a patch to make the release notes more clear, I guess?

I get this impression that this shouldn't be in the 1.18 milestone. Maybe the next 1.17 milestone? (We're currently trying to clear out 1.18 now that we've entered the freeze.)

I think this was fixed by CL 350416.

I think the original issue reported here has not been fixed. The release notes still only say: "AdditionalInheritedHandles is a list of additional handles to be inherited by the new child process." The issue being reported here is that prior to go 1.17, all inheritable file handles would be inherited (if NoInheritHandles is false), whereas in 1.17 only stdin/out/err and the handles listed in AdditionalInheritedHandles are inherited. This behavior change is not mentioned anywhere in the release notes.
(It also makes it much harder for our go binary that wants to transparently pass all inheritable handles through to the subprocess; I can file a separate issue about that if you'd prefer).

I think the original issue reported here has not been fixed. ... This behavior change is not mentioned anywhere in the release notes.

Agree. The release notes did not get updated. Jason started https://golang.org/cl/350412 but abandoned it.
@brianryner8 would you like to change the docs yourself? Here is how to contribute https://go.dev/doc/contribute
@bcmills I will reopen this issue. Feel free to close it if you disagree and think the doco is good enough.

(It also makes it much harder for our go binary that wants to transparently pass all inheritable handles through to the subprocess; I can file a separate issue about that if you'd prefer).

I don't think you want every inheritable handle of your parent process to silently be inherited by every child. This will make a lot of unexplained bugs, like "unclosed handles". But please file a new issue, if you still have any bug or suggestion to discuss.
Thank you.
Alex


Madison workshop

Start planning the Madison workshop.

Looks good to me. I'll wait for the checks to pass since you added some AppVeyor fixes in here.

uh...

+1


Error when running command "az vmss update"

Describe the bug
Command Name
az vmss update --resource-group myrgname --name myssname --add virtualMachineProfile.networkProfile.networkInterfaceConfigurations[0].ipConfigurations[0].ApplicationGatewayBackendAddressPools '{"id": "/subscriptions/mysubid/resourceGroups/myrgname/providers/Microsoft.Network/applicationGateways/Myappgwname/backendAddressPools/mybpnanme"}'
Errors:
Unable to build a model: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get', DeserializationError: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get'
Traceback (most recent call last):
python3.6/site-packages/msrest/serialization.py, ln 1291, in _deserialize
    found_value = key_extractor(attr, attr_desc, data)
python3.6/site-packages/msrest/serialization.py, ln 1081, in rest_key_case_insensitive_extractor
    return attribute_key_case_insensitive_extractor(key, None, working_data)
...

python3.6/site-packages/msrest/serialization.py, ln 1104, in attribute_key_case_insensitive_extractor
    return data.get(found_key)
msrest.exceptions.SerializationError: Unable to build a model: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get', DeserializationError: Unable to deserialize to object: type, AttributeError: 'str' object has no attribute 'get'

To Reproduce:
Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

Put any pre-requisite steps here...
az vmss update --resource-group myrgname --name myssname --add virtualMachineProfile.networkProfile.networkInterfaceConfigurations[0].ipConfigurations[0].ApplicationGatewayBackendAddressPools '{"id": "/subscriptions/mysubid/resourceGroups/myrgname/providers/Microsoft.Network/applicationGateways/Myappgwname/backendAddressPools/mybpnanme"}'

Expected Behavior
Environment Summary
Linux-4.15.0-1089-azure-x86_64-with-debian-stretch-sid
Python 3.6.10
Installer: DEB

azure-cli 2.8.0

Additional Context

hi @qwordy could you pls have a look

Hi vishal8k , please try this command
az vmss update --resource-group myrgname --name myssname --add virtualMachineProfile.networkProfile.networkInterfaceConfigurations[0].ipConfigurations[0].ApplicationGatewayBackendAddressPools id="/subscriptions/mysubid/resourceGroups/myrgname/providers/Microsoft.Network/applicationGateways/Myappgwname/backendAddressPools/mybpnanme"

Any update?

Close the issue. Please open a new issue if there is any problem.


Tracking uncommitted but impending contributions

I am currently preparing the branch for Compiler Design, so just wanted to let the users know, so that there is no confusion. I think this issue can be used to declare upcoming work, until a time, that the repo is no longer in active development. What say @AB1908

That's a good idea for now. I'll add this to the contributing guidelines. Meanwhile, I've cooked up a few other ideas as well:

In your fork, you could make a commit modifying the readme where you'd add the topic and label it as upcoming with your GitHub handle and create a pull request for that. As soon as I merge that PR, everyone gets a heads up about what's coming up and who's contributing it!
Fork and create a pull request immediately with the appropriate label. I'll include in the readme to check PRs for upcoming content.

Any other suggestions?

I am currently preparing the branch for Compiler Design

I felt an itch to dive into it just a few hours ago. Thanks for the heads up.

Seems good. Also, the compiler design branch along will be added in a short while. I ran into some other work, so haven't been able to complete.

No problem. I'm busy as well, so I've only been creating branches and making minimal changes.

I guess we are almost done with courses until the 6th sem. There.s just DBMS left? I'll take care of it, if you haven't started yet, @AB1908

I have already created a branch. You can continue from there.

Seems like my branch was local and I hadn't pushed the changes. I'm working on it now anyway, so let me know if you want to do it.

@AB1908 , got a couple of hours free at night. It shouldn't take too long anyway. So its your call. Anyway, if you have progressed significantly, its better that you push your commits, and if I have to suggest any changes, I'll PR the changes later.

Sure thing


issues: added in option

Adding the in option to groups and projects when getting issues, as it is a functionality that is provided but not documented.
Have made a MR at Gitlab for documenting this - https://gitlab.com/gitlab-org/gitlab-ce/merge_requests/30053 .

My mistake @svanharmelen , should be fixed and squashed now to keep the history clean!

All good! Thanks @eyJhb!


[Performance]: speed regression 0.6.2 => 0.6.3?

Report of performance regression
Using your benchmark
git clone https://github.com/vllm-project/vllm
cd vllm/benchmarks
wget https://huggingface.co/datasets/anon8231489123/ShareGPT_Vicuna_unfiltered/resolve/main/ShareGPT_V3_unfiltered_cleaned_split.json
mkdir results
python benchmark_serving.py \
    --backend vllm \
    --model meta-llama/Meta-Llama-3-8B-Instruct \
    --dataset-name sharegpt \
    --dataset-path ShareGPT_V3_unfiltered_cleaned_split.json \
    --port 9999 \
    --save-result \
    --result-dir results \
    --result-filename test.json \
    --num-prompts 2000 \
    --request-rate inf \
    --seed 42

vllm==0.6.2
python -m vllm.entrypoints.openai.api_server \
    --host <IP_ADDRESS> --port 9999 \
    --model meta-llama/Meta-Llama-3-8B-Instruct \
    --tokenizer meta-llama/Meta-Llama-3-8B-Instruct \
    --dtype=bfloat16 \
    --seed 42 \
    --num-scheduler-steps 8 \
    --disable-log-requests \
    -tp 2

============ Serving Benchmark Result ============
Successful requests:                     2000
Benchmark duration (s):                  37.56
Total input tokens:                      453502
Total generated tokens:                  377235
Request throughput (req/s):              53.24
Output token throughput (tok/s):         10042.39
Total Token throughput (tok/s):          22115.08
---------------Time to First Token----------------
Mean TTFT (ms):                          13418.88
Median TTFT (ms):                        13693.80
P99 TTFT (ms):                           27527.70
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          19.64
Median TPOT (ms):                        18.37
P99 TPOT (ms):                           74.99
---------------Inter-token Latency----------------
Mean ITL (ms):                           137.43
Median ITL (ms):                         136.93
P99 ITL (ms):                            506.59
==================================================

vllm==0.6.3.post1
python -m vllm.entrypoints.openai.api_server \
    --host <IP_ADDRESS> --port 9999 \
    --model meta-llama/Meta-Llama-3-8B-Instruct \
    --tokenizer meta-llama/Meta-Llama-3-8B-Instruct \
    --dtype=bfloat16 \
    --seed 42 \
    --num-scheduler-steps 8 \
    --disable-log-requests \
    -tp 2

============ Serving Benchmark Result ============
Successful requests:                     2000
Benchmark duration (s):                  43.14
Total input tokens:                      453502
Total generated tokens:                  378114
Request throughput (req/s):              46.36
Output token throughput (tok/s):         8764.77
Total Token throughput (tok/s):          19277.04
---------------Time to First Token----------------
Mean TTFT (ms):                          16705.17
Median TTFT (ms):                        15436.29
P99 TTFT (ms):                           36631.51
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          36.76
Median TPOT (ms):                        33.73
P99 TPOT (ms):                           110.12
---------------Inter-token Latency----------------
Mean ITL (ms):                           30.63
Median ITL (ms):                         23.22
P99 ITL (ms):                            318.24
==================================================

Thanks.
Using 2x H100s

@stas00 could you try also adding --multi-step-stream-outputs=False? This will mean tokens are streamed back in chunks of 8 (as was the case by default I think in 0.6.2), rather than individually.

Thank you, @njhill
Indeed adding --multi-step-stream-outputs=False brings it back into the 0.6.2 ballpark - thank you!

How can the users keep track of the default knobs being turned on/off between releases? (other than reading each release's notes)
Do you plan to have a page where you discuss which optimization flags should be turned on/off/specific-values for which use-cases?

vllm==0.6.3.post1 w/ --multi-step-stream-outputs=False
python -m vllm.entrypoints.openai.api_server \
    --host <IP_ADDRESS> --port 9999 \
    --model meta-llama/Meta-Llama-3-8B-Instruct \
    --tokenizer meta-llama/Meta-Llama-3-8B-Instruct \
    --dtype=bfloat16 \
    --seed 42 \
    --num-scheduler-steps 8 \
    --multi-step-stream-outputs=False \
    --disable-log-requests \
    -tp 2

============ Serving Benchmark Result ============
Successful requests:                     2000
Benchmark duration (s):                  39.33
Total input tokens:                      453502
Total generated tokens:                  377187
Request throughput (req/s):              50.85
Output token throughput (tok/s):         9590.81
Total Token throughput (tok/s):          21122.10
---------------Time to First Token----------------
Mean TTFT (ms):                          13810.20
Median TTFT (ms):                        12929.04
P99 TTFT (ms):                           28807.02
-----Time per Output Token (excl. 1st token)------
Mean TPOT (ms):                          20.19
Median TPOT (ms):                        19.50
P99 TPOT (ms):                           73.49
---------------Inter-token Latency----------------
Mean ITL (ms):                           144.42
Median ITL (ms):                         145.11
P99 ITL (ms):                            533.96
==================================================

Thank you, @njhill
Indeed adding --multi-step-stream-outputs=False brings it back into the 0.6.2 ballpark - thank you!

How can the users keep track of the default knobs being turned on/off between releases? (other than reading each release's notes)

I guess we should make this much clearer in the release notes, in general though I think we try not to change the default behaviour. This particular default change was actually the subject of some debate between myself and @mgoin :)
The hope is that we can make some more follow-on changes to hide/mitigate this additional overhead, but just didn't manage to get to that in time for the release. The rationale for it is to match the non-multistep behaviour when streaming of having a response message per output token. However what's interesting is that I would have expected the TTFT at least to have improved but it seems to have also got worse. I will look into this more closely.

Do you plan to have a page where you discuss which optimization flags should be turned on/off/specific-values for which use-cases?

I'm not sure that we do have something clear/comprehensive, partly because all of these optimizations are changing so fast and there are many permutations depending on model / GPU / topology, etc. But we definitely should have such a guide imo.

Also this isn't related to the version change, but the serving benchmark always uses response streaming assuming it's supported by the server. If you are interested more in the non-interactive case then you may be able to improve performance further by disabling that in the test, change from True to False here.
Thank you for doing these experiments by the way!

Hmm, I wonder if perhaps at the very least you could maintain a single page with a few recipes for the common use cases? So as you are creating new performance improvements you have to measure those anyway - so why not share that process with the users? Just an idea of course.
To clarify - rather than asking for an exhaustive performance flags doc - just have something like:
best flags recipe for:

online serving: smallest TTFT / highest throughput
offline gen: highest throughput

Surely there might be a few more common goals, but those 3 could probably be the best low-hanging fruit?

Thank you for doing these experiments by the way!

The pleasure is all mine. I enjoy running benchmarks and finding ways to make things faster ;)

Also this isn't related to the version change, but the serving benchmark always uses response streaming assuming it's supported by the server. If you are interested more in the non-interactive case then you may be able to improve performance further by disabling that in the test, change from True to False here.

It's probably more than that since changing the flag makes the util fail:
--- a/benchmarks/backend_request_func.py
+++ b/benchmarks/backend_request_func.py
@@ -238,7 +238,7 @@ async def async_request_openai_completions(
             "best_of": request_func_input.best_of,
             "max_tokens": request_func_input.output_len,
             "logprobs": request_func_input.logprobs,
-            "stream": True,
+            "stream": False,
             "ignore_eos": request_func_input.ignore_eos,
         }
         headers = {

Traceback (most recent call last):
File "/data/stas/faster-ttft/core/dawn/exp/infer/faster-ttft/vllm/benchmarks/benchmark_serving.py", line 966, in
main(args)
File "/data/stas/faster-ttft/core/dawn/exp/infer/faster-ttft/vllm/benchmarks/benchmark_serving.py", line 667, in main
benchmark_result = asyncio.run(
File "/env/lib/conda/stas-inference/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/env/lib/conda/stas-inference/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/data/stas/faster-ttft/core/dawn/exp/infer/faster-ttft/vllm/benchmarks/benchmark_serving.py", line 426, in benchmark
raise ValueError(
ValueError: Initial test run failed - Please make sure benchmark arguments are correctly specified. Error: Traceback (most recent call last):
File "/data/stas/faster-ttft/core/dawn/exp/infer/faster-ttft/vllm/benchmarks/backend_request_func.py", line 291, in async_request_openai_completions
output.latency = latency
UnboundLocalError: local variable 'latency' referenced before assignment

Yes completely agree about the kind of page you're describing. I'll open an issue for this.

It's probably more than that since changing the flag makes the util fail:

Ah apologies, I'll check and get back to you, I don't expect it would be much more that needs changing. We can hopefully add it as a param to the script.

Thank you for finally addressing this issue!
We really need a simple guide to achieve optimal speed. The versions of vLLM iterate quickly, but the updated features and their corresponding parameters are not clearly documented. Here are some parameters that seem related to performance, but as an average user with limited knowledge of model architecture, it's hard to understand how to configure them:
--enable-prefix-caching
--enable-chunked-prefill
--max-num-batched-tokens
--max-num-seqs
--max-seq-len-to-capture
--num-scheduler-steps
--multi-step-stream-outputs

I understand that different model architectures require different configurations and experimentation, but it would be incredibly helpful if there were at least a basic  recipe for common models like Llama、Mistral that users could follow.

Just wanted to follow up on this issue so that it does not get stale, but is there somewhere we can track the progress of the creation of a "best practices/paramaters" recipe/documentation for scenarios where we need lowest latency or highest throughput? Having it documented by the maintainer team will help reduce the noise and ambiguity from similar performance-related questions.
cc @simon-mo @mgoin @njhill @DarkLight1337


Code editor: select next occurrence after Replace

Related to https://github.com/godotengine/godot/issues/32795

Thanks!


OS does not start up even

OS does not start up even
xenial-mate-rock64-0.5.10-118-arm64.img.xz

It does for me.

I'm wrong.
OK
xenial-mate-rock64-0.5.15-136-arm64.img.xz
xenial-mate-rock64-0.5.10-118-arm64.img.xz
NG
xenial-mate-rock64-0.6.1-141-arm64.img.xz
xenial-mate-rock64-0.6.0-140-arm64.img.xz

the prereleases 0.6.x also don't work for me

This is expected. 0.6.x is not yet stable.


DScanner: Enable a check blacklist for a gradual transition

Continuing the discussion at https://github.com/dlang/phobos/pull/5495#issuecomment-309636396
The idea is

to allow for better crowd-source collaboration on such tasks like improving the docs (properly_documented_public_functions) or providing public examples (has_public_example) as anyone can do this one module at a time
to prevent regressions in the already checked modules (e.g. a new function in module X will be verified against the enabled checks even if there aren't globally enabled)

So instead of the all-or-nothing state for a Dscanner check before, a gradual, distributed transition can happen. While DScanner has a lot of great and useful checks, of course, a bit of care needs to be taken, so that they don't stand in the way of collaborators.
Hence, for convenience I copy/pasted my summary of currently "actionable" checks:
Actionable

asm_style_check: only std.math
could_be_immutable_check: @JackStouffer and @BBasile are big fans of this
exception_check: Pokemon exception catching style (silent sinkhole for bugs), only a few occurrences
function_attribute_check: duplicate attributes @safe @safe void foo() {}
has_public_example: high-level vision, "Make sure every function in Phobos has an example" (this is still in the DScanner queue)
label_var_same_name_check: very confusing to read, only a few occurrences
long_line_check: fixed in https://github.com/dlang-community/D-Scanner/pull/465
mismatched_args_check: prevents myFunction(counter, obj) for void myFunction(A)(A obj, int counter)
object_const_check: checks whether opEquals, opCmp and toHash are const
opequals_tohash_check checks for aggregates which have opEquals defined, but not toHash
properly_documented_public_functions: high-level vision ("Make sure every function in Phobos has Params: and Returns: sections"), call to action)
redundant_attributes_check: see https://github.com/dlang/phobos/pull/5477
undocumented_declaration_check: prevents accidentally leaked symbols, unfortunately a lot has happend in the past. I tried to fix std.algorithm once: https://github.com/dlang/phobos/pull/4312
unused_label_check: this check doesn't support mixins yet (https://github.com/dlang-community/D-Scanner/issues/467)
unused_variable_check: mostly artifacts from debug sessions

To be discussed

auto_ref_assignment_check: only two occurrences, false positives?
imports_sortedness: I still need to modify the Dscanner check to ignore selective sortedness ...
length_subtraction_check: these may be unsigned and could lead to integer underflow
logical_precedence_check: if (a && b || c) // bad, if (a && (b || c)) // good
number_style_check: big numbers should use an underscore
style_check: many, many symbols aren't named according to the DStyle
useless_initializer: seems to be a quite common pattern in Phobos
vcall_in_ctor: only four occurrences

Boring details

the blacklist was generated
The phobos branch at DScanner isn't vanilla anymore, but I am trying to get the bug fixes and features to upstream/master
For simplicity (and maybe the future) I commented the discussed checks, but left the blacklist in (I used "+disabled" instead of setting the check to disabled, to have all (partially) disabled checks at one place)
Before someone asks, inifiled doesn't support multilines (yet).

One concern is that bulk code additions (e.g. adding a binding module as those in etc, or moving in some code from another project such as Druntime or tools) is going to be painful, as the submitter will need to keep running Dscanner to get a list of rules to which they will need to add their module.
It would be a bit nicer if the rules file was inverted: instead of a list of modules per rule, have a list of rules per module; then, such additions could just blacklist the entire module, and allow style fixes or finer-grained blacklists be done in follow-up PRs.
Otherwise this looks good to me. Would be nice to improve in how style failures are communicated to users to reduce submitter frustration (a comment from dlang-bot telling the user how to run the style check locally is a start; inline comments on failing lines would be better; inline comments with suggestions on how to fix them, or automatic pull requests to their branch, would be best).

One concern is that bulk code additions (e.g. adding a binding module as those in etc, or moving in some code from another project such as Druntime or tools) is going to be painful, as the submitter will need to keep running Dscanner to get a list of rules to which they will need to add their module.

Hehe, ideally new submissions should adhere the existing checks ;-)

It would be a bit nicer if the rules file was inverted: instead of a list of modules per rule, have a list of rules per module; then, such additions could just blacklist the entire module, and allow style fixes or finer-grained blacklists be done in follow-up PRs.

They can always do sth. like make -f posix.mak | cut -d':' -f2  | sort | uniq to get a list of all occurring checks ...
Apart from that I am not sure whether this will be a real problem, as imho (1) we should move modules away from Phobos not into, (2) criteria for std.experimental got so harsh that passing the static code analysis checker is a minor issue.

Otherwise this looks good to me. Would be nice to improve in how style failures are communicated to users to reduce submitter frustration (a comment from dlang-bot telling the user how to run the style check locally is a start; inline comments on failing lines would be better; inline comments with suggestions on how to fix them, or automatic pull requests to their branch, would be best).

This is on @MartinNowak's and my TODO list, e.g.

https://github.com/dlang-bots/dlang-bot/pull/57
https://github.com/dlang-bots/dlang-bot/issues/8

Of course help is welcome.

Fine by me; going to leave this one open for a while more in case others have any opinions/arguments to present.


allow creating multi-page PDFs

IIUC the package currently aims to create one PDF file per page in the output file group.
It would be a thrill to have an option to concatenate all pages into one multi-page PDF file (with proper PDF meta-data about the physical page numbers).

Adding the option to generate a multipage pdf shouldn't be this hard. With proper meta-dat, idk.

Note that prima-page-to-pdf already has a multi-page mode. However, it is somewhat restrictive:

it assumes the -xml argument is a directory name, filtering files in there ending with .xml
it assumes the -image argument is a directory name (possibly the same), matching the PAGE-XML filenames to it, with .xml replaced by either .jpg, .png or .tif
when an image file is missing (or has another type or filename), then both arrays will become inconsistent (without proper warning)

This is not good enough for our general OCR-D requirements though. (Files can have arbitrary names, they are merely linked by their METS pageId.)
So we should instead merge the individual PDF files into one, maybe with pdfunite from libpoppler?

So we should instead merge the individual PDF files into one, maybe with pdfunite from libpoppler?

Or with pdfjoin from pdfjam, which also allows to use the original physical pageId strings.

So we should instead merge the individual PDF files into one, maybe with pdfunite from libpoppler?

Or with pdfjoin from pdfjam, which also allows to use the original physical pageId strings.

Or do everything in Python with PyPDF2, both joining the PDF pages and adding meta-data from METS. (Cmp. this SO answer)

I am currently working on a solution just with a little python script and gs (ghostscript), which seems to provide everything we need and the user doesn't need install an extra library.

If gs is already needed by the PRImA lib and also allows setting meta-data, then that would be perfect. Anyway, these were just suggestions...

So a first version is implemented: 659cd280539ebe20b4a8070aee2bd24b44cac845
I need to know what exactly for information are required to write to the pdf. I could name the page as their "pageid" or maybe as their "physicalPagenumber"? And i added the option to add automatically or manually more metadata e.g.  author, title ..

I added also a little feature, that if the pdf already exist in the mets file, that it gets removed and created new. idk if there is an offical "ocr-d-way"?

So a first version is implemented: 659cd28

Nice!
I'll look into it. (Hint: next time if you commit your changes into a feature branch and then open a PR on it, then you can ask others for a review, discussing changes with them and making commit suggestions or even adding commits.)

I need to know what exactly for information are required to write to the pdf. I could name the page as their "pageid" or maybe as their "physicalPagenumber"?

The pageId exposed by ocrd workspace should be identical to the physical page ID in the METS structMap. And yes, that would be ideal for the PDF page name (as the document structure meta-data like TOC will later refer to those).

And i added the option to add automatically or manually more metadata e.g. author, title ..

Great. We can work it from there. There's a whole trove of information in the MODS...

I added also a little feature, that if the pdf already exist in the mets file, that it gets removed and created new. idk if there is an offical "ocr-d-way"?

I saw it. No, that's not OCR-D conforming. (Everything is supposed to be just additive, except if the user wants to destroy annotations.)
There's a long-standing issue on allowing processors to overwrite their output file group with an additional option.
But for now, the user is expected to workspace remove or workspace remove-group herself.

So i did some reworks for the multipage option.
Thanks for the hints in the previews pull.


docs(quickstart): add generic quickstart guide

This adds a new generic build quickstart guide, analogous to the existing Cargo-focused quickstart guide. I tried to keep it simple to follow, so I pared down some of the details from the original guide that I felt were a little too fiddly to present to first-time users. My goal was to get people from 0 to their first release as conveniently as possible.

Rebased on #1271. I've also adapted most of the language from the Rust version.


Release/1.1.4

All improvements in 1.1.3-RCs which weren't included in 1.1.3 release due to urgency.

@oguzhanunlu I tried to build this branch with Carthage and it seems that the archive phase is failing.
Build settings from command line:
    CARTHAGE = YES
    CLANG_ENABLE_CODE_COVERAGE = NO
    CODE_SIGN_IDENTITY =
    CODE_SIGNING_REQUIRED = NO
    GCC_INSTRUMENT_PROGRAM_FLOW_ARCS = NO
    ONLY_ACTIVE_ARCH = NO
    SDKROOT = iphoneos13.1
    SKIP_INSTALL = YES
    STRIP_INSTALLED_PRODUCT = NO

note: Using new build system
note: Planning build
note: Constructing build description
error: ***/Carthage/Checkouts/snowplow-objc-tracker/Pods/Target Support Files/Pods-Snowplow-iOS/Pods-Snowplow-iOS.release.xcconfig: unable to open file (in target "Snowplow-iOS" in project "Snowplow") (in target 'Snowplow-iOS' from project 'Snowplow')
error: ***/Carthage/Checkouts/snowplow-objc-tracker/Pods/Target Support Files/Pods-Snowplow-iOS/Pods-Snowplow-iOS.release.xcconfig: unable to open file (in target "Snowplow-iOS" in project "Snowplow") (in target 'Snowplow-iOS' from project 'Snowplow')
error: ***/Carthage/Checkouts/snowplow-objc-tracker/Pods/Target Support Files/Pods-Snowplow-iOS/Pods-Snowplow-iOS.release.xcconfig: unable to open file (in target "Snowplow-iOS" in project "Snowplow") (in target 'Snowplow-iOS' from project 'Snowplow')

** ARCHIVE FAILED **

Hi @mkotsiandris , thanks for the notice and sharing error log. I'll be looking at this and update you as quickly as possible.

Hi @mkotsiandris we've just pushed an update to this branch which should enable it to work with Carthage. I'm still aware of some macOS build issues, so installing with a platform specific command should give you better luck:
carthage update --platform iOS

Hi @mkotsiandris , my colleague @paulboocock has resolved the issue with building using carthage. We've just published 1.1.4-rc2 and we expect that you should be able to use this version successfully in your project. Please let us know how it works out on your end. Thanks in advance!

@oguzhanunlu @paulboocock Thank you for your quick response and sorry for the delayed answer. I tried it. The issue that I faced is that the header files for the public classes of SnowplowTracker are not exposed properly under the umbrella header file.
In order to bypass that issue and test the framework is working as expected I had to generate a bridging header file for my application and link the bridging header you have under the framework. Would it be possible to resolve that issue?

Thanks for letting us know how it worked on your end @mkotsiandris ! Glad to hear about the functionality. We'll look at the issue with header files and get back as soon as possible.


Can't open file error on Windows 10 / WinUAE + WB3.1, Blitz 2.1 & SuperTed 2.52

Hi,
I believe I have found the issue of Ted randomly complaining it can't open a file as I can recreate it reliably every time and have also fixed it manually - the path that is being piped through by VSCode contains Windows backslashes rather than Amiga forward slashes for directory separators. These just need to be substituted - see my screenshot.
Thanks very much for this excellent piece of work, I'm trying to get it working on Linux but FS-UAE isn't playing ball with me there!
Adam

Ok. Tx for identifying the issue. Not hard to fix, most of my time would be dedicated to setup the env within windows.

Fix, done. need to do https://github.com/MickGyver/vscode-amiga-blitzbasic/issues/14 for full windows compatibility to publish.

@adam-ainsworth correction published in version 0.9.1 : https://marketplace.visualstudio.com/items?itemName=MickGyver.amiga-blitzbasic2

@adam-ainsworth correction published in version 0.9.1 : https://marketplace.visualstudio.com/items?itemName=MickGyver.amiga-blitzbasic2

Thanks very much for sorting this out!


fix: pin fastparquet version

Closes #642 by setting fastparquet version. The library was updated and is not compatible anymore with LCBench dataset. If we want to update to more recent version, we could update the processing done to lcbench.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Thanks! I'm wondering why we didn't catch this, and it looks like launch_lcbench_simulated is not run as an example. I'm looking at why


Fix reachability test race condition

#16913 introduced a race. See https://prow.istio.io/view/gcs/istio-prow/logs/integ-race-native-tests_istio_postsubmit/14

@howardjohn: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

release-test_istio
6bc5148f41d4cf9ba577a75f2bc2a4adcd1e7c8e
link
/test release-test_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

release-test_istio
6bc5148f41d4cf9ba577a75f2bc2a4adcd1e7c8e
link
/test release-test_istio

integ-mixer-k8s-tests_istio
6bc5148f41d4cf9ba577a75f2bc2a4adcd1e7c8e
link
/test integ-mixer-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

/retest

@howardjohn: The following tests failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

release-test_istio
6bc5148f41d4cf9ba577a75f2bc2a4adcd1e7c8e
link
/test release-test_istio

integ-mixer-k8s-tests_istio
6bc5148f41d4cf9ba577a75f2bc2a4adcd1e7c8e
link
/test integ-mixer-k8s-tests_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@howardjohn: The following test failed, say /retest to rerun them all:

Test name
Commit
Details
Rerun command

pilot-multicluster-e2e_istio
98ecb2e66d182e50bf19f1e75db572b5386bea23
link
/test pilot-multicluster-e2e_istio

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.


Server Side Rendering (initialNavigation: 'enabledBlocking') seems to stop the module from working

Following the README, I have integrated this ngx-transloco-router and everything seems to work fine when it comes to remembering the selected language (via a lang picker).
What seems not to be working is adding the language to the current path, I'm not sure if this might be an Angular 15 problem or if the package does not work with SSR? As soon as I remove initialNavigation: 'enabledBlocking', the package is working fine.
My AppRoutingModule looks like this and I'm importing the TranslocoRootModule in my AppModule:
@NgModule({
  imports: [
    RouterModule.forRoot(routes, {
      initialNavigation: 'enabledBlocking',
      scrollPositionRestoration: 'enabled',
      anchorScrolling: 'enabled',

    }),
    LocalizeRouterModule.forRoot(routes)
  ],
  exports: [RouterModule, LocalizeRouterModule],
  providers: [
    {
      provide: LOCALIZE_ROUTER_CONFIG,
      useValue: localizeRouterConfig({
        translateRoute: true,
      })
    }
  ]
})
export class AppRoutingModule { }

Hello, the implementation is pretty much the same as the ported library. You need to enable initial navigation as per their documentation https://github.com/gilsdav/ngx-translate-router#deal-with-initialnavigation

It seems to me that ngx-transloco-router expects only one argument (the routes) for forRoot(), also visible in the localize-router.module.d.ts:
static forRoot(routes: Routes): ModuleWithProviders<LocalizeRouterModule>;
As a result the compiler is  complaining that Expected 1 arguments, but got 2.

Ah I see, so a bit different from how ngx does it. Thank you for pointing this out :)
When following your snippet (and removing initialNavigation from RouterModule.forRoot) I get the following error:
ERROR TypeError: Cannot read properties of undefined (reading 'afterPreactivation')
    at /Volumes/source/<project-dist-dir>/server/vendor.js:198738:54
    ....

Seems like there's actually a lot of changes under Angular's 15 (around 15.0.3 and onwards) routing due to refactoring.
I've published 15.0.0-beta.1 which should hopefully fixes those issues.
I'm not entirely sure how many users are using this library. This project started for a specific client requirement that was already using transloco and no plan to update from angular 13. So I am relying on you/community to test it out, I've only tested in my demo app and bare min setup for SSR.

Thank you for the quick response and cutting a new beta release! :)
I'm unfortunately not able to use this release in my project, as it seems that the index.d.ts is missing in the library: Cannot find module '@penleychan/ngx-transloco-router' or its corresponding type declarations
Despite from that it seems that you already have a good audience for this library, given that I can see on npm roughly 100 weekly downloads. But I'm of course not in your shoes when it comes to maintaining this, it has been THE missing feature for me in transloco though.

Sorry about that, mistake on my end. Could you try 15.0.0-beta.3
Unfortunately majority of the downloads skewed due to bots scrapping npmjs. Anyways thanks for using this library.

Yeah I see, npm stats might be tainted and I guess CI/CD pipelines can add more to it.
The latest version that you cut seems to work with Angular 15 without errors, I only see this warning in the console when I load a page:
DEPRECATED: DI is instantiating a token "LocalizedRouter" that inherits its @Injectable decorator but does not provide one itself.
This will become an error in a future version of Angular. Please add @Injectable() to the "LocalizedRouter" class.

What I still see though - and I'm not 100% sure if that is the exptected behavior - is that the website flickers from the default language (English) to the selected different language on a refresh. I can also see that the HTML that is generated by SSR contains all the strings in English and the translation is only happening on the javascript client side in the browser.

Thanks for that error message.
Flickering issue is caused by SSR not able to determine the value of the default lang on first render. Default cache mechanism is Localstorage and server side apps don't have "localstorage". So, to get around for example in NodeJS server side with express is you would use cookies, and you can change the cache mechanism to "Cookie". So this is really depends on your implementation. See ngx-translate-router#server-side.

Isn't that just the case if I call  without a path?
I should have added that I'm already calling /it/ (or any other language) so that the language does not need to be determined by the browser environment. While the app is (rightfully) ignoring my browser language on client side and is switching (with flickering) from the default language English to any other language, the same thing does not seem to work on the server side.
I can see though that the server is requesting the relevant local from the given transloco resource and is also packaging it in case I add it to the TransferState - just the html is not translated to the given target language.

Can you give<EMAIL_ADDRESS>a try. I think I found the culprit with the SSR issue, tested it and it seems to be translated in the html source:

Tried out 15.0.0 and that seems to fix the issue. I also don't see the warning anymore.
Thanks a lot!


[FR] Calculating Subtotal of Sale Order Items for use in Sale Order Form

Please verify that this feature request has NOT been suggested before.

[X] I checked and didn't find a similar feature request

Problem statement
Below is a code snippet from a sale order template. This snippet generates a table that has the sale order items and a separate area for the line order items. In my case, I am modeling this sale order template after a receipt. I would like to include the total of only the sale order items as a "subtotal" at the base of the sale order items portion of the table. The "total" will appear at the bottom off the extra line portion of the table. Currently, a variable to reference that "subtotal" does not exist. Only the order.total_price variable to get the overall total including extra line items.

            {% trans "Item" %}
            {% trans "Reference" %}
            {% trans "Quantity" %}
            {% trans "Unit Price" %}
            {% trans "Total Price" %}

        {% for line in lines.all %}

                        {{ line.part.name }}

            {{ line.part.IPN }}
            {% decimal line.quantity %}
            {% render_currency line.price %}
            {% render_currency line.total_line_price %}

        {% endfor %}

            {% trans "Subtotal" %}
            {% render_currency subtotal_variable currency=order.currency %}

Suggested solution
After looking through the code on Github I think I have narrowed down the two files that would need to be altered to include this "subtotal" variable. The admin.py and models.py in InvenTree/src/backend/InvenTree/order/. I am trying to build a potential solution to this, but my unfamiliarity with the code and all references is making it challenging.
Here are some of the relevant places I think the code would need altering.
class TotalPriceResourceMixin:
    """Mixin for exporting total price data."""

    total_price = Field(attribute='total_price', column_name=_('Total Price'))

    def dehydrate_total_price(self, order):
        """Return the total price amount, not the object itself."""
        if order.total_price:
            return order.total_price.amount
        return ''

    def save(self, *args, **kwargs):
        """Update the total_price field when saved."""
        # Recalculate total_price for this order
        self.update_total_price(commit=False)

        if hasattr(self, '_SAVING_TOTAL_PRICE') and self._SAVING_TOTAL_PRICE:
            # Avoid recursion on save
            return super().save(*args, **kwargs)
        self._SAVING_TOTAL_PRICE = True

        # Save the object as we can not access foreign/m2m fields before saving
        self.update_total_price(commit=True)

    total_price = InvenTreeModelMoneyField(
        null=True,
        blank=True,
        allow_negative=False,
        verbose_name=_('Total Price'),
        help_text=_('Total price for this order'),
    )

    def update_total_price(self, commit=True):
        """Recalculate and save the total_price for this order."""
        self.total_price = self.calculate_total_price(target_currency=self.currency)

        if commit:
            self.save()

    def calculate_total_price(self, target_currency=None):
        """Calculates the total price of all order lines, and converts to the specified target currency.

        If not specified, the default system currency is used.

        If currency conversion fails (e.g. there are no valid conversion rates),
        then we simply return zero, rather than attempting some other calculation.
        """
        # Set default - see B008
        if target_currency is None:
            target_currency = currency_code_default()

        total = Money(0, target_currency)

        # Check if the order has been saved (otherwise we can't calculate the total price)
        if self.pk is None:
            return total

        # order items
        for line in self.lines.all():
            if not line.price:
                continue

            try:
                total += line.quantity * convert_money(line.price, target_currency)
            except MissingRate:
                # Record the error, try to press on
                _1, _2, _3 = sys.exc_info()

                log_error('order.calculate_total_price')
                logger.exception("Missing exchange rate for '%s'", target_currency)

                # Return None to indicate the calculated price is invalid
                return None

        # extra items
        for line in self.extra_lines.all():
            if not line.price:
                continue

            try:
                total += line.quantity * convert_money(line.price, target_currency)
            except MissingRate:
                # Record the error, try to press on

                log_error('order.calculate_total_price')
                logger.exception("Missing exchange rate for '%s'", target_currency)

                # Return None to indicate the calculated price is invalid
                return None

        # set decimal-places
        total.decimal_places = 4

        return total

Describe alternatives you've considered
The other way to achieve this is to open the template editor and open the respective sales order. Then create a
 or  and manually calculate and insert the value. I was trying to automate this, but this is an option.
Examples of other systems
No response
Do you want to develop this?

[ ] I want to develop this.

I ended up adding a custom filters file to the template tags folder.
I couldn't figure out how to get it to work in the main InvenTree files, so I made a filter instead.
Inside that custom filters file, I added the following code:
import requests
import json
import qrcode as python_qrcode
import report.helpers
from PIL import Image
from django import template
from django.utils.html import format_html

register = template.Library()

@register.simple_tag()
def calculate_subtotal(lines):
    subtotal_sum = 0.0
    for line in lines:
        # Remove any $ or commas from the total_line_price and convert to float
        total_line_price_str = str(line.total_line_price).replace('$', '').replace(',', '')
        try:
            total_line_price_float = float(total_line_price_str)
        except ValueError:
            total_line_price_float = 0.0

        # Add the current line price to the total sum
        subtotal_sum += total_line_price_float

    # Format the total sum as currency
    formatted_subtotal_sum = "${:,.2f}".format(subtotal_sum)
    return formatted_subtotal_sum

I then call the filter in the Sale Order template like so:
{% calculate_subtotal lines.all as subtotal_amount %}

This can also be implemented with a custom ReportMixin plugin by adding a context variable to the report.
from plugin import InvenTreePlugin
from plugin.mixins import ReportMixin
from decimal import *
from djmoney.money import Money
from djmoney.contrib.exchange.models import convert_money
from common.currency import currency_code_default

class InvoiceReportPlugin(ReportMixin, InvenTreePlugin):
    NAME = 'Invoice Report Plugin'
    SLUG = 'invoicereportplugin'
    TITLE = 'Invoice Report Plugin'
    DESCRIPTION = 'A plugin which adds total_no_extra context variable to the report context.'
    VERSION = '0.0.1'

    def calculate_total_no_extra_lines(self, order, target_currency):
        total = Money(0, target_currency)
        for line in order.lines.all():
            if not line.price:
                continue
            total += line.quantity * convert_money(line.price, target_currency)
        return total

    def add_report_context(self, report_instance, model_instance, request, context):
        """Add example content to the report instance."""
        # We can add any extra context data we want to the report
        # Generate a random string of data
        if report_instance.model_type == 'salesorder':
            target_currency = currency_code_default()
            context['total_no_extra'] = self.calculate_total_no_extra_lines(model_instance, target_currency)


module.lookupDep

Codecov Report

Merging #26 (da70c7b) into main (ab7a3a6) will increase coverage by 0.01%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##             main      #26      +/-   ##
==========================================
+ Coverage   93.10%   93.11%   +0.01%
==========================================
  Files          12       12
  Lines        5931     5942      +11
==========================================
+ Hits         5522     5533      +11
  Misses        322      322
  Partials       87       87

Impacted Files
Coverage Δ

import.go
93.43% <100.00%> (ø)

package.go
98.82% <100.00%> (+0.05%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update ab7a3a6...da70c7b. Read the comment docs.


OpenAPI 3: Generate deepObject for complex query parameter

Sample: https://github.com/RSuter/NSwag/issues/1591

This is a much needed feature.
Trying to adopt CQRS pattern, which would greatly simplify API controllers, requires a use of complex types in action queries.
Example (Using ASP.NET Core 3.1).
My controller action:
[ApiController]
[Route("api/[controller]")]
[ApiConventionType(typeof(DefaultApiConventions))]
public class WorkflowsController : ControllerBase
{
  ...
  [HttpGet(nameof(GetWorkflow))]
  public async Task<ActionResult<Workflow>> GetWorkflow([FromQuery] GetWorkflowQuery query)
    {
           var workflow = await Mediator.Send(query);
           return Ok(workflow);
    }
}

Query object:
using MediatR;
public class GetWorkflowQuery : IRequest<Workflow>
    {
        public Guid Id { get; set; }
        public EntityEnvironment Environment { get; set; }
        public bool IncludeOperations { get; set; }
        public bool IncludeDataPaths { get; set; }
        public bool AsNoTracking { get; set; }
    }

public enum EntityEnvironment
    {
        Staging,
        Production,
        Backup
    }

Generated swagger spec:
"/api/Workflows/GetWorkflow": {
      "get": {
        "tags": [
          "Workflows"
        ],
        "operationId": "Workflows_GetWorkflow",
        "requestBody": {
          "x-name": "query",
          "content": {
            "application/json": {
              "schema": {
                "$ref": "#/components/schemas/GetWorkflowQuery"
              }
            }
          },
          "required": true,
          "x-position": 1
        },
        "responses": {
          "200": {
            "description": "",
            "content": {
              "application/json": {
                "schema": {
                  "$ref": "#/components/schemas/Workflow"
                }
              }
            }
          }
        }
      }
    },
....
"GetWorkflowQuery": {
        "type": "object",
        "additionalProperties": false,
        "properties": {
          "id": {
            "type": "string",
            "format": "guid"
          },
          "environment": {
            "$ref": "#/components/schemas/EntityEnvironment"
          },
          "includeOperations": {
            "type": "boolean"
          },
          "includeDataPaths": {
            "type": "boolean"
          },
          "asNoTracking": {
            "type": "boolean"
          }
        }
      },

Generation API Documentation:

Error when trying to execute the action:
TypeError: Failed to execute 'fetch' on 'Window': Request with GET/HEAD method cannot have body.
Error from generated C# client calling the action:
System.Net.ProtocolViolationException: Cannot send a content-body with this verb-type.
   at System.Net.HttpWebRequest.CheckProtocol(Boolean onRequestStream)
   at System.Net.HttpWebRequest.BeginGetRequestStream(AsyncCallback callback, Object state)
   at System.Net.Http.HttpClientHandler.StartGettingRequestStream(RequestState state)
   at System.Net.Http.HttpClientHandler.PrepareAndStartContentUpload(RequestState state)
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at ...API.Clients.WorkflowsClient.<GetWorkflowAsync>d__17.MoveNext()

Is this with the aspnetcore generator?


Invalid endpoint: https://kms..amazonaws.com (double dots)

Getting error  Invalid endpoint: https://kms..amazonaws.com. Any idea where do I find endpoints.json to fix it. Is it part of lambci?
[DEBUG]	2017-08-16T16:31:58.586Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	Looking for credentials via: env

[INFO]	2017-08-16T16:31:58.587Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	Found credentials in environment variables.

[DEBUG]	2017-08-16T16:31:58.587Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	Loading JSON file: /var/runtime/botocore/data/endpoints.json

[DEBUG]	2017-08-16T16:31:58.617Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	Loading JSON file: /var/runtime/botocore/data/kms/2014-11-01/service-2.json

[DEBUG]	2017-08-16T16:31:58.620Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	Loading JSON file: /var/runtime/botocore/data/_retry.json

[DEBUG]	2017-08-16T16:31:58.620Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	Registering retry handlers for service: kms

[DEBUG]	2017-08-16T16:31:58.622Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	Event creating-client-class.kms: calling handler <function add_generate_presigned_url at 0x7f50c4780bf8>

[DEBUG]	2017-08-16T16:31:58.623Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	Assuming an endpoint for kms, : https://kms..amazonaws.com

[DEBUG]	2017-08-16T16:31:58.623Z	4285aa19-3e63-42b5-83c4-17e8ddb10e09	The s3 config key is not a dictionary type, ignoring its value of: None

module initialization error: Invalid endpoint: https://kms..amazonaws.com
END RequestId: 4285aa19-3e63-42b5-83c4-17e8ddb10e09

This is because of #48. It should be fixed but not yet released. Will cut a release now

Thanks for very quick response. I am working with python 3.6. Hope it will be fixed for that too.

New release is here - https://github.com/awslabs/aws-sam-local/releases/tag/v0.1.1. Upgrade with npm and try it out.


[Bug] Visual Map not rendering line color (but it IS rendering correct color in label)

Version
5.4.2
Link to Minimal Reproduction
https://codepen.io/daemach/pen/MWegRvO
Steps to Reproduce
See code pen.
Current Behavior
Line color does not change, although the LABEL color is being mapped correctly according to visual map rules
Expected Behavior
I expect the line color to change based on values in dimension 2 of series 0
Environment
- OS: Linux
- Browser: Firefox && Chrome
- Framework: javascript / angular

Any additional comments?
see #13393

It should be noted that when the series type is 'bar' the bar color matches that defined by dimension not the plotted value

Agree, kind of weird. People want to do line segments coloring in ECharts, but get just line points (+label) coloring.
Solution is to use custom lines - see code example in #18634


[C++][Parquet] Add Stress test for RecordReader SkipRecords

Adding a stress test that will call a random sequence of ReadRecords and SkipRecords.
Reporter: fatemah / @fatemehp
Assignee: fatemah](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=panahi) / @fatemehp
PRs and other links:

GitHub Pull Request #14879

Note: This issue was originally created as PARQUET-2201. Please see the migration documentation for further details.

Micah Kornfield / @emkornfield:
Issue resolved by pull request 14879
https://github.com/apache/arrow/pull/14879


Tests for non-compilable cases

Unsupported cases, as <tag @property="<%=property1%><%=property1%>"> should raise CompilerError during the compilation. Provide a tests for this cases.

reserved symbol @using_assigns


Dynamically adding a Source to a Pipeline in a Playing state fails to link to the Stream-muxer

The AddChild method in the PipelineSourcesBintr is using the incorrect LinkToSink function on dynamic add.

Merged into v0.21.alpha


Bug 2074854: Fix redirect to CIM edit flow

Signed-off-by: Rastislav Wagner<EMAIL_ADDRESS>

/lgtm


feat: create contentContainer component

Recreate contentContainer from monolith to gamut-elements
To be enable use cross-repo.

Merging your changes
When you are ready to merge to master, use the "squash and merge" button in GitHub, and follow the commit message guide to write your commit message.

📬Published Alpha Packages:@codecademy/brand-components@0.8.6-alpha.740994.0+74099439@codecademy/gamut-templates@2.5.5-alpha.740994.0+74099439<EMAIL_ADDRESS>
📬Published Alpha Packages:@codecademy/brand-components@0.8.6-alpha.b691e7.1+b691e7e5@codecademy/gamut-templates@2.5.5-alpha.b691e7.1+b691e7e5<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://5e8510bc9646f293cc713ed9--codecademy-styleguide.netlify.comDeploy Logs

🚀 Styleguide deploy preview ready!https://5e8511c46aaab8a312681e49--codecademy-styleguide.netlify.comDeploy Logs

📬Published Alpha Packages:@codecademy/brand-components@0.8.6-alpha.21ed64.3+21ed643a@codecademy/gamut-templates@2.5.5-alpha.21ed64.3+21ed643a<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://5e8513fd7d20fed28513caf1--codecademy-styleguide.netlify.comDeploy Logs

📬Published Alpha Packages:@codecademy/brand-components@0.8.6-alpha.970678.4+9706787f@codecademy/gamut-templates@2.5.5-alpha.970678.4+9706787f<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://5e8516ae9646f20bf9713b54--codecademy-styleguide.netlify.comDeploy Logs

📬Published Alpha Packages:@codecademy/brand-components@0.8.6-alpha.f401aa.6+f401aa19@codecademy/gamut-templates@2.5.5-alpha.f401aa.6+f401aa19<EMAIL_ADDRESS>

A bunch of nits because I'm rude, but this is looking swell. Yay for standardizing!

They're valid points! Will ship as is for now though to allow for it to be used on the static site work.

📬Published Alpha Packages:@codecademy/brand-components@0.8.6-alpha.18f47d.7+18f47dda@codecademy/gamut-templates@2.5.5-alpha.18f47d.7+18f47dda<EMAIL_ADDRESS>
🚀 Styleguide deploy preview ready!https://5e85f9d0b01a737737342285--codecademy-styleguide.netlify.comDeploy Logs


Navigation replacement

Would it drastically affect the project I’d say I were to change the Modo navigation to the one provided by Compose(NavHost)? Modo doesn’t seem to support passing args to other screens

For anyone else who might have a similar question: yes it is possible


ArcLoader requires locale directory to exist to even use fallbacks

Using the ArcLoader is really picky and I'm having a hard time setting it up for use (adding Fluent functions to a tera CLI). The shared resources feature works, as do fallback locales. The trouble is neither work if a directory in the locales path does not exist for every possible locale variant that might be used. Even if the message data would otherwise fall back to another locale or the shared resources, an empty directory needs to exist to stuff the bundles.
c.f. #41

I haven't dug up this project to see if I can get farther now, but does v0.9.1 relate to this? It looks possible that this is the same issue.