Order by date, priority

Hello,
Is there an option which would allow to order records by date, priority? Overdue first, then all records for Today, and so on, records with higher priority on the top.

same question here!

Hey guys,
Sorry for the delay. I've added the ability to sort differently in different ways. Try out the different sort types and let me know if that's the behavior you're looking for. If not, can you give me an example of how you want it sorted and I'll add the option.
Chris
Get the latest code and add this to your config.
sortType
This will determine the sorting method used when displaying your Todos.
Possible values:
"todoist" - Sort based on the order in Todoist.
"dueDateAsc" - Sort based on the Due Date of the Todo Ascending. (Oldest date first)
"dueDateDesc" - Sort based on the Due Date of the Todo Descending. (Newest date first)
Default value: "todoist"

That's it, thank you!

Awesome! Thanks!


TERM variable in default-config is ignored

When I use a default-generated config file the declared TERM environment variable is ignored. This manifests in that whenever one launches contour TERM is set to vt100.

OS
FreeBSD hades 14.1-RELEASE-p2 FreeBSD 14.1-RELEASE-p2 releng/14.1-n267685-73905162d0ab GENERIC amd64

contour version
<IP_ADDRESS>68

I built contour through my FreeBSD Port at https://github.com/herrhotzenplotz/ports-overlay/tree/master/x11/contour/.

What happens when you do export TERM=contour inside the shell?
Also, we are trying to set TERM to contour while starting terminal, so maybe terminfo file is missing, and exort TERM=contour will tell you about it

What happens when you do export TERM=contour inside the shell?

Of course that works. It's just annoying that everytime I use an ncurses application it starts up in black/white and scrolling in less(1) is dead-slow. Then you quit said application, update your environment and start over.

Also, we are trying to set TERM to contour while starting terminal, so maybe terminfo file is missing

the terminfo is installed correctly:
nico@hades:~ $ find /usr/local/share/terminfo/
/usr/local/share/terminfo/
/usr/local/share/terminfo/c
/usr/local/share/terminfo/c/contour
/usr/local/share/terminfo/c/contour-latest
nico@hades:~ $

and exort TERM=contour will tell you about it

running export TERM=... will tell you nothing. It just sets the shell variable and then exports it to the environment. Unless this is something shell/OS-specific.

Thanks @herrhotzenplotz, will try to figure out what is happening
When you export TERM, shell checks if you can set it
➜  ~ export TERM=asdfasdf
zsh: can't find terminal definition for asdfasdf

You're assuming that I am using zsh. However I am using FreeBSD's /bin/sh (which itself is an enhanced version of ash).

You're assuming that I am using zsh. However I am using FreeBSD's /bin/sh (which itself is an enhanced version of ash).

oh, right, i forgot that this is only zsh implementation

Ok, after some debugging and reading I figured out that the problem is that contour has a hardcoded terminfo search path for /usr/share/terminfo.
It completely ignores its installation prefix which is where the CMakeLists.txt outputs the compiled terminfo file.
So, rather looking for /usr/share/terminfo (which doesn't even exist on FreeBSD) it should be looking in ${PREFIX}/share/terminfo.
This will also fix the problem on other BSDs.

This is from man tic
       Libraries that read terminfo entries are expected to check in succession

       •   a location specified with the TERMINFO environment variable,

       •   $HOME/.terminfo,

       •   directories listed in the TERMINFO_DIRS environment variable,

       •   a compiled-in list of directories (/usr/share/terminfo), and

       •   the system terminfo database (/usr/share/terminfo).

If you have a custom path for terminfo files add it into TERMINFO_DIRS

If you have a custom path for terminfo files add it into TERMINFO_DIRS

It is not a custom path. The manual page you're looking at is generated at the build time of ncurses. Here I get the following:
       The results are normally placed in the system terminfo database
       /usr/local/share/terminfo.  The compiled terminal description can be
       placed in a different terminfo database.  There are two ways to achieve
       this:

       •   First, you may override the system default either by using the -o
           option, or by setting the variable TERMINFO in your shell
           environment to a valid database location.

       •   Secondly, if tic cannot write in /usr/local/share/terminfo or the
           location specified using your TERMINFO variable, it looks for the
           directory $HOME/.terminfo (or hashed database $HOME/.terminfo.db);
           if that location exists, the entry is placed there.

       Libraries that read terminfo entries are expected to check in
       succession

       •   a location specified with the TERMINFO environment variable,

       •   $HOME/.terminfo,

       •   directories listed in the TERMINFO_DIRS environment variable,

       •   a compiled-in list of directories
           (/usr/local/share/terminfo:/usr/local/share/site-terminfo), and

       •   the system terminfo database (/usr/local/share/terminfo).

       The Fetching Compiled Descriptions section in the terminfo(5) manual
       goes into further detail.

(this is from the man page installed from ports)
Linux systems generally dump everything in /usr. *BSD-Systems among other Unix systems split out things under different prefixes for some neat seperation. As for FreeBSD, ports and binary packages are installed in /usr/local, on NetBSD pkgsrc typically installs into /usr/pkg.
Every terminal emulator in ports (that I use and have used) installs its terminfo files under $PREFIX/share/terminfo and it works flawlessly.
Note that there is also a system-provided tic in /usr/bin/tic and it has the same paths as you noted above.
However since we're building contour as a package we want the terminfo not to leak outside the prefix and it should thus be installed in /usr/local/share/terminfo and contour should, just like any other terminal emulator, look there.


Fix broken avatars since #15301

There was a missing * from the avatars routes in #15301.
Fix #15727
Signed-off-by: Andrew Thornton<EMAIL_ADDRESS>

@wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf maybe you can confirm this.

@wULLSnpAXbWZGYDYyhWTKKspEQoaYxXyhoisqHf maybe you can confirm this.

on it

thanks @zeripath  && @silverwind , this definitely fixes #15727 🚀
user-supplied user avatars: shown ✔
org gravatars: shown ✔
user-supplied repo avatars (those were missing, too :D): shown ✔

I've added a testcase to ensure that they don't break again

make lgtm work


Cannot do picking correctly.

I Think theres something wrong with the batchId in output files and the mesh cannot be picked correctly in cesium.js.

The picking issue is solved. However, I used the total batch length as the bath length of each tile. This can cause significant GPU memory waste.


A consistent strategy for documentation

Summary
Terasology's documentation is currently spread-out over several different locations using a mixture of docsify and GitHub wikis:

MovingBlocks/Terasology wiki (GitHub wiki)
Tutorial modules (Some using docsify and others using GitHub wikis)
NUI (docsify)
Documentation on individual modules, for example:

Health (docsify)
DynamicCities (GitHub wiki)

JavaDoc comments (published by Jenkins)
Discussions on Discord

In general, the more documentation there is, the harder it becomes to find what we're looking for. Having multiple locations to search also makes it more difficult to find useful information when needed.
Engine documentation
Engine documentation is primarily found on the wiki. GitHub wikis have some limitations and can quite easily become disorganised due to their flat structure. Hierachies are enforced through listing pages on the sidebar, although this requires people to update the sidebar manually whenever a new page is added. There is a lot of useful content on the wiki but most of it is not referenced from the sidebar, for example the Documentation Guide.
Alternative: In-repo documentation
An alternative might be to move the documentation directly into the repository (under the docs/ directory). We could then publish it using GitHub Pages, possibly using docsify or a static site generator (so long as it is consistent).
Pros:

Working closer to the code. Could be more tightly integrated.
Compatible with existing processes for reviews, approvals, linting, etc.
More flexibility with styling and layout

Cons:

Breaks existing links
Everything would need to be migrated from the engine wiki. Having both the wiki and the docs folder co-existing would be unacceptable.
Loss of the GitHub wiki editor to edit wiki pages

Alternative: Improve the wiki pages
We could also continue to try and improve the state of the engine wiki, possibly by rearranging pages and adding information that was previously missing.

hello am from MLHhackathon open soucre week. could you provide a sample of how you want the documentation to be structured

Hi @lokytech5,
actually that's what we want to discuss in this issue.
As mentioned in the issue description, we already have various places of documentation and with that also various ways of how this documentation is structured.
What we need to figure out is how we can structure the documentation we have in a better way to make it easy to navigate for different types of users and easy to maintain.

Going through the documentation, I can provide you with a sample of how I can make it easy to navigate.

please note. I will be using actual markdown throughout the sample.

Table of Contents

Introduction
Community
Installation

Minimum Requirements
Alternative Installation Methods

Development

Requirements
Workspace Setup
Contributing

License
Credits
Contributor Covenant Code of Conduct

Introduction
The Terasology project was born from a Minecraft-inspired tech demo and is becoming a stable platform for various types of gameplay settings in a voxel world. The creators and maintainers are a diverse mix of software developers, designers, game testers, graphic artists, and musicians. We encourage contributions from anybody and try to keep a warm and friendly community and maintain a code of conduct.

Community
If you want to get in contact with the Terasology community and the whole MovingBlocks team, you can easily connect with us, share your ideas, report and solve problems. We are present in nearly the complete round-up of social networks. Follow/friend us wherever you want, chat with us, and tell the world.

Installation
For easy game setup (recommended) you can use our launcher - download it here.
Minimum Requirements

System (OS): Windows, MacOS, Linux (64 bit)
Processor (CPU): dual-core CPU
Memory (RAM): 4 GB
Graphics (GPU): Intel HD Graphics (Gen 7), GeForce 8xxx series (or higher), Radeon HD 2000 series (or higher) with OpenGL 3.3
Storage (HDD): 1 GB
Please note, if you have both integrated (chip) and dedicated (card) graphics, ensure you're using your dedicated graphics when running Terasology.

Alternative Installation Methods
If you already have a Java Development Kit (JDK) installed, you may use a direct download release as an alternative to using the launcher. Java version 11 is required. Direct download stable builds are uploaded here on GitHub while the cutting-edge develop version can be downloaded here.

Development
Requirements

Technical Requirements:

Java SE Development Kit (JDK) 11. The CI will verify against this baseline version.
Git to clone the repo and commit changes.

Non-Technical Requirements:

Familiarity with Git.
Familiarity with GitHub, especially forks.

Workspace Setup
To run Terasology from source, follow the Contributor Quick Start Guide designed for IntelliJ IDEA. Note that Terasology workspace is a multi-repo workspace.
Contributing
Detailed information on how to contribute can be found in CONTRIBUTING.md. All submissions must be licensed under Apache License, Version 2.0. If you find errors or issues in any resources, report them using GitHub issues.

License
Terasology is fully open source and licensed under the Apache License, Version 2.0 for code and Creative Commons for art.

Credits
(Credits section)

Contributor Covenant Code of Conduct
(Conduct section)

it can be managed easily but the only drawback I notice is that links need to be updated regularly, I think is better if you have a style guide to follow to maintain consistency, and last but not least, as the documentation grows, the table of content we become complex. Those are just the drawback I notice and I can improve on.

@jdrueckert, thanks for clarifying and directing me to the documentation. going through it, a lot of information is going on there, with links directing to different sections in the content. Going through the outline, I think if we can group related topics together, then users, players, developers or maintainers can easily find the sections most relevant to them, and beginners won't find it intimidating if they want to get started when going through the documentation. Another approach I still think we make it easy is hierarchical categorization, as it can be helpful by providing a hierarchical structure that logically groups topics. tell me what you think about this. I await your response

Going through the documentation, I can provide you with a sample of how I can make it easy to navigate.

@lokytech5 For doing so, please consider the individual target audiences (players, developers, maintainers) and what kind of information they would be looking for. Every audience group should have an easy to find entrypoint that allows them to find the most relevant info as fast as possible and navigate to a deeper level of detail if needed.
If you have questions or need us to shed more light on the various documentation resources and topics we have and their respective audiences, please ask :slightly_smiling_face:

Thank you for the guidance. Based on our discussion, I've outlined a proposed structure for the documentation that considers our three main audiences:
[## Documentation Structure
1. Players
Entry Point: A dedicated 'Player's Guide' landing page with:

Game Introduction
Basic Game Mechanics and Controls
FAQs
Troubleshooting for Players

2. Developers
Entry Point: A 'Developer's Hub' landing page with:

Quickstart for Modding
Tutorials (like those listed under 'Tutorial Modules')
API Documentation (Entity System, Events, Module System, etc.)
Troubleshooting for Developers

3. Maintainers
Entry Point: A 'Maintainer's Dashboard' with:

Code and Documentation Conventions
Release Processes
Testing and Quality Assurance Guides
Maintenance Procedures
]

To maintain a clean and hierarchical flow, all other detailed topics will be nested under these main categories. Before moving forward, it would be great to understand any specific priorities or topics you think should be featured prominently. Also Considering the breadth of content and to ensure quality, I plan to work on each section gradually. This phased approach will allow for more focused work, potential feedback iterations, and corrections if needed. Please review the proposed structure and let me know your thoughts. If this aligns with your vision for the documentation, I would appreciate a green light to start working on the "Player's Guide" section as the initial phase.

Hi @lokytech5
I believe the suggested structure make sense as long as we make sure that the respective "landing pages" / "dashboards" are well-accessible for the respective audience groups.
From your perspective, what do you think would be a suitable entry point for the three audience groups to get to the landing page / dashboard that's relevant for them?
Also, if you do this, I would request that you implement this using docsify as we started in a subset of our modules already, for instance Health (see especially the docs folder  and the resulting documentation) as it allows you to experience the contributor workflow including PRs and reviews and it allows us to easier provide feedback, track the changes and compare before eventually disabling the wiki or rollback to it if necessary (e.g. if the contribution is not completed and we don't have sufficient capacity to complete it.
A big concern that we currently have when it comes to migrating the documentation to docsify (or anywhere else really) is that existing links to the current wiki break. Do you have any ideas on how you would approach this problem?
I'd like to discuss your current proposal and any ideas, comments, or concerns you might have on what I wrote above in our weekly reviver meeting on Sunday (6PM UTC on Discord). If the other available revivers are fine with it as well, I don't see a reason not to give you a green light :slightly_smiling_face:

Hello @jdrueckert,
I hope you're well. In response to your question, I have come up with the following suggestions to ensure our audience groups have easy access to their respective "landing pages" or "dashboards":
Players:

A banner on the documentation's main homepage directing users to the 'Player's Guide'.
A clearly labelled "Start Playing" button or icon on the homepage.
For newcomers, an interactive guide on the main game page that introduces basic mechanics.

Developers:

A dedicated "Developers" tab in our top navigation leading directly to the 'Developer's Hub'.
Engaging code snippets on the homepage to draw attention to development sections.
A regular "Developer Updates" section highlighting new features, modding opportunities, and other relevant news.

Maintainers:

A direct link to the 'Maintainer's Dashboard' in the website's footer.
In our navigation sidebar, a section explicitly labelled for maintainers for easy access.
Alerts or notifications on the homepage for any critical updates or changes that maintainers should be aware of.

I also recommend enhancing search functionality to ensure users can navigate to their desired sections easily. Additionally, tailoring the homepage based on user roles or profiles (players, developers, maintainers) could further enhance the user experience.
Addressing the concern of migrating documentation without breaking existing links — while I haven't encountered this issue before, I've done some research. With the pros and cons of each method in mind, I believe a combination of the following approaches might be suitable:

  Retain Original URL Structures where possible.

  Use 301 Permanent Redirects for content that needs to be moved. This can be combined with a link map to ensure comprehensive coverage.

  Implement an enhanced 404 fallback page to guide users back on track if they encounter broken links.

  Engage the Community: Let the community spot and report issues as they use the documentation. We can then quickly address such reported issues.

Regarding docsify, I've looked over the repository you shared. I observed that the docs folder contains all the documentation, with each markdown file representing a page. I also noticed that the repository's homepage aligns with the documentation's homepage. While exploring its plugins, many can be leveraged to structure the documentation effectively. However, I will need some time to familiarize myself with its features and plugins. Considering the link issues, creating a custom redirection plugin seems a potential solution.
Lastly, regarding the weekly reviver meeting on Sunday at 6PM UTC on Discord, I'm eager to join and discuss further. Could you please provide details or an invitation to attend? Thank you.

I haven't properly caught-up on everything here yet but I can address at least one point.

Lastly, regarding the weekly reviver meeting on Sunday at 6PM UTC on Discord, I'm eager to join and discuss further. Could you please provide details or an invitation to attend? Thank you.

@lokytech5
The discord information is in the project README. Just in-case you missed it:

Invite Link: https://discord.gg/terasology
Meeting Channel: #epheremeral-discussions

Documentation is likely to be discussed in the meeting today (6PM UTC).

@lokytech5 First of all, good news. You have our official go :wink:
As mentioned earlier, we'd like you to use docsify for the documentation improvements. #5155 migrates the existing documentation in its current state from the wiki to docsify in the docs folder. It also mentions in its description how to locally test it.
Until #5155 is merged, you'll need to base any PR of yours on that PR to have the migrated base state available. We'll try to get #5155 merged as soon as possible (once we fixed our infra issues) so you can also work on latest develop.
Looking forward to your contributions! Feel free to work in small iterations and open PRs as drafts as soon as possible so we can check if it's going into the right direction and also have a basis for discussion and answering questions. Once you're happy with your current iteration, you can set the PR to ready-for-review. Explicitly asking for a review on our Discord can help getting faster feedback.

Regarding the suggested access points for the different target groups:

Players:

A banner on the documentation's main homepage directing users to the 'Player's Guide'.

How would a player find the documentation's homepage?

A clearly labelled "Start Playing" button or icon on the homepage.

What exactly would that button do?

For newcomers, an interactive guide on the main game page that introduces basic mechanics.

Gameplay mechanics are very dependent on the activated modules, which is why this kind of documentation should be in-game rather than outside of the game. At most, more detailed explanations could be in module documentation, but not in the main (engine wiki) docs... At some point in the future, module dosc should be integrated at least to a degree with the main (engine wiki) docs, though.

Developers:

A dedicated "Developers" tab in our top navigation leading directly to the 'Developer's Hub'.

I'd propose to already engage the different target audience groups on the main page of the docs for easy navigation to their "documentation hubs".

Engaging code snippets on the homepage to draw attention to development sections.

What do you consider an "engaging code snippet"?

A regular "Developer Updates" section highlighting new features, modding opportunities, and other relevant news.

We do have that in the form of release notes and blog posts, the former of which could be considered documentation to a degree while the latter is outreach. I don't think this kind of information should be part of the (engine wiki) docs.

Maintainers:

A direct link to the 'Maintainer's Dashboard' in the website's footer.

Why in the website's footer? :thinking:
I would expect people that visit the website to rather be interested / first-time players and developers rather than active maintainers... wouldn't you?

In our navigation sidebar, a section explicitly labelled for maintainers for easy access.

:+1: Same as for developers, I would already link to that section from the main page of the docs.

Alerts or notifications on the homepage for any critical updates or changes that maintainers should be aware of.

That, again, is not really documentation. But communicating critical updates or changes (for instance current blockers or required actions etc) is an important and relevant topic. It's just out-of-scope for this documentation-focused issue :wink:

Regarding the broken link concern:

Retain Original URL Structures where possible.

Absolutely, that will at least help with relative links within the wiki/docs.

Use 301 Permanent Redirects for content that needs to be moved. This can be combined with a link map to ensure comprehensive coverage.
Implement an enhanced 404 fallback page to guide users back on track if they encounter broken links.

We actually don't have too much control over that. Anything hosted on GitHub pages will probably also drop HTML redirect logic for safety reasons. Naturally, we can check what's possible and what is not.

Engage the Community: Let the community spot and report issues as they use the documentation. We can then quickly address such reported issues.

We already encourage that. All docsify pages should also have an "Edit on GitHub" link at their top to allow people easy access to the underlying resource and creating a PR with changes.

Thank you for the clear instructions and the green light on this. I am currently reviewing PR #5155 and am familiarizing myself with Docsify. I'll begin working on the documentation improvements in small iterations as suggested and will open draft PRs for early feedback.
I'll keep an eye on the status of PR #5155 and ensure my contributions are based on it until it's merged. If I have any questions or need further clarification, I'll reach out, especially on Discord, for quicker feedback.
I look forward to contributing and enhancing our documentation!
About the points you raised, I am looking into them slowly and will take care of each of them carefully.

Hello @jdrueckert,
I've looked into the docs folder and observed that there's no index.html file or .nojekyll file present. from Docsify documentation, these are automatically generated by Docsify upon initialization. In your merge of PR #5155, you mentioned transitioning the wiki to Docsify.
Before I proceed, I'd like some clarification:

  Should I initialize Docsify within the docs folder, potentially overwriting any existing configuration?

  Or would you prefer I manually create the necessary files?

Looking forward to your guidance on this.

@lokytech5 if you were checking your locally checked-out workspace, please pull the latest changes.
As you can see from https://github.com/MovingBlocks/Terasology/tree/develop/docs, both index.html and .nojekyll are present.

Should I initialize Docsify within the docs folder, potentially overwriting any existing configuration? Or would you prefer I manually create the necessary files?

No, docsify is already properly initialized. With #5155 merged, the wiki content is already migrated to docsify, so you can start working directly on the content structure.

good day, @jdrueckert. I have worked on the homepage, before i make a PR, please kindly go through the images of the new home page I came up with,if it meets your requirements and with your permission,I we create a PR before going to Discord.

![Uploading Screenshot 2023-11-04 at 9.31.21 PM.png…](homepage design)
![Uploading Screenshot 2023-11-04 at 9.32.44 PM.png…](homepage design)
![Uploading Screenshot 2023-11-04 at 9.33.50 PM.png…](homepage design)
![Uploading Screenshot 2023-11-04 at 9.34.51 PM.png…](homepage design)
am currently working on the sidebar.

Good day, @jdrueckert. I have worked on the homepage, before I make a PR, please kindly go through the images of the new home page I came up with, if it meets your requirements and with your permission, I will create a PR before going to Discord.

I am currently working on the sidebar.

Good day, @jdrueckert. I have worked on the homepage, before I make a PR, please kindly go through the images of the new home page I came up with, if it meets your requirements and with your permission, I will create a PR before going to Discord.
link: https://drive.google.com/drive/folders/1aismU43av-mgVyZ_q2TuYe1gh0Ic9hth?usp=sharing
I am currently working on the player Guides. i await your reply.

@lokytech5 please just open a PR and I'll review that. No need for an additional roundtrip with screenshots.


pip3 install and cargo issues

Hello,
Thank you for this script.
I'm trying to build ffmpeg with '--enable-gpl-and-non-free' on my Ubuntu  23.10 x64 machine.
Unfortunately I got stuck 2 times.
First time I've received:
'error: externally-managed-environment'
when script run:
$ pip3 install pip setuptools --quiet --upgrade --no-cache-dir --disable-pip-version-check
So I 've installed python3-venv,  created a new virtual environment 'python3 -m venv myenv' and activated the virtual environment 'source myenv/bin/activate'.
The script moved on but stops building cargo and errors again appear indicating 'undefined reference to ...' some functions/files
See full script log:
https://www.dropbox.com/scl/fi/wr1xvwasm5sjdru4pt89i/script.log?rlkey=0agotbpxlpovsgad7f2qxa9s6&raw=1
What should I do move forward?

rav1e is so sloow, I recommend you to disable it with env var SKIPRAV1E
if you want to use av1 codec - use svtav1

Thanks for leading me to a temporary solution to sacrifice the rarely used librav1e AV1 encoder. I simply uninstalled rust/cargo
rustup self uninstall
and the building script continued to the end.

I'm seeing this issue also on Debian 12 bookworm.
It's not possible to disable SKIPRAV1E using the "Quick install and build" method. There should be a better fix for this.


sklearn is deprecated

Pip has started to refuse to install sklearn sometimes. I believe the dependecy should be updated to scikit-learn?
Step #0 - "Build": Collecting sklearn==0.0.post1
Step #0 - "Build":   Downloading sklearn-0.0.post1.tar.gz (3.6 kB)
Step #0 - "Build":   Preparing metadata (setup.py): started
Step #0 - "Build":   Preparing metadata (setup.py): finished with status 'error'
Step #0 - "Build":   error: subprocess-exited-with-error
Step #0 - "Build":
Step #0 - "Build":   × python setup.py egg_info did not run successfully.
Step #0 - "Build":   │ exit code: 1
Step #0 - "Build":   ╰─> [18 lines of output]
Step #0 - "Build":       The 'sklearn' PyPI package is deprecated, use 'scikit-learn'
Step #0 - "Build":       rather than 'sklearn' for pip commands.
Step #0 - "Build":
Step #0 - "Build":       Here is how to fix this error in the main use cases:
Step #0 - "Build":       - use 'pip install scikit-learn' rather than 'pip install sklearn'
Step #0 - "Build":       - replace 'sklearn' by 'scikit-learn' in your pip requirements files
Step #0 - "Build":         (requirements.txt, setup.py, setup.cfg, Pipfile, etc ...)
Step #0 - "Build":       - if the 'sklearn' package is used by one of your dependencies,
Step #0 - "Build":         it would be great if you take some time to track which package uses
Step #0 - "Build":         'sklearn' instead of 'scikit-learn' and report it to their issue tracker
Step #0 - "Build":       - as a last resort, set the environment variable
Step #0 - "Build":         SKLEARN_ALLOW_DEPRECATED_SKLEARN_PACKAGE_INSTALL=True to avoid this error
Step #0 - "Build":
Step #0 - "Build":       More information is available at
Step #0 - "Build":       https://github.com/scikit-learn/sklearn-pypi-package
Step #0 - "Build":
Step #0 - "Build":       If the previous advice does not cover your use case, feel free to report it at
Step #0 - "Build":       https://github.com/scikit-learn/sklearn-pypi-package/issues/new
Step #0 - "Build":       [end of output]
Step #0 - "Build":
Step #0 - "Build":   note: This error originates from a subprocess, and is likely not a problem with pip.
Step #0 - "Build": error: metadata-generation-failed
Step #0 - "Build":
Step #0 - "Build": × Encountered error while generating package metadata.
Step #0 - "Build": ╰─> See above for output.

Hello @lukehsiao,
Firstly, thank you for your work on the pdftotree project – it's been incredibly helpful. I've noticed that there are some dependency issues in the current version, and it seems that there are several pull requests addressing these concerns.
I understand that maintaining a project can be time-consuming, and I truly appreciate the effort you've put into it so far. Would it be possible to consider merging the pending changes that fix the dependency issues? This would greatly benefit the user community and ensure the continued success of pdftotree.
Your contributions are invaluable, and we're grateful for your ongoing dedication to the project. Thank you for your time and consideration.
Best regards,

@HodeiG I'm sorry, I'm afraid I no longer have write access to this repository, consequently, I have no way to help maintain it even if I wanted to.
If you're using this repository, I suspect this is a good candidate for forking, as I would not expect anyone to continue to maintain it in the current organization.


refactor: remove gor and sn-tn networks

Summary
This PR removes gor and sn-tn networks as Goerli got deprecated.
Closes: https://github.com/snapshot-labs/sx-monorepo/issues/324
How to test

Everything should keep working.

Waiting for thegraph index fix, to test space creation

Waiting for thegraph index fix, to test space creation

Is it still broken? It should work.

Was able to create eth sepolia as usual, but hit an error for starknet sepolia

@wa0x6e have you use Whitelist voting strategy there?
Only thing I could see causing it is if you used whitelist strategy with invalid format (it needs to be address:VP in each line).
I tried reproducing otherwise and had no issues - probably should add some kind of validation for it, but would be great if you can confirm if it was this issue or something else.


issues installing

tall"
npm ERR! node v7.0.0
npm ERR! npm  v3.10.8
npm ERR! code ELIFECYCLE
npm ERR<EMAIL_ADDRESS>postinstall: bash build_ios_frameworks.sh
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>postinstall script 'bash build_ios_frameworks.sh'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the react-native-ble-plx package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     bash build_ios_frameworks.sh
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs react-native-ble-plx
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls react-native-ble-plx
npm ERR! There is likely additional logging output above.

Solved by using Git BASH. Windows doesn't support .sh files on command line.

I'll definitely check out and fix that issue on Windows.

I found this solution.
http://www.windowscentral.com/how-install-bash-shell-command-line-windows-10

Darwin 15.3.0
npm ERR! argv "/usr/local/bin/node" "/usr/local/bin/npm" "install" "--save" "react-native-ble-plx"
npm ERR! node v7.9.0
npm ERR! npm  v4.2.0
npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>postinstall: bash build_ios_frameworks.sh
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>postinstall script 'bash build_ios_frameworks.sh'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the react-native-ble-plx package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     bash build_ios_frameworks.sh
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs react-native-ble-plx
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls react-native-ble-plx
npm ERR! There is likely additional logging output above.
npm ERR! Please include the following file with any support request:
npm ERR!     /Users/marwazribi/.npm/_logs/2017-05-05T14_59_43_032Z-debug.log
and i'm a mac user

Can you check version 0.5.0 if this still happens?

132426 verbose stack Error<EMAIL_ADDRESS>postinstall: bash build_ios_frameworks.sh

Is it your full error? Can you enter library directory and run this script by hand?:
cd ./node_modules/react-native-ble-plx/
./build_ios_frameworks.sh

@Cierpliwy I ran the script on windows and I cant run sh without bash.

Probably I will need to create node based script to be platform agnostic. If you want this feature faster, maybe you can tackle this problem and send me PR?

@Cierpliwy I have the same error on examples/ReactBLEScanner

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR<EMAIL_ADDRESS>postinstall: bash build_ios_frameworks.sh
npm ERR! Exit status 1
npm ERR!
npm ERR! Failed at the<EMAIL_ADDRESS>postinstall script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

xcode 9


Speed up cloning of Git submodules

We can use --jobs parameter with git submodule update --init --recursive to set the number of parallel git clone jobs. This can speed up the deployments.
Not sure which value is safe though. On my local I use --jobs=6 and I never had an issue.
I can make a PR if this proposal sounds good to you.

Hi @Leksat
Good idea, came across the issue of slow submodule checkouts too. Can you create a PR. We'll be happily review that!

PR is ready. Test is green, but I have no idea if it tests the cloning of git submodules.

yea, there is no real possibility to test this, so all good! merged, thanks a lot.


IDomainCommand.ExpectedVersion

This would be helpful in addressing situations that could have race conditions. This way the command handler can check if the aggregate has changed since the time the command was issued by the client.
int ExpectedVersion { get; }

I was thinking about this feature for a long time.
I'll add it in one of the next versions, probably 5.2.

It will be nullable

I think it makes more sense to add an "OriginalVersion" instead of "ExpectedVersion" because an expected version implies a calculation that should be done in the event store not at the command level.

I'd suggest that ExpectedVersion matches the usual semantics I've seen in various CQRS projects I've done or have seen in the wild. For me, calling it OriginalVersion would be increased friction.
The intended use case, as I've thought of it anyway, is for a CommandHandler to compare the 'command.ExpectedVersion' with the 'aggregate.Version'. If they don't line up, it means the aggregate has changed since the command was issued, and appropriate action should be taken per business rules.

Okay, I got your point and I can see 2 possible scenarios.

You need to take an appropriate action based on your business rules. In this case you might need the "ExpectedVersion" and the command handler should manage the exception.
You NOT need to take any actions, just throw a concurrency exception if needed. In this case you might just need an "OriginalVersion" and let the framework handle the validation and the exception for you.

Released


Support user-created data fields

Highcharts supports user-created data fields for sticking extra data into tooltips; see e.g. this Stack Overflow thread.
The trick in the accepted answer does not seem to work with python-highcharts, though.
Could you add such support?

I was stuck with the same problem: in my case, not even "standard" attributes like "low", "high", "y" would work when creating the data list "by hand". I then figured out that the problem was with my datatypes, i.e. timedate-like values. I got errors like:

Traceback (most recent call last):
File "tweets.py", line 201, in
chart.add_data_set(newdata, 'columnrange', 'Time Span', marker={'enabled': False})
File "/home/francesca/anaconda3/lib/python3.4/site-packages/highcharts/highcharts/highcharts.py", line 203, in add_data_set
series_data = Series(data, series_type=series_type, **kwargs)
File "/home/francesca/anaconda3/lib/python3.4/site-packages/highcharts/highcharts/highchart_types.py", line 629, in init
if SeriesOptions.validate_options(k,v,DATA_SERIES_ALLOWED_OPTIONS[k]):
File "/home/francesca/anaconda3/lib/python3.4/site-packages/highcharts/highcharts/highchart_types.py", line 512, in validate_options
raise OptionTypeError("Option Type Currently Not Supported: %s" % k)
highcharts.highcharts.highchart_types.OptionTypeError: Option Type Currently Not Supported: x

I temporarily solved this by commenting out the line setting the accepted types for 'x' in DATA_SERIES_ALLOWED_OPTIONS.  Now both standard and custom attributes to data work fine. This is clearly just a patch and by no means intended to be the final solutions, but it shows that custom data attributes are supported already. Hope it helps.


Is this compatible with Eagle 5.0 & pycoral API?

Looks like it's complaining for "edgetpu" => ModuleNotFoundError: No module named 'edgetpu'
Is there a quick workaround for using the previous API instead? Like flashing Day 13?
Thx

Hello, yes, we are working on getting this repo fixed ASAP with our new API.
For now, please use my fork: https://github.com/Namburger/project-posenet

@mike122112 Is this issue still reproducible ?


[7.x] Allow Eloquent Model can self Morphology without pivot

How to use Self Morph Model Trait
Sometime we need an eloquent model can self morph by the database table config.
1. Base class
<?php
use Illuminate\Database\Eloquent\Concerns\SelfMorphModel;

class Product extends \Illuminate\Database\Eloquent\Model
{
use SelfMorphModel;
}

class SimpleProduct extends Product
{
}

class BundleProduct extends Product
{
}

2. Register in Provider class
<?php

//on Provider class to register
class Provider extends \Illuminate\Support\ServiceProvider
{
   public function register()
   {
      SimpleProduct::registerMorph('simple');
      BundleProduct::registerMorph('bundle');
   }
}

3. Prepare database table 'products'

//products table
id morph_type name
1  simple.        simpleproductname1
2  bundle.       bundleproductname1

4. Self Morph result
Product::find(1).    will return instance of SimpleProduct
Product::find(2).    will return instance of BundleProduct

Looks like a signature change to a public method. If so, would need to go to master.
In 7.x, causes the following:
Declaration of App\Models\Model::newInstance($attributes = Array, $exists = false) should be compatible with Illuminate\Database\Eloquent\Model::newInstance($attributes = Array, $exists = false, $fillAttributes = true)

Thanks for your pull request to Laravel!
Unfortunately, I'm going to delay merging this code for now. To preserve our ability to adequately maintain the framework, we need to be very careful regarding the amount of code we include.
If possible, please consider releasing your code as a package so that the community can still take advantage of your contributions!
If you feel absolutely certain that this code corrects a bug in the framework, please "@" mention me in a follow-up comment with further explanation so that GitHub will send me a notification of your response.


wallet-ext: move loading api env from storage at app init

to avoid unnecessary requests use the saved api env during initialisation of the store. (We only need the saved env once)

Moved back in TODO since this is not done

Just to clarify about https://github.com/MystenLabs/sui/pull/2913. It fixes the issue of loading twice when the stored ApiEnv is different from the default one but the race condition is still there. I think we should simplify this and initialise the state from the storage or the default env and always use the value from state. It's not urgent now that loading twice is addressed but we should update it at some point (potentially when moving the API functionality to the background service)
Closing this after discussing with @Jibz1 (to avoid confusion) and opened #2936


Improve error messages for missing languages in CMS

Currently, when you create a slot with no content in system default language (e.g. only adding stuff in german while english is default, you only get a totally unspecific “something went wrong” error, leaving the users totally in the dark.
This should be tested with page, section and block duplication
We should improve this.

[public] Shopware automated response: This issue is linked to the internal issue: NEXT-39089
[created from NEXT-39089, comment 504539]


Update README to reflect DynamoDB migration

As a result of the work done by me and Sean today the database has
moved from postgres in PaaS land to DynamoDB, there are new steps that a
developer needs to take to get this running locally, these changes are
now reflected in the README

Just testing this out locally...

thanks all 👍


DM-39348: Fixup for v25

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

3 of 3 (100.0%)  changed or added relevant lines in 2 files are covered.
43 unchanged lines in 3 files lost coverage.
Overall coverage decreased (-0.4%) to 87.293%

Files with Coverage Reduction
New Missed Lines
%

spectractor/extractor/spectrum.py
6
91.84%

spectractor/fit/fitter.py
16
88.58%

spectractor/fit/fit_multispectra.py
21
67.39%

Totals

Change from base Build<PHONE_NUMBER>:
-0.4%

Covered Lines:
7158

Relevant Lines:
8200

💛  - Coveralls


Recovery cases

Fixed the doughnut chart on the dashboard & the recovered cases has been replaced with real data.
Fixed bugs like age calculation and id to email.
Temporarily I've removed the upcoming events and the buttons with it

should merge to develop first

now merging to develop, good


add aliyun source , speedup install

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.zhenghong seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.


Test issue #15

Trying to force GitHub to re-calculate the issue count in order to make the ghost "1 issue needs help" label go away

Finally, the ghost "1 issue needs help" label is gone


reduce dataset size

I use following command to train pointTr
 bash ./scripts/train.sh 0 --config ./cfgs/KITTI_models/PoinTr.yaml --exp_name example.
The training speed is slow, I tried to reduce both the size of KITTI dataset, PCN dataset and batch size of PointTr using following command to fast the training speed, but the training speed is still slow. what else should I to modify to reduce the dataset?
# open terminal in Colab, type the below in the command line:
# open /content/pointr/cfgs/KITTI_models/PoinTr.yaml
# set total_bs : 2 max_epoch : 100
# open /content/pointr/cfgs/dataset_configs/PCNCars.yaml
# open /content/pointr/cfgs/dataset_configs/KITTI.yaml
# set N_points: 163
# open /content/pointr/datasets/PCNDataset.py
# set data['gt'][:163]

/content/pointr
/usr/local/envs/myenv/bin/python
+ GPUS=0
+ PY_ARGS='--config ./cfgs/KITTI_models/PoinTr.yaml --exp_name example'
+ CUDA_VISIBLE_DEVICES=0
+ python main.py --config ./cfgs/KITTI_models/PoinTr.yaml --exp_name example
2022-11-23 22:49:54,392 - PoinTr - INFO - Copy the Config file from ./cfgs/KITTI_models/PoinTr.yaml to ./experiments/PoinTr/KITTI_models/example/config.yaml
2022-11-23 22:49:54,393 - PoinTr - INFO - args.config : ./cfgs/KITTI_models/PoinTr.yaml
2022-11-23 22:49:54,393 - PoinTr - INFO - args.launcher : none
2022-11-23 22:49:54,393 - PoinTr - INFO - args.local_rank : 0
2022-11-23 22:49:54,393 - PoinTr - INFO - args.num_workers : 4
2022-11-23 22:49:54,393 - PoinTr - INFO - args.seed : 0
2022-11-23 22:49:54,393 - PoinTr - INFO - args.deterministic : False
2022-11-23 22:49:54,394 - PoinTr - INFO - args.sync_bn : False
2022-11-23 22:49:54,394 - PoinTr - INFO - args.exp_name : example
2022-11-23 22:49:54,394 - PoinTr - INFO - args.start_ckpts : None
2022-11-23 22:49:54,394 - PoinTr - INFO - args.ckpts : None
2022-11-23 22:49:54,394 - PoinTr - INFO - args.val_freq : 1
2022-11-23 22:49:54,394 - PoinTr - INFO - args.resume : False
2022-11-23 22:49:54,394 - PoinTr - INFO - args.test : False
2022-11-23 22:49:54,394 - PoinTr - INFO - args.mode : None
2022-11-23 22:49:54,394 - PoinTr - INFO - args.experiment_path : ./experiments/PoinTr/KITTI_models/example
2022-11-23 22:49:54,394 - PoinTr - INFO - args.tfboard_path : ./experiments/PoinTr/KITTI_models/TFBoard/example
2022-11-23 22:49:54,394 - PoinTr - INFO - args.log_name : PoinTr
2022-11-23 22:49:54,394 - PoinTr - INFO - args.use_gpu : True
2022-11-23 22:49:54,394 - PoinTr - INFO - args.distributed : False
2022-11-23 22:49:54,394 - PoinTr - INFO - config.optimizer = edict()
2022-11-23 22:49:54,394 - PoinTr - INFO - config.optimizer.type : AdamW
2022-11-23 22:49:54,394 - PoinTr - INFO - config.optimizer.kwargs = edict()
2022-11-23 22:49:54,394 - PoinTr - INFO - config.optimizer.kwargs.lr : 0.0001
2022-11-23 22:49:54,395 - PoinTr - INFO - config.optimizer.kwargs.weight_decay : 0.0005
2022-11-23 22:49:54,395 - PoinTr - INFO - config.scheduler = edict()
2022-11-23 22:49:54,395 - PoinTr - INFO - config.scheduler.type : LambdaLR
2022-11-23 22:49:54,395 - PoinTr - INFO - config.scheduler.kwargs = edict()
2022-11-23 22:49:54,395 - PoinTr - INFO - config.scheduler.kwargs.decay_step : 21
2022-11-23 22:49:54,395 - PoinTr - INFO - config.scheduler.kwargs.lr_decay : 0.9
2022-11-23 22:49:54,395 - PoinTr - INFO - config.scheduler.kwargs.lowest_decay : 0.02
2022-11-23 22:49:54,395 - PoinTr - INFO - config.bnmscheduler = edict()
2022-11-23 22:49:54,395 - PoinTr - INFO - config.bnmscheduler.type : Lambda
2022-11-23 22:49:54,395 - PoinTr - INFO - config.bnmscheduler.kwargs = edict()
2022-11-23 22:49:54,395 - PoinTr - INFO - config.bnmscheduler.kwargs.decay_step : 21
2022-11-23 22:49:54,395 - PoinTr - INFO - config.bnmscheduler.kwargs.bn_decay : 0.5
2022-11-23 22:49:54,395 - PoinTr - INFO - config.bnmscheduler.kwargs.bn_momentum : 0.9
2022-11-23 22:49:54,395 - PoinTr - INFO - config.bnmscheduler.kwargs.lowest_decay : 0.01
2022-11-23 22:49:54,395 - PoinTr - INFO - config.dataset = edict()
2022-11-23 22:49:54,395 - PoinTr - INFO - config.dataset.train = edict()
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train._base_ = edict()
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train._base_.NAME : PCN
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train._base_.CATEGORY_FILE_PATH : data/PCN/PCN.json
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train._base_.N_POINTS : 163
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train._base_.N_RENDERINGS : 8
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train._base_.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train._base_.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train._base_.CARS : True
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train.others = edict()
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train.others.subset : train
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.train.others.bs : 2
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.val = edict()
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.val._base_ = edict()
2022-11-23 22:49:54,396 - PoinTr - INFO - config.dataset.val._base_.NAME : PCN
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.val._base_.CATEGORY_FILE_PATH : data/PCN/PCN.json
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.val._base_.N_POINTS : 163
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.val._base_.N_RENDERINGS : 8
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.val._base_.PARTIAL_POINTS_PATH : data/PCN/%s/partial/%s/%s/%02d.pcd
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.val._base_.COMPLETE_POINTS_PATH : data/PCN/%s/complete/%s/%s.pcd
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.val._base_.CARS : True
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.val.others = edict()
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.val.others.subset : test
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.test = edict()
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.test._base_ = edict()
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.test._base_.NAME : KITTI
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.test._base_.CATEGORY_FILE_PATH : data/KITTI/KITTI.json
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.test._base_.N_POINTS : 163
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.test._base_.CLOUD_PATH : data/KITTI/cars/%s.pcd
2022-11-23 22:49:54,397 - PoinTr - INFO - config.dataset.test._base_.BBOX_PATH : data/KITTI/bboxes/%s.txt
2022-11-23 22:49:54,398 - PoinTr - INFO - config.dataset.test.others = edict()
2022-11-23 22:49:54,398 - PoinTr - INFO - config.dataset.test.others.subset : test
2022-11-23 22:49:54,398 - PoinTr - INFO - config.model = edict()
2022-11-23 22:49:54,398 - PoinTr - INFO - config.model.NAME : PoinTr
2022-11-23 22:49:54,398 - PoinTr - INFO - config.model.num_pred : 1600
2022-11-23 22:49:54,398 - PoinTr - INFO - config.model.num_query : 100
2022-11-23 22:49:54,398 - PoinTr - INFO - config.model.knn_layer : 1
2022-11-23 22:49:54,398 - PoinTr - INFO - config.model.trans_dim : 384
2022-11-23 22:49:54,398 - PoinTr - INFO - config.total_bs : 2
2022-11-23 22:49:54,398 - PoinTr - INFO - config.step_per_update : 1
2022-11-23 22:49:54,398 - PoinTr - INFO - config.max_epoch : 100
2022-11-23 22:49:54,398 - PoinTr - INFO - config.consider_metric : CDL1
2022-11-23 22:49:54,398 - PoinTr - INFO - Distributed training: False
2022-11-23 22:49:54,398 - PoinTr - INFO - Set random seed to 0, deterministic: False
2022-11-23 22:49:54,408 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02958343, Name=car]
2022-11-23 22:49:54,458 - PCNDATASET - INFO - Complete collecting files of the dataset. Total files: 5677
2022-11-23 22:49:54,466 - PCNDATASET - INFO - Collecting files of Taxonomy [ID=02958343, Name=car]
2022-11-23 22:49:54,466 - PCNDATASET - INFO - Complete collecting files of the dataset. Total files: 150
2022-11-23 22:49:54,467 - MODEL - INFO -  Transformer with knn_layer 1
2022-11-23 22:49:57,825 - PoinTr - INFO - Using Data parallel ...
2022-11-23 22:49:58,171 - PoinTr - INFO - padding while KITTI training
2022-11-23 22:49:59,764 - PoinTr - INFO - [Epoch 0/100][Batch 1/2838] BatchTime = 1.929 (s) DataTime = 0.336 (s) Losses = ['171.1774', '217.5374'] lr = 0.000100
2022-11-23 22:50:13,609 - PoinTr - INFO - [Epoch 0/100][Batch 101/2838] BatchTime = 0.181 (s) DataTime = 0.002 (s) Losses = ['49.5538', '81.0673'] lr = 0.000100
2022-11-23 22:50:26,799 - PoinTr - INFO - [Epoch 0/100][Batch 201/2838] BatchTime = 0.098 (s) DataTime = 0.002 (s) Losses = ['45.2846', '51.2348'] lr = 0.000100
2022-11-23 22:50:37,271 - PoinTr - INFO - [Epoch 0/100][Batch 301/2838] BatchTime = 0.101 (s) DataTime = 0.002 (s) Losses = ['42.0884', '43.5485'] lr = 0.000100
2022-11-23 22:50:49,402 - PoinTr - INFO - [Epoch 0/100][Batch 401/2838] BatchTime = 0.192 (s) DataTime = 0.002 (s) Losses = ['53.3049', '49.0148'] lr = 0.000100
2022-11-23 22:51:04,518 - PoinTr - INFO - [Epoch 0/100][Batch 501/2838] BatchTime = 0.105 (s) DataTime = 0.002 (s) Losses = ['40.6047', '41.6563'] lr = 0.000100
2022-11-23 22:51:14,983 - PoinTr - INFO - [Epoch 0/100][Batch 601/2838] BatchTime = 0.106 (s) DataTime = 0.002 (s) Losses = ['41.7752', '37.6137'] lr = 0.000100
2022-11-23 22:51:25,518 - PoinTr - INFO - [Epoch 0/100][Batch 701/2838] BatchTime = 0.146 (s) DataTime = 0.002 (s) Losses = ['40.7025', '31.4210'] lr = 0.000100
2022-11-23 22:51:41,937 - PoinTr - INFO - [Epoch 0/100][Batch 801/2838] BatchTime = 0.099 (s) DataTime = 0.002 (s) Losses = ['39.4911', '42.2679'] lr = 0.000100
2022-11-23 22:51:52,448 - PoinTr - INFO - [Epoch 0/100][Batch 901/2838] BatchTime = 0.103 (s) DataTime = 0.002 (s) Losses = ['42.6667', '31.6663'] lr = 0.000100
2022-11-23 22:52:02,845 - PoinTr - INFO - [Epoch 0/100][Batch 1001/2838] BatchTime = 0.110 (s) DataTime = 0.002 (s) Losses = ['41.8228', '32.8068'] lr = 0.000100
2022-11-23 22:52:18,260 - PoinTr - INFO - [Epoch 0/100][Batch 1101/2838] BatchTime = 0.188 (s) DataTime = 0.002 (s) Losses = ['67.8683', '63.7105'] lr = 0.000100
2022-11-23 22:52:34,076 - PoinTr - INFO - [Epoch 0/100][Batch 1201/2838] BatchTime = 0.151 (s) DataTime = 0.002 (s) Losses = ['48.7156', '49.9901'] lr = 0.000100
2022-11-23 22:52:46,241 - PoinTr - INFO - [Epoch 0/100][Batch 1301/2838] BatchTime = 0.178 (s) DataTime = 0.002 (s) Losses = ['43.3571', '40.0096'] lr = 0.000100
2022-11-23 22:53:02,159 - PoinTr - INFO - [Epoch 0/100][Batch 1401/2838] BatchTime = 0.102 (s) DataTime = 0.002 (s) Losses = ['40.2321', '33.9443'] lr = 0.000100
2022-11-23 22:53:12,618 - PoinTr - INFO - [Epoch 0/100][Batch 1501/2838] BatchTime = 0.106 (s) DataTime = 0.001 (s) Losses = ['46.2681', '50.6158'] lr = 0.000100
2022-11-23 22:53:23,167 - PoinTr - INFO - [Epoch 0/100][Batch 1601/2838] BatchTime = 0.104 (s) DataTime = 0.002 (s) Losses = ['38.4925', '40.9973'] lr = 0.000100
2022-11-23 22:53:39,609 - PoinTr - INFO - [Epoch 0/100][Batch 1701/2838] BatchTime = 0.108 (s) DataTime = 0.002 (s) Losses = ['38.7526', '42.9651'] lr = 0.000100
2022-11-23 22:53:49,840 - PoinTr - INFO - [Epoch 0/100][Batch 1801/2838] BatchTime = 0.106 (s) DataTime = 0.002 (s) Losses = ['41.1473', '35.3604'] lr = 0.000100
2022-11-23 22:54:00,102 - PoinTr - INFO - [Epoch 0/100][Batch 1901/2838] BatchTime = 0.114 (s) DataTime = 0.002 (s) Losses = ['37.8629', '46.1669'] lr = 0.000100
2022-11-23 22:54:14,742 - PoinTr - INFO - [Epoch 0/100][Batch 2001/2838] BatchTime = 0.144 (s) DataTime = 0.002 (s) Losses = ['40.1596', '41.9645'] lr = 0.000100
2022-11-23 22:54:27,402 - PoinTr - INFO - [Epoch 0/100][Batch 2101/2838] BatchTime = 0.104 (s) DataTime = 0.002 (s) Losses = ['41.6201', '34.9512'] lr = 0.000100
2022-11-23 22:54:37,745 - PoinTr - INFO - [Epoch 0/100][Batch 2201/2838] BatchTime = 0.102 (s) DataTime = 0.001 (s) Losses = ['39.5171', '41.0923'] lr = 0.000100
2022-11-23 22:54:50,710 - PoinTr - INFO - [Epoch 0/100][Batch 2301/2838] BatchTime = 0.180 (s) DataTime = 0.002 (s) Losses = ['40.5638', '43.5947'] lr = 0.000100
2022-11-23 22:55:05,328 - PoinTr - INFO - [Epoch 0/100][Batch 2401/2838] BatchTime = 0.183 (s) DataTime = 0.002 (s) Losses = ['41.9312', '34.2122'] lr = 0.000100
2022-11-23 22:55:21,392 - PoinTr - INFO - [Epoch 0/100][Batch 2501/2838] BatchTime = 0.104 (s) DataTime = 0.002 (s) Losses = ['43.6710', '37.7401'] lr = 0.000100
2022-11-23 22:55:36,679 - PoinTr - INFO - [Epoch 0/100][Batch 2601/2838] BatchTime = 0.150 (s) DataTime = 0.002 (s) Losses = ['43.4865', '35.1678'] lr = 0.000100
2022-11-23 22:55:47,951 - PoinTr - INFO - [Epoch 0/100][Batch 2701/2838] BatchTime = 0.100 (s) DataTime = 0.002 (s) Losses = ['47.0441', '47.0863'] lr = 0.000100
2022-11-23 22:55:58,329 - PoinTr - INFO - [Epoch 0/100][Batch 2801/2838] BatchTime = 0.111 (s) DataTime = 0.002 (s) Losses = ['39.8397', '32.3641'] lr = 0.000100
2022-11-23 22:56:02,181 - PoinTr - INFO - [Training] EPOCH: 0 EpochTime = 364.346 (s) Losses = ['43.7548', '45.2192']
2022-11-23 22:56:03,683 - PoinTr - INFO - Save checkpoint at ./experiments/PoinTr/KITTI_models/example/ckpt-last.pth

Hi,
How many gpus are used in your env?  and what kind?
BTW, reduce dataset size may inference the performance of the trained models.

open /content/pointr/cfgs/dataset_configs/PCNCars.yaml
open /content/pointr/cfgs/dataset_configs/KITTI.yaml
set N_points: 163

This operation can not speed up the PoinTr, since we process the point cloud into a fixed number of point proxies. To speed up, you can try to reduce the depth of Transformer decoder and encoder. reduce the hidden dim of transformer (384 -> 192), reduce the hidden channels of FoldNet in PoinTr...

I use colab to run the training and I use single GPU, The GPU information is here.
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.32.03    Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla T4            Off  | 00000000:00:04.0 Off |                    0 |
| N/A   37C    P8     9W /  70W |      0MiB / 15109MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

I use following code to train.
bash ./scripts/train.sh 0 --config ./cfgs/KITTI_models/PoinTr.yaml --exp_name example

consider to set the batch_size to 32 (https://github.com/yuxumin/PoinTr/blob/master/cfgs/KITTI_models/PoinTr.yaml#L33).
trans_dim can be set to 192 for speeding up the training phase


Rfactor fix

Fixed rfactor for certain cases.

Reduction using add, mul, sub, div can have some op done on the lhs or
rhs.
For example:
product = tvm.comm_reducer(lambda x, y: x + y * y,
    lambda t: tvm.const(1, dtype=t), name="product")
y is multiplied before sum.
In cases like this, if rfactor is applied on reduction axis it produces
wrong result. This PR fixes the case when such operations re done on
rhs.
Does not fix this case which is still broken:
product = tvm.comm_reducer(lambda x, y: x * x+ y,
    lambda t: tvm.const(1, dtype=t), name="product")

CC: @ZihengJiang, @tqchen

@ZihengJiang, @tqchen, sending a gentle reminder. Thanks!

Agree with @yinghai , we do need the reducer to be communicative when using rfactor, and the case you mentioned is not and requires a strict scan order.

@yinghai @tqchen agreed that the pattern in not commutative, however the particular pattern of compute I am doing is for layer norm's mean and variance calculation, in which I would like to iterate over the array once and compute. So right now, comm reducer, provides the way to do both sum of elements as well sum of product of elements. Do you think it is better to introduce a separate kind of reducer?

For your case, I think we could introduce a separate kind of comm_reducer that takes tuple elements and produce both https://docs.tvm.ai/tutorials/language/tuple_inputs.html#sphx-glr-tutorials-language-tuple-inputs-py

Is this enough?
https://gist.github.com/yinghai/8c212fe26294fc574c76494423f60eb5

@tqchen, yes that would be nice. I tried to do something like this inspiring from your example.
import tvm
n = tvm.var("n")
m = tvm.var("m")
A0 = tvm.placeholder((m, n), name='A0')
k = tvm.reduce_axis((0, n), 'k')
B0, B1 = tvm.compute((m, ), lambda i: (tvm.sum(A0[i, k], axis=k), tvm.sum(A0[i, k] * A0[i, k], axis=k)), name='B')
s = tvm.create_schedule(B0.op)
print(tvm.lower(s, [A0, B0, B1], simple_mode=True))

But that does not quite work out of box. I tried some hacks but I think this requires new compute pattern as you suggested.


nci-openmpi: use env.append_flags()

Use env.append_flags() instead of env.append_path().
Thanks to Aidan Heerdegen for finding env.append_flags().

With Spack v0.20, the bug was:
  Imported target "MPI::MPI_Fortran" includes non-existent path

    "/apps/openmpi/4.0.2/include/Intel:-I/apps/openmpi/4.0.2/include/Intel"

With this patch on Spack v0.20:
OMPI_FCFLAGS -I/apps/openmpi/4.0.2/include/Intel
OMPI_LDFLAGS -L/apps/openmpi/4.0.2/lib -L/apps/openmpi/4.0.2/lib/Intel

With this patch on Spack v0.21:
OMPI_FCFLAGS -I/apps/openmpi/4.0.2/include/Intel -I/apps/openmpi/4.0.2/include/Intel -I/apps/openmpi/4.0.2/include/Intel -I/apps/openmpi/4.0.2/include/Intel
OMPI_LDFLAGS -L/apps/openmpi/4.0.2/lib -L/apps/openmpi/4.0.2/lib/Intel -L/apps/openmpi/4.0.2/lib -L/apps/openmpi/4.0.2/lib/Intel -L/apps/openmpi/4.0.2/lib -L/apps/openmpi/4.0.2/lib/Intel -L/apps/openmpi/4.0.2/lib -L/apps/openmpi/4.0.2/lib/Intel

The duplication in Spack v0.21 appears to be due to Spack commit 72b36ac14446782f47519db8ee5cff6631b2b4c9.

Closes https://github.com/ACCESS-NRI/spack-packages/issues/62


README: hint to clone git without history

The git history has several hundred MB which is not needed in most cases, so we should add the hint here

Nice @rubo77, I like it! I appreciate your help :)


IMPRO-1011: Merge different file times

Deals with two problems: partial merging of files using iris.load, and equalisation of (time) coordinate bounds ranges before blending using the TriangularTimeBlending plugin.
Testing:

[x] Ran tests and they passed OK
[x] Added new tests for the new feature(s)

Waiting on https://github.com/metoppv/improver/pull/777 so that tests consistently pass.


Theming support

In preparation for colorblindness mode, styled-components theming support is needed.

Take a look at what I did at https://hodl.camp/, there is a low-vision icon on the top that toggles between different palettes. The palettes come from https://colorbrewer2.org/, which are built in into chroma-js.


feat(oauth2) pkce

Summary
Add support for Proof Key for Code Exchange (PKCE). This implementation adds a field that distinguishes whether a client is public or confidential, and imposes rules around issuing codes/tokens based on the client_type. PKCE was introduced as a method for public applications that were susceptible to the authorization code interception attack.
Full changelog

Adds a client_type to oauth2_credentials
Adds support for PKCE

Issues resolved
Fix #1121

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.amberh seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.You have signed the CLA already but the status is still pending? Let us recheck it.

On:
https://tools.ietf.org/html/draft-ietf-oauth-security-topics-13
It is noted that PKCE should be used for everything. Not only for public.

@bungle I changed the PKCE implementation so that it is available for all clients and required for public (non-confidential) clients.  I stopped short of requiring it for authorization code grants since there are too many OAuth2 client implementations that do not support PKCE today.
Should I add a configuration option to require it for all authorization code flows?

@dave-shawley, @amberheilman, I am focusing this on next 7 days in hopes we can merge this. So first thing we need is to rebase this to latest next. I will focus to changes tomorrow.

@dave-shawley, it might make sense to add option for PKCE.
Let's think the cases:

client requests pkce
client does not request pkce

If client requested pkce then the pkce should be enforced by server.
If client didn't request pkce then it should not be enforced by server.
But of course in secure environments, you might want to enforce it on server always, or perhaps by client type. So it sounds we like to have an options. config.pkce could be the parameter, perhaps with None, Lax, Strict.
None if client request PKCE we will do it but we will not error if client gives wrong data to token endpoint, thus checks code_verifier, but only writes warning or notice to logs but still gives tokens if everything is otherwise ok.
Lax, we will only enforce this on public clients, for confidential see None.
Strict, we will enforce this on all the clients.
@dave-shawley, @amberheilman, how does that sound?

@bungle I rebased our branch over next instead of master and run the tests with pongo.  Is pongo a valid test ... it seems to only use images from hub.docker.com and there isn't a next published there.

@bungle looks like the plugin tests succeeded but the integration suite failed.  I'm going to chat with @amberheilman when I get into the office (about 30 minutes) before I respond to the configuration question.  Thanks for finding some time to look at this.

@bungle the configuration changes make sense.  The only one that I don't completely understand is why PKCE is not enforced for confidential clients when the appropriate bits are in the authorization request in Lax mode.  I think that it would be more useful if Lax mode fully implements PKCE if the client includes the challenge information in the authorization request and requires it for public clients.  Otherwise the only way to use PKCE for confidential clients is to require it for all clients which (I believe) is going to make it difficult to use Strict mode.
In any case, we aren't going to be able to slot in the changes until our next two-week sprint/cycle.  One of us might be able to get to it before then, but I can't guarantee much action on this for another week or two.  Any chance at getting a review started on #5264 since that is the other side of our implementation?  The implementation might address #5252 as well.

@dave-shawley, thanks for getting back! Yes, our testsuite is currently broken on integration tests. Sorry about it. It is good enough if plugin tests for this plugin pass.
Yes, I think your comment on my strange/mixed rules is spot on, let's try again:
Client | None | Lax | Strict
Public Client with PKCE | PKCE Enforced on Token Endpoint | PKCE Enforced on Token Endpoint | PKCE Enforced on Token Endpoint
Public Client without PKCE | PKCE Not Enforced | Fails on Authorization Endpoint | Fails on Authorization Endpoint
Confidential Client with PKCE | PKCE Enforced on Token Endpoint | PKCE Enforced on Token Endpoint | PKCE Enforced on Token Endpoint
Confidential Client without   PKCE | PKCE Not Enforced | PKCE Not Enforced | Fails on Authorization Endpoint
So the Only difference being the Lax.

Is work still ongoing with this?

@bungle @evansmwendwa I have updated to include a configuration option for pkce. I believe all comments have been addressed here and this is up for review.

Yes, I think we are almost there to merge this. I gave some small change request above. If those get worked, I think we can merge this.

@amberheilman did you see my latest comments above ^

@bungle awesome, thanks for reviewing again. I will try to get these changes done today.

@bungle all comments are addressed, this is ready for another pass

@bungle has this been assigned to a milestone or release?  wasn't sure if this is part of the 2.1.0 Milestone or not.


[XLA] Add DiagSlice HLO instruction

This is a PR from JIZHI, the AI platform in Tencent.
@jlebar
There is the purpose of the new HLO instruction:
Generate more efficient kernel for DiagPart and MatrixDiagPart. For example:

%iota.3 = s32[1024,1024]{1,0} iota(), iota_dimension=0
%iota.2 = s32[1024,1024]{1,0} iota(), iota_dimension=1
%compare.0 = pred[1024,1024]{1,0} compare(s32[1024,1024]{1,0} %iota.3, s32[1024,1024]{1,0} %iota.2), direction=EQ
%param_0.3 = f32[1024,1024]{1,0} parameter(0)
%constant_0 = f32[] constant(0)
%broadcast.0 = f32[1024,1024]{1,0} broadcast(f32[] %constant_0), dimensions={}
%select.0 = f32[1024,1024]{1,0} select(pred[1024,1024]{1,0} %compare.0, f32[1024,1024]{1,0}
%param_0.3, f32[1024,1024]{1,0} %broadcast.0)
ROOT %reduce.0 = f32[1024]{0} reduce(f32[1024,1024]{1,0} %select.0, f32[] %constant_0), dimensions={0}, to_apply=%add_F32.17

Here is the hlo code for tf.diag_part(X), where X is tensor with shape 1024x1024. the generated kernel's launch dimension is 1024x1024, and it has 'compare' and 'select' operation to get the right index to output. and the more expensive operation 'reduce' to get the right value to output. The kernel is less efficient for diag-part op. Actually it just needs 1024 threads to get the right value from the diag part of the source matrix which just determined by offset.

Do you have an example where the 1024x1024 kernel launch was not already the result of another operations where the select and reduce could not be fused into the previous operation?
Either way this should be split into two parts.

Add the Hlo and make an evaluator implementation and move the expander into an Hlo pass.
Add the implementation for the CPU/GPU backends.

Can we make this a generic optimization that works as a lowering strategy for fused reductions?  For instance can we recognize this kind of access pattern and use a smarter elemental lowering for the fused kernel?
/CC @bixia1

@blakehechtman The current implementation could be fused into the previous operations. However, it is hard to be fused into the following operations because of reduction. In this PR, the diag-part is lowing to a slice. The slice operation is more easy to be fused into previous operations and following operations.
@sanjoy It's a good question, but I have no idea now. I think 'slice' is a more straight expression of diag-part than 'reduce'.

Can we make this a generic optimization that works as a lowering strategy for fused reductions? For instance can we recognize this kind of access pattern and use a smarter elemental lowering for the fused kernel?

I think the diag-part is basically an index transformation as transpose, slice and reshape. Should we design a 'shuffle' instruction with a nested computation which describes the index transformation for general index transformation situation?

I think the diag-part is basically an index transformation as 'transpose', 'slice' and 'reshape'. Should we design a 'shuffle' instruction with a nested computation which describes the index transformation for general index transformation situation?

Sorry I responded without fully understanding the IR snippet.
The IR snippet looks like it should be easily representable as a gather_nd with an iota index?  At least on CPU/GPU the iota index should fuse into the gather and the gather should further fuse into its users.
I agree using a reduction this was seems odd.

squash the commits that resolve conflicts.


Questa complains about newlines in ASSERT macros

(Spin-off from #622 by @danghai)
Compiling Ibex with Questa gives the following warning:
** Warning: ** while parsing macro expansion: 'ASSERT' starting at /home/danghai/hhoangda/ibex/rtl/ibex_compressed_decoder.sv(286)
** at /home/danghai/hhoangda/ibex/rtl/ibex_compressed_decoder.sv(286): (vlog-2643) Unterminated string literal continues onto next line.

The code in question is
// in ibex_compressed_decoder.sv
  `ASSERT(IbexInstrLSBsKnown, valid_i |->
      !$isunknown(instr_i[1:0]))

// in prim_assert.sv
`define ASSERT(__name, __prop, __clk = `ASSERT_DEFAULT_CLK, __rst = `ASSERT_DEFAULT_RST) \
`ifdef INC_ASSERT                                                                        \
  __name: assert property (@(posedge __clk) disable iff (__rst !== '0) (__prop))         \
    else `ASSERT_RPT(`PRIM_STRINGIFY(__name), `PRIM_STRINGIFY(__prop))                   \
`endif

// Converts an arbitrary block of code into a Verilog string
`define PRIM_STRINGIFY(__x) `"__x`"

`define ASSERT_RPT(__name, __msg)                                         \
`ifdef UVM                                                                \
  assert_rpt_pkg::assert_rpt($sformatf("[%m] %s: %s (%s:%0d)",            \
                             __name, __msg, `__FILE__, `__LINE__));       \
`else                                                                     \
  $error("[ASSERT FAILED] [%m] %s: %s", __name, __msg);                   \
`endif

Given this code, I'm assuming the error message actually comes from the fact that prop (the second argument to ASSERT) is converted to a string before being used in ASSERT_RPT, and this use of prop with newlines isn't liked by Questa.
According to SV2017, the definition of a string literal is

Which indicates our code is actually wrong. I see a couple of options:

Remove the reporting of the property in ASSERT_RPT, and only report the message.
Add backslashes to the end-of-line of each multi-line property. I'm not sure if that's actually valid for the non-string case?
Avoid newlines in properties when writing ASSERT macros.

@danghai:

First, let's check if the hypothesis holds: Does the warning go away if you remove the ASSERT_RPT() call from the definition of the ASSERT macro?

Can you try to add a backslash at the end of the property in ibex_compressed_decoder.sv, e.g.
  `ASSERT(IbexInstrLSBsKnown, valid_i |-> \
      !$isunknown(instr_i[1:0]))

Questa version info: QuestaSim-64 qrun 2019.4 Utility 2019.10 Oct 15 2019
CC @msfschaffner @sriyerg

Looks like its between having the convenience of having the whole failing property showing up in the logs v/s striving for a cleaner code that is supported on all the tools.
I personally like the convenience of being able to look at the failing property at a quick glance. It shows up in our reports as well:
https://reports.opentitan.org/hw/ip/usbdev/dv/latest/results.html
Although, the error message does indicate the file name and line number, so its not a lot of work to quickly lookup the failure by opening the file.
Here's my preference for dealing with this:

Make all properties single line. I think 80-90% of the cases would probably fit within the 100 char line limit. For the rest, we use a backslash (I am trying this with VCS & Xcelium).
Remove the stringified property from the error message. Less preferred, but I am ok with this.

Thinking about this, there's one other option I can see: use an "ifdef Questa" to remove the property from the error message, but just for Questa builds.

Well, it's invalid SV syntax, so I'd rather not hope that tools other than Questa silently continue to accept it.
@danghai please let us know the answers to the two questions above if you have time!

Hi @imphil

I do not see ASSERT_RPT() in rtl source code. I only see ASSERT.
I got the error when I try to do:

  `ASSERT(IbexInstrLSBsKnown, valid_i |-> \
      !$isunknown(instr_i[1:0]))

The error is:
 ** Error: (vlog-13069) ** while parsing macro expansion: 'ASSERT' starting at /home/danghai/ibex/rtl/ibex_compressed_decoder.sv(286)

Thanks for testing the backslashes! So they don't work (as expected).
For the other test, can you apply this patch, which removes the use of ASSERT_RPT from the ASSERT macro.
diff --git a/shared/rtl/prim_assert.sv b/shared/rtl/prim_assert.sv
index f569292..9b5668f 100644
--- a/shared/rtl/prim_assert.sv
+++ b/shared/rtl/prim_assert.sv
@@ -82,7 +82,6 @@
 `define ASSERT(__name, __prop, __clk = `ASSERT_DEFAULT_CLK, __rst = `ASSERT_DEFAULT_RST) \
 `ifdef INC_ASSERT                                                                        \
   __name: assert property (@(posedge __clk) disable iff (__rst !== '0) (__prop))         \
-    else `ASSERT_RPT(`PRIM_STRINGIFY(__name), `PRIM_STRINGIFY(__prop))                   \
 `endif
 // Note: Above we use (__rst !== '0) in the disable iff statements instead of
 // (__rst == '1).  This properly disables the assertion in cases when reset is X at

I got a lot of errors: (sample)
** Error: (vlog-13069) ** while parsing file included at /home/danghai/ibex/dv/uvm/core_ibex/env/core_ibex_env_pkg.sv(10)
** at /home/danghai/ibex/dv/uvm/core_ibex/env/core_ibex_rvfi_if.sv(27): near "endinterface": syntax error, unexpected "SystemVerilog keyword 'endinterface'".
** Error: (vlog-13069) ** while parsing file included at /home/danghai/ibex/dv/uvm/core_ibex/env/core_ibex_env_pkg.sv(11)
** at /home/danghai/ibex/dv/uvm/core_ibex/env/core_ibex_csr_if.sv(12): near "endinterface": syntax error, unexpected "SystemVerilog keyword 'endinterface'".
-- Importing package ibex_mem_intf_agent_pkg
-- Importing package irq_agent_pkg
** Error: (vlog-13069) /home/danghai/ibex/dv/uvm/core_ibex/env/core_ibex_env_pkg.sv(23): near "endpackage": syntax error, unexpected "SystemVerilog keyword 'endpackage'".

Let's try this instead:
diff --git a/shared/rtl/prim_assert.sv b/shared/rtl/prim_assert.sv
index f569292..92a0545 100644
--- a/shared/rtl/prim_assert.sv
+++ b/shared/rtl/prim_assert.sv
@@ -82,7 +82,7 @@
 `define ASSERT(__name, __prop, __clk = `ASSERT_DEFAULT_CLK, __rst = `ASSERT_DEFAULT_RST) \
 `ifdef INC_ASSERT                                                                        \
   __name: assert property (@(posedge __clk) disable iff (__rst !== '0) (__prop))         \
-    else `ASSERT_RPT(`PRIM_STRINGIFY(__name), `PRIM_STRINGIFY(__prop))                   \
+    else $error("assert failed");                                                        \
 `endif
 // Note: Above we use (__rst !== '0) in the disable iff statements instead of
 // (__rst == '1).  This properly disables the assertion in cases when reset is X at

I see the warning vlog-2643 is removed when I apply this patch. However it still has other warnings (it does not relate to this patch)
 18 ** Warning: /home/danghai/ibex/rtl/ibex_cs_registers.sv(730): (vlog-2583) [SVCHK] - Extra checking for conflicts with always_comb     and always_latch variables is done at vopt time.
 19 ** Warning: /home/danghai/ibex/rtl/ibex_cs_registers.sv(732): (vlog-2583) [SVCHK] - Extra checking for conflicts with always_comb     and always_latch variables is done at vopt time.
 20 ** Warning: /home/danghai/ibex/rtl/ibex_cs_registers.sv(739): (vlog-2583) [SVCHK] - Extra checking for conflicts with always_comb     and always_latch variables is done at vopt time.
 21 ** Warning: /home/danghai/ibex/rtl/ibex_cs_registers.sv(742): (vlog-2583) [SVCHK] - Extra checking for conflicts with always_comb     and always_latch variables is done at vopt time.
 22 ** Warning: /home/danghai/ibex/rtl/ibex_cs_registers.sv(745): (vlog-2583) [SVCHK] - Extra checking for conflicts with always_comb     and always_latch variables is done at vopt time.

Thanks for your tests @danghai. So we are back to the options @sriyerg outlined in https://github.com/lowRISC/ibex/issues/669#issuecomment-596685769: Remove the property from the message, or restrict all properties in ASSERT calls to a single line.
I looked through the uses of ASSERT in Ibex and in OpenTitan (we want to keep the two aligned), and I've seen a fair number of asserts which are significantly longer than a single line. That's especially true for the asserts we're using in FPV, which use the same mechanisms. (If you want to have a look yourself, run git grep -C3 'ASSERT(' in opentitan.)
With that I don't see options other than removing the property from the message. I'll prepare a PR for that to see how it looks.

@danghai I've filed #696 to look into the last remaining warning in https://github.com/lowRISC/ibex/issues/669#issuecomment-597823073 more closely.


[Enhancement] Ability to disable mentioning from MC

Existing Issues Searched

[x] I confirm that I have searched for existing issues before reporting to avoid duplicate reporting.

Description
Currently, it is fully possible to mention users via either @Username or @everyone/@here, I have tried the latest build from the Actions page and disabled modifyChatMessages and it still persists
I suggest a config option that disables this, making @everyone, @here, and @Username not function when sent from Minecraft, maybe allowMentions?
Image of me mentioning with modifyChatMessages set to false

you seemed to only disable it for mentioning users, mentioning @everyone/@here still works

Fixed

thank you, it works as intended now

could this get separated into two different permissions, one to disable user pings and another to disable everyone/here/role pings? also, if the bot has administrator permissions, it will still ping everyone, here or roles no matter the mention status in the config file

@lexd0g
You should not give the "Administrator" permission to bots.
Since you are not using the webhook function, removing the "Mention Everyone" permission of the bot solves your needs.


Complex text layout, in particular with TeX input [was: MathJax does not support Complex text layout.]

Because MathJax looks at individual code points it has trouble dealing with scripts that require bidirectionality, context shaping etc. This is visible whenever trying to use hebrew or arabic for instance.
It would be good if MathJax would be able to identify these ranges and be able to keep those as blocks instead of dividing it into individual characters. At the very least in \text mode.
http://en.wikipedia.org/wiki/Complex_text_layout

@hartman do you think you could poke @amire80 sometime? We'd love to improve this, especially if Wikipedia wants to roll out the SVG output more widely.

I'm right here :)
How can I help?
Testing? - Gladly, just tel me what to test exactly.
Examples of how non-Latin scripts work in formulas? - It's not used in Hebrew textbooks, but it is used in textbooks in Arabic and Persian. Maybe @ebraminio can chime in here.
Anything else?

Thanks for stopping by @amire80 :-)

How can I help?

I'm hoping we can improve handling of combined characters in non-Latin scripts. This has come up on WMF bugzilla/phabricator repeatedly. To quote Davide from https://github.com/mathjax/MathJax/issues/474#issuecomment-38324717 :

Ideally, MathJax would put each sequence that forms one group into a single  or , just as it does for single Latin letters now. I've looked into this to some degree, and there are some difficulties handling it. It is possible to have combining characters grouped with their preceding characters, but it is not clear to me how some characters work. For example, it seems that the virama (U+0D4D) combines not just the character on its left, but also on the right, though I might be misunderstanding it. It also seems that some of these grouping are handled by ligatures within the fonts, not by combining characters. Unfortunately, MathJax does not have access to ligature information from the fonts. While it would be possible to add ligature data to MathJax's font tables, this could be a significant amount of data very little of which would be used by any one page.
I'm really not familiar enough with the languages that use these features to know if what I'm trying out would be sufficient or not. I'm wondering if it is possible to get some examples from a variety of languages that show the range of situations that need to be accommodated.

So our question would be: does anyone have expertise they can share with us? @hartman was kind enough to point to you ;-)
(Perhaps we should split this out into a separate issue.)

The (very) basic idea of virama is that the sequence of consonant + virama + consonant has three Unicode characters, which appear as occupying the space of one glyph (but it can get far more complicated).
More generally, I'd love to understand MathJax's current situation. What should I do to test the current rendering? Install my own instance? Or is there an online instance where a current version can be tested?

consonant + virama + consonant has three Unicode characters, which appear as occupying the space of one glyph

Right. Combined characters are common enough in mathematical layout so we understand the situation in general.

(but it can get far more complicated).

That's our problem. We lack the specifics for most natural language, non-Latin scripts.

Or is there an online instance where a current version can be tested?

You can do this on MediaWiki (using the MathML/SVG mode of the math extension), in the browser ((this sample](http://cdn.mathjax.org/mathjax/latest/test/sample-dynamic-2.html) or this codepen) or use a local copy of MathJax -- whichever you like.
A basic example: ത്ര will be converted to ത്ര and since we don't have any routines to identify these kinds of combined characters, the TeX input converts this internally to MathML as
<math xmlns="http://www.w3.org/1998/Math/MathML">
  <mrow class="MJX-TeXAtom-ORD">
    <mo>ത</mo>
  </mrow>
  <mrow class="MJX-TeXAtom-ORD">
    <mo>്</mo>
  </mrow>
  <mrow class="MJX-TeXAtom-ORD">
    <mo>ര</mo>
  </mrow>
</math>

Which the MathJax output will in turn split across three span's (in the HTML outputs) or three g's (in the SVG output) -- and of course this breaks the rendering of the combined character.
(I just noticed that Firefox sometimes combines the spans in the HTML outputs e.g., ത്ര but not the subscript in കു_ശ. Chrome is more "consistent" in that nothing is combined)
So for us the problem is: is there a concise set of data (or some efficient heuristic) that we could use to identify all relevant situations where we need to re-combine into one mi/mo element in the MathML? Once we have that, the rendering will work as well.

So for us the problem is: is there a concise set of data (or some efficient heuristic) that we could use to > identify all relevant situations where we need to re-combine into one mi/mo element in the MathML?

Sorry for the long comment, bringing a bit of off site discussion back to the issue tracker.
How feasible/expensive would it be to make the Unicode UCD database
combining class available to mathjax for each character? Basically (or
at least as a good first approximation) any character with non zero
combining class (field 4 in UnicodeData.txt) needs to stay with the
preceding one, and in addition if it's class 9 (virama) the following
character needs to be kept together as well.
It's probably also worth noting that tex, even unicode tex like xetex
or luatex are almost certainly not going to get this right without
markup
that is you will need \text{abc} or \mathit{abc} or some other such
command to force a string of characters to be typeset as text with a
single font rather than TeX's normal habit of splitting things up
character by character. Even if the construct looks like a single
character to the author.
In classic tex it is not an issue as fonts can only have 256 characters
and while composed characters can be supported with various macro remapping tricks
composing characters following the base are basically not supportable even for simple
composing accents like acute.
Support in unicode tex variants such as xetex and luatex seems a bit variable. In text, xetex
hands things over to the HarfBuzz library so does pretty well. luatex handles it internally and currently does less well with the virama. In math both require a font with an opentype MATH table to do anything very useful and I couldn't find such a font that had a virama.
The following latex document is using kartika in text and latin modern math in math, you will note that
even european accents typically fail in math, but even the virama example works if you add some markup \mbox here or mi or mtext equivalently in MathML
The image shows xetex at the top and luatex at the bottom.
So while not requiring something like \text{..} or \mbox{...} around such character strings would be desirable, it would put your unicode support a long way ahead of what TeX can currently achieve
so it depends a bit on what the specification of the "tex-like syntax" is, how far beyond what TeX can do is it reasonable to push it?
\documentclass{article}

\usepackage{fontspec}
\usepackage{unicode-math}
\setmainfont{kartika.ttf}

\begin{document}

U+0d24 U+0d4d U+0d30 outputs e.g., ത്ര but

abc $abc \mbox{ത്ര} $  U+0063

abç $abç \mbox{ത്ര} $ U+00e7

abç $abç \mbox{ത്ര} $  U+0063 U+0327

\end{document}

I'm not really sure if I understand what the discussion is about, but if the idea is to identify what sequence of characters constitute a single unit, then Unicode Grapheme clustering should provide the needed information..

Yes - what @khaledhosny says sounds like the right thing to me, although I'm not every experienced with it. Maybe @santhoshtr can contribute more details.
Santhosh, I think that what @pkra wrote three comments above explains the problem best.

On 3 March 2015 at 12:05, Khaled Hosny<EMAIL_ADDRESS>wrote:

I'm not really sure if I understand what the discussion is about, but if
the idea is to identify what sequence of characters constitute a single
unit, then Unicode Grapheme clustering
http://unicode.org/reports/tr29/#Grapheme_Cluster_Boundaries should
provide the needed information..

Yes but I suppose the question is how far it makes sense for a javascript
library to do that
by hand if the underlying platform doesn't make the unicode properties
available
and if it's emulating tex syntax how far would tex go? You know as much
about the tex support as anyone. How far would it be reasonable in xetex to
have such a cluster do anything sensible in math without escaping to text
with \text{..} or some such command, given that you can't assign a
\mathclass to such a cluster?

I found a CoffeeScript implementation for graphemes.
https://github.com/devongovett/grapheme-breaker
Might be useful.

Thanks for all the useful comments. To summarize,

xetex/luatex do not handle input the way requested in this issue, i.e., without extra markup such as \text
it's not clear (to me at least) if there are plans to handle it this way
a solution could start with the simple approach David C outlined or potentially build on grapheme-breaker (thanks @hartman!)

To add to that,

On the other hand, a quick test with LaTeXML and pandoc indicates that they do handle such characters as requested here, i.e., not like xetex/luatex.

So it seems to me that a solution can't be in the core TeX input but needs to be an extension. That's not a problem, of course, since it probably would have ended up an extension anyway.
It would be good to hear from MediaWiki/WMF community if they actually want to delineate from the TeX-engines here.

Again it would be good to get more feedback.

At TeX folks, is handling characters in math mode without extra markup the future direction of xetex/luatex/etc?
At MediaWiki / WMF folks: is non-standard TeX behavior actually desired by the relevant communities?

Without more feedback, I think we should punt on this / move it out of the 2.6 milestone.

Let me understand the issue here, people want to do things like $x+y=<complex character>$ where <complex character> is possibly a multi-code point grapheme, and have <complex character> treated as a math identifier, right? If so, then I think that is a reasonable expectation and if current Unicode TeX engines do not handle it correctly (they probably don’t) it is likely a bug or a missing feature, not something by design.
Or is it that people want to do things like $<complex text string>$, where <complex text string> is a multi-character text string that possibly needs complex text layout, and get proper text layout (bidi, shaping etc.)? I don't think that is a reasonable expectation and some kind of markup is needed here to indicate that this is a regular text string that needs to be treated as such.

Thanks, @khaledhosny!

[...] people want to do things like $x+y=$ where  is possibly a multi-code point grapheme, and have  treated as a math identifier, right?

Yes, that's how I understand it as well. (It's a bit difficult to say since this is originally a request from the Wikipedia end).

I think that is a reasonable expectation

Thanks!

if current Unicode TeX engines do not handle it correctly (they probably don’t) it is likely a bug or a missing feature, not something by design.

Thanks for that, too. The "they probably don’t" part worries me slightly but if you and @davidcarlisle agree that it's the desired behavior in Unicode TeX engines, then that's enough for us, I think.

Still hoping the MediaWiki/WMF/Wikipedia side will chime in.

As per F2F, we're removing this from the v2.6 Milestone (i.e., the upcoming release).
It's not clear what the right approach is, in particular, in terms of compatibility with TeX/LaTeX (or rather XeTeX/LuaTeX). It's also not clear what the WMF and the Wikipedia community really want here.
To be clear, we're not closing this issue and we are still interested in figuring out how complex layout might work in the TeX input.

Blast from the future: there's a TC39 proposal "Unicode segmentation" to allow (among other things) to split strings by grapheme https://github.com/tc39/proposal-intl-segmenter. The repository includes a link to a polyfill  (and there's also a non-standard Chrome feautre apparently).

Cool.  Thanks, @pkra.

No problem. The polyfill is unfortunately useless -- it only covers Enligsh. But for those who want to try it out, the chrome build-in might be useful.

Another blast from the future: Chrome and Safari have supported Intl.Segmenter for a while now https://caniuse.com/?search=Intl.Segmenter.

The segmented looks interesting, and something to keep in mind, though until it is available in Firefox, it probably won't be able to be used in MathJax.
One of the PRs for the font update includes changes (8eeeaa2e) to have multiple unknown characters placed in a single container rather than each one in a separate one, so that will help with this, though currently it still requires \text{} to group them into one MathML item (but you don't need to use mtextInheritFont any more).
Of course, this is really an input issue, so that multiple combining characters are treated as a single unit by the TeX input jax so they are all placed in the same MathML token element initially, which is where the Segmenter would come in handy.  It is on the list of things to do, and I haven't forgotten about it.  Perhaps the font update is a good place for that.

The segmented looks interesting, and something to keep in mind, though until it is available in Firefox, it probably won't be able to be used in MathJax.

It could be an (opt-in) progressive enhancement. For NodeJS it would already be useful now (not that I need it, I just wanted to point that out).

Note from the future: Intl.Segmenter is how widely supported https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Intl/Segmenter/Segmenter