Ajustes de SEO, acessibilidade e estilo

Adiciona pacote react-fade-in
Adiciona hook useLocalStorage
Cria páginas de produtos novos e on sale
Adiciona SEO e acessibilidade em todas as páginas

Codecov Report

Merging #9 into master will not change coverage by %.
The diff coverage is n/a.

@@            Coverage Diff            @@
##            master        #9   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files            8         8
  Lines           94        94
  Branches        30        30
=========================================
  Hits            94        94

Impacted Files
Coverage Δ

src/components/NumberSelect/NumberSelect.js
100.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 89396f0...75e77f4. Read the comment docs.


Auto filter freeze

Add the ability to add an autofilter and lock the first row.    This is for when the first row of your CSV are headers.    I basically did this because I got tired of entering Alt-A,T and Alt-W,F,R everytime I open a csv file.   :-)
Fair warning, most of this code was written by my friend Claude with me adding comments and direction, so there might be some funny comments I missed.   So feel free to criticize, you won't hurt my feelings, but AI's have very long memories, so I recommend to my friends to be nice to them now while we're still in control.   :-)

Hi there!
I appreciate the contribution. However, it doesn't appear the new code compiles. It also appears that your good friend Claude may be using quite an outdated version of the Py03 documentation for its contributions and has also broken some of my existing code with old deprecated methods of the Py03 library.
Claude also introduced a number of additional bugs that would cause the program to panic instead of allowing the consumer to handle the errors. (Such as replacing ? with .ok()) . Finally, although I could not directly test this as the code doesn't compile, it appears a number of the changes would cause quite a significant performance regression (such as through running the unnecessary check of ensure_sheet_data_started() on each write_row(). This goes against the goal of the excel-rs project, which is to be the most performant excel writer.
As such, I don't think I can accept the PR in its current state unfortunately.
The autoFilter feature does seem like it would be an interesting feature to add, and it appears it might be possible to add it without introducing any major performance regressions. Could possibly just append the AutoFilter tag https://learn.microsoft.com/en-us/dotnet/api/documentformat.openxml.spreadsheet.autofilter?view=openxml-3.0.1 to the end of a SpreadSheet file. However, I've not looked very deep into this personally so I'm not sure if that is all it takes to implement that feature.
Once again, thank you so much for the contribution! I would be happy to receive and review any further changes that address the above issues.

How odd.  It certainly compiles and runs for me.  This is my first
experience with Rust, so perhaps I'm missing something.  This is great
feedback, let me take another run at it and get back to you.
mp
On Mon, Oct 28, 2024, 1:33 AM Carl Ian Voller @.***>
wrote:

Hi there!
I appreciate the contribution. However, it doesn't appear the new code
compiles. It also appears that your good friend Claude may be using quite
an outdated version of the Py03 documentation for its contributions and has
also broken some of my existing code with old deprecated methods of the
Py03 library.
Claude also introduced a number of additional bugs that would cause the
program to panic instead of allowing the consumer to handle the errors.
(Such as replacing ? with .ok()) . Finally, although I could not directly
test this as the code doesn't compile, it appears a number of the changes
would cause quite a significant performance regression (such as through
running the unnecessary check of ensure_sheet_data_started() on each
write_row(). This goes against the goal of the excel-rs project, which is
to be the most performant excel writer.
As such, I don't think I can accept the PR in its current state
unfortunately.
The autoFilter feature does seem like it would be an interesting feature
to add, and it appears it might be possible to add it without introducing
any major performance regressions. Could possibly just append the
AutoFilter tag
https://learn.microsoft.com/en-us/dotnet/api/documentformat.openxml.spreadsheet.autofilter?view=openxml-3.0.1
to the end of a SpreadSheet file. However, I've not looked very deep into
this personally so I'm not sure if that is all it takes to implement that
feature.
Once again, thank you so much for the contribution! I would be happy to
receive and review any further changes that address the above issues.
—
Reply to this email directly, view it on GitHub
https://github.com/carlvoller/excel-rs/pull/3#issuecomment-2440868958,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANXRUCVJBJP4DSIYQNS5ETZ5XZHHAVCNFSM6AAAAABQWJNJZKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINBQHA3DQOJVHA
.
You are receiving this because you authored the thread.Message ID:
@.***>

So pretty certain the row lock must be added first before any of the actual data, and then the auto filter is tagged on to the bottom.   But on a 50K row file, I'm still running about 30% slower than the original code.   Some comments on that inline.   I haven't modified typed_sheets.rs at this point (inheritance seems a little different in Rust) and I backed out the broken PyO3 changes  but will do that work if we can get past the current performance issues.

Also, just wanna add, I'm completely open to features that may impact the performance of excel-rs, as long as these features are implemented as optional flags that DO NOT affect the performance of excel-rs when they are disabled. A good benchmark for this would be:
I want to convert a huge CSV to XLSX. If I add this optional feature but don't use the feature, the basic conversion of CSV to XLSX should not be impacted.
If this means you have to create multiple versions of essentially the exact same code, I would prefer that over any performance hits to the base functionality of this package.


Adds command purview retentionevent add, Closes #4373

This PR adds the command purview retentionevent add.
Closes #4373

Great work @nicodecleyre, we'll try to review it ASAP!

Thank you for your awesome review! Updated the PR with your comments!

Merged manually, thanks!! 👏


WARN log using MariaDB driver

This is not a major issue or anything, but maybe there's a quick fix for it. Flyway is warning "You are connected to a MySQL database using the MariaDB driver..." when using MariaDB driver with AWS Aurora (based on MySQL), even though it is the recommended driver to use (see here).

The reason we introduced this warning is because the MariaDB driver is unable to connect with MySQL 8.0. In the case of Aurora, which is currently only compatible with MySQL 5.7, this is not yet an issue.

I've recently upgraded to the latest 5.1.4 and it appears there is an auto detect problem,  All of my config files have the following pattern for flyway.url:
flyway.url=jdbc:mysql://<dbhost>/<db_schema>?autoReconnect=true&useOldAliasMetadataBehavior=true&dontTrackOpenResources=true

# Fully qualified classname of the jdbc driver (autodetected by default based on flyway.url)
# flyway.driver=

According to the sample flyway.conf file, the driver should be autodetected based on the format of flyway.url and the comment in the flyway.conf file shows the MySQL url format as:
# MySQL             : jdbc:mysql://<host>:<port>/<database>?<key1>=<value1>&<key2>=<value2>...
Yet when I run it I get the warning:
C:\Projects\GitLab\dbmigration>flyway -configFiles=conf/acc/flyway_hunters.conf -target=001.0.2 -user=admin -password=lint1234 migrate
Flyway Community Edition 5.1.4 by Boxfuse

Database: jdbc:mysql://mrdb-acc01.wp.bluecoat.com/hunters (MySQL 5.6)
WARNING: You are connected to a MySQL database using the MariaDB driver. This is known to cause issues. An upgrade to Oracle's MySQL JDBC driver is highly recommended.
Successfully validated 3 migrations (execution time 00:01.986s)
Current version of schema `hunters`: 001.0.2
Schema `hunters` is up to date. No migration necessary.

This has now been fixed automatically as a consequence of #2144

@axelfontaine doesn't that fix apply to the CLI version only?

@GregTurner No.

The reason we introduced this warning is because the MariaDB driver is unable to connect with MySQL 8.0. In the case of Aurora, which is currently only compatible with MySQL 5.7, this is not yet an issue.

This is now no longer the case - see #2865 and https://mariadb.com/kb/en/about-mariadb-connector-j/


Add data binding support for RabbitMQ services

$title
Design doc: https://docs.google.com/document/d/1AS1BD0qPdP1E870GKowwvcsoFb8ij9aS78X3y6Zop-Y/edit?usp=sharing

dupplicate of https://github.com/ballerina-platform/ballerina-standard-library/issues/2812


Unexpected Error Occurred while Connecting

renderer1.log
main.log
Issue Type: Bug
I just got this error and was told to log it to github
After the error, I was able to connect
Azure Data Studio version: azuredatastudio 1.30.0 (59c4b8e90cf2d3a24bed72623197f10f8d090fdc, 2021-06-16T19:04:08.858Z)
OS version: Windows_NT x64 10.0.19042

System Info

Item
Value

CPUs
Intel(R) Core(TM) i7-10610U CPU @ 1.80GHz (8 x 2304)

GPU Status
2d_canvas: enabledgpu_compositing: enabledmultiple_raster_threads: enabled_onoop_rasterization: enabledopengl: enabled_onprotected_video_decode: enabledrasterization: enabledskia_renderer: enabled_onvideo_decode: enabledvulkan: disabled_offwebgl: enabledwebgl2: enabled

Load (avg)
undefined

Memory (System)
31.78GB (18.46GB free)

Process Argv

Screen Reader
no

VM
0%

Extensions: none

Thanks! Given that you were able to connect and we have the logs I'll close this one, let us know if you encounter it again though!


Reflect on subtypes

Hello,
in this comment (https://github.com/dart-lang/reflectable/issues/7#issuecomment-101576838), it is mentioned that there will be a @ReflectSubtypes(reflector) annotation to mark automatically all subtypes? Is this still planned, is there a way around this at the moment? I am using a reflector from a library, so I can't just add the subtypeQuantifyCapability, mentioned here: https://github.com/dart-lang/reflectable/issues/123#issuecomment-378559376
Thanks,
Niklas

The @ReflectSubtypes(reflector) design was not implemented (or even designed in detail), but the ability to cover subtypes was added in the shape of subtypeQuantifyCapability. There are no plans to introduce a new mechanism for doing this. So you'd need to use the quantifier, perhaps by creating a modified copy of the reflector that you are currently using.
If your target class set can be specified conveniently (say, they are located in a small set of libraries and/or have names that match a regexp) then you could also use @GlobalQuantifyCapability.

Thanks @eernstg for your response!
Using a @GlobalQuantifyCapability is not possible because classes don't share any comman naming pattern.
Could you please clarify the following:

So you'd need to use the quantifier, perhaps by creating a modified copy of the reflector that you are currently using.

To get a bit more concrete. We want to set only one annotation for a base class, to enable serialization for all subtypes automatically. The reflector below is part of a library. How can we "add" the subtypeQuantifyCapability to the reflector below and still be able to use the library as intended.
Because the reflector is provided by the library, we can't modify the reflector.
/// [jsonSerializable] is used as shorthand metadata, marking classes targeted
/// for serialization / deserialization, w/o "()"
const jsonSerializable = JsonSerializable();

/// [JsonSerializable] is used as metadata, marking classes as
/// serialization / deserialization capable targets
class JsonSerializable extends Reflectable {
  const JsonSerializable()
      : super(
            instanceInvokeCapability,
            metadataCapability,
            reflectedTypeCapability,
            newInstanceCapability,
            typeRelationsCapability,
            declarationsCapability);
}

I guess you're saying that this library does not only declare jsonSerializable and JsonSerializable, but it also uses jsonSerializable to perform some reflective operations, so you don't have a way to make them use a different reflector? Otherwise, if you control the code where the reflective operations are performed then you should be able to simply create your own reflector with the added subtypeQuantifyCapability, and use that.

so you don't have a way to make them use a different reflector?

That's exactly the case. Is this a third party library limitation or should this be addressed in the reflectable library?

This is about the semantics of capabilities, so it's not about a third party library, it is about reflectable.
I think you have a good case: The ability to apply an existing reflector to an entire subtype graph rather than just a single class is quite natural (and evidently useful), but isn't covered by the capabilities that we currently have.
So we could add @GlobalQuantifySubtypeCapability(C, reflector) which would associate reflector with the class C as well as all subtypes of C. That would give us the desired effect, and it seems to be the form that fits best into the current design of capabilities. However, I do not have time to work on that right now.


Add some videos about V8 and libuv

Added some video links so that awesome-nodejs can be effective for understanding Node.js internals.

By submitting this pull request, I promise I have read the contribution guidelines twice and ensured my submission follows it. I realize not doing so wastes the maintainers' time that they could have spent making the world better. 🖖
⬆⬆⬆⬆⬆⬆⬆⬆⬆⬆

Have you actually watched these videos and can confirm that they're useful and of high quality (content-wise)?

@sindresorhus
Yes I've watched all the videos and, in my opinion, these were great resources compared to other videos!

Google I/O videos by SWEs at Google

The videos from 2009 and 2012 are a bit old in some areas, but they are very useful for understanding the basics of V8, like Inline Caching, Hidden Class, and Bailout.
The video from 2013 will help you when optimizing your application performance with V8 knowledge.

Node JS Internal Architecture

I've seen other videos about libuv or V8, but this video is the one that gives clear and correct information about both.

Introduction to libuv by libuv maintainer, libuv Cross platform asynchronous i/o by libuv core contributor

Here is a video about libuv with source code and correct information about thread pool.

You Don't Know Node

This video will give you a niche but important knowledge about Node.js.


Fix bug where request is nil and we get a panic

@jdoliner

Bleh, my bad I introduced this. There's the same bug in InternalAPIServer.PutFile would you mind fixing that one too?

Other than that this is LGTM.


Extend Resource prototype

I'd really love to be able to extend Resource by adding my own methods to the prototype.
(Related request, though not one I need: Might be nice to be able to provide your own Resource class to the parser.)

Couple of options to make this possible. In halfred.js, either exporting Resource or something like this would do the trick:
extend: function(callback) {
  callback({
    Resource: Resource,
    Parser: Parser
  });
}

Any thoughts?

Would be okay for me. Both options seem okay, I have a slight preference for simply exporting it like this: https://github.com/basti1302/traverson/blob/2.0.1/traverson.js#L13
Is there any use case for exporting the Parser?

Btw, you're welcome to go forward with a PR on this :-)

Exporting would be much simpler, so if you're fine with that, I am as well. The alternative was just in case you didn't want it that visible. Done: https://github.com/basti1302/halfred/pull/2
As for the Parser, I have no use for it being exported, but I can easily imagine someone else. Though since all of the logic is hidden away in private functions, it would be difficult to extend.


Changed to om.next

Maybe related: https://github.com/dmotz/natal/issues/17
I'm playing a lot with om.next lately and found that it works fairly good in natal / react native as well. We should probably wait before merging this in until next hits stable, but this code works great as reference.

Looks like I messed this one up. A lot of code was still pre-loaded so it seemed to work when in reality it didn't.
To make it work, we need to abstract ReactDOM (or wait until RN is using something similar that we could just plug in). In my tests overwriting it with React seems to work ok-ish because the API is still very similar. There are some things that don't work but it will get around the initial errors.
Afterwards we'll have to get rid of gdom/isElement inside om/add-root!, but after that I ran into You cannot render into anything but a top root.

David was kind enough to push alpha10 out. That allows us to specify a separate render / unmount function to hook in with react native.
It still doesn't work out of the box and needs another small change inside next.cljs, but in my tests when I patched it manually I got om.next running with native.
I'm confident that with the next alpha (alpha11?) we can get it running out of the box.

Excellent work, and a perfect way to get familiar with Om Next.
I've been planning on adding an option to Natal that allows you to specify which React wrapper to use for the app skeleton. Once that's done and your changes are working without having to patch Om, we can merge this in and give users the option between Om Now and Om Next (and eventually others) during the init phase.
Thanks for doing the trailblazing here.

As soon as I mentioned it in the om slack David pushed alpha11 with the needed changes. With that it now works out of the box 😊

I added the ability to switch React interface templates like this:
$ natal init TestApp --interface om-next

Om stable is still the default, but we'll probably switch to Om Next once it's ready.
Thanks again for the PR.


KNL-1361 - Add support for @System prefix in sakai.properties

If a property has the @System prefix as a suffix, it is turned into
a system-wide JVM property instead of a Sakai property.

Please review this carefully in case there is a more Spring-ish way to do this. Thanks.

I've been thinking of a larger idea to provide meta data to configuration via property files. I don't particularly like the "@" annotation since these properties are typically passed to Spring.
Maybe we can have a discussion on this topic?

I was hoping to have a conversation about it.   The @properties get passed to Spring.  Our code controls that and we could "eat" the @System properties at a really clean place.  But the code that hunts for a bean with the @name and passed theproperty to it does not care if the @bean does not exist:
<EMAIL_ADDRESS>Perhaps I should name the suffix something like @SystemLevelProperty so as to make sure we never hit a bean with that name.
I am also happy to change the code that passed the properties into beans and stop the @SystemLevelProperty ones from being forwarded to Spring.
I would be rather opposed to some new place for these kinds of properties - it is jsut one more thing to drag around when supporting lots of clients.   There needs to be some convention in sakai.properties IMHO.

Lets chat in JIRA so if this PR is closed and resubmitted we have all the conversation in one place.
https://jira.sakaiproject.org/browse/KNL-1361


[1210] Implement content_status in courses table

This is now being pulled in from the API and displayed in the table.
After

 EPIC: Courses table in manage-courses-frontend

 EPIC: Courses table in manage-courses-frontend


⚠ Jenkins Plugins Artifactory has degraded performance

In dad2910, Jenkins Plugins Artifactory (https://get.jenkins.io/plugins/artifactory/) experienced degraded performance:

HTTP code: 200
Response time: 1184 ms

Resolved: Jenkins Plugins Artifactory performance has improved in 519f428 after 25 minutes.


Defer image decoding when scrolling fast.

Description
A List or GridView of images can end up consuming fast amounts of memory, particularly when the user "flings" the list causing the scrollable to go ballistic.
The basic problem is that we push a ton of decoding work onto the IO thread of the engine, and then end up not even needing those images because the children aren't in the tree anymore.  This causes thrash on the image cache, wasted CPU load for decoding those images, and in some cases OOM death.
I've resolved this by adding a new ImageProvider implementation that allows the caller to defer or cancel loading the image. The ImageState now uses this to wrap its incoming image provider, and defers loading until scrolling has slowed down enough that we would be seen on screen (defined as scrolling velocity < the physical number of pixels on the longest side of the screen). If we're no longer in the tree at that point, we cancel the load. This provider is cache-aware, and will just return the image in the cache if at any point during its cycle the image appears in the cache.
/cc @hugosantos @ignatz @alml  @cvarvara
/cc @liyuqian @chinmaygarde
Related Issues
https://github.com/flutter/flutter/issues/32143
https://github.com/flutter/flutter/issues/44510
b/144232910
Tests
I added the following tests:

Tests that the image cache can tell us whether it has a pending or completed image for a given key
Tests that the DeferringImageProvider can cancel, defer, or resolve an image
Tests that the DeferringImageProvider will check the cache (in case someone pre-cached the image, or in case another widget managed to cache it for us since we last checked).
Tests that a large grid view of images driven by a high velocity ballistic simulation do not try to load images during the high velocity part of the simulation.

Checklist
Before you create this PR confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I signed the CLA.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I updated/added relevant documentation (doc comments with ///).
[x] All existing and new tests are passing.
[x] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Right now this is - I'm open to feedback suggesting ways to make it not be.
Will add link to design doc shortly - will add link to migration guide if this remains an API break before landing.
Did any tests fail when you ran them? Please read Handling breaking changes.

[ ] No, no existing tests failed, so this is not a breaking change.
[x] Yes, this is a breaking change. If not, delete the remainder of this section.

[x] I wrote a design doc: https://flutter.dev/go/deferred-image-decoding
[ ] I got input from the developer relations team, specifically from: Replace with the names of who gave advice
[ ] I wrote a migration guide: Replace with a link to your migration guide

Related: https://github.com/flutter/flutter/issues/32156


Ziti Tunneler shows in start menu

Even when i click 'off' the app (and it disappears) the "MainWindow" shows in the task bar

looking at the code i now see that this is the expected behavior when in 'detached' mode. closing


Change void main to int main

Nice Catch. Thanks.

Nice Catch. Thanks.

Well it was actually the Clang compiler that caught this for me. ;)


Update TypeDoc comments for botFrameworkAdapter.ts

Description
Copy edited the documentation comments for public and protected members.

Fixes broken formatting, such as malformed links.
Attempts to normalize the reference documentation comments.

Specific Changes

Replaces jsDoc comments with TypeDoc comments.
Attempted to bring the JavaScript ref documentation more in line with the C# ref documentation.

Testing
I performed a local build of the reference documentation using a tool chain similar to what the publishing team uses.

Pull Request Test Coverage Report for Build 78885

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 88.594%

Totals

Change from base Build 77828:
0.0%

Covered Lines:
4428

Relevant Lines:
4804

💛  - Coveralls


Improve ISDB-Tb time fields handling

As @lelegard said in #602, ISDB chose to reuse parts of DVB while adding modifications like using JST instead UTC. During its first years it served for their purpose as it was only used at Japan.
When Brasil decided to use ISDB as the base for their digital terrestrial TV system that later was standardized as ISDB-Tb, they chose to deviate even further by not using UTC but... UTC-3! (Brasilia Time). All South American ISDB-Tb deployments are based in this standard (ABNT NBR-15603-2, see references)
With the current version of TSDuck, using tsanalyze or tsdate on ISDB-Tb streams TS1globo.ts or TS2band.ts available on website will report semantically invalid time fields. For example:
$ tsdate TS1globo.ts
* TOT UTC time: 2015/12/07 10:02:46.000
  Country: BRA, region: 3
  Local time:   2015/12/07 10:02:46.000, local time offset: 00:00
  Next change:  2012/02/18 00:00:00.000, next time offset:  00:00
...
$ tsdate -v TS1globo.ts
* TOT, TID 115 (0x73), PID 20 (0x0014)
  Short section, total size: 29 bytes
  - Section 0:
    UTC time: 2015/12/07 10:02:46
    - Descriptor 0: Local Time Offset (0x58, 88), 13 bytes
      Country code: BRA
      Region id: 3 (0x03), polarity: east of Greenwich
      Local time offset: 00:00
      Next change: 2012/02/18 00:00:00
      Next time offset: 00:00
    CRC32: 0x72901546 (OK)
...

If the TS is played at 01:30, one can see that the original local time was 10:04 = 13:04 UTC.
I see two alternatives to fix this issue:

Internally convert UTC-3 to UTC

Displayed UTC is the correct UTC
External programs expecting UTC will get the correct time
Local time offsets will require conversion to keep local times correct.

Keep UTC-3 as-is internally but change output texts (UTC/UTC-3/JST) depending of the detected standard or what is provided as an assumption via command line

Local time conversions are straightforward
Processors and external programs expecting real UTC may misbehave.

According to DiBEG, ISDB-T deployments outside Japan (using JST) and South America (using UTC-3) like the ones made in Botswana, Philippines and Maldives chose to use their local times as the time reference, complicating this situation even further.
References
http://www.telemidia.puc-rio.br/~rafaeldiniz/public_files/normas/SBTVD/pt_BR/ABNTNBR15603-2_2007Vc_2008Ing.pdf - 2008 version
https://www.abntcatalogo.com.br/norma.aspx?ID=329893 - 2015 version (requires registration)
(Both documents are English versions.)

Hi @nicst35
Thanks for reporting this. It's good to have a contact in Brazil for ISDB-Tb specificities. Currently, TSDuck testers for ISDB  were all in Japan. With their help, TSDuck implemented Japanese specificities. Character set support was the most difficult part. Let's now do the same thing with ISDB-Tb, in Brazil and elsewhere.
It is clear that ISDB people had very little consideration for the concept of "international standard" and interoperability in a connected world. They invented their own world, reusing parts of DVB while changing the semantics of SI (and even the syntax sometimes!)
Concerning the time reference, there are two different contexts: TOT/TDT and EIT.
In TOT/TDT, the field is named UTC_time and should be consequently understood as UTC. Note that the ABNT standard dares to write "UTC_time: [...] current time and date in Brazil (UTC-3)" (respectively JST in the ARIB version). So, TSDuck tries to adjust the time to UTC to get a consistent UTC_time field everywhere.
In several commands, the option --japan was added to indicate some presets for Japan. First, the options --isdb --default-charset ARIB-STD-B24 --hf-band-region japan are automatically set. Second, the time in TDT/TOT is interpreted as JST. The field utc_time in EIT C++ objects and the corresponding attribute in <EIT> from XML files were automatically updated back and forth. So, a UTC_time is always UTC in C++ and XML but the binary serialization and deserialization are affected by option --japan.
Note that there were two restrictions with option --japan. First, it was missing from the tsdate command. Second, the UTC adjustment was not done when displaying TDT/TOT with tstables and similar commands. I just fixed those two bugs.
I understand your concern about ISDB-Tb. So, I generalized the UTC adjustment using a new option --time-reference which specifies the type of adjustement from UTC to apply to TDT/TOT. The default is none, as specified by DVB. The help text is:
--time-reference name
    Use a non-standard (non-UTC) time reference in TDT/TOT. This is
    typically used in ARIB ISDB and ABNT ISDB-Tb standards. The specified
    name can be either 'UTC', 'JST' (Japan Standard Time) or 'UTC+|-hh[:mm]'.
    Examples: 'UTC+9' (same as 'JST' for ARIB ISDB), 'UTC-3' (for ABNT
    ISDB-Tb in Brazil).

As a convenience, the field default.time has been added in the user's configuration file to allow a global default for users always working on TS with a specific non-standard time reference.
So, in an ISDB context, three possibilities:

Option --japan now means --isdb --default-charset ARIB-STD-B24 --hf-band-region japan --time-reference JST.
The new option --brazil means --isdb --time-reference UTC-3.
A specific --time-reference can be used anywhere else.

In EIT, the time field is named start_time. The description of the semantics differs in the various standards but the field name does not enforce a specific one. Event information is just informational, typically for an EPG. Because this is a very "local" concept without impact on the stucture of the transport stream, TSDuck does not interpret it. The stored value is identically used, without particular semantics, in the EIT C++ objects, in the <EIT> in XML files and displayed by tstables and similar commands. The options --japan, --brazil and --time-refernce do not affect the interpretation of the events start time.
Please provide any useful feedback on TSDuck for ISDB in South America. Did you have the opportunity to test ISDB tuners or modulators with TSDuck?

Thanks.

Note that the ABNT standard dares to write "UTC_time: [...] current time and date in Brazil (UTC-3)"

Note that the Spanish and Portuguese(the original one) versions put that field as UTC-3_time. There are slight differences between them (along with PDF rendering errors).
(http://www.telemidia.puc-rio.br/~rafaeldiniz/public_files/normas/SBTVD/)

So, I generalized the UTC adjustment using a new option --time-reference which specifies the type of adjustement from UTC to apply to TDT/TOT

This creates a problem with ts::TOT::localTime(). Since it acts on UTC time, should it apply the inverse offset along with the local offset? tsanalyze is now reporting an incorrect TOT local time

The new option --brazil means --isdb --time-reference UTC-3

It should also imply --default-charset ISO-8859-15.

In EIT, the time field is named start_time. The description of the semantics differs in the various standards but the field name does not enforce a specific one.

Semantics of information for event information section shall be according to EN 300 468:2007, subclause 6.2.4; except for the fields below:
⎯ table_id: see Table 6;
⎯ start_time: 40-bit field that shall contain the start time of the event in the UTC-3 and the date in “modified julian date” (MJD) (see Annex A)…

My country (CL) is on DST right now so our local time is UTC-3, so EIT times would match the real ones.
The only two broadcasters in my zone that are transmitting digital TV rushed the deployment so no EIT for now.

A good way to check it would be getting a TS from a peruvian broadcaster, since they use UTC-5 (and -4 during summer) so they should always have an offset, and I am assuming they didn't change the standard.

Hi @nicst35
Thanks for your feedback. With your help, I will try to make TSDuck as compliant as possible with ISDB-Tb.

Note that the Spanish and Portuguese(the original one) versions put that field as UTC-3_time. There are slight differences between them (along with PDF rendering errors).

Inconsistencies between translated versions is a pain. I can't speak Portuguese and I lost all Spanish I learnt at school, so I can only use the English versions. It was worse with ARIB since English versions are outdated and don't even exist for some documents. I had to rely on Japanese users for help (as well as Google Translation for a few specific differences in the most recent and Japanese-only versions of the standard). All technical international standards should be written in English, full stop, regardless of little national susceptibilities. English is the de-facto reference technical international language, the only language which is understood by all engineers all over the world, just as Latin was in the middle age. And I say that as a Frenchman, not a native English speaker ;) But I am so often pissed off by those little French susceptibilities on linguistic pride...

This creates a problem with ts::TOT::localTime(). Since it acts on UTC time, should it apply the inverse offset along with the local offset?

Yes, that makes sense. The local_time_offset_descriptor is defined as "offset from UTC" by DVB but should be intepreted here as "offset from whatever is in the field misnamed UTC_time".
Just for the fun, look at what is written in the ABNT English version for local_time_offset_descriptor:

local_time_offset_polarity: [...] When set to "0", the local time shall advance from UTC time
local_time_offset: [...] current offset time from UTC-3

UTC in one case, UTC-3 the other one...
Anyway, I fixed ts::TOT::localTime() as you suggested.

It should also imply --default-charset ISO-8859-15.

We should discuss this more in details. I do not see any precise description of how strings should be handled in the ABNT documents. In the ARIB documents, it is clear that everything should be ARIB STD-B24. But this useful for the Japanese language only.
Searching for "8859" in ABNT NBR 15603-1, -2 and -3, I can find references to ISO-8859-15 in the various ISO_639_language_code fields, country_code fields and the text fields in the short_event_descriptor and extended_event_descriptor
In the case of ISO_639_language_code and country_code, this is useless. Quote:

ISO_639_language_code: [...] specified by ISO 639-2. Each character is encoded [..] in accordance with ISO 8859-15
country_code: [...] specified in ISO 3166-1. Each character will be coded [...] according to ISO 8859-15

Both ISO 639-2 and ISO 3166-1 specify simple short strings with ASCII characters only. The original standards which define them, ISO 15818-1 and ETSI EN 300 468 specify the use of ISO 8859-1. Since ISO 8859-1 and 8859-15 are supersets of ASCII and all language or country codes use ASCII characters only, we can safely ignore this.
Concerning text fields in the two event descriptors, this is understandable. However, there are other types of strings in ISDB SI. Just a few examples: service_descriptor, network_name_descriptor, bouquet_name_descriptor, etc. Those strings are obviously very "local" names with possibly specific characters. I assume that they use ISO 8859-15 by default as well. However, this is not clearly specified.
The question should be then raised for other uses of ISDB, outside Japan and South America. ISO 8859-15 is the preferred encoding for western european languages (Spanish, Portuguese, French). So it makes sense as default for South America. Some French satellite providers even violate DVB standards and use it as default, without leading charset code (see option --europe in TSDuck). But what about Asian or African countries using ISDB?
Finally, with --brazil, shall we use charset ISO-8859-15 or RAW-ISO-8859-15? The difference is subtle but important. With DVB methods, there are various ISO-xxx charsets (plus UTF-8 and -16). The ISO-xxx charset is selected by the first one or two bytes (if lower than 0x20). Without such leading byte, the default is ISO-6937 (or, more precisely, a slightly modified version of it). The names ISO-xxx in option --default-charset designate the default charset but it does not prohibit the usage of other charsets.
So, with --default-charset ISO-8859-15, if the first byte is greater than 0x1F, then ISO-8859-15 is used as default instead of ISO-6937. But if the string starts with a binary sequence to select another charset, then the regular DVB rules apply.
This is for decoding strings. Now, for encoding strings, TSDuck tries a few charsets to see which one can fit. In fact, not all characters can be encoded in each ISO-xxx. You have to try until one works. With --default-charset ISO-8859-15, the option indicates which charset to try first. But if it does not work (because some characters in the string cannot be encoded with that charset), it will try others. An in all cases, the proper binary selection sequences are correctly generated. If all charsets fail, then TSDuck falls back to UTF-8 which can encode everything.
Now come the RAW-ISO-xxx charsets. Here, we leave the DVB scheme with its optional initial selection sequences. The complete binary sequence in the string is unconditionally interpreted (deserialization) or generated (serialization) using the selected ISO-xxx. Specifically, in the case of the generation, no selected sequence is generated and characters may be absent if not representable in that charset.
This is why I ask shall we use charset ISO-8859-15 or RAW-ISO-8859-15?
To summarize, ARIB and ABNT standards are very hard to understand when you are used to the strictness and precision of ISO, ITU and ETSI (DVB) documents because of the uncertainty, contradictions and lack of strictness.

Semantics of information for event information section

About EIT, the start_time remains as it is, without specific semantics. So, the displayed time (as well as values in XML files and C++ objects) shall be interpreted according to the local standard. This was already the case with ARIB. So, I did not change it for ABNT. Do you think we should change anything?

To illustrate the various character sets, we can use the command tscharset, a useful debug tool I added while working on ARIB STD-B24 support.
By default, DVB uses ISO-6937. Let's start with pure ASCII codes:
$ tscharset --encode "-c-a-e-i-e-"
2D 63 2D 61 2D 65 2D 69 2D 65 2D

This is encoded as default ISO-6937, using pure ASCII codes.
Let's add a special character 'µ' which is available in ISO-6937:
$ tscharset --encode "-c-a-e-i-µ-"
2D 63 2D 61 2D 65 2D 69 2D B5 2D

We are still in the default character set ISO-6937 with 'µ' being encoded as B5.
Now let's use western european characters:
$ tscharset --encode "-ç-à-é-ï-è-"
0B 2D E7 2D E0 2D E9 2D EF 2D E8 2D

This time, some characters are not representable as ISO-6937 and TSDuck automatically switched to ISO-8859-15 (leading binary sequence 0B as specified in section A.2 of ETSI EN 300 468).
Now let's try to use ISO-8859-1 by default.
$ tscharset --encode "-ç-à-é-ï-è-" --default-charset ISO-8859-1
10 00 01 2D E7 2D E0 2D E9 2D EF 2D E8 2D

This works because all characters are representable in ISO-8859-1 but in a more complicated way (the leading sequence for ISO-8859-1 is 10 00 01).
We can verify that this alternate ISO-8859-1 encoding is correctly decoded:
$ tscharset --decode "10 00 01 2D E7 2D E0 2D E9 2D EF 2D E8 2D"
-ç-à-é-ï-è-

Now let's try to use ISO-8859-2 by default or forcing the default ISO-6937.
$ tscharset --encode "-ç-à-é-ï-è-" --default-charset ISO-8859-2
0B 2D E7 2D E0 2D E9 2D EF 2D E8 2D
$ tscharset --encode "-ç-à-é-ï-è-" --default-charset ISO-6937
0B 2D E7 2D E0 2D E9 2D EF 2D E8 2D

Nope, some characters are not representable and TSDuck selected ISO-8859-15 (leading 0B).
Now, if we use the RAW- versions instead, we force to use a specific charset, without adding any DVB-compliant leading binary selection sequence and omit characters which cannot be represented in the forced charset:
$ tscharset --encode "-ç-à-é-ï-è-" --default-charset RAW-ISO-8859-2
2D E7 2D 2D E9 2D 2D 2D
$ tscharset --encode "-ç-à-é-ï-è-" --default-charset RAW-ISO-6937
2D 2D 2D 2D 2D 2D

In the case of RAW-ISO-8859-2, the characters 'à', 'ï' and 'è' are not representable and deleted. In the case of RAW-ISO-6937, no special character can be represented and only the dashes remain.

I think RAW-ISO-8859-15 should be the charset to use for, because of what I read from the standard (and the operational guidelines), other charsets are not considered.

But what about Asian or African countries using ISDB?

They simply "patched" the standard and put their preferred encoding.
https://ntc.gov.ph/wp-content/uploads/2018/MC/MC-07-12-2014-Attachment.pdf, page 15. They chose UTF-8 for SI tables.
https://www.bocra.org.bw/sites/default/files/documents/ISDB-T Standard for Botswana - ISDB-Tbw.pdf, not as radical as the first one but they use UTF-8 for captions/subtitles and data broadcasting (BML).

I think RAW-ISO-8859-15 should be the charset to use

Alright.
I started to have a look at the Philippines document. I noticed the UHF frequency mapping and updated the TSDuck HF bands file.
Do you know what is the UHF frequency mapping over South America, country by country? Any difference between countries? The required information is summarised here in the new entry for the Philippines:
  <hfband type="UHF">
    <region name="philippines"/>
    <channels first_channel="14" last_channel="44" base_frequency="473,000,000" channel_width="6,000,000"/>
  </hfband>

For countries or regions with identical mapping, several names can be specified.

Hi @lelegard.
I think the ABNT frequency table should be used, at least for South America, that means:
<hfband type="VHF">
    <region name="abnt"/> <!-- or 'south-america' ? -->
    <channels first_channel="7" last_channel="13" base_frequency="177,142,857" channel_width="6,000,000"/>
</hfband>
...
<hfband type="UHF">
    <region name="abnt"/>
    <channels first_channel="14" last_channel="36" base_frequency="473,142,857" channel_width="6,000,000"/>
    <channels first_channel="38" last_channel="69" base_frequency="617,142,857" channel_width="6,000,000"/>
</hfband>

Taken from https://www.abntcolecao.com.br/normavw.aspx?ID=1162, pages 58-59 (PDF, not the printed ones)
Now for some country-specific allocations.

Chile: https://www.cntv.cl/wp-content/uploads/2020/11/CNTV-Decreto-que-contiene-CNTV-Plan_de_radiodifusion_televisiva-167_y_95.pdf, page 11,

Channel number, frequency range and central carrier frequencies are described. Even if it's a subset of the ABNT channels, frequencies match perfectly.

Argentina: https://www.enacom.gob.ar/multimedia/normativas/2013/Resolucion-7_13-Anexo 3.pdf, page 7.

Same situation as Chile, only a subset of the whole bands used but frequencies are the same.
A related document describes something called SATVD (Argentinian Digital TV System), basically provisions for 576/50i and p, 720/50p and 1080/25p and 50i broadcasting. Full text is available at https://www.enacom.gob.ar/multimedia/normativas/2013/Resolucion-7_13-Anexo 1.pdf (warning: Spanish-only).

Ecuador: https://www.arcotel.gob.ec/wp-content/uploads/2016/03/norma-television-digital-terrestre.pdf, page 7, same frequencies, but with gaps.
Paraguay: https://www.conatel.gov.py/conatel/wp-content/uploads/2019/10/rd-1599-2016-160907-modificacion-norma-tecnica-servicio-television.pdf, pages 1 and 3.

Note that channel 37 is reserved for radio astronomy here and can't be used. The frequencies are the same, but the lowest and highest usable channels are different. Personally I think putting every country in the frequency mapping is overkill and the general frequency plan should be included in the table.

Hi @nicst35
Thank a lot for all these reference information. Spanish documents are good enough for frequency tables and some useful comments are phonetically so close to French that I can easily understand the idea.
I have updated the HF band files in TSDuck.

Personally I think putting every country in the frequency mapping is overkill and the general frequency plan should be included in the table.

It is not a big deal to add different sections for different countries. Even if the general frequency layout is the same all over South America, I did it precisely because the number of allocated channels is different by country. This makes a huge difference when using tsscan. This is usually a lengthy process and avoiding unused channels in a specific country is usually appreciated.
I noted that Botwana uses 8 MHz channels (like DVB countries) while all other ISDB countries use 6 MHz channels.
I think that we now have a fairly complete logical view of ISDB outside Japan.
Do you have any ISDB-T USB tuner to try TSDuck with? It could be nice to have a snapshot of the Chilean terrestrial TV in the TSDuck stream repository.

Yes, a Geniatech (MyGica) S270. Works well with minor quirks: OFDM modes and Guard Intervals are reported as "Auto" (they're available via debugfs though).
I can send you two streams (~700MiB) but they're quite basic, only the minimum required SI tables are sent, meaning there's no EIT is available on my area.
If someone living in Santiago can receive a channel transmitting additional tables (EIT, SDTT, etc.), please respond.

Yes, a Geniatech (MyGica) S270. Works well with minor quirks

Does it work with TSDuck on Windows?
I have one too, a generous gift from a Japanese user. But, being in Europe, I have no ISDB-T network to test it and no ISDB-T modulator to build a test bed. I can only see how it is recognized by TSDuck on the various OS.
On Linux, it is correctly reported as an ISDB-T demodulator:
$ tslsdvb
0: /dev/dvb/adapter0 ("Siano Mobile Digital MDTV Receiver, MDTV Receiver 2.00", ISDB-T)

But, on WIndows, it is seen as a DVB-T demodulator:
$ tslsdvb
0: "BDA Tuner Filter" (DVB-T, DVB-T2)

So, I wonder how it can work on Windows.

Maybe there's something useful: https://github.com/tsduck/tsduck/issues/389#issuecomment-541480324
As mentioned there, so many ISDB-related interfaces exist (https://docs.microsoft.com/en-us/previous-versions/windows/desktop/api/dvbsiparser/nf-dvbsiparser-iisdbterrestrialdeliverysystemdescriptor-gettransmissionmode), that it's pretty unlikely that it's not supported in tuning.
I would start looking at the registry after having confirmed that some other Windows application is working with the tuner.

@SS-FF, yes, I have used many useful information you provided for ISDB tuner on Windows (without being able to test in my continent, unfortunately).
All the code is normally in TSDuck. However, on this specific tuner, after connecting the tuner filter to the network provider filter, I query the ITunerCap interface of the network provided and get only one network type: DVBTNetworkProvider while I would have expected to get ISDB_TERRESTRIAL_TV_NETWORK_TYPE.

I don't know anything better than you in that case.
What I wanted to suggest is to look into the registry where the tuning spaces are defined to (hopefully) get an idea of how it's handled.

(maybe requires that another application has been running before)

Look at the following. I never had any ISDB device installed, and the tuning space is still listed.
And what's interesting: The CLSID for the DVB-T tuning space is the exact same as for the ISDB-T tuning space:

So most likely, this is all handled just like DVB-T..?

PS: All tuning spaces in that list have the same CLSID. But for DVB-S it's different.

The CLSID for the DVB-T tuning space is the exact same as for the ISDB-T tuning space:

Yes, but the problem is slightly different.
The method ITunerCap::get_SupportedNetworkTypes returns network types.
From uuids.h:
// {216C62DF-6D7F-4e9a-8571-05F14EDB766A}
OUR_GUID_ENTRY(CLSID_DVBTNetworkProvider,
0x216c62df, 0x6d7f, 0x4e9a, 0x85, 0x71, 0x5, 0xf1, 0x4e, 0xdb, 0x76, 0x6a)

From bdamedia.h:
#define STATIC_ISDB_TERRESTRIAL_TV_NETWORK_TYPE \
    0x95037f6f, 0x3ac7, 0x4452, 0xb6, 0xc4, 0x45, 0xa9, 0xce, 0x92, 0x92, 0xa2
DEFINE_GUIDSTRUCT("95037F6F-3AC7-4452-B6C4-45A9CE9292A2", ISDB_TERRESTRIAL_TV_NETWORK_TYPE);
#define ISDB_TERRESTRIAL_TV_NETWORK_TYPE DEFINE_GUIDNAMED(ISDB_TERRESTRIAL_TV_NETWORK_TYPE)

#define STATIC_ISDB_T_NETWORK_TYPE \
    0xfc3855a6, 0xc901, 0x4f2e, 0xab, 0xa8, 0x90, 0x81, 0x5a, 0xfc, 0x6c, 0x83
DEFINE_GUIDSTRUCT("fc3855a6-c901-4f2e-aba8-90815afc6c83", ISDB_T_NETWORK_TYPE);
#define ISDB_T_NETWORK_TYPE   DEFINE_GUIDNAMED(ISDB_T_NETWORK_TYPE)

There is one value for DVB-T and two values for ISDB-T (why two?). All three values are different.
And for the MyGica S270, the only returned value is CLSID_DVBTNetworkProvider. Maybe we could force some usage for ISDB-T tuning spaces. But, at some point, TSDuck needs to know which type of tuner it is. If DirectShow defines distinct values for DVB-T and ISDB-T and nonetheless an ISDB-T tuner identifies itself as supporting DVB-T only, we cannot do much.
Note: On Linux, this tuner is correctly identified as ISDB-T only, right after one single ioctl(), without running thousands of lines of code, building complex graphs and praying the magic bat at sunset...

You know I haven't gone through this myself, but I think I got a few leads to the right solution for this.
I think part of the solution for this is in the code that you had dropped a while ago (which was right, though).
But let's start somewhere else. Look at the following enum:

What this tells us, it that ISDB is handed just like another variant of DVB (unlike ATSC for example).
And then, do you remember the following code?

One step earlier, we have been enumerating the system-defined (existing in the registry) tuning spaces and tried one after another whether it's compatible (via tuner->put_TuningSpace).
Then we had changed that (as I had advised at that time) to constructing our own tuning spaces (GetTuningSpaceFromNetworkType).
In the code above, when tuner->put_TuningSpace succeeds, we were querying for the IID_IDVBTuningSpace interface.
And the IDVBTuningSpace interface has a method get_SystemType. And what does that method return?
=> A value from the enum at the top of this post - which can also be ISDB-X.
Conclusion
My guess would be that ISDB-T is implemented using

The DVBTuningSpace
A DVBTLocator
Where

DVBTuningSpace::SystemType== ISDB_Terrestrial
DVBTuningSpace::_NetworkType == STATIC_ISDB_TERRESTRIAL_TV_NETWORK_TYPE
(or the other one, I really don't know why there are two of them)

To test what kind of tuner we have, we will probably need to construct such tuning spaces and use tuner->put_TuningSpace to check which one the tuner will accept.

As a kind of proof why I'm quite confident that it will work like that, let's go back to the screenshot from GraphStudioNext that I have posted above:
The first property page is showing various tuning spaces (precisely, they are all DVBTuningSpace inheritors), including an ISDB-T tuning space.
But when you go to the second property page, there is just one or two tuning spaces remaining
=> those that are compatible with the tuner
And that's the mechanism you're looking for.
(now that I'm done, I start to remember that I've written almost the same text a while before...)

Hi, given that the most basic issues with ISDB-Tb handling were solved. Can we mark this issue as closed?

Hi, given that the most basic issues with ISDB-Tb handling were solved. Can we mark this issue as closed?

Yes, I think so.

@lelegard - Have you been able to try whether my suggested procedure works?

Have you been able to try whether my suggested procedure works?

Honestly, no, I didn't. First, I have no ISDB network to test and anything I could see on a system won't be tested up to the packet reception. So, there will still be a risk that all the work is incomplete.
I must say that I am positively fed up with DirectShow programming.
I just checked the source code:

On Linux, the overall code to handle tuners is 2500 loc, including comments, and verbose code to set/get parameters. Apart from enhancements for new modulations or new forms of tuner identification, the code was written once and worked.
On Windows, the same code uses 6700 loc. Most of it was developed the hard way, without precise documentation, using extremely long trial and error cycles. A large part of the code was written more than once, especially when DirectShow-experienced people like you provided advices which were not otherwise available in the documentation. Each time a new type of tuner appears, this is not just a matter of a bunch of new parameters to add. We must reconsider again and again new approaches, new hypotheses, new tests, maybe, let's try, etc. And the result is still less reliable, with less performance than on Linux.

So, I give up with DirectShow. The code is good enough to capture a stream from a Windows laptop when you move to some regional location. But for stable reception operations, I have been recommended Linux for 15 years now and every new issue just confirms this position again and again.
I would be happy to merge PR's which prove to improve the tuner implementation on Windows. But, as far as I am concerned, I am simply disgusted by DirectShow once more. TSDuck is a free and open source project which is only powered by enthusiasm and DirectShow clearly kills any form of enthusiasm. So, I give up, full stop.

But, as far as I am concerned, I am simply disgusted by DirectShow once more.

I totally understand that and I acknowledge that you still got that far with tsDuck.

Have you been able to try whether my suggested procedure works?

Honestly, no, I didn't. First, I have no ISDB network to test and anything I could see on a system won't be tested up to the packet reception. So, there will still be a risk that all the work is incomplete.

I thought you'd be working on fixing the detection of the user's MyGica tuner, that's why I had responded to your question in that comprehensive way.
(also hoping it would confirm my theory which I had posted here about a year ago ;-)


Next.js : Prop id did not match. Server: "pattern-61302" Client: "pattern-75795"

Describe the Bug
When trying basic integration within next.js, the following error returns.

Warning: Prop id did not match. Server: "pattern-61302" Client: "pattern-75795"
at pattern
at svg
at Background (webpack-internal:///./node_modules/react-flow-renderer/dist/esm/index4.js:45:27)
at Provider (webpack-internal:///./node_modules/zustand/esm/context.js:12:5)
at Wrapper (webpack-internal:///./node_modules/react-flow-renderer/dist/esm/index.js:3701:23)
at div
at eval (webpack-internal:///./node_modules/react-flow-renderer/dist/esm/index.js:3749:20)
at WorkflowConfigurator (webpack-internal:///./components/workflows/Configurator.tsx:68:62)
at div
at div
at main
at div
at div
at Layout (webpack-internal:///./components/layout/Layout.tsx:79:23)
at AddCompanyPage
at MaxFormsWeb (webpack-internal:///./pages/_app.tsx:44:27)
at ErrorBoundary (webpack-internal:///./node_modules/next/dist/compiled/@next/react-dev-overlay/client.js:8:20584)
at ReactDevOverlay (webpack-internal:///./node_modules/next/dist/compiled/@next/react-dev-overlay/client.js:8:23125)
at Container (webpack-internal:///./node_modules/next/dist/client/index.js:357:9)
at AppContainer (webpack-internal:///./node_modules/next/dist/client/index.js:791:26)
at Root (webpack-internal:///./node_modules/next/dist/client/index.js:915:27)

Code used is the basic example:
`import { useState } from 'react';
import ReactFlow from 'react-flow-renderer';
const initialNodes = [
{
id: '1',
type: 'input',
data: { label: 'Input Node' },
position: { x: 250, y: 25 },
},
{
id: '2',
// you can also pass a React component as a label
data: { label: Default Node },
position: { x: 100, y: 125 },
},
{
id: '3',
type: 'output',
data: { label: 'Output Node' },
position: { x: 250, y: 250 },
},
];
const initialEdges = [
{ id: 'e1-2', source: '1', target: '2' },
{ id: 'e2-3', source: '2', target: '3', animated: true },
];
function Flow() {
const [nodes, setNodes] = useState(initialNodes);
const [edges, setEdges] = useState(initialEdges);
return ;
}
export default Flow;`
Your Example Website or App
No response
Steps to Reproduce the Bug or Issue
Install Next.js and react-flow and use the default example
Expected behavior
To be able to integrate with next.js
Screenshots or Videos
No response
Platform

OS: MaxOS
Browser: Chrome
Version: all

Additional context
No response

To extend on the issue description:
The background generates a unique id on the server- and client-side which results in non-matching ids -> throws Error.
You can currently work around that though by making the Background a non-ssr component.
import dynamic from "next/dynamic";

const NoSSRBackground = dynamic(
  () => import("react-flow-renderer").then((im) => im.Background),
  { ssr: false }
);

<ReactFlow ...>
  <NonSSRBackground color="#aaa" gap={16} />
</ReactFlow>

Here's a sandbox with the NonSSRBackground.
You can force the error in the sandbox by just using the regular Background component.

thanks for bringing this up. we are no longer use math random for generating the ids. Fix released in v10.1.2


fes-es.txt

add Friedrich-Ebert Schule

@mmetzler1 First stage of verification passed, official domain is fes-es.de as far as I verified. This is just an informational note. Pull request is still in the review. I appreciate your patience while waiting for the final decision.

@mmetzler1 Thank you for your patience while waiting for a review and processing your request. Pull request merged.


Merged manually, thanks for the review @aromanenko-dev !


Add mode on the RE that pauses on error

As a developer I would like a mode for plan development that goes into pause on error, rather than cleaning up after itself. This will allow me to work out what state EPICS is in when the error occurred rather than having to piece it back together after the RE has cleaned up.
It looks like finalize_wrapper and contingency_wrapper already support this:
https://github.com/bluesky/bluesky/blob/6116a09dc0a292dc5d08902f6fa7e25816c73b61/src/bluesky/preprocessors.py#L514-L516
Unfortunately the wrappers that call it like stage_wrapper do not expose pause_for_debug. I propose that the default argument for pause_for_debug becomes None, which means "Take your value from an environment variable", with the values True and False meaning what they currently mean. This would maintain backwards compatibility for forcing it on or off in custom plans, while supporting reading the environment variable in standard plans.
I suggest calling the environment variable BLUESKY_PAUSE_BEFORE_CLEANUP, and:
if pause_for_debug is None:
    pause_for_debug = os.environ.get("BLUESKY_PAUSE_BEFORE_CLEANUP", "NO").upper().startswith("Y")

@tacaswell @danielballan thoughts?
Acceptance criteria:

When environment variable is set to "YES" and an error occurs, the RE is put into pause state
If environment variable is not set existing behaviour is preserved
Docs are updated with this information

It is not global, but the finalize wrapper has a flag for this: https://github.com/bluesky/bluesky/blob/6116a09dc0a292dc5d08902f6fa7e25816c73b61/src/bluesky/preprocessors.py#L496-L516

It is not global, but the finalize wrapper has a flag for this:
https://github.com/bluesky/bluesky/blob/6116a09dc0a292dc5d08902f6fa7e25816c73b61/src/bluesky/preprocessors.py#L496-L516

GitHub has a race condition, creating an issue from a task tells people about it before you get a chance to write a description!

sorry for jumping the gun on an empty issue body!
I'm :+1: on making the default controlled by an ENV.

sorry for jumping the gun on an empty issue body!

No problem! I wish "create issue from task" gave you a UI to fill this in...

I'm 👍 on making the default controlled by an ENV.

Is the name ok? Any advance on BLUESKY_PAUSE_BEFORE_CLEANUP?

BLUESKY_PAUSE_ON_PLAN_EXCEPTION might be better?  "cleanup" is too generic and we have too many phases of cleanup!
One fun side effect of doing this globally might that you pause in every layer on the way out.  Might be worth the complexity of an additional escape hatch.

BLUESKY_PAUSE_ON_PLAN_EXCEPTION might be better? "cleanup" is too generic and we have too many phases of cleanup!

That sounds like a good name, let's go with that.

One fun side effect of doing this globally might that you pause in every layer on the way out. Might be worth the complexity of an additional escape hatch.

Do you mean that if you both @bpp.stage_decorator and @bpp.run_decorator and raise an exception in the plan then it will pause twice? That's probably ok, as we are likely to kill the plan on the first raise rather than resume.
We probably want to log the exception too on the first debug so that we can see why we paused.
Side note: why is finalize_wrapper not implemented as a call to contingency_wrapper?

Do you mean that if you both @bpp.stage_decorator and @bpp.run_decorator and raise an exception in the plan then it will pause twice? That's probably ok, as we are likely to kill the plan on the first raise rather than resume.

or any of the other places it might be used.  If you resume or stop then the exception will keep bubbling up through the stack and be handled by the next layer.  I can see why this would be useful (to investigate all of the clean up steps) but also super annoying (I found the problem just let me out!).  You can get out via RE.halt() but then you need to remake your RE to resume debugging.

Side note: why is finalize_wrapper not implemented as a call to contingency_wrapper?

We wrote finialize_wrapper first and did not go back to refactor it when we added contingency_wrapper.

Do you mean that if you both @bpp.stage_decorator and @bpp.run_decorator and raise an exception in the plan then it will pause twice? That's probably ok, as we are likely to kill the plan on the first raise rather than resume.

or any of the other places it might be used. If you resume or stop then the exception will keep bubbling up through the stack and be handled by the next layer. I can see why this would be useful (to investigate all of the clean up steps) but also super annoying (I found the problem just let me out!). You can get out via RE.halt() but then you need to remake your RE to resume debugging.

Maybe we should set a global flag on entry to finalize_wrapper if debug_on_pause or environment variable is set, then only pause if set, and clear the flag before raise?

Side note: why is finalize_wrapper not implemented as a call to contingency_wrapper?

We wrote finialize_wrapper first and did not go back to refactor it when we added contingency_wrapper.

Should we refactor as part of this ticket?

Yes to both

We discussed it with @yxrmz and have a clear idea how to implement it.


New package: MS v0.0.1

Registering package: MS
Repository: https://github.com/ctarn/MS.jl
Created by: @ctarn
Version: v0.0.1
Commit: 5d69193c2b601d27d25e070e8919ac5faf505bd2
Git reference: HEAD

The repository isn't accessible anymore


Crash when steam is closed

Checklist

[x] I have read the appropriate section in the contributing
guidelines
[x] I believe this issue is a problem with polybar itself and not a misconfiguration on my part.
[x] I have searched for other open and closed issues that
may have already reported this problem.
[x] I have checked the known issues page for this problem.

Describe the bug
When steam is closed, polybar is still running correctly, but the dwm module disappears. Closing here means closing it completely, not just clicking the x and leaving it in the tray.
Expected behavior:
dwm module crashes and disappears.
Actual behavior:
dwm module should not crash.
Was it working before?

No, I just installed polybar-dwm two days ago

To Reproduce
Start steam and close it by clicking: Steam > Exit

Polybar Log

polybar|notice:  Parsing config file: /home/aakodadi/.config/polybar/config
polybar|notice:  pulseaudio: using default sink alsa_output.pci-0000_27_00.6.analog-stereo
polybar|notice:  Loaded font "Liberation Mono:size=10" (name=Liberation Mono, offset=1, file=/usr/share/fonts/liberation/LiberationMono-Regular.ttf)
polybar|notice:  Loaded font "Font Awesome 6 Free:size=10" (name=Font Awesome 6 Free, offset=1, file=/usr/share/fonts/TTF/fa-regular-400.ttf)
polybar|notice:  Loaded font "Font Awesome 6 Free:style=Solid:size=10" (name=Font Awesome 6 Free, offset=1, file=/usr/share/fonts/TTF/fa-solid-900.ttf)
polybar|notice:  Loaded font "Font Awesome 6 Brands:size=10" (name=Font Awesome 6 Brands, offset=1, file=/usr/share/fonts/TTF/fa-brands-400.ttf)
polybar|error: module/dwm: [dwmipcpp] Client with window id 27263019 not found
polybar|notice:  Stopping 'module/dwm'...

Screenshots

Before closing steam

After closing steam

Environment:

WM: dwm
Distro: Artix Linux x86_64
Kernel: 5.18.0-artix1-1
Output of polybar -vvv:

polybar 3.5.2-100-g0c3e139a

Features: +alsa +curl +dwm +i3 +mpd +network(libnl) +pulseaudio +xkeyboard

X extensions: +randr (+monitors) +composite +xkb +xrm +xcursor

Build type: Release
Compiler: /usr/bin/c++
Compiler flags: -march=x86-64 -mtune=generic -O2 -pipe -fno-plt -fexceptions         -Wp,-D_FORTIFY_SOURCE=2 -Wformat -Werror=format-security         -fstack-clash-protection -fcf-protection -Wp,-D_GLIBCXX_ASSERTIONS -Wall -Wextra -Wpedantic -O3 -DNDEBUG
Linker flags: -Wl,-O1,--sort-common,--as-needed,-z,relro,-z,now

Additional context
I installed polybar-dwm-module from the AUR

Duplicate of #26


The progress bar appeares when I was using vim, and the speed becames slow

When I use the shell normally, such as git, the progress bar returns the progress very well.
But when I use vim, the progress bar reflects the line where I am, which causes a problem, vim becomes slow.
I mean, I feel that when I change the cursor up and down in VIM, terminus will refresh the progress bar, which takes time.
I used two kinds of vim, one is vim in windows environment based on msys2, which has a slow progress bar, and the other is vim from SSH to ubuntu without a progress bar, which is very fast.
Is there any way I can use that variable to control not to use this progress bar in vim? Or you have other suggestions to improve the speed of vim. Or can you explain the relevant code, and can I try to mention PR.

Hi, the progress bar is triggered when a program outputs a string that matches the \s\d+% pattern (i.e. a number and a percent sign), and only when the program is not in the "alternate screen" mode (the fullscreen mode that vi/nano/etc use). It looks like the alternate screen mode isn't detected properly with msys2. Can you please try doing the same under e.g. WSL or cygwin and see if the progress bar still appears then? Also changing the "Use ConPTY" setting under Settings -> Shell could help.

Great, indeed this is the key to the problem. When I opened it with mintty, he recognized the alternate screen, and the scroll wheel can be used in the vim environment, but terminus is not available at the moment. I will carefully see if I can find a solution from mintty .


Add continuous integration

Continuous integration to run tests and publish to npm.

Added unit tests and workflow to publish to npm.


Display frames for second

Instead of drawing the current fps on the screen, print the average fps to the log when the engine is shut down. Because it's simple.


--isolate_polyfills option in compiler does not exist

I need to use isolated polyfills if some file. I found polyfills description in the wiki and this methods does not works (https://github.com/google/closure-compiler/wiki/Polyfills)

Enable with --isolate_polyfills or options.setIsolatePolyfills(true);

But compiler says:
"--isolate_polyfills" is not a valid option
"options.setIsolatePolyfills(true);" in my js file does nothing.
Also, I cannot find responsible code in the repo.
How can I isolate polyfills?

Sorry about that, we forgot to add the --isolate_polyfills flag to https://github.com/google/closure-compiler/blob/master/src/com/google/javascript/jscomp/CommandLineRunner.java.
I'll do that now.


[Do not merge until v0.8] Removed src_dir from CoverallsConfiguration.

This change will go into the v0.8 release. Don't merge for v0.7 line.

Rebased, so it should pass the tests

Oops. I needed to remove src_dir from the test yaml files.

0.7 is still not released.
I can not understand why yo dont want to merge it into 0.7 ?

Because this is a breaking change. The old config files wil trigger an error.

And then, current version is 0.6. And you want to release 0.7. And that version change allows you to make BC breaks.

Strictly saying, that's true, but because it's been a while from the last stable release, I assume many people are using dev-master. If we merge this, their build will break. I would release 0.7.0 and start encouraging using that stable version, then will introduce breaking change in 0.8.0.

If sb using dev-master he is asking for it :D But I can accept that.
BTW, 0.7 won't be stable version... stable will be 1.0.

Ya, right.

Rebased

This change will go into 1.0.0 release. @keradus Do you want to take a look or is it good to merge?

I think you should keep it for 0.8 (as @keradus said, if somebody is using dev-master, they're asking for it) but meh, it looks good though.

:+1:

Thanks @keradus and @Taluu
Merging


Modify leaf appearance curve

Currently the leaf appearance curve works ok with branching varieties but it underestimates the number of leaves for the non-branching varieties (see example below for TME 419).

James suggested to use a linear curve and reduce the leaf appearance rate when the assimilates, water or nitrogen are not enough to supply the demand. Currently we are reducing the leaf appearance when there is water stress and nitrogen stress.
These are some testing for non-branching varieties using linear regression (in Excel):

And the model Wang-Engel:

Now we have two independent curves for branching vs. non-branching or late branching varieties (thermal time for first branching >1200):
IF (PDL(1) < 1200.) THEN
                IF (DAE > 0) THEN
                    IF (ISWWAT == 'Y') THEN
                        IF (ISWNIT /= 'N') THEN
                            LNUMG = ((1.048488E6*LNSLP)/((((3.5986E3))+DAWWP)**2))*TT* (AMIN1(WFG,NFG)) * WFGREA
                        ELSE
                            LNUMG = ((1.048488E6*LNSLP)/((((3.5986E3))+DAWWP)**2))*TT* WFG  * WFGREA                                      !LPM 31JUL2015 to consider water stress
                        ENDIF
                    ELSE
                        LNUMG = ((1.048488E6*LNSLP)/((((3.5986E3))+DAWWP)**2))*TT                                              !LPM 21/02/2015 leaf number curve
                    ENDIF
                ELSEIF (DAG > 0) THEN
                    IF (ISWWAT == 'Y') THEN
                        IF (ISWNIT /= 'N') THEN
                            LNUMG = ((1.048488E6*LNSLP)/((((3.5986E3))+DAWWP)**2))*TTGEM*(AMIN1(WFG,NFG)) * WFGREA                               !LPM 31JUL2015 to consider water stress
                        ELSE
                            LNUMG = ((1.048488E6*LNSLP)/((((3.5986E3))+DAWWP)**2))*TTGEM * WFG * WFGREA
                        ENDIF

                    ELSE
                        LNUMG = ((1.048488E6*LNSLP)/((((3.5986E3))+DAWWP)**2)) * Tfgem                                              !LPM 21/02/2015 leaf number curve
                    ENDIF
                ENDIF
            ELSE
                IF (DAE > 0) THEN
                        IF (ISWWAT == 'Y') THEN
                            IF (ISWNIT /= 'N') THEN
                                LNUMG = LNSLP * tfd * (AMIN1(WFG,NFG)) * WFGREA
                            ELSE
                                LNUMG = LNSLP *tfd * WFG * WFGREA                                       !LPM 31JUL2015 to consider water stress
                            ENDIF
                        ELSE
                            LNUMG = LNSLP * tfd                                              !LPM 21/02/2015 leaf number curve
                        ENDIF
                    ELSEIF (DAG > 0) THEN
                        IF (ISWWAT == 'Y') THEN
                            IF (ISWNIT /= 'N') THEN
                                LNUMG = LNSLP * Tfgem * AMIN1(WFG,NFG) * WFGREA                             !LPM 31JUL2015 to consider water stress
                            ELSE
                                LNUMG = LNSLP * Tfgem * WFG * WFGREA
                            ENDIF
                        ELSE
                            LNUMG = LNSLP * Tfgem                                              !LPM 21/02/2015 leaf number curve
                        ENDIF
                    ENDIF
            ENDIF


Short term solution for selecting a (sub)region for a scenario

The current interface for selecting a region in the ETModel interface is becoming messy.  That’s the reason to rethink this and find a new implementation.
We have to keep in the back of our heads that this implementation will not be suitable for 15.000+ datasets but stating that we are far far away from that happening, this will have to do in the mean time.
I have two basic ideas:
The first idea is adding a select box and splitting the regions from the countries:

So when Netherlands is selected, you'd have another select box with regions from The Netherlands. If however anther country is selected like Poland, there are no other regions to pick from:

Upsides:

Clear division of regions and countries

Downsides:

A user needs to perform an extra step to create a scenario for a local region.

Another approach to this problem might be a 'hierarchy':

Upsides:

Somewhat clear division of regions and countries

Downsides:

It won't look pretty, stating that in the current HTML-spec it's not possible to create a hierarchy in an option list. However, there are probably customizations available online which can make it look nicer than the default html select box implementation.

The current interface for selecting a region in the ETModel interface is becoming messy. That’s the reason to rethink this and find a new implementation.

Could we go back a step and consider in what way it is messy? I agree that it doesn't look very pretty, but how would you describe the problem with the current option list? Too many entries? Not clear why there are three different sections? ...

but how would you describe the problem with the current option list?

Too many entries? Not clear why there are three different sections?

Well those two things definitely. Also it's now pretty unclear that the third section of regions are all regions within The Netherlands. For an outsider it might not be immediately clear that that's the case.

Using optgroups would kind of solve those issues – except for there still being too many entries – and would be really simple to implement. I think it's also easier for visitors to not show them a separate "Region" list.

If having too many entries is a problem, then this won't work (and I think the second idea – "hierarchy" – is probably therefore also a no-go).
I don't have a strong opinion to be honest, do you have a preference?

I think it's also easier for visitors to not show them a separate "Region" list.

Yes.
I think the optgroup solution might actually work for the time being. Also since the only supported regions are the one's from The Netherlands this will hold. When the list becomes larger than >100 items this idea might need to be revisited.


Building and running Lacework CLI in Docker-less environments using podman and buildah

Hi,
Please find below how I build and run Lacework CLI containers in Docker-less environments using podman and buildah:
Buildah from scratch:
https://github.com/marcredhat/lacework/blob/master/demo4.sh
Buildah from Dockerfile:
https://github.com/marcredhat/lacework/blob/master/demo3.sh
Hope this helps,
Marc

Hey Marc, thank you for bringing these demos to our attention and to play with the Lacework CLI! 💯
I have a couple of things to mention, the first one is that we introduced a new feature to the
lacework configure command to run non-interactively. I saw that your shell scripts are executing
the command:
lacework configure --account "<account>" --api_key "<API key>" --api_secret "<API secret>"

I would recommend you to pass the flag --noninteractive to the previous command to avoid the
interactions with a prompt.
The second thing to mention is that we do have a certified container for our agent:

https://catalog.redhat.com/software/containers/lacework/lacework-agent/5e8b82e2ecb5246c09fe95db

But, we do not have a container for our CLI, which is definitely a feature we would like to have and that you
have already contributed with your scripts. 🥇 Thank you for that!
Last, we were wondering what was the motivation to work on this? what are you using the Lacework CLI for?
Is there anything we can help you with?


Ensure the vmq_bridge is properly restarted after a crash

This should fix #785

Removing the unsubscribe mechanism could be a problem, as one could state in vernemq.conf to use clean_session=off for a bridge, which would potentially result in a remote subscription that couldn't be removed anymore.

I belive this is already a problem today if one would shutdown the broker, change the subscriptions and then restart the broker?
But isn't this behaviour what should be expected?


Defining variables containing other data seem to be a proxy to the original

Version
2.5.13
Reproduction link
http://git.dannysmc.com/snippets/61
https://dannysmc.com/vuejs-issue.html
Steps to reproduce
Just go to the link, put it into a html file and load it in a browser, then look at the content, currently it is true or false (which is correct), then click save and you will see all the true and false switch to 1 or 0 respectively, even though I am not directly setting it.
What is expected?
It should just update the data variable and not affect the someData, it should only update the data and that is it.
What is actually happening?
It seems to be proxying the someData through the data variable, therefore you're unable to take variables from vue's data object and change them in a new variable, because they proxy back to the original place.

I could be wrong and this could be expected behaviour in my eyes, these seems wrong, as that doesn't make sense as to why that happens. I have tested on 2.5.16 as well and the same happens. If this is not expected can this be fixed and why does it happen? and if it is expected, how do you create a separate variable containing the contents of a variable from the vue's data object?
I noticed it because I am using this issue within something I am building for my company as we are making extensive use of Vue for many reasons, but what I was doing is when I have a lot of data, I was preparing it and setting it to the correct values for the database before sending it up (using vue-resource) but when I try and prepare the data and update it, it was breaking the page because of this problem.
On looking I can see that me assigning the data like that means that the variable I assign the data to becomes an observer, but is there no way to not do this?

This is a feature of JavaScript not related to Vue. Both variables have the same reference for the same object. Look up deep clone to solve the problem

You're setting it in the doSave method. As @pandemm said, it's about references in javascript 🙂


Add your own button and function to it

Do you think is it possible to add my own button ?

Yes, it is possible.
If you want to add your own button and implement it by Java,  just implement the interface named Button
icarus-android base on Simditor, so you cant implement buttons by Javascript too. See the  Simditor docs

Thanks for you detailed answer. It want to a a button that creates a custom
tag and sets color to a the selected text.
2016-09-10 8:13 GMT+03:00 Dyson Woo<EMAIL_ADDRESS>
Yes, it is possible.
If you want to add your own button and implement it by Java, just
implement the interface named Button
https://github.com/mr5/icarus-android/blob/master/library/src/main/java/com/github/mr5/icarus/button/Button.java
icarus-android base on Simditor, so you cant implement buttons by
Javascript too. See the Simditor docs
http://simditor.tower.im/docs/doc-usage.html
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mr5/icarus-android/issues/15#issuecomment-246089711,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AI2nVqi9qbXbZ1kAoDYPNFo75rUv54OOks5qojx5gaJpZM4J3v1K
.

It is awesome.  You can create a pull request after you finished it, if you don't mind.

The docs are not very specific and I'm a begginer at js. Sorry. I will try
another repository.
2016-09-10 8:26 GMT+03:00 Dyson Woo<EMAIL_ADDRESS>
It is awesome. You can create a pull request after you finished it, if you
don't mind.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/mr5/icarus-android/issues/15#issuecomment-246090179,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AI2nVutn5oxLNkwP2tdvH1NZQkcuRoiCks5qoj9ogaJpZM4J3v1K
.


Do not retry failed tests in CI

Tests that do not consistently pass are broken tests and should be fixed. Failed tests should not be retried to avoid accidentally adding new broken tests.
This is more of an experiment if this is achievable and WIP.

I think some of the breakage between tests happens because the paths in tests.constants are shared between all tests in a session and some tests write to these paths, influencing other tests. It might make sense to turn that module into a fixture and therefore make these paths test-specific.

All tests cleanup any remaining widgets and windows after themselves now and this is enforced through an autouse fixture. It looks like this has fixed issues with seemingly unrelated tests failing due to interfering Qt signals.
There are some heisenbugs remaining and the fact that the non-xdist pytest run hangs until the GitHub actions timeout is reached. I have a guess that this might be related to improper cleanup of multiprocessing.Process'es though.

The non-xdist test suite seems to be good now. The xdist-enabled test suite still has some flaky tests and I am not sure why yet.

the hdf failures are also strange.
We check if a port can be used and there should be nothing to get in between when we use it then. You could try if it makes a difference not checking for the serial none xdist tests.
I introduced the xdist tests to get a faster result. When we get the normal tests faster, we can set them optional in the docs.

For what it's worth, I had serial test suite running over night on my laptop and it managed to complete all tests 74 times without any failures, and it is still running. I am therefore confident that the test suite is pretty robust now, in the serial and reverse case. The CI error seems to be just another case of the GitHub runners being rather slow and running into a timeout that is a bit too tight. Specifically, it ran into the 10 second timeout of https://github.com/Open-MSS/MSS/blob/f8dad9c598bca19ea980cd5d59dcbd9665c18bfd/tests/utils.py#L188
and I think that can be increased a bit. I am also wondering if we even want these additional timeouts when we already wrap everything with pytest-timeout...

We check if a port can be used and there should be nothing to get in between when we use it then. You could try if it makes a difference not checking for the serial none xdist tests.

I am pretty sure that the port selection mechanism is correct under the assumption that no other software is trying to get one of those ports (at least for the serial test suite and with xdist's loadfile, other --dist options should have issues). Ideally I would like for it to just use port 0, let the OS pick a suitable free port and discover it programmatically, but I think that would require a few more extensive changes.

There seems to be something wrong in mslib/msui/mpl_{map,qtwidget}.py. Those tend to always turn up in the stacktrace when there is a segfault in the tests, and ultimately it seems to be coming from somewhere in basemap.

I have a suspicion that many of the remaining intermittent failures are due to the mscolab process running in a multiprocessing.Process, something which the used eventlet wsgi server simply does not support.

The serial tests passed four times in CI now and another 75 times overnight on my laptop, so it looks like that is pretty stable now (famous last words after the last time, but whatever).

There is a race condition in mslib.mscolab.file_manager.py::FileManager::save_file. If the server is running using multiple threads (e.g. through werkzeug.serving.make_server with threaded=True) then it can happen that two threads try to write to the same "main.ftml" file, corrupting the file in the process. This led to the tests in tests._test_msui.test_mscolab_version_history.py randomly failing, because the corrupted file produced a mslib.mscolab.models.Change in the database pointing to a commit sha that showed a corrupted file that read back as being empty.
This does not happen with the eventlet wsgi server since that one is running single-threaded (at least by default). Depending on how mscolab is deployed in production this issue might appear there though.

@ReimarBauer the general idea of the test suite overhaul here is finished and if you want you can review some or all of it.
Almost all tests pass reliably now, both in the serial test suite and the parallel one.
There is a segfault issue remaining that I do not understand yet and which might be the same bug as or at least related to #2107. Do you have any idea on how that could be fixed?
I'll keep this as a draft for now because I am not sure yet if I want to rewrite the commit history to make a little more sense before this is considered to be merged (and I would really like to get this segfault bug fixed as well).

Good to know. This failure should be unrelated to the skipif removals, since there was none in that file. Maybe the refactoring to remove flask_testing broke it. Can you confirm that this error does not happen on develop?

Good to know. This failure should be unrelated to the skipif removals, since there was none in that file. Maybe the refactoring to remove flask_testing broke it. Can you confirm that this error does not happen on develop?

In develop are not enough skipped, all which have that  - ValueError: cannot find context for 'fork' can't succeed on windows.
FAILED tests/_test_mscolab/test_mscolab.py::Test_Mscolab::test_handle_db_reset - PermissionError: [WinError 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess ver...
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_url_combo - PermissionError: [WinError 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess ver...
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_connect_except[ConnectionError] - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_connect_except[InvalidSchema] - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_connect_except[InvalidURL] - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_connect_except[SSLError] - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_connect_except[Exception] - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_connect_denied - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_connect_success - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_disconnect - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_login - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_login_with_different_account_shows_update_credentials_popup - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_logout_action_trigger - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_logout - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_add_user - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_add_users_without_updating_credentials_in_config_file - ValueError: cannot find context for 'fork'
ERROR tests/_test_msui/test_mscolab.py::Test_Mscolab_connect_window::test_add_users_with_updating_credentials_in_config_file - ValueError: cannot find context for 'fork'
=================== 1 failed, 3 passed, 27 skipped, 491 deselected, 4 warnings, 16 errors in 49.16s ===================

and test_handle_db runs on windows in develop into the same
    def _rmtree_unsafe(path, onerror):
        try:
            with os.scandir(path) as scandir_it:
                entries = list(scandir_it)
        except OSError:
            onerror(os.scandir, path, sys.exc_info())
            entries = []
        for entry in entries:
            fullname = entry.path
            if _rmtree_isdir(entry):
                try:
                    if entry.is_symlink():
                        # This can only happen if someone replaces
                        # a directory with a symlink after the call to
                        # os.scandir or entry.is_dir above.
                        raise OSError("Cannot call rmtree on a symbolic link")
                except OSError:
                    onerror(os.path.islink, fullname, sys.exc_info())
                    continue
                _rmtree_unsafe(fullname, onerror)
            else:
                try:
                    os.unlink(fullname)
                except OSError:
                    onerror(os.unlink, fullname, sys.exc_info())
        try:
>           os.rmdir(path)
E           PermissionError: [WinError 32] Der Prozess kann nicht auf die Datei zugreifen, da sie von einem anderen Prozess verwendet wird: 'C:/Users/LOCAL_~1.BAU/Temp/17/tmp_fquflnzmsui83b39851af/colabTestData\\filedata\\Example'

@r.bauer Since I am not actually getting closer to having the tests reliably pass without retries I am considering splitting this up into multiple PRs to get at least some improvements merged already, e.g.:

refactor mscolab and mswms servers into fixtures
refactor Qt application setup into a fixture
config reset for each test
...

What do you think?

@ReimarBauer Since I am not actually getting closer to having the tests reliably pass without retries I am considering splitting this up into multiple PRs to get at least some improvements merged already, e.g.:

refactor mscolab and mswms servers into fixtures
refactor Qt application setup into a fixture
config reset for each test
...

What do you think?

@matrss
yes, there is already a lot good done in this PR, split it.
This is a big improvement when we can merge parts of it.


Docker images appear to be missing

Docker images from the previous repo appear to be missing in: https://hub.docker.com/r/innogames/graphite-ch-optimizer

That's unfortunate. I'll try to create the images in ghcr.io asap

It looks like InnoGames doesn't have any image anymore https://hub.docker.com/u/innogames

Thank you @Felixoid

You can build your own image from #26, at least the process works good there already.


Fixes #6163

It looks like this has been resolved by #6311, @Ruud-Zuiderlicht can you reopen if this is still an issue?