Investigate using filetype other than CSV for intermediate outputs of the backend process, to potentially speed up score generation process

Is your feature request related to a problem? Please describe.
This was originally mentioned in a design document for The Big Refactor: Change filetype of intermediate outputs from csv to another filetype.
Using feather or parquet would be really simple code changes but could speed up I/O.
For instance, every ETL class loads data from an external data source, and then writes a big output file to CSV. The score generation process loads 24-ish of these big CSV files and combines them, generates a bunch of scores, then writes them to CSV. Then the tile generation process loads the big score CSV and turns it into tiles and downloads. Then the comparison tool loads the big score CSV and runs a bunch of reporting on it and writes Excel files.
Everywhere in the above sentence that says "CSV" could be replaced with feather, parquet, or something similar to dramatically speed up the time it takes to read and write the files.
Describe the solution you'd like

An investigation of whether such a change would speed up the score generation process
A proposal for the best path for moving forward

Describe alternatives you've considered

Leaving CSVs as the intermediate file format

This comment seems to indicate the speedup of some of these approaches would be substantial:

feather with "zstd" compression (for I/O speed): compared to csv, feather exporting has 20x faster exporting and about 6x times faster importing. The storage is around 32% from the original file size, which is 10% worse than parquet "gzip" and csv zipped but still decent.

Wow, this post implies a 150x speed improvement.

That’s a drastic difference — native Feather is around 150 times faster than CSV.


Cannot destructure property 'message_id' of 'message' as it is undefined.

i had a problem suck like this
 error: Uncaught TypeError: Cannot destructure property 'message_id' of 'message' as it is undefined.
    const { message_id: messageId, chat, text } = message as Message;
                        ^
    at Bot._handleUpdate (https://deno.land/x/telegram/bot.ts:100:25)
    at Array.forEach (<anonymous>)
    at Bot._handleUpdates (https://deno.land/x/telegram/bot.ts:109:13)
    at Bot._fetchUpdates (https://deno.land/x/telegram/bot.ts:118:10)
    at async Bot.launch (https://deno.land/x/telegram/bot.ts:137:5)

```
my deno version is 1.0.2

It looks like you using outdated denogram version, try to clear Deno cache

this is work thankyou


Relationship Request status 'failed' meaning; explicit rejection?

Hi all,
is there a way that the customer can explicitly reject the relationship request, or is not accepting it all they can do?
Does the 'failed' status for the request indicate that the customer rejected it?
Thanks

Document Details
⚠ Do not edit this section. It is required for learn.microsoft.com ➟ GitHub issue linking.

ID: 6d433d1a-290b-d150-b055-71c1e5f7a807
Version Independent ID: a6b5d0be-d9ac-d406-d9ab-632b12a2e17a
Content: delegatedAdminRelationshipRequest resource type - Microsoft Graph v1.0
Content Source: api-reference/v1.0/resources/delegatedadminrelationshiprequest.md
Product: partner-customer-administration
Technology: microsoft-graph
GitHub Login: @koravvams
Microsoft Alias: MSGraphDocsVteam

Hi @1nickt, currently, there is no option for customer to reject a relationship. "failed" status used if there is any internal processing error. Hope this helps. Thanks.


How To Learn In Private

title: How To Learn In Private
slug: learn-in-private
category: essay
tags: [Advice, Learning, Learn in Public]
date: 2020-01-17
author: swyx
description: Of course I don't think that everything should be public. I don't even think everyone should Learn In Public. The majority of the time you are still learning in private. Here are some thoughts on how to do it well.
This is a raw public draft - a full version is now published in The Coding Career Handbook.
I'm known for my advice on Learning in Public. It is easy to assume that I think everybody should put everything in public, or they are Doing Life Wrong.
Of course I don't think that everything should be public. I don't even think everyone should Learn In Public, as I have taken great pains to explain in every talk and podcast I do.
The more realistic phrasing of "Learn In Public" is "Learn More In Public", by which I'm merely saying most people would benefit from going a little more public than "100% Private", which is the default for the vast majority of people. In practice, this really only means going from 0% Public to maybe 1% or 5% or 10% or 30% Public, so of course the majority of the time you are still learning in private.
So here are some thoughts on how to Learn In Private well.
Improving What Goes In
People over Content: Follow people over content sources. When you come across a quality blog, podcast, or talk, notice the person behind it and follow what else they do. A lot of how content works these days is through aggregators, like Reddit or Hacker News, or an industry blog or podcast or conference, and the quality may vary quite a lot despite the best curation efforts. Titles and upvotes may be gamed for SEO. But a quality person is likely to be quality for life. You already know you are the average of the X people you spend time with - so curate those people. RSS feeds help here.
The expanded form of this is Following the Graph - not only following people, but following the people they follow (the best way to get in their heads), and tracing back work histories and tracing forward new workstreams. Most things are made by only a small set of prime movers and they are usually (by definition) not hard to find.
Read Good Books Cover To Cover: Authors spend a lot of time nailing down details in books because of how permanent it is. They also spend a lot of time organizing the structure and relative weight of the content with some educational goal in mind. If you come across a highly recommended book (or just one that starts off very well), do it the justice of going cover-to-cover. You may get bad chapters or sections, but you won't know til you plow through it. Particularly pay attention to endings, since people often don't make it to the end but some authors put The Good Stuff at the end. Usually this is the "unimportant details" that they didn't want to clutter the introduction with. This is especially relevant if you are Intermediate at whatever you're doing. Think about it: Beginners need tutorials as they give the shortest path to success, Experts just need to tinker and keep up to date, but Intermediates need context and clear explanation of details. (My Tweets about this)
Read Source Code: People often treat the open source movement as "code that I can use for free", forgetting the premise that it's also "code that you can freely fork for your needs". But we are so spoiled that we even forget that it is "code that you can read to learn from masters". As Ryan Florence has noted: "I get asked all the time 'how do I level up?' Read code. Read lots of code." It's simple, but not easy. One of Paul Irish's early hits was 10 Things I Learned from the jQuery Source (followed by 11 more things) - granted, this was a public talk, but it could easily be private learning. I curate the list of open source React Project Ideas on /r/reactjs for this reason.
Subscribe to Repos and Issues: Just like there is a metalanguage behind the language, behind the code, there is metacode - discussions and attempts on Github Repos and Issues by collaborators, all in the open. If Reading Source Code is reading code-frozen-in-time, then subscribing to repos and issues is reading code-in-motion. Since a developer's job is very evidently to put code-in-motion, learning how to do this by watching others is a great idea.
Getting More Out of What Goes In
Have a Goal: You will go further if you put more effort learning one thing than to diffuse your efforts learning random assortments of things. You even gain the superpower of having the permission to ignore things if they're not on-goal for now - you can always come back later if your goals change.
I do believe that Systems are more important than Goals. This whole thing you're reading is a proto-System. But Systems can be focused by Goals. Sometimes you can stuff a system into a goal - Ultralearning seems to have caught some fire recently as a form of super intense, immersive learning. Joe summarizes it as such: "Set a goal to experiment say with a new technology for 31 days. Check it out. See what you think. Write down what you learn."
Note: I'm not saying you have to have a goal at all times. Sometimes undirected learning is fun, and nice, and leads to serendipity.
Take Notes: A syllogism:

There's no point learning anything if you can't recall it - or rather, the more you can recall, the more you actually learned
There is a natural limit on how much you can recall at any time
So you must take notes to learn more than your brain can hold

This is currently in vogue as Building a Second Brain, but it should be no surprise that "Personal Knowledge Management" is a key life habit for knowledge workers. It's not a fad, it's something professionals have always learned to do to get good.
I have a bias toward digital notes, because search is a superpower. But search is no substitute for organization, which at its base form is a tagging system, but at its best is a "minimum spanning taxonomy" of the subject matter.
For the same amount of content stored, an organized mind will perform far superior to a disorganized one.
Spaced Repetition: Of course you can also just try to increase how much you can recall. Michael Nielsen is famous for making very bombastic claims about the capacity of Long Term Memory, but the basic principle is sound: use spaced repetition learning. Anki is the leader here. I'll be honest, I don't do this much. In a way, having a daily habit of pursuing the same topic, and making sure to take searchable notes leads you toward spaced reptition anyway - but of course, I could always do a better job of this because I often have to recall things on the spot in interviews and workshops.
Deliberate Practice: A lot has been made of Gladwell's popularization of the 10,000 hour rule, but it is undisputed that you grow the most when you practice at the edge of your abilities rather than the stuff you already know. Enough said? Enough said. However I wanted to highlight 2 specific ways to home in the exact limits of your ability:

Pick Up What They Put Down: I have called this the Ultimate Hack for Learning in Public, but of course you can also do this in private. Follow up on anything and everything your unknowing mentor says is good, or is yet to be done, or straight up just replicate their work. Which leads to...
The Ben Franklin Method: I've read this before, but was recently reminded by Mike White. The "Ben Franklin" method is shorthand for a Dissect and Reconstruct process for training yourself to notice differences in quality between your output and your input.  Ira Glass calls this The Gap. The Ben Franklin method is a feedback loop for Closing The Gap.

The Feynman technique is a similar framing of this, which is more self oriented rather than focused on replicating techniques of others.

Go Meta
Active over Reactive: Active Learning is Pull-based - You try to do something, find gaps, and go seek out what you need to get it done. Most learning starts out Reactive - you see something new, you learn it. Push-based. The problem is the content industry has swung waaaay too far towards drowning you under a deluge of content. Additionally, Reactive Learning isn't serving your goals - it serves someone else's. Though part of learning is learning what to learn, and Serendipity is a wonderful thing, Just-In-Case learning isn't very productive. The right mix of learning will probably include more Active Learning, Just-in-Time, than is natural or feels comfortable.
I confess I don't do this well at all and would welcome ideas for how to make active learning more of a habit.
Compare and Contrast Multiple Perspectives: I find a shocking number of devs cannot objectively hold competing ideas in their head at the same time. They just loudly state their point of view over and over and over again and attack the weakest argument of their opponents. This is unacceptable to me. The 5th Habit of the 7 Habits is Seek First to Understand, then To Be Understood. You are more effective if you can summarize the best arguments of all major parties in a way that THEY agree with. This is an unjudgmental "schools of thought" based mode of learning I wish more people adopted.
As a bonus, once you can do this well, you also become more persuasive to people who disagree with you. Because you took the time to understand them. This is sometimes called Steel-manning (more from Andrew Chen), as opposed to Straw-man arguments.
Distinguish Game vs Metagame: Much of learning is focused on how to do better at the Game you're playing, whatever the game is. But the best players recognize that the rules of the game are changing even while it is being played. If they are smart, or passionate, they are also actively involved in changing the rules of the game while they are playing it. If you don't want to keep "fighting the last war", and arriving at the top of your game only to have the game change on you, you must also pay attention to how the game is changing while you play it. Often, this takes the form of realizing that there's a bigger game out there than the one you once thought your whole world.

See Also: Big L Notation (expanding on why "Learn In Public" is the fastest way to learn)

Further References

Sarah Drasner: Learning to Learn

you have some very valid points. while reading I stopped several times to think about them. Excellent article!


Translations update from Hosted Weblate

Translations update from Hosted Weblate for Fortran programming language/Fortran webpage.
Current translation status:

@awvwgk
thank you for merge!
I found several broken URL-links in the original English index.po file and left comments about it on webplate appropriate strings. Could you please fix it?

broken cmake url : "," and "\" are required to be dropped.

not visible allocatable arrays url: ".html" extension should be removed from url.

@band-a-prend Thanks will fix it.


feat: Add validation for ignores_time_zone field in FieldContent struct

Title
Add ignores_time_zone field to FieldContent struct with validation
Type of Change

[x] New feature

Description
This pull request introduces the ignores_time_zone field to the FieldContent struct. The ignores_time_zone field is a boolean value that determines whether the date and time displayed in the field should respect the current device's time zone (false) or the time zone associated with the provided date value (true). The default value is false if not specified.

The field adds flexibility in controlling how time zones are handled in field displays.
A validation function ensures that only valid boolean values are accepted.

Testing

Unit tests were added to validate the behavior of the ignores_time_zone field.

Verified that the field accepts valid boolean values (true, false).
Tested the default behavior when the field is not provided (defaults to false).
Ensured that invalid inputs raise appropriate errors.

Impact

The FieldContent struct now includes the ignores_time_zone field, which could impact parts of the codebase that rely on date and time handling.
This change provides developers with more control over how date/time is displayed based on time zone behavior.

Additional Information
None.
Checklist

[x] My code adheres to the coding and style guidelines of the project.
[x] I have performed a self-review of my own code.
[x] I have commented my code, particularly in hard-to-understand areas.
[x] I have made corresponding changes to the documentation.
[x] My changes generate no new warnings.

closed: #150


Several issues

Dear gin66
Thank you for your great ideas in this library that takes care to off load the cpu, when running more steppers.
I have integrated v. 0.22 of the library into our real time controllers for our stepper controlled devices at DTU (Danish Technical University) and have found a couple of problems. I have wrapped your interface into what we are using (with 3 steppers) and have performed some tests of the library.
In short are the found problems:

The current speed is wrong when the stepper is moving
The current speed is wrong when the stepper is stopped (Used by runForward() and keepRunning())
The current position is wrong when the stepper is moving
The current acceleration is wrong when the stepper is moving
The sign of the current speed is always positive no matter what direction the motor is running
The sign of the current acceleration is always positive no matter what direction the motor is running
The current acceleration > 0 when the motor is not running

All problems are documented in the attached file (WrappedExample_210228.ino). I have used v. 0.22 of the library. I have chosen not to separate the problems into different issues before you have looked into them.
I know that some of the issues (problem1 and 3) are documented to be known, but they are essential to function correctly, if the library shall be used for real applications.
Hope you will look into the tests/problems, so we can get a superior and full functioning stepper library out of it. Please do not hesitate to contact me, in case I can help.
Best regards
Soren
WrappedExample_210228.ino.txt

Result is from execution on my hw:
rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:2
load:0x3fff0018,len:4
load:0x3fff001c,len:1044
load:0x40078000,len:10124
load:0x40080400,len:5828
entry 0x400806a8
Starting ...
Acceleration test
    Error - Wrong value - A1: 40.00  80.00  120.00
  A2: 40.00  80.00  120.00
  A3: 40.00  80.00  120.00
  Error - Should be 0 - A4: 40.00  80.00  120.00
Speed test
  Error - Vrong values - S1: 39.27  81.81  122.72
  Error - Should be 0 - S2: 39.27  81.81  122.72
Position test
  P1: -85.78  -140.06  482.75
  P2: -25.13  25.13  33.24
  P3: 25.13  -25.13  -25.13
  P4: -25.13  25.13  25.13

BTW: Thanks for the detailed report and test program. I have added some more information, so I better understand what is going on:
Starting ...
>>> Set speed of stepper 1 to speed in rad/s=50.00, steps/s=50929
>>> Set acceleration of stepper 1 to steps/s^2=50930
>>> Set speed of stepper 2 to speed in rad/s=50.00, steps/s=50929
>>> Set acceleration of stepper 2 to steps/s^2=50930
>>> Set speed of stepper 3 to speed in rad/s=50.00, steps/s=50929
>>> Set acceleration of stepper 3 to steps/s^2=50930
Acceleration test
>>> Set speed of stepper 1 to speed in rad/s=120.00, steps/s=122230
>>> Set speed of stepper 2 to speed in rad/s=120.00, steps/s=122230
>>> Set speed of stepper 3 to speed in rad/s=120.00, steps/s=122230
>>> Set moveByAcceleration of stepper 1 to steps/s^2=40744
>>> Set moveByAcceleration of stepper 2 to steps/s^2=-81487
>>> Set moveByAcceleration of stepper 3 to steps/s^2=122231
    Error - Wrong value - A1: printAcceleration: 40.00  80.00  120.00
  A2: printAcceleration: 40.00  80.00  120.00
>>> Set moveByAcceleration of stepper 1 to steps/s^2=-40744
>>> Set moveByAcceleration of stepper 2 to steps/s^2=81487
>>> Set moveByAcceleration of stepper 3 to steps/s^2=-122231
  A3: printAcceleration: 40.00  80.00  120.00
>>> StopMove for stepper 1
>>> StopMove for stepper 2
>>> StopMove for stepper 3
  Error - Should be 0 - A4: printAcceleration: 40.00  80.00  120.00

Speed test
>>> Set speed of stepper 1 to speed in rad/s=40.00, steps/s=40743
>>> Set speed of stepper 2 to speed in rad/s=80.00, steps/s=81487
>>> Set speed of stepper 3 to speed in rad/s=120.00, steps/s=122230
  Error - Vrong values - S1:
printSpeed: 40.91  81.81  122.72
>>> StopMove for stepper 1
>>> StopMove for stepper 2
>>> StopMove for stepper 3
  Error - Should be 0 - S2:
printSpeed: 40.91  81.81  122.72

Position test
>>> Set position of stepper 1 to 25600
>>> Set position of stepper 2 to -25600
>>> Set position of stepper 3 to -25600
  P1: printPosition: >>> Stepper 1 is still running
-72.93  >>> Stepper 2 is still running
-131.32  >>> Stepper 3 is still running
474.33
Expected 8*PI=25.13

>>> Set position of stepper 1 to -25600
>>> Set position of stepper 2 to 25600
>>> Set position of stepper 3 to 25600
  P2: printPosition: -25.13  25.13  >>> Stepper 3 is still running
31.58
Expected 8*PI=-25.13 25.13 25.13

>>> Set position of stepper 1 to 25600
[WrappedExample.ino.txt](https://github.com/gin66/FastAccelStepper/files/6057913/WrappedExample.ino.txt)

>>> Set position of stepper 2 to -25600
>>> Set position of stepper 3 to -25600
  P3: printPosition: 25.13  -25.13  -25.13
Expected 8*PI=25.13 -25.13 -25.13

>>> Set position of stepper 1 to -25600
>>> Set position of stepper 2 to 25600
>>> Set position of stepper 3 to 25600
  P4: printPosition: -25.13  25.13  25.13
Expected 8*PI=-25.13 25.13 25.13

Based on the above, here are my comments to your findings:

The current speed is wrong when the stepper is moving

Relevant part from the log:
Speed test
>>> Set speed of stepper 1 to speed in rad/s=40.00, steps/s=40743
>>> Set speed of stepper 2 to speed in rad/s=80.00, steps/s=81487
>>> Set speed of stepper 3 to speed in rad/s=120.00, steps/s=122230
  Error - Vrong values - S1:
printSpeed: 40.91  81.81  122.72

Here the issue is the resolution from integer calculation in unit [us]. 40473 steps/s is 24.708 us. Retrieving the speed in us yields 24. Calculating back in rad/s gives 40.91. Similar the other values. Internally the esp32 uses 16 MHz and using setSpeedInHz() makes use of this higher resolution. The corresponding function is getSpeedInTicks().

The current speed is wrong when the stepper is stopped (Used by runForward() and keepRunning())

>>> StopMove for stepper 1
>>> StopMove for stepper 2
>>> StopMove for stepper 3
  Error - Should be 0 - S2:
printSpeed: 40.91  81.81  122.72

The maximum speed is still valid. I assume, you need a function similar to int32_t getActualSpeed()
BTW: keepRunning() is not needed, because runForward() and runBackward() implies keepRunning().

The current position is wrong when the stepper is moving

Here I do not get, what the problem is..

The current acceleration is wrong when the stepper is moving

The relevant log is:
Acceleration test
>>> Set speed of stepper 1 to speed in rad/s=120.00, steps/s=122230
>>> Set speed of stepper 2 to speed in rad/s=120.00, steps/s=122230
>>> Set speed of stepper 3 to speed in rad/s=120.00, steps/s=122230
>>> Set moveByAcceleration of stepper 1 to steps/s^2=40744
>>> Set moveByAcceleration of stepper 2 to steps/s^2=-81487
>>> Set moveByAcceleration of stepper 3 to steps/s^2=122231
    Error - Wrong value - A1: printAcceleration: 40.00  80.00  120.00
  A2: printAcceleration: 40.00  80.00  120.00
>>> Set moveByAcceleration of stepper 1 to steps/s^2=-40744
>>> Set moveByAcceleration of stepper 2 to steps/s^2=81487
>>> Set moveByAcceleration of stepper 3 to steps/s^2=-122231
  A3: printAcceleration: 40.00  80.00  120.00

Except the sign, the values are equal.

The sign of the current speed is always positive no matter what direction the motor is running

Yes. The speed in context of the ramp generator is the maximum speed in both directions. getSpeedInUs() does not retrieve the current speed but this maximum speed. As this is applicable for both directions, the speed is a scalar and not a vector. What you need is getActualSpeed() or similar.

The sign of the current acceleration is always positive no matter what direction the motor is running

True. The system is not intended to provide the "current" acceleration. It uses just one acceleration as scalar in both directions. The function moveByAcceleration() translates the sign into corresponding target direction and uses the absolute value as scalar for the acceleration. If the motor is coasting at maximum speed, then the actual
acceleration is 0. Reading back the acceleration value just retrieves the scalar acceleration.

The current acceleration > 0 when the motor is not running

Yes, as said before.
My takeaway is:

Needed feature is the possibility to retrieve the current direction of the stepper e.g. getCurrentDirection()with results: +1, 0, -1
Needed feature is the possibility to retrieve the current direction of the acceleration e.g. getHeadingTo() with +1, 0, -1

Separation in two values, because the stepper can run forward (+1), but is decelerating (-1).
Any better alternatives ?

Thank you for your fast feed back, and I'm sorry for my late reply (Hmmm ... too busy these days).
I guess that the problem is, that we are using the stepper driver for real time control. That is why it is necessary to be able to read the actual position, actual speed and actual acceleration at any time and get correct values for them. We have not seen this problem in other drivers, as they always return actual values for position, speed and acceleration.
I am not sure that your suggested current direction and heading are necessary, as these shall be included as the sign of the actual values for position, speed and acceleration.
Re. "The current speed is wrong when the stepper is moving": My mistake. I was not aware of the truncations, so you are absolutely right on this.
Will it be possible to add functions for getting these actual values for position, speed and acceleration correctly from the driver ?

For speed I have an idea, how to improve this and perhaps even get rid of getCurrentSpeedAtQueueEnd(). For acceleration adding the sign and even return zero while coasting, should be possible, too. For current position it is a bit more complicated, but for esp32 there is another possibility to make use of available pulse counters. With three motors, there are actually five pulse counters free.

It sounds great.
We are really interested in using your driver principle, as the drivers we use pt. (AccelStepper and FlexyStepper) takes one of the CPU kernels on an ESP32 just to send the pulses to the steppers, where yours practically will free up half of the CPU resources.
I am looking forward to your findings and will intensively perform practical tests of them when they are ready.

Please try the latest, not yet official release. This adds the functions: getCurrentSpeed()  and getCurrentAcceleration().
Any issues, please report accordingly. Just finished the implementation and have not done any test for these two functions.
In addition the getCurrentPosition() is improved on esp32 to take the current ongoing command's issued steps in consideration. From my (limited) test, I see at max one count off from a second attached pulse counter.

I have made a few tests and found one problem in the getCurrentSpeedInUs().
Please see the attached program, which gives the following output:
Starting ...
Speed test0
S: inf  inf  inf
S: inf  inf  inf
S: inf  inf  inf
S: 0.54  -0.54  0.54
S: 0.54  -0.54  0.54
S: 0.54  -0.54  0.54
S: 0.54  -0.54  0.54
S: 0.63  -0.63  0.63
S: 0.70  -0.70  0.70
It seems as the first 3 speed readings gives getCurrentSpeedInUs() = 0.
WrappedExample_210304.ino.txt

getCurrentSpeedInUs() returns 0 for stopped motor. This special case needs to be taken care for in 'readCurrentSpeedInRadianPerSecond()'.
Why it takes three loops until the speed is not zero, I need to check.

Actually the implementation of getCurrentSpeedInUs() is broken due to a mistake in a condition. Good proof, that every feature should be tested....
This evening can upload a fixed version.

Yes, right :-).
Agree, it will be excellent to have getCurrentSpeedInHz() and in MilliHz.
I will update the wrapper now and make some more tests this afternoon.
Looking forward to your update.

Done. Please check on your side. There is only getCurrentSpeedInMilliHz() for now

Thank's. I have included getCurrentSpeedInMilliHz() and used setSpeedInHz() in the attached file.
Output:
Starting ...
Set speed in rad/s (Hz): 40.00
Set speed in rad/s (Hz): -80.00
Set speed in rad/s (Hz): 120.00
Speed test after 0 s:
Current speed in rad/s (Hz): 0.00
Current speed in rad/s (Hz): -1.25
Current speed in rad/s (Hz): 1.63
Speed test after 3 s:
Current speed in rad/s (Hz): 40.07
Current speed in rad/s (Hz): -80.14
Current speed in rad/s (Hz): 120.83
It looks much better now, though there still are significant truncation errors.
WrappedExample_210305.ino.txt
Will continue with more tests.

the example still does this division based in us instead of using getCurrentSpeedInMilliHz():
void readCurrentSpeedInRadianPerSecond(int stepper, double & currentSpeedInRadPerSecond) {
  switch (stepper) {
    case 1: { currentSpeedInRadPerSecond = 1000000.0 / (stepper1->getCurrentSpeedInUs() * stepsPerRad); }; break;
    case 2: { currentSpeedInRadPerSecond = 1000000.0 / (stepper2->getCurrentSpeedInUs() * stepsPerRad); }; break;
    case 3: { currentSpeedInRadPerSecond = 1000000.0 / (stepper3->getCurrentSpeedInUs() * stepsPerRad); }; break;
  }
}

I have attached a test file.
Problem1: The current speed is changed though the acceleration is 0.
Notice the following lines:
A: 40.0003  0.0000  0.0000    S: 119.9081  -120.8305  120.8305    P: 177.6512  -268.2704  298.4248
A: 0.0000  0.0000  0.0000    S: 119.9081  -120.8305  120.8305    P: 178.7448  -269.3719  299.5263
A: 0.0000  0.0000  0.0000    S: 119.9081  -120.8305  120.8305    P: 179.8375  -270.4735  300.6278
A: 0.0000  0.0000  0.0000    S: 120.8305  -120.8305  120.8305    P: 180.9381  -271.5740  301.7284
The speed of stepper1 is changing though the acceleration of stepper1 is 0.
Problem 2: The current position of the stepper is moving though the current speed is 0.
Notice the output lines:
A: -40.0003  0.0000  0.0000    S: 1.0142  0.0000  0.0000    P: 436.3829  -436.5636  437.0937
A: 0.0000  0.0000  0.0000    S: 0.0000  0.0000  0.0000    P: 436.3898  -436.5636  437.0937
A: 0.0000  0.0000  0.0000    S: 0.0000  0.0000  0.0000    P: 436.3937  -436.5636  437.0937
A: 0.0000  0.0000  0.0000    S: 0.0000  0.0000  0.0000    P: 436.3947  -436.5636  437.0937
The position of stepper1 is changing though the speed of stepper1 is 0.
WrappedExample_210305b.ino.txt

Yes, the current acceleration is based on the ramp state, which is related to queue end.
The speed is for speeds <4096 us the current command's speed and for slower speeds, actually from queue end.
The difference between queue end and current command is just a couple of ms.
Any other issue identified ?

I expect that it is ok to have a time deviation of a few ms on reading the current values as long as the sampling time in a control loops is bigger than 10 ms.
I have to verify this on a practical system.

I can see a significant deviation in setting the max speed, please see the attached wrapper + test output.
WrappedExample_210306b.ino.txt
Test output:
Starting ...
Test1:
Set max speed in rad/s (Hz): 120.00  Set max speed in (INT) mHz): 122230996  Read max speed in rad/s (Hz): 122.72
Test2:
Set max speed in rad/s (Hz): 120.00  Set max speed in (INT) Hz): 122231  Read max speed in rad/s (Hz): 120.83
Test3:
Set max speed in rad/s (Hz): 180.00  Set max speed in (INT) mHz): 183346494  Read max speed in rad/s (Hz): 187.00
Set max speed in rad/s (Hz): 180.00  Set max speed in (INT) Hz): 183346  Read max speed in rad/s (Hz): 180.55
Test4:
Set max speed in rad/s (Hz): 200.00  Set max speed in (INT) mHz): 203718327  Read max speed in rad/s (Hz): 180.55
Set max speed in rad/s (Hz): 200.00  Set max speed in (INT) Hz): 203718  Read max speed in rad/s (Hz): 180.55
There is significant deviation between the set max speed and the read max speed. When setting the max speed to 200 rad/s is the read max speed 180.55 rad/s.
The read max speed is the one reached when reaching a target speed and reading it with readCurrentSpeed.

Thanks for highlighting this. The internal calculation is 250*16000000/speed_in_milli_hz*4, because 16,000,000,000 does not fit into a 32bit. I have changed the calculation by use of another 32bit multiplication and division to improve the result. In addition, the result is rounded, too.
Please check the changed version. Should be fine now within the limits of 0.0625us granularity of the time base.

Running the same test again now gives the following output:
Starting ...
Test1:
Set max speed in rad/s (Hz): 120.00  Set max speed in (INT) mHz): 122230996  Read max speed in rad/s (Hz): 119.91
Test2:
Set max speed in rad/s (Hz): 120.00  Set max speed in (INT) Hz): 122231  Read max speed in rad/s (Hz): 119.91
Test3:
Set max speed in rad/s (Hz): 180.00  Set max speed in (INT) mHz): 183346494  Read max speed in rad/s (Hz): 180.55
Set max speed in rad/s (Hz): 180.00  Set max speed in (INT) Hz): 183346  Read max speed in rad/s (Hz): 180.55
Test4:
Set max speed in rad/s (Hz): 200.00  Set max speed in (INT) mHz): 203718327  Read max speed in rad/s (Hz): 180.55
Set max speed in rad/s (Hz): 200.00  Set max speed in (INT) Hz): 203718  Read max speed in rad/s (Hz): 180.55
There are some things to notice here:
The speed setting of 120 rad/s has become significant better and has now a 0.1% error.
The speed setting of 180 rad/s is unchanged and still has a -0.3% error.
The speed setting of 200 rad/s is unchanged and has a 10.8% error, which is unusable. It seems as there still are something wrong in the calculations of the speed for higher speeds.
The 0.1% - 0.3% error is acceptable but still quite large.
We use the faster speeds (i.e. 200 - 300 rad/s) when using the stepper driver with an O-drive driving an AC-motor with tachometer, so these speeds should be working correctly.
I have executed the program with low speeds with the following output:
Starting ...
Test1:
Set max speed in rad/s (Hz): 0.01  Set max speed in (INT) mHz): 10186  Read max speed in rad/s (Hz): 0.010000
Test2:
Set max speed in rad/s (Hz): 0.01  Set max speed in (INT) Hz): 10  Read max speed in rad/s (Hz): 0.009817
Test3:
Set max speed in rad/s (Hz): 0.20  Set max speed in (INT) mHz): 203718  Read max speed in rad/s (Hz): 0.200000
Set max speed in rad/s (Hz): 0.20  Set max speed in (INT) Hz): 204  Read max speed in rad/s (Hz): 0.200278
Test4:
Set max speed in rad/s (Hz): 1.00  Set max speed in (INT) mHz): 1018592  Read max speed in rad/s (Hz): 0.999997
Set max speed in rad/s (Hz): 1.00  Set max speed in (INT) Hz): 1019  Read max speed in rad/s (Hz): 1.000379
This seems to be ok for all speeds set with setSpeedInMilliHz().
Test2 using setSpeedInHz() has a deviation of approx. 2%, so it seems as there are more deviation when using setSpeedInHz().
I cannot see any gains from using setSpeedInHz() relative to setSpeedInMilliHz(), so I will suggest to remove it from the interface and always use setSpeedInMilliHz() for speed setting.

Actually test4 gives the same result at test3, which indicates, that setting of the new speed value has not been executed.
200 rad/s translates to 203718 Steps/s, which is above the library limit of 200kSteps/s.
Please check the return value of setSpeedInHz() and setSpeedInMilliHz(). For 200rad/s it will indicate an error.
The 0.1%-0.3% error is surprisingly good. The tick value are with 120-200 rad/s around ~100. The tick has a granularity of 1. So the error can be expected with proper rounding up to 0.625% at fastest tick value of 80 aka 200 kHz.

Sure, I will check the return value from the setSpeed... (). And I see your point.
Yes, right, there must be some errors when the speed setting is high.
Will continue ...

Not sure, how complicated it would be for implementation, but there could be a scheme to go for an average speed. This means, toggling between two slightly different speeds (differ by one tick, aka approx. 1%) with the average speed being the desired one. The change between those two speeds would be around 1ms.
Not sure, if it is worth the effort and if the constant speed switch is acceptable or not.

We can read the actual set speed so we do know what it is.
Getting the "constant" speed exact in average is more a matter of principle.
I do not have any use cases where it is necessary, so I do not think it is worth the effort.
I will include the driver in one of our systems, and verify it on a real model. It will take some time to do this. I will close this issue and open a new if I find more problems.


user-data capability

Hi,
I downloaded v.2.4.0 and tested it on a standalone Ubuntu and I have a question regarding the user-data parameters.
As "--user-data-file" works just fine with a local YAML file I couldn't figure out how the "--user-data-raw" works. In the documentation I found the description that it expects the user-data in a single line but I couldn't find a way how to do that. Would be great to get a hint how that works.
Best regards
Hendrik

Hi @agepi477, --user-data-raw is originally designed to be used with the Rancher, allowing to pass contents of the user data file, not a path to the file. We are thinking about merging those two flags into a single one in the future (#65). I hope, this answers you question :slightly_smiling_face:

Hi,
thanks for your quick reply. That’s fine. Would be good to have something similar to the ECS creation wizard on the OTC console. Can you make an estimation what „future“ means?
Best regards
Hendrik

Am 17.03.2021 um 16:33 schrieb Anton Kachurin @.***>:

Hi @agepi477, --user-data-raw is originally designed to be used with the Rancher, allowing to pass contents of the user data file, not a path to the file. We are thinking about merging those two flags into a single one in the future (#65).
I hope, this answers your question 🙂
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or unsubscribe.

What exactly do you mean by "something similar to the ECS creation wizard"?

When you deploy a ECS via OTC Web Console you can inject user data using this dialogue:
[ https://docs.otc.t-systems.com/usermanual/ecs/en-us_topic_0032380449.html | https://docs.otc.t-systems.com/usermanual/ecs/en-us_topic_0032380449.html ]
I think it would be grat to have something similar in the rancher node template for the OTC.
In the vSphere node driver this is already implemented like that:
Cheers, Hendrik
Von: "Anton Kachurin" @.>
An: "opentelekomcloud/docker-machine-opentelekomcloud" @.>
CC: "Hendrik Praedicow" @.>, "Mention" @.>
Gesendet: Donnerstag, 18. März 2021 08:34:48
Betreff: Re: [opentelekomcloud/docker-machine-opentelekomcloud] user-data capability (#69)
What exactly do you mean by "something similar to the ECS creation wizard"?
—
You are receiving this because you were mentioned.
Reply to this email directly, [ https://github.com/opentelekomcloud/docker-machine-opentelekomcloud/issues/69#issuecomment-801700622 | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AMLCSTZIG65XF5BQ4V3PGJTTEGUJRANCNFSM4ZKZOWPA | unsubscribe ] .

@agepi477 We already planned to implement this in the OTC node driver UI :slightly_smiling_face:

This is great to hear. Thank you very much for your time.
Von: "Anton Kachurin" @.>
An: "opentelekomcloud" @.>
CC: "Hendrik Praedicow" @.>, "Mention" @.>
Gesendet: Donnerstag, 18. März 2021 12:59:47
Betreff: Re: [opentelekomcloud/docker-machine-opentelekomcloud] user-data capability (#69)
[ https://github.com/agepi477 | @agepi477 ] We already planned to implement this in the OTC node driver UI 🙂
—
You are receiving this because you were mentioned.
Reply to this email directly, [ https://github.com/opentelekomcloud/docker-machine-opentelekomcloud/issues/69#issuecomment-801869153 | view it on GitHub ] , or [ https://github.com/notifications/unsubscribe-auth/AMLCST6V4ECDTTHB7ELQXRTTEHTLHANCNFSM4ZKZOWPA | unsubscribe ] .


Add Pull verb

From the discussion on Build and Deploy, it seems we should have a Pull verb.

cc @joeyaiello for description

I assume Get is too generic then, just curious :-)

So what is the opposite of Push then? Pop or Pull?

Is this not simply Get? 😄

Not totally remembering the justification for this. Going to close it given there's no pressing need right now.


memory leak due to EditableInputConnection

Describe the bug
memory leak due to EditableInputConnection
Screenshots

Would you like to work on the issue?
YES

@iamareebjamal let me know what do you plan to do with this issue

I dont think this is solvable. Its an internal Android framework bug. Please see this thread https://issuetracker.google.com/issues/37043700
There are workarounds though. Please refer to https://issuetracker.google.com/issues/37043700#comment17.
Should I implement it?
@adityastic @iamareebjamal @mariobehling

I dont think this is solvable. Its an internal Android framework bug. Please see this thread https://issuetracker.google.com/issues/37043700
There are workarounds though. Please refer to https://issuetracker.google.com/issues/37043700#comment17.
Should I implement it?
@adityastic @iamareebjamal @mariobehling

I dont think this is solvable. Its an internal Android framework bug. Please see this thread https://issuetracker.google.com/issues/37043700
There are workarounds though. Please refer to https://issuetracker.google.com/issues/37043700#comment17.
Should I implement it?
@adityastic @iamareebjamal @mariobehling

We can leave it as is, let's leave it to google to fix. Didn't cause any spike in leaks as well. Let me close it

I dont think this is solvable. Its an internal Android framework bug. Please see this thread https://issuetracker.google.com/issues/37043700
There are workarounds though. Please refer to https://issuetracker.google.com/issues/37043700#comment17.
Should I implement it?
@adityastic @iamareebjamal @mariobehling

We can leave it as is, let's leave it to google to fix. Didn't cause any spike in leaks as well. Let me close it


[ja] Translate concepts/security/controlling-access/ into Japanese

This is a Feature Request
Translate concepts/security/controlling-access/ into Japanese.

What would you like to be added

Why is this needed
This page has not been translated.
Comments

/language ja
/assign

/triage accepted


Add a page embedding the Wiki provides by Eclipse

My proposal is to consolidate the documentation in the Kuksa website (currently prepared in staging branch) and thus to only link to that documentation from the Eclipse Wiki. As a consequence, we would not embed the wiki into the website and would close this issue. What do the other think?

Sounds good! Let's include the link in the wiki as soon as staging is merged.

Obsolete


Ch.2 L2: Scripting component

In Chapter 2 users will need to write some code that they can then run. This particular challenge will have the user implement a hashing function that finds a hash starting with n amount of zeros to teach them the concepts of proof of work.
To facilitate this, the engineering team needs to implement a component that allows users to write scripts.
Some research is currently going into:

What programming languages we want to support.
Whether the solution should be client- or server-side.
Whether the solution will support full blown REPL capabilities or whether its a simulation.
What frontend libraries could prove to be useful (i.e. monaco).
What backend infra is needed to support this feature if we decide to go for full-blown REPL.
etc.

This issue is blocked until these matters have been resolved.

Here are my two cents on this:

What programming languages we want to support.

Supporting Python and Javascript should suffice for our case. They both are versatile, commonly used, and easy to implement and understand languages.

Whether the solution should be client- or server-side.

Server-side rendering would be the ideal solution because every user will have access to equal resources and processing power. However, this approach will increase the load on the server. So client-side rendering should be a more practical approach.

Whether the solution will support full blown REPL capabilities or whether its a simulation.

Though doing simulation is the favored solution till now, Going with a functional IDE will be a better solution, IMO. Because:
There can be multiple ways to solve a problem. So writing test cases for all of them will not be practical.
If the user takes an approach, which is correct, but that we didn’t predict could be used would be deemed wrong. This scenario might lead to breaking immersion for the user.
With a full-blown IDE, we can encourage users to play around with the code and do experimentation with it. This will not be possible with the simulated environment.

What frontend libraries could prove to be useful (i.e. monaco).

I researched this point, and ace-editor is a favorable solution for implementing the IDE. (GitHub link of the package)

What backend infra is needed to support this feature if we decide to go for full-blown REPL.

With using ace-editor, the backend infrastructural changes shall be minimum. However, I haven’t researched this point thoroughly and there might be some edge cases that can require some further infra changes.

Hi @shaavan, I actually already implemented a similar approach using monaco editor (same thing thats running VSCode), you can take a look at a preview here. It seems to work fine but if we run into any issues with it we can give ace-editor a shot.

@GBKS Here's a screenshot of Kevin's spike on supporting multiple dev languages. Wanted to flag it for you wrt to design.

Closed in https://github.com/saving-satoshi/saving-satoshi/pull/327


Android Support Library > v23.1.1

In my app, Calligraphy works fine when using support library v23.1.1. However when I upgrade to the latest version (v23.3.0), the title font of newly launched AppCompatActivity's is no longer set to a custom font. Note: the title of my MainActivity (launcher, which uses a CoordinatorLayout) is displayed correctly though and all other fonts are set to my custom font as well. I'm building with Calligraphy v2.2.0 and I initialise each activity correctly by calling:
super.attachBaseContext(CalligraphyContextWrapper.wrap(newBase));

Any thoughts?

Thats a little vague, can you post your xml layout?

Same problem here with support library version 23.2.1

My MainActivity uses a CoordinaterLayout (and works perfectly fine):
`

`
The other activities extend AppCompatActivity and have the following layout:
`

        < ... />

`

I'm actually seeing this too. I'm using the 23.3.0 support libs and Calligraphy 2.2.0

OK I'll take a look this evening.
On Fri, 15 Apr 2016, 21:54 Bryan Stern<EMAIL_ADDRESS>wrote:

I'm actually seeing this too. I'm using the 23.3.0 support libs and
Calligraphy 2.2.0
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/chrisjenx/Calligraphy/issues/268#issuecomment-210642866

I'm stepping through this now with a debugger, but I'm unfamiliar with the source, so I'm not sure if this is helpful. It appears that CalligraphyFactory#onViewCreated is never passed the action bar.

You're going to have to put together a sample as I can't reproduce this.

Oh do you mean the auto inflated ActionBar doesn't work from the AppCompat Activity?

Yes, sorry it wasn't clear
On Fri, Apr 15, 2016, 6:59 PM Christopher Jenkins<EMAIL_ADDRESS>wrote:

Oh do you mean the auto inflated ActionBar doesn't work from the AppCompat
Activity?
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/chrisjenx/Calligraphy/issues/268#issuecomment-210675736

Hi Chris, i seem to have the same problem. I'm currently on 'com.android.support:appcompat-v7:23.3.0' and
@Override
protected void attachBaseContext(Context newBase) {
super.attachBaseContext(CalligraphyContextWrapper.wrap(newBase));
}
works for lower version of the support library just fine. I'm using calligraphy:2.2.0

I haven't supported the action bar for ages. Not sure why your seeing this
now.
Use the toolbar. My guess is that they changed the underlying
implementation or its another thing they don't inflate.
On Sat, 16 Apr 2016, 11:13 sebichondo<EMAIL_ADDRESS>wrote:

Hi Chris, i seem to have the same problem. I'm currently on
'com.android.support:appcompat-v7:23.3.0' and
@Override https://github.com/Override
protected void attachBaseContext(Context newBase) {
super.attachBaseContext(CalligraphyContextWrapper.wrap(newBase));
}
works for lower version of the support library just fine. I'm using
calligraphy:2.2.0
—
You are receiving this because you commented.
Reply to this email directly or view it on GitHub
https://github.com/chrisjenx/Calligraphy/issues/268#issuecomment-210785132

Thanks Chris, decided to go with the  option for now as i try to figure out what exactly changed

Is there anyone fixed this error?

@gilbertparreno don't use ActionBar's, I'm not going to support them.


chore: wrap relationship fields in ArrayField

Description of changes:

Wrap relationship fields in ArrayField
Init snapshot testing for HAS_ONE relationship

Delimitations:

Since Autocomplete doesn't exist yet, this will break E2E.
Tasks to map props to the Autocomplete and ArrayField outstanding.

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Codecov Report

Merging #719 (2930f90) into tagged-release/cms-support (1196013) will decrease coverage by 0.00%.
The diff coverage is 82.35%.

@@                      Coverage Diff                       @@
##           tagged-release/cms-support     #719      +/-   ##
==============================================================
- Coverage                       91.54%   91.54%   -0.01%
==============================================================
  Files                             101      101
  Lines                            3952     3963      +11
  Branches                         1106     1111       +5
==============================================================
+ Hits                             3618     3628      +10
  Misses                            312      312
- Partials                           22       23       +1

Impacted Files
Coverage Δ

.../codegen-ui-react/lib/forms/react-form-renderer.ts
80.23% <50.00%> (-0.75%)
:arrow_down:

.../lib/amplify-ui-renderers/amplify-form-renderer.ts
40.86% <100.00%> (+1.04%)
:arrow_up:

...codegen-ui-react/lib/forms/form-renderer-helper.ts
98.21% <100.00%> (+0.01%)
:arrow_up:

packages/codegen-ui-react/lib/primitive.ts
100.00% <100.00%> (ø)

...s/codegen-ui-react/lib/react-component-renderer.ts
100.00% <100.00%> (ø)

packages/codegen-ui-react/lib/forms/form-state.ts
99.15% <0.00%> (+0.84%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1196013...2930f90. Read the comment docs.


docs: Add Podman support alongside Docker in installation instructions

Pull Request Checklist
Before submitting, make sure you've checked the following:

[x] Target branch: Please verify that the pull request targets the dev branch.
[x] Description: Provide a concise description of the changes made in this pull request.
[x] Changelog: Ensure a changelog entry following the format of Keep a Changelog is added at the bottom of the PR description.
[x] Documentation: Have you updated relevant documentation Open WebUI Docs, or other documentation sources?
[x] Dependencies: Are there any new dependencies? Have you updated the dependency versions in the documentation?
[x] Testing: Have you written and run sufficient tests for validating the changes?
[x] Code review: Have you performed a self-review of your code, addressing any coding standard issues and ensuring adherence to the project's coding standards?
[x] Prefix: To clearly categorize this pull request, prefix the pull request title, using one of the following:

docs: Documentation update or addition

Changelog Entry
Description

Added Podman support to all Docker commands in the Quick Start and installation instructions for Open WebUI. This provides users with an alternative containerization option using Podman, alongside Docker, for setting up and running Open WebUI.
Updated the README to reflect both Docker and Podman options for installation, GPU support, and OpenAI API configuration.

Added

Podman support for running Open WebUI with default configurations.
Podman commands for setting up Open WebUI with Ollama and CUDA support.
Podman examples for troubleshooting connection issues and keeping installations up to date.

Changed

Modified the installation instructions in the README to include both Docker and Podman commands for all relevant sections, including OpenAI API usage, Nvidia GPU support, and Ollama integration.
Expanded troubleshooting section to include both Docker and Podman networking solutions.

Fixed

Clarified usage of host.containers.internal for Podman where Docker uses host.docker.internal.

Security

No new security updates or changes.

Additional Information

These changes enhance user flexibility by supporting both Docker and Podman, allowing users to choose their preferred container runtime.
No breaking changes introduced.

Related discussions: Discussion thread on adding Podman support.

Screenshots or Videos

N/A

Feel free to update https://docs.openwebui.com/getting-started/#installing-with-podman


Support video urls with timestamps

Should support URLs with timestamps like
https://youtu.be/YgSSql6MAKo?t=89
and
https://www.youtube.com/watch?v=YgSSql6MAKo&t=109s

Added timestamp support for first video in a50c579aa9dbb5dfd48d005107c489bd945b5728
Need to change how playlists store videos or change ID parsing to support storing the timestamp. Alternatively there could be an ID-timestamp lookup table/dictionary.


DPL-738 As SSM (Richard R) I would like to reuse existing NovaSeq 6000 pipelines in Limber but submit samples on to the new NovaSeqX sequencing machine.

User story
As SSM (Richard R) I would like to reuse existing NovaSeq 6000 pipelines in Limber but submit samples on to the new NovaSeqX sequencing machine.
Who are the primary contacts for this story
Richard R
Who is the nominated tester for UAT
Liz C
Tris B
Acceptance criteria
To be considered successful the solution must allow:

[ ] Users can take a samples on an existing pipeline process (e.g. GBS) and submit them to the new NovaSeqX machine.
[ ] Users can still submit samples on these pipelines to NovaSeq 6000, HiSeq 2500 and MiSeq machines.

Dependencies
This story is blocked by the following dependencies:

https://github.com/sanger/General-Backlog-Items/issues/287

Additional context
Note the new NovaSeqX has a different flow cell type and different number of lanes per flow cell than the NovaSeq 6000 machines.
It is likely that completing DPL-711 will give this feature for free - however this story exists to check that happens, make any modifications needed and test the submission functionality from existing pipelines works as expected.
If it turns out that existing pipelines cannot use a different sequencing machines - we can clone the existing NovaSeq 6000 pipelines and make the changes required to submit to the NovaSeqX under this user story.

Done as part of https://github.com/sanger/General-Backlog-Items/issues/287


Feature Invoke Attrs and multi-sign

What current issue(s) does this address, or what feature is it adding?

Adds ability to attach tx attributes to SC invokes
Adds the abililty to attach an arbitrary number of signing 'owners' to a transaction
Other various fixes

How did you solve this problem?

Add optional invoke_attrs and owners parameters to the invoke routines

How did you make sure your solution works?

tests

Are there any special changes in the code that we should be aware of?

no

Please check the following, if applicable:

[x] Did you add any tests?
[x] Did you run make lint?
[x] Did you run make test?
[x] Are you making a PR to a feature branch or development rather than master?
[ ] Did you add an entry to CHANGELOG.rst? (if not, please do)

Coverage decreased (-0.8%) to 78.049% when pulling 3cb05538a5991833f8c6e88efc6764c03b681000 on feature-multisign into 12c15b3bbed6475f63d54b5a259dda8d6f0f4d69 on development.

Thanks for the feedback!  Will do that and also add some tests


Cordova Android duplicated uses-permissions from two plugins

Bug Report
Problem
I try to update the plugin cordova-plugin-camera 6.0.4 -> 7.0.0 with Android13 bug corrections. BUt i've got a probleme: Cordova Android duplicated uses-permissions from two plugins with permission "WRITE_EXTERNAL_STORAGE". In fact, this permission is add with another cordova plugin without maxSdkVersion like cordova-plugin-file v6.0.2 or cordova-plugin-x-social-sharing v6.0.4. This 2 last plugins add "WRITE_EXTERNAL_STORAGE" wihtout maxSdkVersion and the AndroidManifest result contains 2 same uses-permission:
- <uses-permission android:name="android.permission.WRITE_EXTERNAL_STORAGE" />
- <uses-permission android:maxSdkVersion="32" android:name="android.permission.WRITE_EXTERNAL_STORAGE" />

What is expected to happen?
What does actually happen?
Information

Command or Code

Environment, Platform, Device

Version information

Checklist

[x ] I searched for existing GitHub issues
[ ] I updated all Cordova tooling to most recent version
[ x] I included all the necessary information above

https://github.com/apache/cordova-plugin-camera/issues/854#issuecomment-1715932306
Closing as duplicated.
I would suggest filing a big ticket with the third-party plugin. Ask them to update the plugin for Android 13 and set the maxSdkVersion to 32 for those permissions.
In the mean time, you can look at the above link. It contains a gist link to code example that you can use in a hookscript to remove the duplicated permissions.


Xero sample

Thanks for this great library!
The code below might help others who are googling "Xero oauth2 integration with google sheets". It's clunky so I'm not going to try a PR with a sample... maintainers please close this issue as you see fit.
You just need to trigger showAuthLink() function to get a sidebar with an authentication link.
var secret = '';
var clientid = '';

// Xero Oauth2 endpoints.
var authorizeURL = 'https://login.xero.com/identity/connect/authorize';
var requestTokenURL = 'https://identity.xero.com/connect/token';

// Get an Oauth2 service on demand.
function getXeroService() {
  return OAuth2.createService('xero')
    .setAuthorizationBaseUrl(authorizeURL)
    .setTokenUrl(requestTokenURL)
    .setClientId(clientid)
    .setClientSecret(secret)
    .setCallbackFunction('authCallback')
    .setScope('accounting.contacts.read accounting.transactions.read accounting.settings.read accounting.reports.read accounting.journals.read payroll.employees.read payroll.payruns.read payroll.payslip.read')
    .setPropertyStore(PropertiesService.getUserProperties());
}

// Opens a sidebar with a link to authorise.
function showAuthLink() {
  var xeroService = getXeroService();
  if (xeroService.hasAccess()) {
    var url = 'https://api.xero.com/connections'
    var token = xeroService.getAccessToken();
    var options = {
      'method': 'GET',
      'headers': {
        'Accept': 'application/json',
        'Authorization': 'Bearer ' + token,
      }
    };
    var response = UrlFetchApp.fetch(url, options);
    // Output the tenant data for convenience.
    var template = HtmlService.createTemplate('
You appear to be authenticated with Xero already.

<?= tenantData ?>
');
    template.tenantData = response.getContentText();
    var page = template.evaluate();
    SpreadsheetApp.getUi().showSidebar(page);
  }
  else {
    var authorizationUrl = xeroService.getAuthorizationUrl();
    var template = HtmlService.createTemplate('
<a href="<?= authorizationUrl ?>" target="_blank">Authenticate for 30 mins access to Xero.
.');
    template.authorizationUrl = authorizationUrl;
    var page = template.evaluate();
    SpreadsheetApp.getUi().showSidebar(page);
  }
}

// Opens a sidebar with a link to authorise.
function authCallback(request) {
  var xeroService = getXeroService();
  var isAuthorized = xeroService.handleCallback(request);
  if (isAuthorized) {
    return HtmlService.createHtmlOutput('Xero authenticated! You can close this tab.');
  }
  else {
    return HtmlService.createHtmlOutput("Something went wrong with the authentication.");
  }
}

Once you are authenticated, here's a sample of getting data.
  var xeroService = getXeroService();
  var url = 'https://api.xero.com/api.xro/2.0/Contacts';
  var token = xeroService.getAccessToken();
  // I hardcode this with a fixed value (you run showAuthLink() when you're authenticated to get the value of tenant ID). I think this is a fixed value per organisation, but I'm not 100% sure the best way to get it.
  var tenantId = ''abcd1234-aaaa-4b83-bbbb-cccccccccccc';

  var options = {
    'method': 'GET',
    'muteHttpExceptions': false,
    'headers': {
      'Accept': 'application/json',
      'Connection': 'keep-alive',
      'Authorization': 'Bearer ' + token,
      'Xero-tenant-id': tenantId
    }
  };

  var response = UrlFetchApp.fetch(url, options);
  var responseData = JSON.parse(response.getContentText());
  Logger.log(responseData);

Thanks for the suggestion! I've added a Xero sample to the repo, although it's a bit simpler and doesn't assume a spreadsheet.

Nice - Theres also a new app out there for folks not looking to build and maintain themselves. https://apps.xero.com/us/search/app/flatly

Hi,
Trying to set something similar to this. And getting the following error
Error code: 500
Error: unauthorized_client : Invalid redirect_uri

I have used the following redirect url in my xero app: https://script.google.com/d/16DdBbfNxsfZPK6MQ2tqUaXw1bcuji44e-7m0pVwH_R_fEWSULKB2R1ip/usercallback


Update sbt-protobuf to 0.7.1

Updates com.github.gseitz:sbt-protobuf from 0.6.5 to 0.7.1.
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "com.github.gseitz", artifactId = "sbt-protobuf" } ]

labels: sbt-plugin-update, early-semver-major, semver-spec-minor, commit-count:1

Superseded by #593.


September 19th, 2022 Community Meeting

Please add a topic in this thread and add a link to the GitHub issue associated with the topic.
Please make sure you give folks enough time to review/discuss the topic offline on GitHub before coming into the meeting
(optional) Paste the image of an animal 😸

Meeting was cancelled, per few attendees and none topics to discuss.


Refactor project structure

Codecov Report

Merging #173 into master will decrease coverage by 0.06%.
The diff coverage is 76.86%.

@@            Coverage Diff             @@
##           master     #173      +/-   ##
==========================================
- Coverage   57.13%   57.06%   -0.07%
==========================================
  Files          13       18       +5
  Lines        1269     1281      +12
==========================================
+ Hits          725      731       +6
- Misses        544      550       +6

Impacted Files
Coverage Δ

dpemu/filters/sound.py
0% <0%> (ø)

dpemu/plotting_utils.py
22.97% <100%> (ø)
:arrow_up:

dpemu/filters/time_series.py
100% <100%> (ø)

dpemu/filters/image.py
63.68% <63.68%> (ø)

dpemu/filters/common.py
94.62% <94.62%> (ø)

dpemu/filters/text.py
96.55% <96.55%> (ø)

dpemu/filters/__init__.py
96.66% <96.66%> (ø)

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2c3f474...3bdb5cf. Read the comment docs.


Add PGX Logo

Add Pegaxy Logo

Initially I merged this but I reverted it.

Please add the logo to #948
Make sure your contract is verified first


[release-calendar] Adding stylelint

This is supposed to add stylelint to format scss and css files.
Hi @estherkim, I'm going to ask you about this tomorrow morning.

Hiya, instead of having a separate script, let's use the stylelint prettier plugin so we can "prettify" all files in the same process.
https://github.com/prettier/stylelint-prettier


optimization debian package manager tweaks

What does this PR (Pull Request) do?

[x] This PR is not related to any Issue

By default, Ubuntu-based or Debian-based apt or apt-get system installs recommended - but not suggested - packages .
By passing --no-install-recommends"\ option, the user lets apt-get know not to consider recommended packages as a dependency to install.
This results in smaller downloads and installation of packages .
Refer to blog at Ubuntu Blog .
This PR inserts this option into calls to apt/apt-get in various places throughout the code base.
Which Traffic Control components are affected by this PR?

CDN in a Box
Documentation

What is the best way to verify this PR?
Build documentation and build and run affected Docker image specifications.
The following criteria are ALL met by this PR

[x] Only affects dependency management; building/running is the only testing that makes sense.
[x] This PR includes documentation
[x] An update to CHANGELOG.md is not necessary
[x] This PR includes any and all required license headers
[x] This PR does not include a database migration
[x] This PR DOES NOT FIX A SERIOUS SECURITY VULNERABILITY

closing as stale with conflicts; please re-base and re-open or open a new PR to get these changes merged.


Feature request

Is it possible to add scrolling in the picture as a feature? maybe hold ctrl and hover on it and scroll?
Thanks for the extension!

nope


Add website preview

This could be Netlify or the other one that @CIPHERTron  suggested... what was that again?

@yochannah , It was Feature Peek but there's a bad news. Netlify acquired Featurepeek and it kinda removed its existence completely XD
Previously, it was at https://featurepeek.com/ but now, its redirecting to https://netlify.com
But no worries, netlify's preview is also great and easy to configure or you can say, it requires 0 configuration.

ah lol - everything comes down to Netlify in the end 😂

this is live at https://journal-policy-tracker.netlify.app/


libnode.so: cannot open shared object file: No such file or directory when trying to run

I just extracted the tarball and when I go to run it I get this
./discord: error while loading shared libraries: libnode.so: cannot open shared object file: No such file or directory
I'm on ubuntu 15.10 X64

Do you not have libnode.so in that directory? It should be there. You're sure you downloaded the right architecture? Can you post the results of ldd ./discord?


Add missing jszip to package.json

Hey guys, to solve the problem about jszip it's not a constructor on importing xlsx, the jszip should be in package.json. I just wanna know, why don't? Pls, let me know if this is relevant, because I solved this issue like that and I have my fork ready for send PR, adding the missing lib to xlsx npm packages.
Obs.: this is going on with too much people.

Check that you are using the latest version, since we changed the dist script configuration going into 0.11.0.
For straight script tag insertion in a web page, use dist/xlsx.full.min.js instead of the individual scripts.
In NPM, require('xlsx') should just work, as it's how the xlsx CLI tool invokes the library.
Without seeing a project structure, it's hard to say what's going on, but there are various demos showing projects using angular, react, electron, and other frameworks.

@reviewher I'm using the latest version!


ppa:ondrej/php-7.0 is deprecated

This repository will be disabled in the future, please use ppa:ondrej/php as it has the same contents, PHP 5.6 as a added bonus now, and any future version of PHP in the future.

:+1:

https://launchpad.net/~ondrej/+archive/ubuntu/php-7.0


Add surname to registration form

At the request of Marketing, this PR adds an explicit surname field to the registration form.

I think we'll actually need to do something with this value here. Where do they want it to go? Hubspot?

@0xcadams Yep, Hubspot - from the last time I used this page, I thought all values from the form got passed through to Hubspot by default


[FLINK-18974][docs-zh]Translate the 'User-Defined Functions' page of "Application Development's DataStream API" into Chinese

What is the purpose of the change
[FLINK-18974][docs-zh]Translate the 'User-Defined Functions' page of "Application Development's DataStream API" into Chinese
Brief change log

Translate the 'User-Defined Functions' page of "Application Development's DataStream API" into Chinese

Verifying this change
This change is a trivial rework / code cleanup without any test coverage.
Does this pull request potentially affect one of the following parts:

Dependencies (does it add or upgrade a dependency): (no)
The public API, i.e., is any changed class annotated with @Public(Evolving): ( no)
The serializers: (no)
The runtime per-record code paths (performance sensitive): (no)
Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn/Mesos, ZooKeeper: (no)
The S3 file system connector: (no)

Documentation

Does this pull request introduce a new feature? (no)
If yes, how is the feature documented? (no)

Thanks for your review @RocMarshal.I have made some changes according to your suggestions.

@zhuxiaoshang thanks for your contribution, left some comments, please let me know what do you think, thanks.

thanks for your review,i had fixed accroading to your suggestions.

@zhuxiaoshang thanks for your contribution, left some comments based on the item 5. 中文标点符号 of Flink+Translation+Specifications please let me know what do you think ,thanks.

yes,you are right,brackets should be chinese form.

Thanks for the review! Merging ...


[Feature] need record the starttime and endtime for every canary batch in Rollout

so far, user can not know  starttime and endtime for every canary batch/step

Sounds reasonable, are you interested in impl this feature? @StevenLeiZhang

If necessary, I can develop this feature @veophi


Admob and Tabris.js

Tabris.js has cordova as its engine. (https://tabrisjs.com/) I have tried to install this plugin, but it doesnt do his job.

Only "not working "is not a good bug report... Tell me more details. How did you use it? Any error message or logs?

Tabris.js use cordova plus its own virtual box UI. It don`t make use of webview. Read this: https://github.com/eclipsesource/tabris-js/issues/353

if Tabris.js does not have a webview, then it will be a problem, as the plugin is written for Cordova, and it will resize webview when ad banner is on/off.

Tabris seems to be promissor and use cordova as its engine. Maybe you could make and plugin that works on it!
Thanks for your help!

I personally think webview is good enough, as Google and Apple have both improved the performance.
Tabris.js seems not a popular project, as it only get 33 stars, but with 140+ open issues. It seems not wise to spend time porting the plugin especially for Tabris.js. I will consider your suggest when they get 500+ stars. :P

357 and counting :)


How to remove directional triangles from streets?

At the moment streets are plotted with white triagnles that show direction:

However for large maps this is not a useful feature as it strongly decreases visibility, and the road widths remain the same under zoom meaning that even if I zoom in the plot I still can't really tell apart which direction the triangles point.
I'd like to disable the triangles. How can I do that? Can we add a simple keyword argument to the plot function?

p.s.: I've assumed they are directional triangles. I could never seem them with high enough visibility to be sure. They may be something else. Regardless I would prefer to not plot them.

These are the markers for one-way roads. You should be able to disable them by passing graphplotkwargs = (; arrow_show = false).

oh thanks! That was so easy!


Animating dollar amounts leaves out the decimal value when .00

I am using AnimeJS to animate donation totals changing after more donations have been received. The animation works great however when the new donation amount is a full dollar amount, like $50.00, the animation ends by cutting off the .00. I am using the numeral npm package to format all of my data coming in from a JSON call. Here is the code, it is part of a timeline.
.add({       targets: self,       total: newValue.formatted,       round: 100,       easing: 'easeInOutQuad' });
I've also noticed when animating up, say from $20.00 to $40.00, every time the number animates to another .00, the number appears as $30. This throws off the width of the label and makes the animation look a little jittery.

Anime doesn't provide any formatting options.
What you can do instead, is format the value using numeral in the update() function, like this:
var dols = {
  total: 0
}

anime({
  targets: dols,
  total: 100,
  round: true,
  easing: 'easeInOutQuart',
  duration: 3000,
  update: function(a) {
    var value = a.animations[0].currentValue;
    var formated = numeral(value).format('$0,0.00');
    console.log(formated);
  }
});

https://codepen.io/juliangarnier/pen/57c7ca4ab926fb551e5475723a0e04c4/?editors=1111

Ah that is what I needed, thank you very much!


Missing Subscribe/Unsubscribe

I‘m missing a button to subscribe and unsubscribe a issue or pr. So this is a feature request :)

Bug Report Dump (Auto-generated)

Version 1.22.0<PHONE_NUMBER>)
Device: iPhone 6s (iOS 11.4.1)
TestFlight: false

You can subscribe/unsubscribe to an issue or pull request in your inbox. These should have this “loudspeaker” icon.
Where else would you expect to see this? Would you like to be able to subscribe to an issue you’re not subscribed to in the first place?
Sent with GitHawk

Yes, exactly. The loudspeaker icon is new, isn't it? Will it subscribe only for app notification or for GitHub, so that I can view notification is the browser too?

The loudspeaker 📢 is an interesting “shortcut”
You can only see it in your inbox so it means your already subscribed to it.

I feel more likely you read the issue and then want to unsubscribe which means you must go to view all and then unsubscribe? It just feels a bit weird.
I’d suggest removing the loudspeaker 📢 from inbox and moving it into the manager button below the divider along with #2140.
I think then inbox cells can be simplified to just the read button and a more button see #2127 for an example.
Sent with GitHawk

Sub/un in the burger menu would be great! Also removes the need to hide/show the menu button which would clean up a lot of code!
Sent with GitHawk

Would love this as well

@ijm8710 I actually meant to tag this issue in my PR, the PR you tagged is to add this support.

Sweet, would it make sense to have a reference section where it says you’re actively watching/subscribed to said thread like it does on github. Somewhat similar to how you guys created the reference subsections

Not sure I follow.
In the manger content menu it says either : subscribe, unsubscribe so you’ll know your current status
Sent with GitHawk

Meant this
While what you said is absolutely true, I just meant having the blurb so you know without avruoning the manager button

@ijm8710 maybe for a separate PR, focusing on this one first. Good suggestion just wanna do one thing at a time
Sent with GitHawk

Strongly seconded. I very frequently subscribe to issues on repos where I'm "just a user/beta tester", so that I get notified when a fix for a particular problem is available. This very issue would a prime example! (of course, now I've commented on it, which subscribes me to updates, but in most cases, I'm just lurking without a comment).


Add exists accessor

Addresses #220.

Coverage increased (+0.009%) to 97.7% when pulling 7e465c8f50706f52c5b516c2d4692d2f21468a78 on sutori:introduce-exists into 0f20135d16179278eb6fec7b04b505be79f096ef on san650:master.

Coverage increased (+0.009%) to 97.7% when pulling 7e465c8f50706f52c5b516c2d4692d2f21468a78 on sutori:introduce-exists into 0f20135d16179278eb6fec7b04b505be79f096ef on san650:master.

@YoranBrondsema great addition to the library! I also found myself needing something like this.
Thanks for contributing.

@san650 No problem, thanks for merging!


once workflow finishes, parse cromwell output

once a cromwell worfklow completes, read the metadata into the hail batch log

Messy history -

I set the json_output to default to None, which caused a linting error
Also ran a bump2version minor instead of patch

Reverted both and made the small suggested change plus a bump2version patch


Feature request: autodownload PDF and save file using meta data

Great extension! Would be fantastic if it could be automated to automatically save and name PDF downloads based on meta data, e.g. Author Year Title.PDF

Great idea!  I’ll see what I can do (might be a couple weeks before I have time, but I like this a lot!)
maybe going even further: generate a bibtex entry or something as well


Issue#296

Switched to ScheduledExecutorService instead of Time, because of OutOfMemory exception in thread creation.

@dmayboroda  I still see other commits apart from your fix. I don't think you are doing it right.


Refactor Warrior and Wizard movement, jumping and attacking keys pressed on keyboard

Hi, I'm suggesting this change because my keyboard does not have a numeric keypad to get the wizard to attack the warrior. I imagine there are other users of this repo in the same situation.
Suggested change:
Warrior: Movement / Jumping / Attacking
a<-->d / w / z or x
Wizard: Movement / Jumping / Attacking
h<-->k / u / n or m
By adopting an alphabet only approach to keys pressed, playing the game will become more accessible.

This is a good suggestion, I hadn't considered the number keys being an issue. On this occasion, as this repo is the reference source code for my youtube tutorial, I will keep the controls with the numpad as that's how it is explained in the video. This still leaves the option for individual users to modify the controls locally as needed.

Hi, thanks for considering my suggestion 😎
I agree with you that it's best to keep the repo and video, consistent.


Request for building VSCode from source without Visual Studio 2017+

Request for building VSCode from source and change some things
Hello!
I want to build vscode from source and change the code to see what happens behind the scenes and just see what I can do with it.
I just want to tell that, Can I do the above without Visual Studio 2017+, as I don't have enough requirements for that.
Please!

All the details for how to build VS Code are in the wiki: https://github.com/microsoft/vscode/wiki/How-to-Contribute


Build failure: sunshine (with CUDA enabled)

Steps To Reproduce
Steps to reproduce the behavior:

Build sunshine with CUDA enabled (i.e. sunshine.override { cudaSupport = true; })

Build log
The final error is:
[100%] Linking CXX executable sunshine
/nix/store/zsvpmlddl2i5mpzm031a99xfpn410m5b-binutils-2.40/bin/ld: /nix/store/18qyy12yw5jb046jm0qga4j59vj7r310-zeromq-4.3.4/lib/libzmq.so.5: undefined reference to<EMAIL_ADDRESS>collect2: error: ld returned 1 exit status
make[2]: *** [CMakeFiles/sunshine.dir/build.make:686: sunshine-0.20.0] Error 1
make[1]: *** [CMakeFiles/Makefile2:137: CMakeFiles/sunshine.dir/all] Error 2
make: *** [Makefile:156: all] Error 2

Full build log: https://gist.github.com/robwhitaker/829d4cae374e8c9e8bcc31289d581052
Additional context
N/A
Notify maintainers
@devusb
Metadata
Please run nix-shell -p nix-info --run "nix-info -m" and paste the result.
[user@system:~]$ nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-linux"`
 - host os: `Linux 6.1.34, NixOS, 23.11 (Tapir), 23.11.20230628.60a783e`
 - multi-user?: `yes`
 - sandbox: `yes`
 - version: `nix-env (Nix) 2.15.1`
 - channels(root): `""`
 - channels(rhit): `""`
 - nixpkgs: `/nix/store/22qgs3skscd9bmrxv9xv4q5d4wwm5ppx-source`

Think this PR is intended to resolve the build issue -- unfortunately I no longer have an NVIDIA card so unable to verify.

Thanks, I applied the change in that PR myself:
sunshine.override {
  cudaSupport = true;
  stdenv = cudaPackages.backendStdenv;
}

and it works!

Should have been fixed by #293938


error

https://pastebin.com/nPX5JSNU

Your server sent an empty packet, this is commonly down to plugins/mods on the server manipulating packets improperly


[MINOR] Prepare Spark release 3.0.0-preview

What changes were proposed in this pull request?
To push the built jars to maven release repository, we need to remove the 'SNAPSHOT' tag from the version name.
Made the following changes in this PR:

Update all the 3.0.0-SNAPSHOT version name to 3.0.0-preview
Update the PySpark version from 3.0.0.dev0 to 3.0.0.preview

Please note those changes were generated by the release script in the past, but this time since we manually add tags on master branch, we need to manually apply those changes too.
We shall revert the changes after 3.0.0-preview release passed.
Why are the changes needed?
To make the maven release repository to accept the built jars.
Does this PR introduce any user-facing change?
No
How was this patch tested?
N/A

cc @srowen @cloud-fan @dongjoon-hyun @gatorsmile

Test build #112604 has started for PR 26243 at commit 3db7b71.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17569/
Test PASSed.

Test build #112606 has started for PR 26243 at commit ed65ac5.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17571/
Test PASSed.

Test build #112607 has started for PR 26243 at commit 8ac854b.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17572/
Test PASSed.

Right, seems like we'd have to do this for release and then revert. OK

Test build #112604 has finished for PR 26243 at commit 3db7b71.

This patch fails PySpark pip packaging tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112604/
Test FAILed.

Test build #112606 has finished for PR 26243 at commit ed65ac5.

This patch fails PySpark pip packaging tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112606/
Test FAILed.

Test build #112607 has finished for PR 26243 at commit 8ac854b.

This patch fails PySpark pip packaging tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test build #112609 has started for PR 26243 at commit 322fcab.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17574/
Test PASSed.

Hi, @jiangxb1987 . We need to update R/pkg/DESCRIPTION, too.
cc @shivaram , @felixcheung , @HyukjinKwon

Also, this should be reverted this immediately after commit. Otherwise, our Jenkins will try to publish this to our snapshot repository.

Hi, @jiangxb1987 . I update the PR title for a commit log. Since we use Preparing Spark release v2.4.4-rc3, the following will be the better.
Prepare Spark release v3.0.0-preview-rc1

Also, could you review this, @gatorsmile ?

Test build #112609 has finished for PR 26243 at commit 322fcab.

This patch fails SparkR unit tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112609/
Test FAILed.

The R failure is due to this.
Error in sparkR.session(master = sparkRTestMaster, enableHiveSupport = FALSE) :
  (converted from warning) Version mismatch between Spark JVM and SparkR package. JVM version was 3.0.0-preview , while R package version was 3.0.0

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17579/
Test PASSed.

Test build #112614 has started for PR 26243 at commit ebc2345.

Test build #112614 has finished for PR 26243 at commit ebc2345.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112614/
Test FAILed.

Test build #112616 has started for PR 26243 at commit f3731db.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17581/
Test PASSed.

Merged build finished. Test FAILed.

Ideally yes we release Pyspark and SparkR. At least, Pyspark seems pretty important.
If it's a major problem that we're trying to do this in master, then I concede, let's make a branch. Does a name like "3.0.0.preview" cause fewer headaches? we do have to keep the branch if that's what we release, but, if we have to do it we have to.
If the idea is to go ahead and cut branch-3.0, I'd rather not just because of the headache of merging back every change from now until 3.0. But it's not out of the question.

Ah, yea, I meant the branch like branch-3.0-preview or preview-3.0 for instance.

Does a name like "3.0.0.preview" cause fewer headaches?

If this works, yea it should be good. Let us know @jiangxb1987 :D.

I don't think we should release the SparkR preview on CRAN. We can't overwrite versions and the default install from CRAN gives the latest version.

I am not sure CRAN accept preview in the version number also. And what Shivaram says

From: Shivaram Venkataraman<EMAIL_ADDRESS>Sent: Thursday, October 24, 2019 6:35:03 PM
To: apache/spark<EMAIL_ADDRESS>Cc: Felix Cheung<EMAIL_ADDRESS>Mention<EMAIL_ADDRESS>Subject: Re: [apache/spark] Prepare Spark release v3.0.0-preview-rc1 (#26243)
I don't think we should release the SparkR preview on CRAN. We can't overwrite versions and the default install from CRAN gives the latest version.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://github.com/apache/spark/pull/26243?email_source=notifications&email_token=ACENZ6YRTV45MTUH4PXGERDQQJEMPA5CNFSM4JESXMOKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOECG4ZKY#issuecomment-546163883, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ACENZ63XMGEJSF6AC32PHZDQQJEMPANCNFSM4JESXMOA.

Thank you all.

For SparkR, if we are going to skip CRAN, the problem is solved.
For PySpark, the branch was already created as branch-3.0-preview and deleted before due to the side effect (@HyukjinKwon . Please see the mailing list. We discussed the side-effect on dev mailing list).
If there is a way to avoid the side-effect on our merge script and release script. I'm also okay for re-creating.

Please check the feasibility before creating the branch. Maybe, it seems to depend on the branch naming.

Test build #112649 has started for PR 26243 at commit f3731db.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17610/
Test PASSed.

Thanks for pointer @dongjoon-hyun. Alright, then, we don't do PyPI too I guess @holdenk?
/tmp/tmp.RRgnNxjsCA/3.5/lib/python3.5/site-packages/setuptools/dist.py:398: UserWarning: Normalizing '3.0.0.preview' to '3.0.0rc0'
......
Requirement<EMAIL_ADDRESS>looks like a filename, but the file does not exist
Processing ./python/dist/pyspark-3.0.0.preview.tar.gz
Could not install packages due to an EnvironmentError: [Errno 2] No such file or directory<EMAIL_ADDRESS>
This looks just pip packaging test failure. setup.py is only for PIP packaging and we don't probably need it.
Can we just completely skip pip packaging or use some workarounds to disable the normalizing the name (https://github.com/pypa/setuptools/issues/308) for this preview?

If ^ is easy, and can be done in this PR, 1. we can test if the tests pass 2. we release preview
and 3. revert this commit from the master.
Sorry if this was discussed already somewhere but we're not going to vote on it right?

Test build #112649 has finished for PR 26243 at commit f3731db.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112649/
Test FAILed.

Retest this please.

Test build #112661 has started for PR 26243 at commit f3731db.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17620/
Test PASSed.

Test build #112661 has finished for PR 26243 at commit f3731db.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112661/
Test PASSed.

Since we are not going to release the Python and R package, shall the current changes be good enough ?

To be clear, we can release Pyspark's tarball, just not push to pip? I think it's OK.

To be clear, we can release Pyspark's tarball, just not push to pip? I think it's OK.

Yeah, release the packages but not push to pip.

But can we change the PySpark version to something different from 3.0.0?

Seems like we can either skip pip package test or use workaround mentioned at https://github.com/apache/spark/pull/26243#issuecomment-546203075
Seems we're good about this..? I can push some changes into this branch within 2 days.

+1 for @HyukjinKwon 's comment.

But can we change the PySpark version to something different from 3.0.0?

BTW, @HyukjinKwon . Do you know why AppVeyor is still failing? Does this preview break something on Windows?

For AppVeyor failure, will bring some updates within 2 days as well. I'm moving to new house now so dont have laptop with internet yet :p

Hi, All.
It seems that the current release script doesn't generate Hadoop 3.2 profile. (@gatorsmile found the following).

https://dist.apache.org/repos/dist/dev/spark/v3.0.0-preview-rc1-bin/

Hadoop 3.2 profile publishing is critical for RC1. (cc @wangyum ).

I'm testing it and will make a PR soon.

@dongjoon-hyun Thanks for working on the release script, yeah I agree hadoop 3.2 package is a must for 3.0.0-preview release. In the meanwhile, I feel there are also need to release more combinations, I just sent out a email to devlist to discuss. After all, hadoop 3.2 package is definitely necessary. Thank you!

I made a PR.

https://github.com/apache/spark/pull/26260

Hi, @jiangxb1987 .
#26260 is merged to master. Please rebase to the master when you test this PR.

Test build #112738 has started for PR 26243 at commit c512d42.

Test build #112738 has finished for PR 26243 at commit c512d42.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112738/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17683/
Test PASSed.

Retest this please.

Test build #112739 has started for PR 26243 at commit c512d42.

Test build #112739 has finished for PR 26243 at commit c512d42.

This patch fails some tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112739/
Test FAILed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17684/
Test PASSed.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17685/
Test PASSed.

Test build #112740 has started for PR 26243 at commit b10f174.

Test build #112740 has finished for PR 26243 at commit b10f174.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112740/
Test PASSed.

AppVeyor failure seems related with scala-maven-plugin upgrade from a cursory look, which I will continue to investigate at https://github.com/apache/spark/pull/26278.
Since Spark does not currently support to build on Windows officially, I think it's fine to unblock this preview.

Test build #112754 has started for PR 26243 at commit 6459efe.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17699/
Test PASSed.

Test build #112754 has finished for PR 26243 at commit 6459efe.

This patch fails due to an unknown error code, -9.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112754/
Test FAILed.

retest this please

Test build #112761 has started for PR 26243 at commit 6459efe.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17706/
Test PASSed.

Test build #112761 has finished for PR 26243 at commit 6459efe.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112761/
Test PASSed.

I reverted https://github.com/apache/spark/commit/5fc363b307e4a032df87182d477ff63473d6d02d which I merged by myself ... seems it causes the build failure on Windows (https://github.com/apache/spark/pull/26278). Let me rebase it here.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17714/
Test PASSed.

Test build #112770 has started for PR 26243 at commit 6a59acd.

Test build #112770 has finished for PR 26243 at commit 6a59acd.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112770/
Test FAILed.

Test build #112772 has started for PR 26243 at commit 6a59acd.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17717/
Test PASSed.

Got it. Thank you for reverting and keeping this moving forward! @HyukjinKwon .

Test build #112806 has started for PR 26243 at commit ec571bb.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17746/
Test PASSed.

Test build #112806 has finished for PR 26243 at commit ec571bb.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112806/
Test FAILed.

retest this please

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17747/
Test PASSed.

Test build #112808 has started for PR 26243 at commit ec571bb.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17748/
Test PASSed.

Test build #112808 has finished for PR 26243 at commit ec571bb.

This patch fails build dependency tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test FAILed.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112808/
Test FAILed.

Test build #112809 has started for PR 26243 at commit e1808a5.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/17749/
Test PASSed.

Test build #112809 has finished for PR 26243 at commit e1808a5.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Merged build finished. Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/112809/
Test PASSed.

should be good to go.

LGTM

Thanks, merging to master!

FYI, this PR has been reverted by https://github.com/apache/spark/commit/b33a58c0c6a40aa312cbd72f763d5e519182c3d1

Great! Thank you all!