--integer-simple is broken

... and I figured out why while investigating the CI failures from #701. It's not something there's one natural fix for that stands out. We're at a crossroads and we need to pick a solution. Everyone's free to give some input, but it'd be great to in particular hear from @bgamari @snowleopard and @angerman here.
Here's the problem statement. When we build with --integer-simple, stage 1 GHC gets built fine (under _build/stage0/bin/ghc, since it is built by the boot -- stage 0 -- compiler). The libraries that were needed for this got either built or copied from the boot compiler's package db and reside under build/stage0/lib. Then the stage 1 GHC starts building the few packages it needs for building the stage 2 compiler, and while it's doing this, we get errors about mismatched interface files that have to do with Integer and in particular GHC seems to be reaching for integer-gmp interface files. Stage 1 GHC has built integer-simple at that point (under _build/stage1/lib) and should therefore be using it, but instead it gets confused and goes for integer-gmp. This only happens with a GHC three that has this commit.
So what's going on? Everything Integer belong to a fictional package integer-wired-in which gets resolved to some version of integer-gmp or integer-simple depending on the circumstances. Our stage 1 GHC lives under _build/stage0/bin/ghc, and with the current heuristics, the global ("system") package db is determined to be under $topdir/package.conf.d. By default, $topdir is assumed to be at ../lib compared to where the executable lives. So the global package db for our stage 1 ghc is therefore taken to be _build/stage0/lib/package.conf.d, even though those are things built by the boot compiler! You can see how this is going to be problematic...
In initPackages, we end up looking at two package databases, the global one mentionned above, because that's something we always look at by default, unless told not to (-no-global-package-db), as well as the one given on the command line (the right one, _build/stage1/lib/package.conf.d). Here's their contents:
- /home/alp/WT/ghc-diffs/_quickest/stage0/lib/package.conf.d
  - ghc-8.7
  - Cabal-<IP_ADDRESS>
  - parsec-<IP_ADDRESS>
  - text-<IP_ADDRESS>
  - process-<IP_ADDRESS>
  - ghci-8.7
  - template-haskell-<IP_ADDRESS>
  - ghc-boot-8.7
  - binary-<IP_ADDRESS>
  - pretty-<IP_ADDRESS>
  - mtl-2.2.2
  - transformers-<IP_ADDRESS>
  - ghc-heap-8.7
  - hpc-<IP_ADDRESS>
  - terminfo-<IP_ADDRESS>
  - ghc-boot-th-8.7
  - directory-<IP_ADDRESS>
  - containers-<IP_ADDRESS>
  - unix-<IP_ADDRESS>
  - bytestring-<IP_ADDRESS>
  - time-<IP_ADDRESS>
  - deepseq-<IP_ADDRESS>
  - array-<IP_ADDRESS>
  - filepath-1.4.2
  - base-<IP_ADDRESS>
  - integer-gmp-<IP_ADDRESS>
  - ghc-prim-<IP_ADDRESS>
  - rts
- _quickest/stage1/lib/package.conf.d
  - base-<IP_ADDRESS>
  - integer-simple-<IP_ADDRESS>
  - ghc-prim-0.5.3
  - rts-1.0

And in findWiredInPackages, when comes the time to decide what integer-wired-in should resolve to for everything that this ghc session will do, we're faced with a choice between stage0/lib's integer-gmp-<IP_ADDRESS> and stage1/lib's integer-simple-<IP_ADDRESS>. As you can probably guess from the error, GHC ends up picking integer-gmp even though it was built with the boot compiler and not stage 1, and later down the road, whenever we touch something that refers to any entity from integer-wired-in, boom.
Now you may wonder, why on earth do we pick integer-gmp? Well, the "comparison" is done using a function named compareByPreference, whose code is below:
compareByPreference
    :: PackagePrecedenceIndex
    -> PackageConfig
    -> PackageConfig
    -> Ordering
compareByPreference prec_map pkg pkg' =
    case comparing packageVersion pkg pkg' of
        GT -> GT
        EQ | Just prec  <- Map.lookup (unitId pkg)  prec_map
           , Just prec' <- Map.lookup (unitId pkg') prec_map
           -- Prefer the package from the later DB flag (i.e., higher
           -- precedence)
           -> compare prec prec'
           | otherwise
           -> EQ
        LT -> LT

The precedence index thingie is just a map that gives (in our case) higher precedence to packages from _build/stage1/lib/package.conf.d (because it's given on the command line) then to those from _build/stage0/lib/package.conf.d (which is the global one for our stage 1 GHC). So in that precedence index, integer-gmp is given a precedence of 1 and integer-simple has 2, which should in theory make it "win". But look at the code... It compares package versions before anything. It assumes the package names are identical, I guess. And it turns out, integer-gmp version is strictly greater than integer-simple's... so it gets picked. I can get the entire build to succeed by simply changing compareByPreference to the following implementation:
compareByPreference
    :: PackagePrecedenceIndex
    -> PackageConfig
    -> PackageConfig
    -> Ordering
compareByPreference prec_map pkg pkg'
  | packageName pkg == packageName pkg' -- this is the existing logic
  = case comparing packageVersion pkg pkg' of
        GT -> GT
        EQ | Just prec  <- Map.lookup (unitId pkg)  prec_map
           , Just prec' <- Map.lookup (unitId pkg') prec_map
           -- Prefer the package from the later DB flag (i.e., higher
           -- precedence)
           -> compare prec prec'
           | otherwise
           -> EQ
        LT -> LT
  | otherwise -- different package names! let's only rely on precedence here
  = let mprec  = Map.lookup (unitId pkg)  prec_map
        mprec' = Map.lookup (unitId pkg') prec_map
    in case (mprec, mprec') of
         (Just prec, Just prec') -> compare prec prec'
         _                       -> error "I don't know what to do here"

Now, you can see this is not ideal. But it still makes more sense than the existing implementation, which completely ignores the possibility that the packages might be different. However, this situation shouldn't happen to begin with, our stage 1 ghc's global package database should be _build/stage/lib/package.conf.d. So we've got two problems:

compareByPrecedence makes an assumption that bad situations can break, whereas my definition can be a lot more helpful in those cases, especially if the situation is bad enough to make it land in the error branch. And alternative implementation would be to error out if packageName pkg /= packageName pkg' instead of looking at precedences. Regardless, there's an improvement to be made there I think. (1)
_build/stage<N>/bin/ghc's package database lives at _buid/stage<N+1>/lib/package.conf.d but GHC doesn't know about such layouts so _build/stage<N>/bin/ghc ends up thinking its package db is _build/stage<N>/lib/package.conf.d. We override the -package-db` whenever we invoke ghc in hadrian, but that's not going to be the case of people running the ghc executables manually. (2)

For (1), a GHC patch that implements either of my suggestions is enough to fix it. It is however more of a "improve behaviour/problem reporting under buggy circumstances" patch, and is not the core of our problem. The real problem is (2). Note that the build failure can also be addressed by specifying -no-global-package-db in every _build/stage<N>/bin/ghc command, that'd make integer-gmp (and the whole stage0 package db) disappear from the listing above, and there would be no choice left to make, we'd just go for integer-simple. However, we don't want to force our users to do this when using our ghc executables!
What are our options? A few came up in my mind or in discussions with others. But here's the one I would likely suggest we go for.
Let's move _build/stage<N>/bin/ghc to _build/stage<N+1>/bin/ghc. That way, it would sit right next to the libraries that it builds (as opposed to next to the other things that were built by the same compiler that built _build/stage<N>/bin/ghc, like now), the $topdir would correctly resolve to _build/stage<N+1>/lib, the default/global package would correctly resolve to _build/stage<N+1>/lib/package.conf.d and everything would go well. We would have:
_build
├──  _stage0     # libraries/etc built by the boot compiler in order to build the stage 1 compiler
│       ├── lib      # no bin, the stage 0 toolchain is already there before we build GHC
├──  _stage1
│       ├── bin     # the entire stage1 "ghc toolsuite" (ghc, ghc-pkg, ...)
│       ├── lib      # package db with all the libraries built by _stage1/bin/ghc
├──  _stage2
│       ├── bin     # the entire stage2 "ghc toolsuite"
│       ├── lib      # package db with all the libraries built by _stage2/bin/ghc

The alternatives all worse, from what I can tell. We could require the generation and use of wrapper scripts that point to the right topdir with -B. We do want to support wrappers and have some code for that, but requiring them when we can avoid it feels a bit silly. We could teach GHC about those stage<N>/stage<N+1> dirs, but that would be a gross hack. All in all, I think the aforementionned split would be best.
Thoughts on both problems and the solutions that I suggest?

@alpmestan Thank you for the amazing investigation!
I don't understand why simply adding -no-global-package-db to all invocations of GHC by Hadrian is not a good solution. You say:

However, we don't want to force our users to do this when using our ghc executables!

But they won't need to! We will install GHC and the package library in completely separate directory, outside the _build tree, where there will be no way confuse different stages.
I would really prefer to keep everything build in Stage0 under the _build/stage0 directory. It feels the Right Thing to do, because it's easy to explain. Introducing the complexity just as a workaround what I believe is a bug in compareByPreference sounds wrong.

P.S.: I think we can all agree that the current behaviour of GHC where it "prefers" a completely different package is a bug. Let's not capitulate to this bug by complicating Hadrian! :)

The Real Bug (TM) is that the global package db paths that hadrian-produced GHCs compute is wrong.
-no-global-package-db just lets us pretend that package db doesn't exist, so it slips the problem under the carpet, very much literally. Fortunately, we also specify the right -package-db on every invokation, but any command that doesn't have that is also dommed to potentially fail if arbitrary precision integers are involved one way or another... And I do run stage 1 or 2 GHC from the build tree regularly, while investigating bugs typically, so I'd definitely like having the ability to run my own little commands without having to prefix them with some magical package db incantation.
The most important thing to do here is to go back to computing correct global package dbs. And that problem is triggered by the current _build layout. They should be right, from the start, and we possibly might want to change paths when we install GHC but just doing _build/stage0/bin/ghc -e '1 :: Integer' ought to work with --integer-simple GHCs, without the need for a wrapper script or to install things, I think.
To achieve correct computation of global package db, the most logical solution is I think the one I suggested above. Finally, the compareByPreference "bug" is merely a side show here, if we fix (2) nobody ever has to deal with (1), while if we just fix (1) then anyone not doing the magic package db dance on the command line will not be able to build code that involves integer-wired-in, if my understanding is correct.
Reply to your P.S.: I don't think that would be a big hadrian patch, actually. Also with a diagram like the one I made above (with more details) we can probably explain this very easily. Or better yet, an actual good looking picture.

The Real Bug (TM) is that the global package db paths that hadrian-produced GHCs compute are wrong.

Could you clarify how "Hadrian-produced GHC" differs from "Make-produced GHC"?
I think the package database path-computation logic is the same in both, isn't it? So, if you say this is the Real Bug, shouldn't we fix it?
My understanding is that you say that the path-computation logic is wrong in both Hadrian-built and Make-built GHCs, and the only pragmatically viable solution is to just use the same directory structure as Make, even at the cost of extra complexity, because it happens to work well the current path-computation logic.
Maybe you are right, maybe a better path-computation logic is just too complex itself and this is just a trade-off between the complexity in GHC and the complexity in the build system.

I don't think that would be a big hadrian patch, actually.

Sure, I believe it. It's easy to switch a more complex directory structure in Hadrian. But will it be easy to work with this more complex structure in years to come? This is the only thing that bothers me. I was very confused by how build artefacts were organised in Make, and I wanted to move towards a more logical and clean organisation. It would be unfortunate that some of this progress will have to be undone.

Sorry, I forgot to add a paragraph about why make-produced GHCs don't suffer from this problem. The topdir-resolution logic being tailored to the make-produced GHCs, there will never suffer from anything like this since the inplace/ layout is exactly how the GHC code expects bins/libs to be laid out. Stage 1 and 2 GHCs always manage to find the right topdir and package db.

Notice how there's no stage separation and how 1/ all the actual binaries are under inplace/lib/bin 2/ they all have wrapper scripts under inplace/bin (that specifically use -B to point to the lib dir for $topdir). From there, GHC's code takes over and just looks at $topdir/package.conf.d to get the (global) package db.
Whereas in our case, our ghc executables would have to "cross over the stage separation", and we also don't generate wrappers right away yet, you have to ask for a binary distribution, and they're not quite as featureful I think.
All in all, yes, I really recommend the structure I described above. GHC developers will all be surprised regardless, by the stage separation, but I think it'd be great to keep it! Only we would have to change the semantics slightly, to match my "diagram" from earlier. I could try and whip up a slightly nicer diagram (or a picture) that describes it all nicely and add it to the PR that changes this, if we decide that this solution is the one we want. It'll definitely be less surprising for most people IMO that there's no "cross stage" relationships, especially if that happens to make everything work as expected again.

One trouble with the Make build system's layout is that it's unclear how we could do a stage3 build with its own libraries (e.g. using stage2 to build libaries and then building a stage3 compiler linked against them). That being said, this may not be a super important use-case.
However, I agree with @alpmestan that the current scheme is very surprising. In particular, having to reach into the stage0 directory to find the stage 1 compiler is quite mind-bending.

@alpmestan @bgamari OK, let's go ahead with the restructuring.
Is there a simple meaning to what we will then store in _build/stageN? I guess the simplest is "StageN compiler, plus the libraries it built... except for Stage0, where we only have libraries" :)

Well, it makes sense in my head, because we don't build a stage 0 compiler, that's the boot compiler that we start with. While we do build stage 1 & 2 compilers.

Yes. I had initially a +1 logic on binaries.
To have stage0/bin binaries, (those would be the bootstrap binaries, we could theoretically just symlink them there, or have shell wrapper they just call the respective binary hoping to find it in PATH) to have a uniform interface.
The only thing you need to make sure is that we tar/zip up the right lib/bin pair.

@angerman Thanks! Indeed, this is a possible way to make it more uniform. For now I think is not really necessary, let's just go ahead with @alpmestan's plan.
@alpmestan Are you going to implement the change? I'm happy to do this too if you like.

Well, I barely got started but will resume on monday unless you take over before that.

I have some time over the weekend, and feel a bit guilty for not doing much in Hadrian recently :-) This seems like a good high-priority task I could do. I'll give it a try and will let you have a look.

One unclear point: what do we do with unlit and touchy? Right now we build them in lib/bin directory, although we planned to move them to bin, see #570.
I guess I'll move them "up" one stage too, but in principle we could perhaps also move them to bin as part of this restructuring.

Hmm... I'd probably recommend leaving them under lib/bin/ for now, and then perhaps discussing with the larger GHC devs audience whether we want those to live under the usual binary dir? Or at the very least separating this in two PRs.

@alpmestan OK, agreed.
I think I have a working patch -- will create a PR after my Windows build completes.

OK, I've created #703. There is one problem though: with the new structure, Stage2 does not work since there is no package database in _build/stage2/lib:
ghc.exe: can't find a package database at C:\msys\home\nam83\ghc\_build\stage2\lib\package.conf.d

Any thoughts?

@alpmestan I've created an empty package database, i.e. the directory _build\stage2\lib\package.conf.d. This allowed me to finish the build, but I still can't run Stage2 GHC, now getting the following errors:
...
<interactive>:1:111: error:
    Not in scope: data constructor `System.IO.NoBuffering'
    No module named `System.IO' is imported.

<interactive>:1:135: error:
    Not in scope: `System.IO.hSetBuffering'
    No module named `System.IO' is imported.

<interactive>:1:159: error:
    Not in scope: `System.IO.stderr'
    No module named `System.IO' is imported.
...

Also, one interesting side-effect of the changes is that Haddock and ghc-tags are now built in the directory _build/stage3/bin. I guess that's correct, just looks a bit odd.

Hmm, this is all very odd. Let me know if I can help you figure things out.

I think it's worth copying @angerman's comment from the PR here:

Let ghc0 be our bootstrap compiler; and libs0 the libraries shipped with the bootstrap compiler. bootstap0 are the libraries we built with ghc0 to augment libs0 as we deem them to be potentially too old. Further let ghcN, libsN be the respective binaries and libraries built in Stage N, by the Stage (N-1) binaries and libraries.
The fundamental reasoning was the following:

ghc0 + libs0 + bootstrap0 => libs1, ghc1.
ghc1 + libs1 => libs2, ghc2.

For cross compilers building ghc2 or infect any other tool for stage 2 makes no sense, as they would be cross compiled compilers, not cross compilers. That is, for ghc1 it will run on a different host than the one it most likely targets. ghc2 build by the same logic would however run on the target and as such not be a cross compiler anymore.
The, we do need libs2 as those are the libraries against which we will link with ghc1; but we do not need (or even want) ghc2 but instead want ghc1 in it's place.
Does that make sense?
NB: we might want to support the case where you build a cross compiled GHC as well; that is instead of copying ghc1 into stage2, we would build ghc2.

And below is my response:

@angerman Thanks! I'm assuming all the above messages are the same, so responding just to the last one :)

The fundamental reasoning was the following:

ghc0 + libs0 + bootstrap0 => libs1, ghc1.
ghc1 + libs1 => libs2, ghc2.

I'm trying to translate your notation to what should happen in Hadrian. Is the following correct?

ghc0 is what we call Stage0 GHC in Hadrian. It comes with libs0.
ghc1 is Stage1 GHC, which should be _build/stage1/bin/ghc after this PR.
ghc2 is Stage2 GHC, which should be _build/stage2/bin/ghc after this PR.
bootstrap0 live in _build/stage0/lib both before and after this PR.
libs1 live in _build/stage1/lib both before and after this PR.
libs2 live in _build/stage2/lib both before and after this PR.

@alpmestan @angerman @bgamari It appears there is some confusion about Stage2 libraries, which are supposed to go into the directory _build/stage2/lib both right now and after the proposed restructuring.
I think we never built Stage2 libraries before. Make doesn't build Stage2 libraries by default, and Hadrian never built them either.
Here is my analysis of the situation:
Q1: How does Hadrian manage to build Stage2 packages such as Haddock then?
A1: Because we point it to the Stage1 package database via the command line.
Q2: Stage2 GHC does work when executred from the command line after my PR #704. Why?
A2: Because there are no Stage2 libraries, and GHC's path-finding logic cannot figure out that it should look for these libraries in the ../../stage1/lib folder.
Q3: How did Stage2 GHC work previously?
A3: It worked because previuosly it was located in _build/stage1/bin. Therefore it the built-in path-finding algorithm could locate Stage1 libraries in ../lib directory.
Q4: How does Make-built Stage2 GHC work?
A4: In Make, all GHC executables live in the same directory inplace, where we also build the Stage1 library. Both Stage1 and Stage2 GHCs therefore use exactly the same library.
So, if we want to proceed with the restructuring, it appears that we have two options: (1) teach GHC to find Stage1 libraries in the ../../stage1/lib folder; or (2) actually build Stage2 libraries where GHC expects them to be (_build/stage2/lib), which would significantly increase build times.
Neither of the two options looks appealing.

@angerman Yes, this is my understanding too. Yet, I still don't know how to proceed with this issue.
One option would be to simply copy _build/stage1/lib to _build/stage2/lib. It's a bit wierd, but will probably work. In future, this step could be made conditional: when building Stage3 compiler we would actually populate _build/stage2/lib by recompiling all libraries again using the Stage2 compiler.

Hmm. This is a tough situation. Without your PR, the lib path resolution logic hits a wall. But with your PR too. In the first case, well this whole integer-simple business showed us that our stage 1 compiler is a bit messed up, specifically because of the recent integer-wired-in commit and our layout. In the second case, it's stage 2 that's bitten by this.
This indeed either calls for a better lib path resolution logic or a more appropriate layout. We can do anything we want there, as long as it matches with how GHC works. It seems the current separation isn't working all that well. Stage 1 and stage 2 libs should be the same when we're building a stage 2 GHC, but not when building stage 3. The bootstrap compiler comes with its own libs and builds some more. The binaries are more easily classified into stages and the main issue with them is actually only one with ghc and perhaps ghc-pkg: either the stage 1 (without your PR) or stage 2 (with your PR) GHC get the default/global package db path wrong.
Is there a layout we haven't considered that would get us unstuck?

@alpmestan do you have a link for the integer-wired-in commit?
So this brings us back to what I alluded to on IRC, that this might be a special case for the stage1 compiler.
So we could hardcode the package path in hadrian, but that would mean that anyone calling GHC form the source tree would need to pass the same flags as hadrian, am I right?

@angerman https://github.com/ghc/ghc/commit/fc2ff6dd7496a33bf68165b28f37f40b7d647418

I think the PR #704 makes things worse, because:

We still have issues with path-resolution logic, arguably more serious ones: while the integer-simple problem seems to be just a thorny edge-case, a completely broken Stage2 GHC is not easy to fix without some dubious hacks. Also, the Stage3 GHC story becomes more complicated.
In my (relatively isolated so far) opinion, the layout becomes more complicated too. I still prefer a simple rule when everything build in StageN is located in directory _build/stageN.

To elaborate on the above Stage3 GHC remark: with the current layout, building a Stage3 GHC is straighforward -- just run another build stage and everything will fall into right places.
I think #704 just highlighted the hidden complexities of other layouts.

What solution would you like hadrian to go for?

I think I'd just fix GHC's compareByPreference so that it doesn't "resolve" integer-simple to integer-gmp.
This should solve this particular issue.
After that, we could create two new tickets:

You mentioned that you'd like to be able to run Stage1 GHC from command line without pointing it to the right package database with a flag. I don't know how to address this yet, but I think it's not as high-priority as fixing integer-simple.
There are concerns that the current directory layout may be surprising in the sense that Stage1 GHC lives in the directory _build/stage0. We can form a community opinion about this, and plan how to address a potential restructuring in a separate ticket.

@alpmestan What do you think?

You mentioned that you'd like to be able to run Stage1 GHC from command line without pointing it to the right package database with a flag. I don't know how to address this yet, but I think it's not as high-priority as fixing integer-simple.

The usual trick is to use wrapper scripts for that I think, but I really like the idea of being able to use the executables right there directly and reserve the wrapper scripts for the situations that really, really call for them. They bring their own complexity.
Now, regarding the suggestion to fix compareByPreference, I certainly get behind that, as I think the current implementation just leaves too many doors open. Regarding the layout etc, I don't really have a great idea here, staged compilation just keeps screwing with my mind.

The usual trick is to use wrapper scripts

Actually, I think it may indeed be quite convenient to create something like top-level ghc-stage1.sh and ghc-stage2.sh scripts which invoke corresponding binaries from the _build directory with the right flags.
This would isolate GHC developers from changes in build layout and associated issues with flags etc.

Now, regarding the suggestion to fix compareByPreference, I certainly get behind that, as I think the current implementation just leaves too many doors open.

Great, then I guess we can close my PR for now.

Actually, I think it may indeed be quite convenient to create something like top-level ghc-stage1.sh and ghc-stage2.sh scripts which invoke corresponding binaries from the _build directory with the right flags.

I believe those would need to live in _build; we still have a configurable root right?

I submitted the compareByPreference patch on phabricator: https://phabricator.haskell.org/D5266

I believe those would need to live in _build; we still have a configurable root right?

@angerman Yes, the build root is configurable, so these scripts should probably indeed live in the build folder (and should therefore be built by Hadrian).
@alpmestan Awesome, thank you! I've closed #704. Please also close this issue when your GHC patch lands.

I can finally say it: fixed by ghc/ghc@86ee74dc999db6030874ddaf5d10adec23108c02 :-)

@alpmestan Many thanks :tada:


Publish scala 3 version?

To ? https://search.maven.org/search?q=zio-saga

Hey, Andrew! Thanks for the concern, had a few issues with the release, but now it's done! Please see:
https://search.maven.org/artifact/com.vladkopanev/zio-saga-core_3/0.5.0/jar

Yay!


consider renaming applicative to function everywhere

for symmetry
op -> operative
fn -> function
defop -> define operative
defn -> define function

I think this is fine now that the term function is defined (2b4aab275a6b6a85a872c553ab0580eafcf2e2f1); there's a subtle distinction between applicative and function in that functions specifically take a list of arguments, whereas an applicative takes any value (...but in practice it's always a list).

combiner: any value which can be combined with another value to return a different value
operative: a combiner which receives unevaluated operands and the caller's dynamic scope
applicative: a wrapped combiner which evaluates its operands before passing them to the underlying combiner (typically an operative)
function: an applicative which takes specifically a list of arguments


Postgresql changes for single and flexible

terraform commands are creating the database workbook properly but I am receiving a syntax error after the ./deploy-workbook.sh -s 08b0af8b-8743-42cc-b219-e440a64f67a1 -g postgresql.  I think the workbook is correct now using the new extend kql commands.

I've updated MySQL part, so, I'd like you to revise PostgreSQL part based on it.
Please make sure the following items:

[ ] 1. Move the Summary part under Single pane.
[ ] 2. For Summary part, only Flexible.
[ ] 3. Add notification about Single Server being retired.
[ ] 4. Adjust column MySQL to ether Single and Flexible.
[ ] 5. Make sure burstable SKU is production SKU

cf MySQL.:

@tsubasaxZZZ for the question on Burstable if this would be considered production, I wouldnt think so as it is triggered when you change the server to be development.

I changed my label from GeoRedundantBacup to Geo Backup and changed Zone to "AvailabiltyZone".  Some labels have spaces and others dont have spaces so not sure the pattern there.
As far as the labels on the columns and order of them. I have reordered Zone to be in the same order.  If you want more changed please specify exactly the column and name you would like it to be. I have tried to match as many as I see that makes sense for the service.
Added area for warning for Single server to point to  https://learn.microsoft.com/en-us/azure/postgresql/single-server/whats-happening-to-postgresql-single-server
I moved the summary to be under Single Server, and changed the query so it is only including Flexible server now.

@tsubasaxZZZ

@tsubasaxZZZ pushed my changes up, let me know if you see something else. For the summary area, I am not sure there are other properties to add, if you see something please let me know.

@wjpigott Thank you for your update, and I reviewed your update again.

PostgreSQL part is not considered replica. And it need to fix small update. I attach updated MySQL tab, so, please refer to this by importing it.

FTA - Reliability Workbook - v2.9.1 - 0821-databases (1).txt

The following files should be remove to commit.

Added replication role and section in the summary. I am not sure what to summarize on if it is number of primary or ?

These changes were updated in GIT as well so should be good for you to look at. @tsubasaxZZZ

All recommendations added except the one with the backup image as I am not sure what the goal is there. @tsubasaxZZZ

@tsubasaxZZZ changes are complete for backup and other suggestions. Thank you.

I don't use the editor as you are showing only after deploying it, so might have to learn how to do that :-)
So for small to Full I see that is changing from a 1 to a 3.
"size": 3,
"title": "PostgreSQL - Flexible",
I think I have the icons set to unknown for the gray icon with (?) and as a 2 for the orange triangle.
From past conversations you said not to put any icons on Replication Role, so those are not set for me.
Environment is still colors if you want that to be this way or not.
I didnt have anything as unavailable to test with but set it properly I think.

These changes are pushed up.

@tsubasaxZZZ  also, how did you get the replication role to say geoasyncreplica vs.   "replicationRole": "AsyncReplica" as in the JSON output of the postgresql replica?

On the GeoAsyncReplica I found a way to make this work by doing this in one line with !contains = "AsyncReplica" this seems to work. I had to change the sum on the GeoBackup to look for "not configured" vs. "disabled" as well.

@tsubasaxZZZ

@tsubasaxZZZ  GeoBackup property has been changed to state disabled when the backup is not configured and not contain asyncreplica. Thanks!

Change has been pushed up for changing disabled for GeoBackup summary.

@tsubasaxZZZ

@wjpigott Could you revert to build/template_kql/storage/storage_account_resources_details_summary.kql?
You need to consider that Storage Accounts are deployed whether or not they are in the 3+0 region.
You don't need to modify build/template_kql/storage/storage_account_resources_details_summary.kql. And you need to fix condition of the part: https://github.com/Azure/reliability-workbook/blob/e4ee6edc13e0d709cfed73feb9970dc8ccc4a3d9/build/template_kql/common/extend_resource.kql#L311C1-L315C11
Please add 3+0 region to in statement. You can check the target region the following document:
https://learn.microsoft.com/en-us/azure/reliability/cross-region-replication-azure#regions-with-availability-zones-and-no-region-pair
And please create new PR to fix this first to identify other issue.

@wjpigott Could you revert to build/template_kql/storage/storage_account_resources_details_summary.kql? You need to consider that Storage Accounts are deployed whether or not they are in the 3+0 region.
You don't need to modify build/template_kql/storage/storage_account_resources_details_summary.kql. And you need to fix condition of the part: https://github.com/Azure/reliability-workbook/blob/e4ee6edc13e0d709cfed73feb9970dc8ccc4a3d9/build/template_kql/common/extend_resource.kql#L311C1-L315C11 Please add 3+0 region to in statement. You can check the target region the following document: https://learn.microsoft.com/en-us/azure/reliability/cross-region-replication-azure#regions-with-availability-zones-and-no-region-pair
And please create new PR to fix this first to identify other issue.

@tsubasaxZZZ  This was reverted back and pushed up. I will let the storage owner make the other changes to ZRS to make this acceptable when time permits. Thanks!


Issues with SELinux on Fedora 29.

Similar to #567, #437.
The dnscrypt-proxy package for Fedora 29 currently does not work out of the box due to SELinux. I found that another user that posted a command that fixes it:
sudo ausearch -c '(pt-proxy)' --raw | audit2allow -M my-ptproxy

@eclipseo
Do you still maintain this package? Can we look at perhaps getting a fix so users don't need to worry about running this?

Yes I maintain it. I have reported the SELinux issue upstream: https://github.com/fedora-selinux/selinux-policy/issues/231 but no action has been taken yet.
I'll push an update with a workaround for now.


TSLint Airbnb

Is there any way of integration with Airbnb TS Styleguide?

What exactly do you mean, @mauriciord? Airbnb's JavaScript Style Guide is, as the name suggest, intended for JavaScript and not TypeScript. I don't think there is a TypeScript version. So by that regard the answer would be "no". That being said, since the two languages are so similar, most of the recommendations in it also apply to TypeScript, and I actually think that the code in this repository already mostly follows the guide. Do you have anything particular in mind?
If you are asking if [tslint-config-airbnb](https://github.com/progre/tslint-config-airbnb/#readme] could be turned on, assume that the answer is "yes". I recently switched from TSLint to ESLint, though, since TSLint is being deprecated, but that doesn't block you to going to back to using TSLint.

Does this answer your question? Can I close this issue, @mauriciord?

Yep @janaagaard75


fix Fix start_fe.sh missing METADATA_FAILURE_RECOVERY with foreground (#31258)

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall


🐛 BUG: workers.api.error.wrong_plan for publish / dev workers

Which Cloudflare product(s) does this pertain to?
Wrangler
What version of Wrangler are you using?
2.9.0
What operating system are you using?
WSL
Describe the Bug
I am logged to the wrangler via wrangler login on the account : ACCOUNT_PRIVATE with an ID let's say 123 and I have free version of workers
I'm also added to the 'ACCOUNT_WORKER' with an ID let's say 456 and with he Enterpraise plan + unbounded workers.
I want to publish a worker to the  ACCOUNT_WORKER
Mine wrangler.toml:
name = "test"
main = "src/index.ts"
compatibility_date = "2023-02-03"
account_id = "456"
usage_model = "unbound"
workers_dev = true

After running the wrangler publish I'm getting:
wrangler publish
 ⛅ wrangler 2.9.0
-------------------
Total Upload: 0.25 KiB / gzip: 0.17 KiB

✘ [ERROR] A request to the Cloudflare API (/accounts/xxx/workers/scripts/xxx) failed.

  workers.api.error.wrong_plan [code: 10075]

  If you think this is a bug, please open an issue at:
  https://github.com/cloudflare/wrangler2/issues/new/choose

I can add new workers through website and also through direct api calls... I think the problem lies somewhere on the calls that Wangler does

Hi, how did you manage to fix this? I have the same problem, and using the same System "WSL"

Thanks!!!


[WIP]add scheduler config and support out-of-tree scheduler plugin

What type of PR is this?

support scheduler configurable
support add custom plugin

What this PR does / why we need it:
Which issue(s) this PR fixes:

Fixes #281
Special notes for your reviewer:

Codecov Report

Merging #498 (eb707ac) into main (76b0dd0) will increase coverage by 0.02%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##             main     #498      +/-   ##
==========================================
+ Coverage   13.07%   13.10%   +0.02%
==========================================
  Files          65       64       -1
  Lines        7081     7065      -16
==========================================
  Hits          926      926
+ Misses       6077     6061      -16
  Partials       78       78

Impacted Files
Coverage Δ

...m/clusternet/clusternet/pkg/scheduler/scheduler.go
4.07% <0.00%> (-0.28%)
:arrow_down:

...ternet/clusternet/pkg/scheduler/default_plugins.go

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

/hold

/PTAL @dixudx


How to use with multiple service workers

BEFORE YOU SUBMIT please read the following:

[ ] I'm submitting a bug report
[ ] I'm submitting a feature request
[X] I'm submitting a support request

webpack version:
2.2.1
sw-precache-webpack-plugin version:
0.9.1
I have an additional service worker at the moment, that handles push notifications. How can I combine the generated one which handles the fetches and my current one?

Friendly ping to @goldhand :)

How about using importScripts?

Jep just saw it. Closing...


Log Level Support

Set default log level to 'warning'
Respect log level as configured from UI

PR: https://github.com/UniversalDevicesInc/polyglot-v3-nodejs-interface/pull/10

Allows setting of logLevel from UI
Sets logLevel based on logLevel value in server.json
Default logLevel is set to 'warn'

PR was accepted


chevrotrain parser plugin

Goal:
Alternative parse to graphql-js that is faster
Benchmarks that show how it is faster (CI)

Resources:
https://github.com/Chevrotain/chevrotain/tree/master/examples/grammars/graphql

With next major you should be able to pass in custom parse function https://github.com/n1ru4l/envelop/pull/1487 but the issue is that AST from chevotrain is not same as graphql-js so user would also need to create some sort of transform wrapper and then not sure how much difference one would get. A lot of benefit in prod servers by simply using useParserCache plugin


positioning doesn't work with custom template starting with a comment

I have a positioning issue when using custom template that starts with a multi-line comment like that:
<!--
comment
blabla
-->

Notifications are always display on the top left:

positionX parameter is ignored
I haven't test with positionY but it's probably ignored as well
if I have several notifications, instead of being visible one after the other (as a list), only the last one is visible and I have to close it to see the previous one.

Show your code, please.


Investigate missing suppliers on phase-1 keycloak

The phase-1 keycloak workflow has 14,530 / 23,683 have supplier names. We need to understand and investigate why this is not more.
SBOM Quality by Interlynk Score:8.5	components:23683	./enriched-keycloak-sbom-cyclonedx/enriched_keycloak-sbom.cdx.json
+-----------------------+--------------------------------+-----------+--------------------------------+
|       CATEGORY        |            FEATURE             |   SCORE   |              DESC              |
+-----------------------+--------------------------------+-----------+--------------------------------+
| NTIA-minimum-elements | comp_with_name                 | 10.0/10.0 | 23683/23683 have names         |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_supplier             | 6.1/10.0  | 14530/23683 have supplier      |
|                       |                                |           | names                          |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_uniq_ids             | 10.0/10.0 | 23683/23683 have unique ID's   |
+                       +--------------------------------+-----------+--------------------------------+
|                       | comp_with_version              | 9.9/10.0  | 23513/23683 have versions      |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_authors                   | 10.0/10.0 | doc has 2 authors              |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_creation_timestamp        | 10.0/10.0 | doc has creation timestamp     |
|                       |                                |           | 2024-10-01T05:39:04+00:00      |
+                       +--------------------------------+-----------+--------------------------------+
|                       | sbom_dependencies              | 10.0/10.0 | doc has 63307 relationships    |
+-----------------------+--------------------------------+-----------+--------------------------------+

this might be unrelated bc it's SPDX instead of Cyclone, but I noticed this in the Keycloak "Augment Keycloak SPDX" step too:
{"level":"info","ts":1727921916.299821,"caller":"edit/spdx_edit.go:68","msg":"SPDX error updating supplier: not supported"}
{"level":"info","ts":1727921916.2998703,"caller":"edit/spdx_edit.go:68","msg":"SPDX error updating repository: not supported"}


Segmentation fault when using createCaffeImporter

Hi,
I follow this example (http://docs.opencv.org/master/d5/de7/tutorial_dnn_googlenet.html#gsc.tab=0) and got Segmentation Fault error. This issue is already reported in issue #383 and fixed in #391 by @rokm. However, after merged the code from #391 I still get the same error.
This happens even with this simple code
test_dnn.cpp:
#include <opencv2/dnn.hpp>
using namespace cv;
using namespace cv::dnn;

int main (int argc, char **argv)
{
       Ptr<Importer> importer = createCaffeImporter("bvlc_googlenet.prototxt", "bvlc_googlenet.caffemodel");
       return 0;
}

g++ test_dnn.cpp -o test -L/usr/local/lib -I/usr/local/include -lopencv_core -lopencv_dnn

The output from gdb core dumped:
Core was generated by `./test_dnn'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f9677de290f in cv::dnn::LayerFactory::registerLayer(cv::String const&, cv::Ptr<cv::dnn::Layer> (*)(cv::dnn::LayerParams&)) ()
   from /usr/local/lib/libopencv_dnn.so.3.0
Traceback (most recent call last):
  File "/usr/share/gdb/auto-load/usr/lib/x86_64-linux-gnu/libstdc++.so.6.0.19-gdb.py", line 63, in <module>
    from libstdcxx.v6.printers import register_libstdcxx_printers
ImportError: No module named 'libstdcxx'

Hey,
can you rebuild OpenCV with debugging symbols enabled (e.g., -DCMAKE_BUILD_TYPE=RelWithDebInfo) to get more detailed backtrace?

This is weird, I rebuild everything again and the problem is gone. Thanks again for your fix @rokm.


Websocket live feed stops after running for 30 mins

https://community.thefirstock.tech/t/websocket-live-feed-stops-after-running-for-30-mins/89


pip-tools unable to find package in custom index url.

Pip tools is not able to find a package in the given index url.
Environment Versions

OS Type
Python version: 3.6.7
pip version:  19.3.1
pip-tools version: 4.3.0

Steps to replicate
We have a private server similar to PyPi to store our python packages. I am trying to install a package from the private server which is given as an argument to pip-tools as --index-url
Expected result
A requirements.txt file containing the package and its sub dependencies
Actual result
pip._internal.exceptions.DistributionNotFound: No matching distribution found for package==version (from -r requirements.in (line 1))

Hello @sathishbabu96,
Thanks for the issue! If you attach the output of pip-compile -v, we'll try to figure out what's wrong. Do redact sensitive data if it's needed.

Yea, here's the output for the command pip-compile -v
`Using indexes:
http://private-server
Configuration:
-f http://private-server
                      ROUND 1

Current constraints:
cherrypy==13.1.0 (from -r requirements.in (line 1))
Finding the best candidates:
found candidate cherrypy==13.1.0 (constraint was ==13.1.0)
Finding secondary dependencies:
cherrypy==13.1.0          requires cheroot>=5.9.1, portend>=2.1.1, six>=1.11.0
New dependencies found in this round:
adding ['cheroot', '>=5.9.1', '[]']
adding ['portend', '>=2.1.1', '[]']
adding ['six', '>=1.11.0', '[]']
Removed dependencies in this round:
Result of round 1: not stable
                      ROUND 2

Current constraints:
cheroot>=5.9.1 (from cherrypy==13.1.0->-r requirements.in (line 1))
cherrypy==13.1.0 (from -r requirements.in (line 1))
portend>=2.1.1 (from cherrypy==13.1.0->-r requirements.in (line 1))
six>=1.11.0 (from cherrypy==13.1.0->-r requirements.in (line 1))
Finding the best candidates:
Could not find a version that matches cheroot>=5.9.1 (from cherrypy==13.1.0->-r requirements.in (line 1))
No versions found
Was http://cm-server/python_downloads/ reachable?`
The package cheroot-6.0.0 is available in our server.

@atugushev Any update on this?

@sathishbabu96 hard to tell what's wrong by the given log. More questions:

Are you sure about "Was http://private-server/ reachable?"?
Could you CURL the http://private-server/simple/cheroot/?
Do you see cheroot-6.0.0 sources/wheel on http://private-server/simple/cheroot/?
Could you install cheroot directly via pip install cheroot --verbose --index-url http://private-server/? What was the output?

I was facing this issue in macOS, when I tried the same in Linux, everything worked perfectly. The package was not available for macOS version, hence I was facing this issue. I am closing the issue. Sorry for the inconvenience caused!.


Implement local Katana integration

Ensure that user can connect to a local Katana instance
(instead of is_alive use "version" endpoint)

Checklist

[X] Modify plugin/src/components/DevnetAccountSelector/index.tsx ✓ https://github.com/NethermindEth/starknet-remix-plugin/commit/7296dc8f83bb22a5c15ef7983f27de023ce90aca
[X] Check plugin/src/components/DevnetAccountSelector/index.tsx ✓ https://github.com/NethermindEth/starknet-remix-plugin/commit/d92ee01c60d708844bc7a7aae5e62e972b9c20ee
[X] Modify plugin/src/components/EnvironmentSelector/index.tsx ! No changes made

@rjnrohit Does the pr this bot created look correct?

The Katana project is managed from here: https://github.com/dojoengine/dojo/blob/main/crates/katana/README.md


Building with scala 3

Fixes #341

I added a fix for the self-type issue but the tests don't compile cleanly any more, mostly because the new version of specs is not syntax compatible with the old.

I added a fix for the self-type issue

Nice!

but the tests don't compile cleanly any more, mostly because the new version of specs is not syntax compatible with the old.

Ah, we can use a cross-dependency for the tests https://github.com/etorreborre/specs2/issues/848

Cool, I fixed all the compilation issues but now it seems that

some printing facility changed, now escaping unicode characters
ProductFormats don't work any more, probably because Scala compilation changed the way fields are encoded

ProductFormats don't work any more, probably because Scala compilation changed the way fields are encoded

Ultimately, we would want to use derivation in Scala 3, but I guess it would be nice to support it in a simpler way until then?

Fixed the remaining issues, was pretty trivial after all.

/cc @jrudolph this might now be good to go?

Is this dead?


All children tabs remain deselected after their parent is disabled and enabled

Description
All vaadin-tab elements will remain deselected after their parent element (vaadin-tabs) is disabled and enabled
Expected outcome
The previously selected tab should be re-selected, after its parent is re-enabled
Actual outcome
All tabs are visually deselected
Live Demo
Tabs tabs = new Tabs();
tabs.add(new Tab("asd"));
tabs.add(new Tab("asd2"));
tabs.add(new Tab("asd3"));

add(tabs);

Button btn = new Button("Test");
btn.addClickListener(e -> {
  tabs.setEnabled(!tabs.isEnabled());
});

Steps to reproduce

Create a vaadin-tabs element
Add at least one vaadin-tab child element
Disable the vaadin-tabs element
Enable the vaadin-tabs element

Browsers Affected

[x ] Chrome
[ x] Firefox
[ x] Safari
[ x] Edge
[ x] IE 11
[ x] iOS Safari
[ x] Android Chrome

It still happens in Vaadin 14.6.1. The selected attribute is removed from vaadin-tab after vaadin-tabs is disabled. The value of getSelectedIndex()/getSelectedTab() on the server side are not affected.
As a workaround, set the selected attribute again after disabling the tabs.
tabs.setEnabled(false);
int selectedIndex = tabs.getSelectedIndex();
if (selectedIndex>=0) {
   tabs.getElement().executeJs("this.children[$0].selected=true", selectedIndex);
}


Updates istio to v1.13. Kube to v1.23. cert-manager to v1.7

GetRootCertBundle is now a member of security.Client interface, and so is now implemented as part of our "dummy" certmanagerClient client.
https://github.com/istio/istio/blob/284def0c93cc2dc37b07a1d7acd74f26eb24fb63/pkg/security/security.go#L270

/retest


examples/decrypt-cenc: multiple keys

I came across a video with multiple keys:
IV:d805bba9ee360871c54f430e5de9486b key:784f6b59536e7078594f653959715a49 type:2
IV:2890e5c5feb02fb5ec3e6ea4e0758bc6 key:5062524e343151366950744d7a4b4e65 type:2
IV:dc7e67eb7d9b6474fc9914ddeddb9493 key:58526c32716a30586c6c7349466f3639 type:2
IV:a870d9a9b8abd4b8add8c7eec1ac7f8e key:703439466a586a757235576a5069754b type:2
IV:c889acba1c740bb6f28dca890c5980a2 key:4f3532724142445476307876686d7347 type:2

does this module or tool support multiple keys? I can provide video sample if need be.

The example could be enhanced to support multiple keys, but right now it doesn't.

I think it was an error on my part. the license server I was working with, responded with multiple keys.
this has always been the case, but in this situation is was multiple CONTENT keys, where in the past it was only one content key. I resolved this by parsing the PSSH and extracting the Key ID, then comparing that against the IDs returned in the license response. this is working so far, but time will tell.


IOT Cloud Improvements

We should have a clear distinction of Cloud Gateways as Hybrid (Container Ready) and SaaS offering.

Would you help us do a example PR?

Would you help us do a example PR?

Sure.

thanks


Add audit logging of data modify/delete actions

This creates a separate audit log in JSON format
to record the date range/no. of datapoints for
modification actions and the name of the dataset
for modify and delete actions.

So it turns out it's surprisingly hard to unsubscribe from everything after leaving an organisation! Hi everyone!

Hi @paulfurley!  :wave:
https://github.com/watching might be handy

Bosh, thanks @jabley, don't know why I couldn't find that myself! Over and out :)


Correct typo in MIGRATION.md

It seems like the function completstream_asynce_async doesn't exist

Thanks for your report I added the change in https://github.com/mistralai/client-python/pull/144


[Style] Remove trailing spaces

Prevent trailing spaces in text files is almost kind of the most common styling rule in many coding style.
I can also help add the check in CI after the PR was accepted ;)

Coverage remained the same at 95.192% when pulling 3b30c41cd5334b88f6ff55ecc92a4431092d8cde on PeterDaveHello:style/RemoveTrailingSpaces into b9f7727ebdb877dab69e44f0f4f4d17474f1e72a on cla-assistant:master.


Fix json-schema type of objects nested under arrays. Fixes #400

Fix for https://github.com/dry-rb/dry-schema/issues/400
Objects nested under arrays are not neccesailry arrays themselves - the :member option shouldn't be passed down the AST when generating json-schema.
For example for the test schema https://github.com/dry-rb/dry-schema/blob/5853103d6849490ae8ba94158f73631c1fd121a8/spec/extensions/json_schema/schema_spec.rb#L28-L34
the result of the fix is:
 {
   "$schema": "http://json-schema.org/draft-06/schema#",
   "type": "object",
   "properties": {
     "email": {
       "type": "string"
     },
     "age": {
       "type": "integer"
     },
     "roles": {
       "type": "array",
       "items": {
         "type": "object",
         "properties": {
           "name": {
             "type": "string",
             "minLength": 12,
             "maxLength": 36
           },
           "metadata": {
-            "type": "array",
-            "items": {
-              "type": "object",
-              "properties": {
-                "assigned_at": {
-                  "type": "string",
-                  "format": "time"
-                }
-              },
-              "required": [
-                "assigned_at"
-              ]
-            }
+            "type": "object",
+            "properties": {
+              "assigned_at": {
+                "format": "time",
+                "type": "string"
+              }
+            },
+            "required": [
+              "assigned_at"
+            ]
           }
         },
         "required": [
           "name",
           "metadata"
         ]
       }
     },
     "address": {
       "type": "object",
       "properties": {
         "street": {
           "type": "string"
         }
       },
       "required": [

       ]
     },
     "id": {
       "anyOf": [
         {
           "type": "string"
         },
         {
           "type": "integer"
         }
       ]
     }
   },
   "required": [
     "email",
     "roles",
     "id"
   ]
 }

Hey @tomgi just asked about this issue on the zulip chat.
Tried monkey patching your solution on my project and it seems to be working perfectly.
@solnic is there any plans to merge this PR?

Following up on this -- @solnic would merging it be possible? The bug is still in place and this fix is a good one.

I'm sorry it took so long, I'll push a release, thanks for addressing it

Awesome work. Thanks for dedicating your time to fix this :heart:


New release won't install

When installing via NuGet, this is the reaction of the download/installation process:
Error:     Could not install package 'Microsoft.IdentityModel.Clients.ActiveDirectory 3.10.305231913'. You are trying to install this package into a project that targets  '.NETPortable,Version=v4.6,Profile=Profile32', but the package does not contain any assembly references or content files that are compatible with that framework. For more information, contact the package author.
Is there a way to get the latest version installed anyhow? What other (older) version is recommended?

Are you trying to install the package into the sample code, or into your own project?

I'm trying to install in a project of my own. That project is a class library (dll) with WIN8.1/Phone 8.1 as targets. In the mean time I have (at least I think I have) succesfully "installed"  by adding the references to Microsoft.IdentityModel.Clients.ActiveDirectory and Microsoft.IdentityModel.Clients.ActiveDirectory.Platform. both in the 3.9.302261508-alpha version. Can you confirm that this is a reliable way to go as well? Is it an option to upgrade to a later version?

This now was updated with recents ADAL.NET versions. and this works fine.


Some phones doesn't focus

I've issue, that some phones doesn't auto-focus in LiveStream (some Android phones like Samsung Duos).
Unfortunatelly, this issue comes from user beta testing, I'm not able to reproduce it on my phones, but they wrote that camera auto-focused correctly in any other application.
I'm thinking this problem should be caused by constrains: width and height.
Is there possibility to detect which device should use which resolution? Or it doesn't matter? Or is important to detect at least ratio of width x height? Or any other idea?
Edit: My web app needs to be responsive, so I've set width/height to 1280x720 and set width of canvas to 100% (according to width of container). Should it influence scanner? If yes, should be better to use css transform: scale/matrix?
Thank you

Unfortunately there is nothing you can do to trigger the auto-focus of the camera via the WebAPI. You have to rely on the phone/browser combination to work correctly. The only suggestion I can give you is to request a higher resolution image, but crop it and so the user can move further away from the barcode. This should help with the auto-focus issues.

autoFocus is available as a constraint these days, although my limited experience is that Android Chrome isn't very good about honoring it, even when you make it a required constraint.
The LG G5 and later have multiple camera identifiers, and the default camera selection is a wide angle camera that doesn't seem to have auto-focus capability.


Report Problem Button

a link titled ‘Report Problem’; clicking it would trigger a screen capture and a popup that allowed the user to send a quick message which was sent to a group email account (e.g. -<EMAIL_ADDRESS>or<EMAIL_ADDRESS>

Generate a separate iobio page with instructions on how to submit to github issues.


Feature category+banner 2023 10 08

finish add banner component to app!

ok

finish merge branch!


preparations for js-ipfs 0.34

👋 I upgraded ipfs to 0.34.0-rc.0 and ran the tests - they failed :(
  1) app swarm
       each node connections are bounded:

      AssertionError: expected 12 to be at most 10
      + expected - actual

      -12
      +10

      at inboundConnectionCounts.forEach (webpack://peerbase/test/app-swarm.spec.js:98 <- node_modules/aegir/src/config/karma-entry.js:397581:31)
      at Array.forEach (<anonymous>)
      at Context.it (webpack://peerbase/test/app-swarm.spec.js:97 <- node_modules/aegir/src/config/karma-entry.js:397580:29)

https://github.com/ipfs/js-ipfs/issues/1721

@alanshaw Thank you for this!
This was a problem that apparently happened on some tests, and was fixed by #200.
I've installed 0.34.0-rc.0 locally and all the tests are passing consistently.
Thank you!


[POC] refactoring tests

Context
The purpose of this ticket is primarily to allow the developers of demcompare to become familiar with the xdem tool in order to prepare for the refinement of future tasks, including, for example, dem_processings, classification, or statistics.
As part of the federation of DEM comparison tools xDEM and demcompare, we propose this initial ticket to refactor the unit tests and consolidate similar lines in a conftest file. This ticket primarily has an educational purpose. Indeed, mastering the unit test files will allow the main developers of the federation to better understand the architecture of xDEM while offering an initial AGILE implementation.
Tasks

[ ] Add a conftest file where we will mainly store the load functions
[ ] Pass the test folders through all xDEM's quality tools
[ ] Move the handling of affine and CRS parameters to the conftest
[ ] Create a helpers.py file containing the necessary functions for the tests

For the "coreg" folder

[ ] Add a conftest.py file

test_affine.py

[ ] Move the load_examples function to the conftest
[ ] Move the functions from the class to the conftest
[ ] Remove the class and call the functions from the conftest

test_base

[ ] Retrieve load_examples and class variables from the conftest
[ ] Create a file per class or possibly a folder test_base_test_CLASS.py
[ ] Move assert_coreg_meta_equal to helpers

test_biascorr

[ ] Move the load_examples function to the conftest
[ ] Move the functions from the class to the conftest
[ ] Remove the class and call the functions from the conftest

Estimate 3d

@adehecq @duboise-cnes If you agreed for the coreg folder, i will complete this issue for the entire tests folder

Ok for the principle to begin on xdem tests to "feel" the code and be able to understand better xdem for integrating demcompare-like cli. Be aware to think how we can plug coreg, dem_processing, classif, stats (maybe more important) functional parts of a demcompare like CLI during this educational step. This purpose could be added to the issue explicitely.
I let @adehecq say if ok with the approach of refactoring to xdem.
For my point of view
Don't try to be exactly like demcompare. Interesting to take inspiration from pandora2d too which is newer as pytest organization. Helpers can be named another way. Maybe also important to converge on a test structure more clearly described (schema, simple refactoring architecture from existing functions) with amaury in the issue before coding ? Also use pytest fixtures is the key of factorization i think (not only conftest or helpers like module to aggregate functions), I would have put it explicitely also in description....
My 2cts for this issue.

I let @adehecq complete if necessary, but I think I can say that this refactoring also makes great sense to us! :slightly_smiling_face:
Same as @duboise-cnes noted, we had in mind to use pytest fixtures to make the test data loading more consistent than the current redundant implementation, see the issue we opened here and the references we noted especially for this purpose: https://github.com/GlacioHack/xdem/issues/427.
But we're really not experts in how best to do this, we have fairly limited experience and are open to a new structuring.
We have a bit of experience populating directly conftest.py for data download and preparation in other projects. For some reason in xdem we chose early to separate the data download and preparation methods into a separate examples.py module :thinking: (not necessarily the best name, was more oriented towards documentation than test originally). I think the reason might have been to avoid making conftest.py too large a file (@erikmannerfelt can probably comment on that).

I also agree that some improvements can be made in the tests by using conftest.py and fixtures. I'm totally up for restructuring as needed. It's just a bit difficult for me to understand what it involves without a good example, so maybe the best is to do this restructuring for one submodule first and iterate on it before applying the same to other tests?
Or describe a more detailed example of changes in this issue, if the first option is too long.


chore(publish): publish packages on release

What kind of change does this PR introduce?
This is helping us releasing the library in a consistent manner. It will not release automatically, it will release when a github release is created.

What is the current behavior?
A person manually runs the npm publish command.

What is the new behavior?
To publish the package, one now can simply create a release on github, and the package will be published automatically.

Codecov Report

Merging #36 (0f47244) into main (e3e7dbc) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main      #36   +/-   ##
=======================================
  Coverage   80.04%   80.04%
=======================================
  Files           8        8
  Lines         406      406
  Branches       88       88
=======================================
  Hits          325      325
  Misses         44       44
  Partials       37       37

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update e3e7dbc...0f47244. Read the comment docs.


Automated Package Upgrade

:tada: This PR is included in version 21.4.1 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


fix scope issues when replying to PMs

/u/gooeyblob fixed a reddit bug in which the user could not reply to PMs because the /comment endpoint required the submit scope no matter what. Now it checks whether the item is a message and permits the privatemessages scope.
https://www.reddit.com/r/redditdev/comments/3flqqh/praw_incorrect_scopes/ctu84pk

:+1:

:ok_hand:


returnFocus doesn't work in Safari and Firefox

Summary:
Focus returns as expected in Chrome. But it fails in Firefox and Safari.
Steps to reproduce:

Click the button.
Close modal.
The button is not focused (its text should be red).

Expected behavior:
The button is focused (its text is red).
Link to example of issue:
https://codesandbox.io/s/mjy915nqp9
Additional notes:
As I see in returnFocus function, focusLaterElements array consists of <body> in Firefox and Safari instead of <button> (in Chrome).

I want to note that this bug is not from react-modal but from safari and firefox.
It is an expected behavior from the browsers since both do not focus on button elements when you click on them using a mouse (link).
This but is not relevant if a user uses keyboard to navigate through the page. Since when tabbing the buttons are given the focus before you can press on them.
I believe there should be a fine workaround for the mouse users. However manually giving buttons focus for mouse users seems pretty hacky. I am wondering if there is a more elegant workaround.


RepeatMasker parseTagData error

Problem with loading RM database
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 2
Rules claiming more threads will be scaled down.
Job counts:
        count   jobs
        1       repeatmasker
        1

[Sun Dec  2 09:29:30 2018]
rule repeatmasker:
    input: mask/SRR5579964_repeatmasker_3.fa
    output: mask/SRR5579964_repeatmasker_3.fa.masked, mask/SRR5579964_repeatmasker_3.fa.out, mask/S
RR5579964_repeatmasker_3.fa.tbl
    jobid: 0
    wildcards: sample=SRR5579964, n=3
    threads: 2

Activating conda environment: /gpfs/hpchome/taavi74/fastq/prjna361402/.snakemake/conda/5302d956
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
        LANGUAGE = (unset),
        LC_ALL = (unset),
        LC_CTYPE = "UTF-8",
        LANG = "en_US.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to a fallback locale ("en_US.UTF-8").
RepeatMasker version open-4.0.7
Search Engine: NCBI/RMBLAST [ 2.6.0+ ]
parseTagData: ID field not to EMBL spec "Sola3-3_NV repeatmasker; DNA; ???;  BP.
" from DE   RepbaseID: Sola3-3_NVXX

 at /gpfs/hpchome/taavi74/fastq/prjna361402/.snakemake/conda/5302d956/share/RepeatMasker/RepeatMasker line 7611.
Master RepeatMasker Database: /gpfs/hpchome/taavi74/fastq/prjna361402/.snakemake/conda/5302d956/share/RepeatMasker/Libraries/RepeatMaskerLib.embl ( Complete Database: dc20170127-rb20170127 )

[Sun Dec  2 09:29:45 2018]
Error in rule repeatmasker:
    jobid: 0
    output: mask/SRR5579964_repeatmasker_3.fa.masked, mask/SRR5579964_repeatmasker_3.fa.out, mask/SRR5579964_repeatmasker_3.fa.tbl
    conda-env: /gpfs/hpchome/taavi74/fastq/prjna361402/.snakemake/conda/5302d956

RuleException:
CalledProcessError in line 61 of /gpfs/hpchome/taavi74/Projects/vs/rules/mask.smk:
Command 'source activate /gpfs/hpchome/taavi74/fastq/prjna361402/.snakemake/conda/5302d956; set -euo pipefail;
    RepeatMasker -qq -pa 2 mask/SRR5579964_repeatmasker_3.fa -dir mask
    if head -n 1 mask/SRR5579964_repeatmasker_3.fa.out | grep -q "There were no repetitive sequences detected"
      then ln -sr mask/SRR5579964_repeatmasker_3.fa mask/SRR5579964_repeatmasker_3.fa.masked
    fi ' returned non-zero exit status 25.
  File "/gpfs/hpchome/taavi74/Projects/vs/rules/mask.smk", line 61, in __rule_repeatmasker
  File "/gpfs/hpchome/taavi74/miniconda3/envs/snakemake/lib/python3.6/concurrent/futures/thread.py", line 56, in run
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

To fix this, just delete conda environment and let snakemake to install fresh one.


Testing now requires including recorder tests.

Using the homeassistant structure in this package causes a circular import.  Trying to restructure may be complex.

Fixed in #116


DensityMatrix.partial_transpose does not preserve input dimensions

Environment

Qiskit Terra version: to be released in 0.24.0
Python version: 3.10
Operating system: MacOS

What is happening?
The partial_transpose should return a density matrix of the same dimensionality as the input DensityMatrix object. When the dimensions are for systems different than qubits, the dimensions are currently not preserved.
How can we reproduce the issue?
For in the following example, the dimensions of the output density matrix ρ2 are (9,) but they should be (3,3).
>>> import numpy as np
>>> from qiskit.quantum_info import Statevector, DensityMatrix
>>> ψ = Statevector(np.array([1,0,0,0,1,0,0,0,1])*1/np.sqrt(3),dims=(3,3))
>>> ρ1 = DensityMatrix(ψ)
>>> ρ1.dims()
(3, 3)
>>> ρ2 = ρ1.partial_transpose([0])
>>> ρ2.dims()
(9,)

What should happen?
It should be an easy fix. The function should return:
return DensityMatrix(rho, dims=self.dims())

instead of just:
return DensityMatrix(rho)

Any suggestions?
No response

Thanks, Diego.  Would you like to make the PR as well?

Hi @jakelishman. Will do.

Thank you @diemilio for bringing this to our attention


Add back/forward keyboard shortcuts

We need keyboard shortcuts for global back/forward nav. Cmd/Ctrl+arrow key

not sure what modifier to use for Windows. Help seems to indicate that Ctrl and (or?) Alt cannot be used to fire a trigger.

@bcooney93 - Please use custom menus instead of triggers

Done


Upgrade to ES8 locally

Fixes

Fixes #2567
Description

This PR updates our local development stack to use ElasticSearch 8. I've reviewed the ES8 changelog and documentation. It doesn't appear we're using any relevant functionality which was deprecated or altered from ES7-8. Our usage of ES generally is quite simplistic.
The frontend and API both work flawlessly with this ES8 cluster locally. I tried all available frontend filters to ensure queries of all types are working, along with the search syntaxes described in our search guide.
Testing Instructions

Try the API locally; observe that CI passes.
Checklist

[x] My pull request has a descriptive title (not a vague title likeUpdate index.md).
[x] My pull request targets the default branch of the repository (main) or a parent feature branch.
[x] My commit messages follow best practices.
[x] My code follows the established code style of the repository.
[x] I added or updated tests for the changes I made (if applicable).
[x] I added or updated documentation (if applicable).
[x] I tried running the project locally and verified that there are no visible errors.
[x] I ran the DAG documentation generator (if applicable).

Developer Certificate of Origin

Developer Certificate of Origin
Developer Certificate of Origin
Version 1.1

Copyright (C) 2004, 2006 The Linux Foundation and its contributors.
1 Letterman Drive
Suite D4700
San Francisco, CA, 94129

Everyone is permitted to copy and distribute verbatim copies of this
license document, but changing it is not allowed.

Developer's Certificate of Origin 1.1

By making a contribution to this project, I certify that:

(a) The contribution was created in whole or in part by me and I
    have the right to submit it under the open source license
    indicated in the file; or

(b) The contribution is based upon previous work that, to the best
    of my knowledge, is covered under an appropriate open source
    license and I have the right under that license to submit that
    work with modifications, whether created in whole or in part
    by me, under the same open source license (unless I am
    permitted to submit under a different license), as indicated
    in the file; or

(c) The contribution was provided directly to me by some other
    person who certified (a), (b) or (c) and I have not modified
    it.

(d) I understand and agree that this project and the contribution
    are public and that a record of the contribution (including all
    personal information I submit with it, including my sign-off) is
    maintained indefinitely and may be redistributed consistent with
    this project or the open source license(s) involved.

I tried it locally and about 48 tests failed.

@dhruvkb interesting, which tests?

Mostly in the search controller, as to be expected, right?

Here's my local results of just api/test:
❯ just api/test
env COMPOSE_PROFILES="api" just ../up
env COMPOSE_PROFILES="api" docker-compose -f docker-compose.yml -f docker-compose.overrides.yml up -d
[+] Running 8/8
 ✔ Container openverse-cache-1             Running                                                                                0.0s
 ✔ Container openverse-upstream_db-1       Running                                                                                0.0s
 ✔ Container openverse-db-1                Running                                                                                0.0s
 ✔ Container openverse-es-1                Running                                                                                0.0s
 ✔ Container openverse-indexer_worker-1    Running                                                                                0.0s
 ✔ Container openverse-web-1               Running                                                                                0.0s
 ✔ Container openverse-ingestion_server-1  Running                                                                                0.0s
 ✔ Container openverse-proxy-1             Started                                                                                0.4s

================================================================================
                                 Service Ports
================================================================================
webserver (catalog):
-  http://<IP_ADDRESS>:9090  (→ 8080)
ingestion_server (ingestion_server):
-  http://<IP_ADDRESS>:50281 (→ 8001)
plausible (plausible/analytics):
-  http://<IP_ADDRESS>:50288 (→ 8000)
web (api):
-  http://<IP_ADDRESS>:50280 (→ 8000)
-  http://<IP_ADDRESS>:50230 (→ 3000)
es (docker.elastic.co/elasticsearch/elasticsearch):
-  http://<IP_ADDRESS>:50292 (→ 9200)
s3 (minio/minio):
-  http://<IP_ADDRESS>:5010  (→ 5000)
-  http://<IP_ADDRESS>:5011  (→ 5001)
db (postgres):
-  http://<IP_ADDRESS>:50254 (→ 5432)
upstream_db (upstream_db):
-  http://<IP_ADDRESS>:50255 (→ 5432)
cache (redis):
-  http://<IP_ADDRESS>:50263 (→ 6379)
================================================================================
just ../ingestion_server/wait # API profile includes ingestion server
just wait # API waits for ES in entrypoint
env DC_USER="opener" just ../exec web pytest
just dc exec -u opener  web pytest
env COMPOSE_PROFILES="api,ingestion_server,frontend,catalog" docker-compose -f docker-compose.yml -f docker-compose.overrides.yml exec -u opener web pytest
Test session starts (platform: linux, Python 3.11.4, pytest 7.4.0, pytest-sugar 0.9.7)
django: settings: conf.settings (from env)
rootdir: /api
configfile: pytest.ini
plugins: sugar-0.9.7, Faker-18.11.2, django-4.5.2, raises-0.11
collecting ...

Testing LOCAL environment
collected 559 items

 test/test_auth.py ✓✓✓✓✓✓✓✓✓✓✓✓                                                                                2% ▎
 test/test_dead_link_filter.py ✓✓✓✓✓✓                                                                                3% ▍
 test/unit/controllers/test_search_controller.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                32% ███▎
 test/unit/management/commands/test_generatewaveforms.py ✓✓✓✓✓✓                                                                                33% ███▍
 test/unit/models/test_audio.py ✓✓✓✓                                                                                34% ███▌
 test/unit/models/test_media_report.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                40% ████
 test/unit/serializers/test_audio_serializers.py ✓✓✓                                                                                40% ████
 test/unit/serializers/test_media_serializers.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                72% ███████▎
 test/unit/templates/test_admin_base_html.py ✓✓✓                                                                                46% ████▋
 test/unit/utils/test_image_proxy.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                98% █████████▊
 test/unit/utils/test_search_context.py ✓✓✓✓✓✓✓✓✓✓                                                                                48% ████▉
 test/unit/utils/test_throttle.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                54% █████▌
 test/unit/views/test_health_views.py ✓✓✓✓✓✓                                                                                100% ██████████
 test/unit/views/test_image_views.py ✓✓✓                                                                                55% █████▌
 test/unit/views/test_media_views.py ✓✓✓✓✓✓✓✓✓✓                                                                                57% █████▊
 test/test_audio_integration.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                60% ██████
 test/test_backwards_compat.py ✓✓✓✓                                                                                61% ██████▏
 test/test_examples.py ✓✓✓✓✓✓✓✓✓                                                                                62% ██████▎
 test/test_image_integration.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                65% ██████▋
 test/test_v1_integration.py ✓✓ss✓✓✓✓s✓✓✓✓s                                                                                68% ██████▊
 test/unit/configuration/test_link_validation_cache.py ✓✓✓✓✓✓✓✓✓✓✓✓✓✓✓                                                                                71% ███████▏
 test/unit/utils/test_attribution.py ✓✓✓✓✓✓✓                                                                                74% ███████▍
 test/unit/utils/test_check_dead_links.py ✓✓✓✓                                                                                74% ███████▌
 test/unit/utils/test_drf_renderer.py ✓✓✓✓✓✓✓✓✓✓✓                                                                                76% ███████▋
 test/unit/utils/test_help_text.py ✓✓✓✓                                                                                77% ███████▊
 test/unit/utils/test_tallies.py ✓✓✓✓                                                                                99% █████████▉
 test/unit/utils/test_watermark.py ✓✓                                                                                99% █████████▉
 test/unit/utils/test_waveform.py ✓                                                                                99% █████████▉
====================================================================================================================================================== warnings summary ======================================================================================================================================================
../venv/lib/python3.11/site-packages/gunicorn/util.py:25
  /venv/lib/python3.11/site-packages/gunicorn/util.py:25: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html
    import pkg_resources

../venv/lib/python3.11/site-packages/pkg_resources/__init__.py:2871
../venv/lib/python3.11/site-packages/pkg_resources/__init__.py:2871
  /venv/lib/python3.11/site-packages/pkg_resources/__init__.py:2871: DeprecationWarning: Deprecated call to `pkg_resources.declare_namespace('zope')`.
  Implementing implicit namespace packages (as specified in PEP 420) is preferred to `pkg_resources.declare_namespace`. See https://setuptools.pypa.io/en/latest/references/keywords.html#keyword-namespace-packages
    declare_namespace(pkg)

test/unit/models/test_media_report.py::test_deindexing_creates_deleted_image_instance[image-Image-ImageReport-DeletedImage-ImageFactory]
test/unit/models/test_media_report.py::test_deindexing_creates_deleted_image_instance[image-Image-ImageReport-DeletedImage-ImageFactory]
test/unit/models/test_media_report.py::test_deindexing_creates_deleted_image_instance[audio-Audio-AudioReport-DeletedAudio-AudioFactory]
test/unit/models/test_media_report.py::test_deindexing_creates_deleted_image_instance[audio-Audio-AudioReport-DeletedAudio-AudioFactory]
  /venv/lib/python3.11/site-packages/django/db/models/fields/related_lookups.py:52: RemovedInDjango50Warning: Passing unsaved model instances to related filters is deprecated.
    warnings.warn(

test/unit/serializers/test_media_serializers.py::test_media_serializer_sensitivity[image_media_type_config-has_confirmed_report-has_sensitive_text]
test/unit/serializers/test_media_serializers.py::test_media_serializer_sensitivity[image_media_type_config-has_confirmed_report-has_sensitive_text]
test/unit/serializers/test_media_serializers.py::test_media_serializer_sensitivity[image_media_type_config-has_confirmed_report-no_sensitive_text]
test/unit/serializers/test_media_serializers.py::test_media_serializer_sensitivity[image_media_type_config-has_confirmed_report-no_sensitive_text]
test/unit/serializers/test_media_serializers.py::test_media_serializer_sensitivity[audio_media_type_config-has_confirmed_report-has_sensitive_text]
test/unit/serializers/test_media_serializers.py::test_media_serializer_sensitivity[audio_media_type_config-has_confirmed_report-has_sensitive_text]
test/unit/serializers/test_media_serializers.py::test_media_serializer_sensitivity[audio_media_type_config-has_confirmed_report-no_sensitive_text]
test/unit/serializers/test_media_serializers.py::test_media_serializer_sensitivity[audio_media_type_config-has_confirmed_report-no_sensitive_text]
  /api/api/models/media.py:272: DeprecationWarning: The 'body' parameter is deprecated for the 'update' API and will be removed in a future version. Instead use API parameters directly. See https://github.com/elastic/elasticsearch-py/issues/1698 for more information
    getattr(es, method)(

test/unit/utils/test_watermark.py::test_sends_UA_header
test/unit/utils/test_watermark.py::test_long_title_wraps_correctly
  /api/api/utils/watermark.py:203: DeprecationWarning: getsize_multiline is deprecated and will be removed in Pillow 10 (2023-07-01). Use ImageDraw.multiline_textbbox instead.
    _, attribution_height = font.getsize_multiline(text)

-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html

Results (88.72s (0:01:28)):
     555 passed
       4 skipped

Wow, must be something wrong with my setup then!

Closing, we feel this validates exploring ES8 configuration in terraform and staging.

@WordPress/openverse-maintainers, @sarayourfriend pointed out to me that if this is working, we might as well switch to ES8 locally. What do you think?


Removing and adding user to group triggers password reset

Describe the bug
Removing and adding a user to the group triggers password reset even though this makes little sense. UserService:903
GitLab then issues an email to the user that his password was changed. This might look really strange to a user, why was his GitLab password changed.
Expected behavior
The user is updated without password reset unless Artemis password changed...

This is a well known issue in Gitlab. Updating a user (including the password) over the REST API automatically issues this and there seems no way to prevent this.
We could try to avoid that when the password has not changed, e.g. when the user is only added to or removed from a group, but this might not solve all issues (e.g. when the user changes the password in Artemis in case this is allowed)
Best would be, if we could deactivate that users change their password within Gitlab, but up to now, we did not find a way how to do that. Otherwise, it could also happen that the Artemis and Gitlab password diverge, which might be ok for most users, but which also might lead to issues for some (especially novice) users.

Well, I think the email is fine if the user changes his password on Artemis, and then he receives a mail that his GitLab Password was changed. The user will probably understand what is going on.
But today I received an "admin changed your password email" after somebody added me to a course/group. I did not know about it and hence was really confused about who changed by password or why Artemis. When a ask my colleague he told me he added me to a course and I checked the code why Artemis would change my password.
I think at least not resetting the password after adding/removing of a group would be an improvement over the current situation. Also makes little sense to reset the password on that occasion right?

I can provide the PR for this is wanted

Yes, I fully agree, in this case it does not make much sense, in particular if gitlab sends such emails (which it does not in our test instance).
The code in Artemis does not really reset the password, it actually updates the user object, which automatically triggers a password reset (even for unchanged passwords) in gitlab. The method which is responsible for this, already offers a boolean flag shouldSynchronizePassword in GitLabUserManagementService:69, so this should be easy to fix.
We would appreciate a PR. Thanks!

By the way: are you evaluating Artemis for the use in the University of Bonn? Or are you already using it in a real course? ;-)

Kind of evaluated and we wanna use it though the course has not yet started (6. November). Creating exercises now. Of course, we might need to cancel if I discover major issues.

Yes, I fully agree, in this case it does not make much sense, in particular if gitlab sends such emails (which it does not in our test instance).
@krusche I'm actually not sure if we set up SMTP for our gitlab instance, so that might be why we don't receive these mails.

Yes, I fully agree, in this case it does not make much sense, in particular if gitlab sends such emails (which it does not in our test instance).

@krusche I'm actually not sure if we set up SMTP for our gitlab instance, so that might be why we don't receive these mails.

Yes, I think this is not mentioned in the docs. We should be able to prevent the mentioned issue with some small changes

https://forum.gitlab.com/t/managing-git-lab-users-from-external-system/44202 asked on GitLab Forum

Solved in #2248


cell competition

Cell competition sounds like a phenomena that is studied in transformed cell cucltures?
cell competition https://www.ncbi.nlm.nih.gov/pubmed/20644714
which would be out of scope for GO?

PMID:27676435

ok closing


pgx timestamp support is not being generated

Version
1.15.0
What happened?
sqlc does not take advantage of pgx/v4 when generating sql.query.go
Although db.go generates the pgx/v4 import, sql.query.go does not use any of the types supported by pgx/v4. This results in a loss of precision for types like timestamptz - the generated code uses sql.NullTime instead and loses fractional seconds.
Relevant log output
No response
Database schema
-- Example queries for sqlc
CREATE SCHEMA togo;
CREATE TABLE IF NOT EXISTS togo.tasks (
    id BIGSERIAL PRIMARY KEY ,
    name VARCHAR(100) NOT NULL,
    description VARCHAR NOT NULL,
    created_on TIMESTAMPTZ(6) NOT NULL,
    completed_on TIMESTAMPTZ(6) NULL,
    due_date TIMESTAMPTZ(6) NULL
);

SQL queries
-- name: AllTasks :many
SELECT * FROM togo.tasks;

-- name: RemoveTask :exec
DELETE FROM togo.tasks WHERE name = $1;

-- name: AddOrUpdateTask :exec
INSERT INTO togo.tasks (name, description, created_on, completed_on, due_date)
VALUES ($1, $2, $3, $4, $5)
ON CONFLICT (name) DO UPDATE
    SET description = $2, created_on = $3, completed_on = $4, due_date = $5;

-- name: FindByName :one
SELECT * FROM togo.tasks WHERE name = $1;

-- name: FindByDueDate :many
SELECT * FROM togo.tasks WHERE due_date = $1;

-- name: FindOverdueTasks :many
SELECT * FROM togo.tasks WHERE due_date < $1;

-- name: CountTasks :one
SELECT COUNT(*) FROM togo.tasks;

Configuration
version: "2"
sql:
  - engine: "postgresql"
    queries: "query.sql"
    schema: "query.sql"
    gen:
      go:
        package: "postgres"
        sql_package: "pgx/v4"
        out: "."

Playground URL
https://play.sqlc.dev/p/3860b78b43c8e9f605454a012c3c691d8ec20e8999b49ab2e5419fc77d6f42a8
What operating system are you using?
Linux
What database engines are you using?
PostgreSQL
What type of code are you generating?
Go

Ignore me, the issue is not in sqlc's handling of temporal values.


Keystore not appear in the list of signing identity

The keystore does not appear in the list of signing identity after creating or importing. I could find keystore in windows , but it didn't appear in the list of signing identity.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: d22ceddd-69d4-3a5d-2947-a5abddbbd3aa
Version Independent ID: e238051f-b809-5541-4049-f1205dd0a3a5
Content: Signing the Android Application Package - Xamarin
Content Source: docs/android/deploy-test/signing/index.md
Product: xamarin
GitHub Login: @conceptdev
Microsoft Alias: crdun

Thanks for submitting an issue and we apologize for the delayed response. The requested updates haven't been made since the creation of the issue, and the timeline for resolution may vary based on resourcing, so the issue is being moved to an internal repo to better track it.


Autofocus should delay request of focus until the material page is at the forefront

If the navigator page stack contains more than one page, and a page that is not the one displayed requests focus through autofocus, that focus should be delayed until the page is put on the forefront and the focus should stay on the currently viewed page until then.
Repro
Here is a repro repository https://github.com/cedvdb/flutter_repros/tree/autofocus (checkout the autofocus branch)
When you run the app and click on the button you'll see the focus disappear from the page that is shown because the underlying page hides the TextFormField for a second then respawns it.
Alternatively here is the repro code which needs the rxdart dep
import 'dart:async';

import 'package:flutter/material.dart';
import 'package:rxdart/rxdart.dart';

void main() {
  runApp(const MyApp());
}

class MyApp extends StatelessWidget {
  const MyApp({Key? key}) : super(key: key);

  @override
  Widget build(BuildContext context) {
    return MaterialApp(
      title: 'Flutter Demo',
      theme: ThemeData(
        primarySwatch: Colors.blue,
      ),
      home: const MyHomePage(),
    );
  }
}

class MyHomePage extends StatefulWidget {
  const MyHomePage({Key? key}) : super(key: key);

  @override
  State<MyHomePage> createState() => _MyHomePageState();
}

class _MyHomePageState extends State<MyHomePage> {
  BehaviorSubject<int> streamController = BehaviorSubject.seeded(0);

  void _showOtherPage(BuildContext context) async {
    Navigator.of(context).push(
      MaterialPageRoute(
        builder: (ctx) => Scaffold(
          appBar: AppBar(),
          body: TextFormField(
            autofocus: true,
          ),
        ),
      ),
    );
    streamController.add(streamController.value + 1);
    await Future.delayed(const Duration(milliseconds: 800));
    streamController.add(streamController.value + 1);
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: StreamBuilder(
        stream: streamController.stream,
        builder: (ctx, AsyncSnapshot<int> snap) {
          if (!snap.hasData) {
            return Container();
          }
          if (snap.data! % 2 == 0) {
            return Column(
              children: [
                TextFormField(autofocus: true),
                const SizedBox(height: 20),
                ElevatedButton(
                  onPressed: () => _showOtherPage(context),
                  child: const Text('show other page'),
                ),
              ],
            );
          }
          return Container();
        },
      ),
    );
  }
}

workaround
replace page instead of having it in the stack

Your interpretation is slightly off, I'll correct it :
After tapping on show other page, it navigates to another page which has a textfield which does receive the focus since it has autofocus: true. However when the page under it is updated to show a text field which also contains autofocus: true, the page at the forefront loses the focus.
https://github.com/flutter/flutter/issues/98199  and https://github.com/flutter/flutter/issues/92541 are not the same issues

Thanks for the update.

 stable, master flutter doctor -v

[✓] Flutter (Channel stable, 2.10.3, on macOS 12.2.1 21D62 darwin-x64, locale
    en-GB)
    • Flutter version 2.10.3 at /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision 7e9793dee1 (11 hours ago), 2022-03-02 11:23:12 -0600
    • Engine revision bd539267b4
    • Dart version 2.16.1
    • DevTools version 2.9.2

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

[✓] Flutter (Channel master, 2.11.0-0.0.pre.818, on macOS 12.2.1 21D62
    darwin-x64, locale en-GB)
    • Flutter version 2.11.0-0.0.pre.818 at
      /Users/dhs/documents/fluttersdk/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision b5d3c43280 (77 minutes ago), 2022-03-07 19:57:36 -0800
    • Engine revision 0fed94d050
    • Dart version 2.17.0 (build 2.17.0-182.0.dev)
    • DevTools version 2.11.1

[!] Xcode - develop for iOS and macOS (Xcode 12.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    ! Flutter recommends a minimum Xcode version of 13.
      Download the latest version or update via the Mac App Store.
    • CocoaPods version 1.11.2

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] VS Code (version 1.62.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.21.0

[✓] Connected device (5 available)
    • SM G975F (mobile)       • RZ8M802WY0X • android-arm64   • Android 11 (API 30)
    • Darshan's iphone (mobile)  • 21150b119064aecc249dfcfe05e259197461ce23 •
      ios            • iOS 14.4.1 18D61
    • iPhone 12 Pro Max (mobile) • A5473606-0213-4FD8-BA16-553433949729     •
      ios            • com.apple.CoreSimulator.SimRuntime.iOS-14-3 (simulator)
    • macOS (desktop)            • macos                                    •
      darwin-x64     • Mac OS X 10.15.4 19E2269 darwin-x64
    • Chrome (web)               • chrome                                   •
      web-javascript • Google Chrome 98.0.4758.80

[✓] HTTP Host Availability
    • All required HTTP hosts are available

! Doctor found issues in 1 category.

/cc @gspencergoog

Hmm.  I wonder if we should wrap navigator pages in an ExcludeFocus unless they're on top.
Which would also be a good workaround for this, you can wrap the main page with an ExcludeFocus that sets excluding when it's not the top route.

This is also an issue if you have multiple forms inside a page view where the parent widget holding the page view is rebuilt.
if page 1 has auto focus and page 2 does too,  when going to page 2, page one will have the focus.

Thanks for the update.
@cedvdb


Detect Python binary rather than hard-coding.

Since my local Python is /usr/bin/python3, the build failed with:
/bin/sh: 1: python: not found

This change uses the PythonInterp package which sets PYTHON_EXECUTABLE. We use this to great success in the 32blit SDK.

Looks good - thanks.


Give Travis CI and admins read access to the CUR bucket.

The lambda for grouping costs by cost center includes a test that
will download and process the current report without uploading the
results. Give read access to travis and to 'organizations' account
admins for running the test.
PR Checklist:
[x] Describe and explain your intentions for this change
[x] Setup pre-commit and run the validators (info in README.md)

@zaro0508 the lambda has already been given access, but that doesn't affect access for Travis CI while running the lambda's tests.

Closing this because adding a permission model for functional lambda testing is too expensive for the minimal value we get from verifying s3 object existence in testing.