Possibility to resume from specific binlog and position

Hi,
Is it possible to give Spirit binlog file and position from which to resume altering?
Our use case:
We have an issue that binlog accidentaly purged on our instance and spirit could not resume from needed binlog
INFO[27855] closing canal
INFO[27855] syncer is closing...
WARN[27855] releasing metadata lock for x.xx
ERRO[27855] error writing checkpoint: context canceled
INFO[27855] kill last connection id xxx
INFO[27855] syncer is closed
spirit: error: Error 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
[root@ip-xxxxx spirit]# ./spirit --host=${DB_HOST} --username=${DB_USERNAME} --password=${DB_PASSWORD} --database=x --table=xx --alter="PARTITION BY KEY() PARTITIONS 8" --defer-cutover=true --checksum=true --skip-drop-after-cutover=true --threads=32 --target-chunk-time=1s
INFO[0000] Starting spirit migration: concurrency=32 target-chunk-size=1s table=x.xx alter="PARTITION BY KEY() PARTITIONS 8"
INFO[0000] attempting to acquire metadata lock for x.xx
INFO[0000] acquired metadata lock: x.xx
INFO[0000] unable to use INPLACE: ALTER either does not support INPLACE or when performed as INPLACE could take considerable time. Use --force-inplace to override this safety check
INFO[0006] create BinlogSyncer with config {ServerID:1730 Flavor:mysql Host:xxxxxxxxxxx Port:3306 User:xxxxxx Password: Localhost: Charset:utf8 SemiSyncEnabled:false RawModeEnabled:false TLSConfig:0xc000504540 ParseTime:false TimestampStringLocation:UTC UseDecimal:false RecvBufferSize:0 HeartbeatPeriod:0s ReadTimeout:0s MaxReconnectAttempts:0 DisableRetrySync:false VerifyChecksum:false DumpCommandFlag:0 Option: Logger:0xc000046970 Dialer:0x7c2600 RowsEventDecodeFunc:0xa95a20 TableMapOptionalMetaDecodeFunc: DiscardGTIDSet:false EventCacheCount:10240}
WARN[0007] resuming from checkpoint failed because resuming from the previous binlog position failed. log-file: mysql-bin-changelog.x log-pos: xxxxx
INFO[0007] could not resume from checkpoint: reason=binlog position is impossible, the source may have already purged it
We have restored instance from snapshot and possibly want to resume spirit by giving specific binlog and position to start from. Is that possible?

@morgo  Perhaps wider question - Can we resume spirit on restored DB with chkpnt, sentinel, _new tables there and it won't have any data incosistencies between source and new tables?

Also,

can spirit run on read replica? We have B/G deployment and on green environment it runs ok, wondering if on Read Replica it will run as well.
ERRO[27855] error writing checkpoint: context canceled what does this error mean behind the scenes in Spirit? Does it mean binlog is not getting updated to newest checkpoint? Because we have binlog retention period set to 24hrs and even though 24hrs have not passed spirit tried to resume from binlog older than 24hrs.

@morgo Perhaps wider question - Can we resume spirit on restored DB with chkpnt, sentinel, _new tables there and it won't have any data incosistencies between source and new tables?

It may have inconsistencies if it misses some binary logs. This will happen if a modification occurs below the current checkpoint, because a change can occur which is not identified.
However, having said that, the lost change can be identified by the checkpoint, and the checkpoint will repair inconsistencies as they are discovered.
I don't think this is something we will ever support as an official feature, but you can manually edit the checkpoint table to set the binary log value to a current existing position in time. When you resume, you must have the checksum enabled to prevent data loss.

can spirit run on read replica? We have B/G deployment and on green environment it runs ok, wondering if on Read Replica it will run as well.

This is not something we have planned. It is unlikely to benefit Spirit that much, since it uses INSERT IGNORE .. SELECT FROM and doesn't actually read the data into Spirit.

ERRO[27855] error writing checkpoint: context canceled what does this error mean behind the scenes in Spirit? Does it mean binlog is not getting updated to newest checkpoint? Because we have binlog retention period set to 24hrs and even though 24hrs have not passed spirit tried to resume from binlog older than 24hrs.

You are hitting this issue: https://github.com/cashapp/spirit/issues/197
Basically when a checkpoint can't be written (perhaps because the system is overloaded) the migration is allowed to continue. But there's a risk with this, since if there's a failure the tracked position in the binary log (as per the checkpoint table) is way behind where the actual applied position of the binary log is. In future, it should be fixed so that failing to write a checkpoint causes a failure for spirit.

**It may have inconsistencies if it misses some binary logs. This will happen if a modification occurs below the current checkpoint, because a change can occur which is not identified.
However, having said that, the lost change can be identified by the checksum, and the checksum will repair inconsistencies as they are discovered.
I don't think this is something we will ever support as an official feature, but you can manually edit the checkpoint table to set the binary log value to a current existing position in time. When you resume, you must have the checksum enabled to prevent data loss.**
Regarding above - in theory we have binlog file from which we should resume in the output below, right? So perhaps as you suggest editing checkpoint table to set it to that value/file.
WARN[0007] resuming from checkpoint failed because resuming from the previous binlog position failed. log-file: mysql-bin-changelog.x log-pos: xxxxx
This is not something we have planned. It is unlikely to benefit Spirit that much, since it uses INSERT IGNORE .. SELECT FROM and doesn't actually read the data into Spirit.
Could you elaborate on this? Technically green environment instances are also replicas from blue environment, so I would assume if it works on B/G setup it should work correctly on Read Replica which would later could be promoted to master.
**You are hitting this issue: https://github.com/cashapp/spirit/issues/197
Basically when a checkpoint can't be written (perhaps because the system is overloaded) the migration is allowed to continue. But there's a risk with this, since if there's a failure the tracked position in the binary log (as per the checkpoint table) is way behind where the actual applied position of the binary log is. In future, it should be fixed so that failing to write a checkpoint causes a failure for spirit.**
Thank you, it makes more sense now. And there is no way to know what is the actual applied position of the binary log? What happens if in checkpoint table I set it to binlog which is way begind the actual applied position? And can there be any other reasons besides system overload? From metrics I don't see any bottlenecks.

Perhaps on the last point, can replica lag be a factor? This is only metric I am seeing that is having issues(more than 10k seconds lag).
Also, what is best/recommended binlog_retention_period to be set on DB where spirit runs? If it has issues like you described with writing to checkpoint table I reckon bigger retention period could help preventing this from user side.

I don't think this is something we will ever support as an official feature, but you can manually edit the checkpoint table to set the binary log value to a current existing position in time. When you resume, you must have the checksum enabled to prevent data loss.**
Regarding above - in theory we have binlog file from which we should resume in the output below, right? So perhaps as you suggest editing checkpoint table to set it to that value/file.

Yes, provided these points are still available.

WARN[0007] resuming from checkpoint failed because resuming from the previous binlog position failed. log-file: mysql-bin-changelog.x log-pos: xxxxx
This is not something we have planned. It is unlikely to benefit Spirit that much, since it uses INSERT IGNORE .. SELECT FROM and doesn't actually read the data into Spirit.
Could you elaborate on this? Technically green environment instances are also replicas from blue environment, so I would assume if it works on B/G setup it should work correctly on Read Replica which would later could be promoted to master.

If you are using B/G or your read-replicas are writable, you may be able apply the schema changes directly. If it's a blocking change, replication will take some time to catch up after its complete. I have seen this method suggested in the AWS documentation. I don't recommend it though; what they are not telling you, is that Row-based replication depends strictly on the ordinality of columns. If you were to perform a schema change where you remove a column mid-way through, and add a new column to the end of the table, you could end up with data corruption as changes are incorrectly applied on the replica.
Gh-ost for example supports read-replicas for estimates (number of rows in the table, where chunk boundaries should start/stop). We use a different method for table estimates in Spirit which is not quite as accurate, and for chunk boundaries we have 2 different implementations of chunker. But if we need to read-and-estimate I actually think it is beneficial to do it on the writer because it will pre-load those pages into memory, ahead of them being copied shortly after.

**You are hitting this issue: #197
Basically when a checkpoint can't be written (perhaps because the system is overloaded) the migration is allowed to continue. But there's a risk with this, since if there's a failure the tracked position in the binary log (as per the checkpoint table) is way behind where the actual applied position of the binary log is. In future, it should be fixed so that failing to write a checkpoint causes a failure for spirit.**
Thank you, it makes more sense now. And there is no way to know what is the actual applied position of the binary log? What happens if in checkpoint table I set it to binlog which is way begind the actual applied position? And can there be any other reasons besides system overload? From metrics I don't see any bottlenecks.

It may also be that there are some connections in the connection pool which need to be cycled out. When we hit this last, we hadn't added rejectReadOnly to the DSN yet, but it should be there now. We haven't hit this issue in a while, so I can't say with certainty what causes it. But if you have a reproducible testcase that shows it's quite probably we will be happy to prioritize a fix.

@morgo Thanks!
One last thing before we close this case:
INFO[14173] syncer is closing...
WARN[14173] releasing metadata lock for x.xx
ERRO[14173] error flushing binary log: context canceled
INFO[27855] closing canal INFO[27855] syncer is closing... WARN[27855] releasing metadata lock for x.xx ERRO[27855] error writing checkpoint: context canceled INFO[27855] kill last connection id 3479009 INFO[27855] syncer is closed spirit: error: Error 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
Is it logical to think that some other connection(perhaps replication from master) tries to lock same table which we try to partition and spirit prematurely releases metadata lock leading to error which I see in the output above?
I am trying to think of root causes why randomly spirit just dies sometimes for us.

INFO[27855] closing canal INFO[27855] syncer is closing... WARN[27855] releasing metadata lock for x.xx ERRO[27855] error writing checkpoint: context canceled INFO[27855] kill last connection id 3479009 INFO[27855] syncer is closed spirit: error: Error 1205 (HY000): Lock wait timeout exceeded; try restarting transaction
Is it logical to think that some other connection(perhaps replication from master) tries to lock same table which we try to partition and spirit prematurely releases metadata lock leading to error which I see in the output above?
I am trying to think of root causes why randomly spirit just dies sometimes for us.

The metadata lock is a get_lock() call. It's an advisory lock, not an actual lock, so it will only block other spirit processes from running and not user queries.
I think I have a better answer for you! The most pertinent error here is this one:

Error 1205 (HY000): Lock wait timeout exceeded; try restarting transaction

This could either be an InnoDB row-lock (trying to copy a chunk) or a metadata lock (trying to acquire a lock for checksum or cutover).
For InnoDB row-locks, the MySQL default is 50s, but spirit lowers this to 3s here: https://github.com/cashapp/spirit/blob/a04a2eeb48d03f42b3d8c38117e45d3cfe83c8da/pkg/dbconn/dbconn.go#L39
(We don't want spirit to be blamed for impacting production systems, so we are unlikely to make this configurable.)
For metadata lock on cutover, the MySQL default is 8hours, but the spirit default is 30s. We do make this one configurable, but we don't document it. Try --lock-wait-timeout=120s and let me know how it goes!
(Caveat being that this could bring your system offline for 2 minutes if it can't acquire that lock. Again, we default to a lower timeout so spirit doesn't get blamed for taking systems offline.)

@morgo do you recommend changing this on DB(it has param like this) or spirit has this flag? Because in Usage.md I don't see this described

@morgo do you recommend changing this on DB(it has param like this) or spirit has this flag? Because in Usage.md I don't see this described

This is a spirit flag - sorry it's not documented, we can add it.
Changing it on the DB (such as with a parameter group) will have no effect because spirit always overwrites it.

But basically what I can gather from this - this could either be an InnoDB row-lock (trying to copy a chunk) or a metadata lock (trying to acquire a lock for checksum or cutover).
It is still for us in process of copying data(not doing checksum or cutover), so possibly this could either be an InnoDB row-lock (trying to copy a chunk) is probable here.
But how does it work? Does it mean spirit itself kind of is the cause of failure?

FYI, we are trying to partition very large table(more than 15TB). and target-chunk-time is set to 1s(due to high waiting time for rollback transactions to finish when spirit fails).

It is still for us in process of copying data(not doing checksum or cutover), so possibly this could either be an InnoDB row-lock (trying to copy a chunk) is probable here.
But how does it work? Does it mean spirit itself kind of is the cause of failure?

No, this likely has more to do with your application.
Before you modify a row, locks have to be acquired. Those locks are then held for the duration of the transaction (not the statement!), and are only released when you commit or rollback.
Each chunk-copy is it's own transaction, so Spirit is ostensibly performing 1s transactions in your case. Spirit could technically conflict with itself (between applying rows and copier), but the copier won't conflict with itself. The more likely scenario is it conflicts with your application, and the more likely scenario of the two is that the replication applier causes the conflicts, because you tend to modify rows that have recently been modified.
When a row-lock can't be acquired immediately it will queue waiting, and then other row locks will queue waiting behind it. Chunking (at 1s) is for two reasons: replication lag, and yielding locks so that foreground application threads will not have timeouts triggered by spirit. So you can see why the innodb-lock-wait-timeout of 3s is also closely related to this goal. Spirit is likely timing out because an application thread is locking and processing these rows.
You are not seeing these problems in your application, because the default MySQL timeouts are crazy high (50s for innodb locks, and 8 hours for data locks). For many OLTP outages, waiting until these timeouts will basically cause an outage.
You can see running transactions with:
select * from information_schema.INNODB_TRX\G

And locks those transactions hold with:
select * from performance_schema.data_locks\G

Be careful that the data_locks query is pretty expensive prior to 8.0.40.
It might be that your transactions are too long-lived. If you can identify the cause (for example: maybe you are holding a transaction open while you do another network request?), it's not just a fix for spirit, but your application will perform better.


Add livecheck Formula DSL

Homebrew/livecheck provides various automated ways of detecting formulae updates. A livecheck DSL should be added in Homebrew/brew and used by Homebrew/homebrew-core formulae and migrated from Homebrew/livecheck's Livecheckables directory and brew livecheck use this new data.

Hi, I'd like to work on this. Any pointers to get started?

@nandahkrishna If you ask any more specific questions we can help!

For what it's worth, I have some forthcoming changes to livecheck that I will be proposing relatively soon (basically, allowing for some additional ways of matching and manipulating version strings) and this could impact the design/implementation of the DSL in some ways (it may be minor, though).
This isn't to say that you can't or shouldn't work on this now, by any means, but that you would have to keep track of changes to livecheck and incorporate them into the DSL as they're merged.

Alright, thanks. I’m still looking through the code to see how it works. I’ll keep track of the changes in livecheck and make changes when required.

@samford Can you detail what you are thinking of in this thread for discussion?

Hello, I had a look at the code in the livecheck repository and in general, how the brew commands have been written and structured here.
Hope this isn't a silly question, but would our aim here be to create a new livecheck.rb in Library/Homebrew, reimplement the functionality from the livecheck repository and migrate the Livecheckables here?

Hope this isn't a silly question

@nandahkrishna No such thing!

but would our aim here be to create a new livecheck.rb in Library/Homebrew

Perhaps but that would be in Library/Homebrew/dev-cmd and probably be the last thing if we wanted to remove the tap completely. Instead the priority should be:

adding support to formula.rb to be able to handle livecheck in formulae
migrate Livecheckables to each individual formulae in Homebrew/homebrew-core

Make sense?

@MikeMcQuaid correct me if I'm wrong, but currently Livecheckables are made of a method that return an hash. I think we would want something block like this instead in the formulae:
livecheck do
  url "https://aacgain.altosdesign.com/alvarez/",
  regex /aacgain-(\d+(?:\.\d+)+)\.tar/
end

@Moisan Something like that, yeh. Could you detail in here all the different fields/data you see being useful?

@MikeMcQuaid yep that does make sense. I'll look through formula.rb and think it over then. Thanks!

There is currently no other inputs to livecheck outside of the information available in the Formula and an optional Livecheckable. However we could probably help the livecheck heuristic a bit by pointing on the right format. For example something like :using => :gnome  could force to use gnome versioning (which skips odd versions since they are not stable). So far this is done by matching the url on different regex to find the right versioning.
There is also some information hardcoded in livecheck heuristic.rb, such as the formulae to not check with Github versioning even if their url includes github. The equivalent in a formula would be something like :not_using => :github.  Same goes for some sourceforge formulae.
A tag such as allow_pre_release could be used in some very specific case. It goes against Homebrew policy to include pre release code but some software seems to forever be in pre-release (e.g., black. Some recent changes now remove pre-releases from livecheck results.
This is all the details I can think of for now.

Thanks @Moisan, that's really helpful.

Can you detail what you are thinking of in this thread for discussion?

Thierry already mentioned the idea of adding the ability to specify how livecheck's heuristic should approach a formula (rather than guessing), so +1 on that.
Allow for text to be replaced in version strings.  For example, the version in a formula might be 1.2.3 but the version from livecheck may be 1_2_3 and this would always lead to a mismatch.  If we could provide regex/string replacements in the livecheckable, this kind of situation could be addressed.
Create an assortment of built-in version-matching regexes that can be referenced in livecheckables, so we don't have to copy-paste the same regex code over and over (with small changes, if any).
Add prefix and suffix regex options, to be prepended/appended to the main regex within livecheck.  This is somewhat related to the built-in regexes idea, as it would allow us to make livecheckable's that are a bit more DRY.  You could specify the built-in regex to use (or potentially lean on the heuristic) and only have to provide a specific prefix or suffix for matching. For example, a git repo may contain tags for various projects but we only want to match tags starting with projectname-, followed by the version.
Add the ability to completely skip livecheck for a given formula.  Some formulae are important but are unmaintained and won't receive further updates (e.g., imap-uw).  In this case, we want to keep the formula around but we don't need livecheck to waste time/effort only to give an error.  This could be something like, :skip => "Not maintained", where the optional text could explain why the formula is being skipped.
Also +1 on the idea of allowing pre-release versions.  How to best approach this (and whether we would need an explicit allow_pre_release tag) will likely be affected by the above ideas.

I've tried to keep things relatively brief here, so that this issue doesn't get derailed.  If any of these seem like valuable additions, I can create issues in the homebrew-livecheck repo to discuss them further (e.g., interactions between these ideas, design/implementation, etc.).  I'm still thinking about these ideas and experimenting with how they could be implemented, which is why I haven't already created issues.
I think this is everything I've come up so far that would affect the design/implementation of the livecheck DSL.

Allow for text to be replaced in version strings.

👍
Thoughts on an API for this?

Create an assortment of built-in version-matching regexes

I'd see them being perhaps in Homebrew/brew and being referenced with a symbol.

Add prefix and suffix regex options

I think this is something that could be in Homebrew/brew and build upon the built-upon ones. I don't think DRYness between formula is a priority more than explicitness and avoiding coupling.

I think this is everything I've come up so far that would affect the design/implementation of the livecheck DSL.

@samford Could you give a go at suggesting what a full DSL might look like for your options and @Moisan's?

Allow for text to be replaced in version strings.

Thoughts on an API for this?

If it's possible to handle this with a block, that would seem pretty ergonomic to me. Within the block, you could do whatever you need to do with the version string, so long as it's returned in the end. For example:
livecheck do
  alterations do |version|
    version.gsub('_', '.').sub(/-rubbish$/, '')
  end
end

This would likely be used very sparingly and only for substitutions, since we can already isolate versions with a carefully crafted regex. If you need more information than this, let me know.

built-in version-matching regexes

I'd see them being perhaps in Homebrew/brew and being referenced with a symbol.

Referencing with a symbol is what I imagined as well. For example:
livecheck do
  regex :semver
end

Add prefix and suffix regex options

I think this is something that could be in Homebrew/brew and build upon the built-upon ones. I don't think DRYness between formula is a priority more than explicitness and avoiding coupling.

I'm not sure if I explained this well before, so here's an example for illustration:
livecheck do
  regex :semver,
    :prefix => "packagename-",
    :suffix => "-something"
end

If the :semver regex is /v?(\d+(?:\.\d+)+)/ in this example, this would create a regex like /packagename-v?(\d+(?:\.\d+)+)-something/. We have quite a number of livecheckables where the regex is some type of semver matching with something before/after it, so it would be nice to simplify these livecheckables in conjunction with the built-in regexes feature.
As seen in the example above, we could handle regex prefixes and suffixes as an optional hash after the regex. This makes it clear that "prefix" and "suffix" are in reference to the regex, rather than something that is prepended or appended to the version string before being returned. If this is possible from an implementation standpoint, I think this would be the way to go.
One thing to note about this: if the prefix and/or suffix options are provided as a string, we would need to escape any special characters that could be interpreted as regex syntax. For example, if the prefix was "^" and the regex was /test/, you would end up with /^test/ when you actually want /\^test/. If this would be difficult to implement, we can drop the string support and only accept regex prefix and suffix values.

Are you looking for something like this?
livecheck do
  using :git                     # May also want to accept an array of symbols
  not_using :sourceforge         # May also want to accept an array of symbols
  url "https://www.example.com/downloads"
  regex /v?(\d+(?:\.\d+)+)/,     # Also accepts symbol for built-in regex
    :prefix => "packagename-"    # Also accepts regex
    :suffix => "-something"      # Also accepts regex
  alterations do |version|
    version.gsub('_', '.').sub(/-rubbish$/, '')
  end
end

using is eerily highlighted above (seemingly due to Module.using?), so we may have to rename it to something like use and avoid?
using and not_using should accept arrays of symbols, so we can theoretically specify multiple strategies to use or avoid, as appropriate. This likely won't be common but it would be nice to offer.
I've omitted allow_pre_release, since I feel that this would be better handled by the strictness or looseness of the regex. You can use ^/$ or delimiters in text (like " at the end of an HTML attribute) to ensure that we don't match anything after the desired version text (e.g., -beta1, -rc2, etc); the converse is true if you do want to match unstable versions. Additionally, how pre-release versions are allowed is sometimes specific to a given formula's version string format, so I'm not sure that it's something we could reliably handle in the heuristic anyway.

Have I missed anything or misinterpreted any of this? If everyone could review this and provide feedback when you have the time, that would be appreciated.

If everyone could review this and provide feedback when you have the time, that would be appreciated.

This looks great to me! In the case above I'm not sure what I'd think about using using and regex at the same time (assuming that's intended) but otherwise: perfect!

I realized that I forgot to include skip in the full DSL example, so I went back and added it.

In the case above I'm not sure what I'd think about using using and regex at the same time

That last code example is just for the purpose of illustrating everything that would be available and definitely isn't meant to demonstrate a livecheckable that would actually exist in reality, ahah.
However, I can definitely imagine using and regex appearing in the same livecheckable.  using will likely be pretty niche to begin with but, when it does occur, regex would seemingly be included just as often as with any other livecheckable.
My description of why this makes sense got a bit long, so I went ahead and created an issue for using in Homebrew/homebrew-livecheck.  Hopefullly my explanations of the two potential applications help to make it clear that using and regex have different purposes and aren't mutually exclusive.
I'm planning to create more issues in the Homebrew/homebrew-livecheck and Homebrew/brew repos for each of these specific features when I get a chance, so we can iron out specifics of how each feature should work and discuss implementation details.

That last code example is just for the purpose of illustrating everything that would be available and definitely isn't meant to demonstrate a livecheckable that would actually exist in reality, ahah.

Good 😅

My description of why this makes sense got a bit long, so I went ahead and created an issue for using in Homebrew/homebrew-livecheck. Hopefullly my explanations of the two potential applications help to make it clear that using and regex have different purposes and aren't mutually exclusive.

Gotcha, read it and understand now. Agreed!

Hello. I was recently inactive due to some academic commitments, but I've been going through the codebase and think I'm getting the hang of it. I would just like to check if my understanding is right.
To formula.rb in Homebrew/Library, we would add:
  # Within class Formula
  def livecheck; end
  ...
    # Within the Formula DSLs class
    def livecheck(&block)
      define_method(:livecheck, &block)
    end

(I've completely based my guess on how test has been written. Based on my current understanding, we would define additional methods that handle the version matching and conditions like skip, etc.)
And also, in every formula inside homebrew-core's Formula/, we would be adding a livecheck block as previously suggested.
Do let me know if my intuition is right. If it is, and you give me the go-ahead, I could open a PR to start working on it. Thanks!

Do let me know if my intuition is right. If it is, and you give me the go-ahead, I could open a PR to start working on it. Thanks!

Sounds right as a starting point, yep!

Cross-posting from #7179:

It's up to you whether you match or deviate from the existing livecheck syntax in the Homebrew/homebrew-livecheck tap's brew livecheck command. What you want to be doing is making changes in three places:

the Homebrew/brew repository (this one!) to add the livecheck  DSL to formula.rb
the Homebrew/homebrew-core repository to add livecheck to various files in Formula
the Homebrew/homebrew-livecheck tap's brew livecheck command to read from Formula#livecheck/Formula.livecheck instead of the Livecheckables directory

You'll probably need to have all three PRs open at once for your local testing and we'll merge 1. first (which will cause no disruption and need no changes), then 2. for e.g. a single formula, then 3. to use the livecheck DSL if it's available or  fall back to Livecheckables. When all Livecheckables have been moved to Homebrew/homebrew-core the fallback code in brew livecheck can be removed (and we may even consider migrating the command to Homebrew/brew.

I've created a draft PR for a feature to filter versions based on criteria related to parts of the numeric version (e.g., a major version less than or equal to 3, a minor version less than 90, etc.). This wasn't one of the features that we originally specified but it became apparent over time that it was needed and some recent PRs relate to the issue as well. [Once this feature is merged, I'll be continuing to implement the other features but it made sense to tackle this one now.]
A full example would basically look something this for the DSL:
livecheck do
  versions {
    :comparison => :lte, # Default comparison for version parts
    :major      => 4,    # ...or { :comparison => :eq, :version => 4 }
    :minor      => 3,    # ...or { :comparison => :eq, :version => 3 }
    :patch      => 2,    # ...or { :comparison => :eq, :version => 2 }
    :other      => 1,    # ...or array of values/hashes
  }
end

If the Hash syntax doesn't need to be explicit, it could be potentially be something like:
livecheck do
  versions :comparison => :lte, # Default comparison for version parts
           :major      => 4,    # ...or { :comparison => :eq, :version => 4 }
           :minor      => 3,    # ...or { :comparison => :eq, :version => 3 }
           :patch      => 2,    # ...or { :comparison => :eq, :version => 2 }
           :other      => 1,    # ...or array of values/hashes
end

I'm looking for review/feedback on the draft PR for this feature, so I would appreciate it if anyone here who has time could take a look at Homebrew/homebrew-livecheck#536.

livecheck do
  versions {
    :comparison => :lte, # Default comparison for version parts
    :major      => 4,    # ...or { :comparison => :eq, :version => 4 }
    :minor      => 3,    # ...or { :comparison => :eq, :version => 3 }
    :patch      => 2,    # ...or { :comparison => :eq, :version => 2 }
    :other      => 1,    # ...or array of values/hashes
  }
end

I appreciate this is necessary but I have to be honest and say I have no idea what this means by looking at this DSL 😭.  Thanks!

It seems a bit verbose, and something that could potentially limit rather than allow flexibility. For example, what if you want to restrict the minor version to be even?
In terms of homebrew-livecheck, it personally think it could be something like:
livecheck :version => ->(v) { v.major <= 4 && v.minor <= 3 }

For Homebrew/brew, the lambda could be avoided by using a block:
livecheck do
  versions do
    version.major <= 4 && version.minor <= 3
  end
end

(or equivalent notation)

I appreciate this is necessary but I have to be honest and say I have no idea what this means by looking at this DSL 😭.  Thanks!

The draft PR goes into more detail about how things work but basically that contrived example would filter out any matched versions above <IP_ADDRESS>. The :lte symbol in the comparison corresponds to <= and would be used by default when comparing a matched Version's tokens to what's supplied in the livecheckable for filtering (e.g., is the Version's major less than or equal to 4?...is the minor less than or equal to 3?, etc.). The comparison can be overridden for a given part (e.g., minor) by passing a hash like in the comments above.

It seems a bit verbose, and something that could potentially limit rather than allow flexibility. For example, what if you want to restrict the minor version to be even?

I agree that the current draft implementation is potentially verbose and also wouldn't allow for something as seemingly simple as matching even-numbered minor versions (which is useful for most GNOME formulae). Using a proc/lambda/block would certainly improve flexibility and also simplify the implementation.
I'm already planning to use this approach for the alterations feature (above) and I successfully tested out the idea in the past, so it makes sense to me. My only concern initially was that it may be a little less user-friendly to some. That said, I'll try to work on an alternative implementation sometime in the next few days (ideally later today) and report back with the results.
It's worth noting that we don't currently have #major/#minor/#patch methods for Version, so we would be comparing against tokens like:
  livecheck :versions => ->(v) { v.tokens[0] <= 4 && v.tokens[1] <= 3 }

In my comment in #7340, I mentioned that the major/minor/patch meanings break down with formulae that don't use something similar to semantic versioning (e.g., YYYYMMDD version schemes like 20200409). If we wanted to have those methods, we could simply treat them like the major/minor/patch convenience API in the draft PR, where they would only be used when the meanings apply and we would simply use #tokens for non-semantic versions.
Either way, #tokens would still need to be accessible to livecheck, and #major/#minor/#patch would simply be for convenience. However, it's probably better to discuss this in #7340, as it may affect that PR.

The draft PR goes into more detail about how things work but basically that contrived example would filter out any matched versions above <IP_ADDRESS>.

Why/when would you want to do that? I'm trying to understand to suggest a higher/better abstraction.

It's worth noting that we don't currently have #major/#minor/#patch methods for Version

I'd say: that's an argument for having a class other than Version to handle this stuff. Perhaps a subclass in livecheck?
Thanks!

Why/when would you want to do that? I'm trying to understand to suggest a higher/better abstraction.

The general goal of the feature is to filter out versions that aren't appropriate, so livecheck doesn't erroneously list them as the newest. That is, the livecheckable regex is intended to identify all versions from a source and this feature is meant to filter out inappropriate versions. Up to now, we've dealt with this by creating complex regexes to restrict matching and this feature is intended to provide a more ergonomic way of handling this, so we can keep the regex simple (which is useful in the future, keeping the idea of built-in regexes in mind).
As before, the example was merely to illustrate the full API and not a practical real-world example. In practice, this will primarily be used to restrict matched major and minor versions but I wanted it to be flexible enough to potentially restrict any part of the version, in case we encounter any weird situations in the future that can't be easily avoided through modifying the regex.
The most common usage for now will be filtering out matched versions above (or potentially not equal to) a given major version for versioned formulae (e.g., versions above 3 for gtk3, versions above 4 for gtksourceview4, etc.). Similarly, the GNOME version scheme uses even-numbered minor versions below 90 to indicate stable releases but Homebrew/homebrew-livecheck#305 only applies this restriction when using the download.gnome.org strategy. I can create a lengthier list of formulae this would be used for (and why) but suffice to say there's a real need for this feature (e.g., we have 12 open PRs related to version restriction/filtering that I'll be revisiting after this feature is merged).
Just to be clear, this isn't intended as a way of having livecheck artificially avoid displaying a new version simply because it would be inconvenient. That is, if a formula is stuck on a given version until the formula is updated to work with a newer version (or something is addressed upstream), we would still want livecheck to display the newer version. For example, elasticsearch/kibana is currently at 6.8.7 but we wouldn't want to artificially restrict it to avoid showing 7.6.2 as the newest.

I'd say: that's an argument for having a class other than Version to handle this stuff. Perhaps a subclass in livecheck?

That's certainly a possibility, once we can access Version#tokens in that context. livecheck already extends Formula, so there's definitely precedent. However, how would this play out if livecheck (excluding livecheckables) is incorporated into Homebrew/brew as part of the DSL work? Would those extensions just be merged into Versions in the end?

In practice, this will primarily be used to restrict matched major and minor versions but I wanted it to be flexible enough to potentially restrict any part of the version, in case we encounter any weird situations in the future that can't be easily avoided through modifying the regex.

Let's restrict the DSL to only current use-cases. It can be extended if we need it in future (but: YAGNI).

The most common usage for now will be filtering out matched versions above (or potentially not equal to) a given major version for versioned formulae (e.g., versions above 3 for gtk3, versions above 4 for gtksourceview4, etc.). Similarly, the GNOME version scheme uses even-numbered minor versions below 90 to indicate stable releases but Homebrew/homebrew-livecheck#305 only applies this restriction when using the download.gnome.org strategy. I can create a lengthier list of formulae this would be used for (and why) but suffice to say there's a real need for this feature (e.g., we have 12 open PRs related to version restriction/filtering that I'll be revisiting after this feature is merged).

Sounds like a DSL that specifically refers to "prerelease" or "unstable" would be suitable here. There may be situations in which you want to output those versions (e.g. for a tap that tracks the development versions).

That's certainly a possibility, once we can access Version#tokens in that context. livecheck already extends Formula, so there's definitely precedent. However, how would this play out if livecheck (excluding livecheckables) is incorporated into Homebrew/brew as part of the DSL work? Would those extensions just be merged into Versions in the end?

I think there's probably still an argument for live check to have its own subclass to avoid it being exposed to/used by Formula unnecessarily.

Let's restrict the DSL to only current use-cases. It can be extended if we need it in future (but: YAGNI).

If we approach this with a lambda/block like Bo suggested, then I don't think there's a difference in restrictive or expansive implementations (i.e., it would allow one to compare any parts of the version using #tokens). If you're referring to restricting the convenience methods to #major and #minor (maybe #patch), than that's fine by me. As mentioned in #7340, expecting the third token to be patch (rather than an extension, build, date, etc.) may not hold up in practice, so I'm not sure it would be valuable anyway.
At the end of the day, I'm fine with just using #tokens and I'm not personally invested in #major/#minor convenience methods. Having them may confuse some folks anyway (i.e., when do I use v.major and when do I use v.tokens[0]?).

Sounds like a DSL that specifically refers to "prerelease" or "unstable" would be suitable here. There may be situations in which you want to output those versions (e.g. for a tap that tracks the development versions).

Is it possible for a tap to maintain their own livecheckables to override ours?
Livecheck is heavily geared toward only matching stable versions and we primarily avoid other versions at the regex-level, so they're not even part of the matches. It's possible to make the versions that are filtered out as part of version-filtering feature easily available but it would take a lot of effort to make versions that are omitted due to a regex available.
Outside of this feature, being able to provide this information would require looser regexes in livecheckables (or the heuristic) and this may pick up undesirable versions other than just prerelease/unstable. You would seemingly have to develop a way of identifying prerelease/unstable versions, to separate them from stable versions and weird versions that aren't stable but also aren't prerelease/unstable, and this can vary between formulae (if it's feasible at all).
I haven't seen anyone else bring up this issue yet (I don't imagine livecheck is used much beyond us), so right now it would be a lot of work without much benefit. It may be something to revisit in the future (once livecheck is in better shape) but I don't think we should bother with it now. The modifications that only relate to this particular feature are minor and could always be done in the future.

I reimplemented the version filtering feature using a lambda (per Bo's suggestion) and it only took eight lines across both heuristic.rb and utils.rb (compared to ~85 lines for the current implementation). Updating this to use a block in the future will be very easy due to the simplicity.
# heuristic.rb
if livecheck_args.is_a?(Hash) &&
   livecheck_args.key?(:versions) &&
   livecheck_args[:versions].lambda?
  filter_versions(match_version_map, livecheck_args[:versions])
end

# utils.rb
def filter_versions(match_version_map, conditions)
  match_version_map.select! { |_match, version| conditions.call(version) }
end

I've been testing this feature with a draft version of a livecheckable for gtk+ (which needs to be restricted to a major version <= 2 and even-numbered minor versions below 90) and it looks like this for the lambda setup:
class Gtkx
  livecheck :url      => "https://download.gnome.org/sources/gtk+/cache.json",
            :regex    => /gtk\+-v?(\d+(?:\.\d+)+)\.t/,
            :versions => lambda { |v|
              v.tokens[0].value <= 2 &&
                (v.tokens[1].value.even? && v.tokens[1].value < 90)
            }
end

I'm using the lambda { |v| ... } syntax here since it's a little closer to syntax for a block and will potentially make it a little easier to update livecheckable syntax when migrating to the DSL in the future.
What do we think about this? I'm partial to this setup but I'm holding off on updating the draft PR until others are on board as well.

Livecheck is heavily geared toward only matching stable versions and we primarily avoid other versions at the regex-level, so they're not even part of the matches.

I don't think that's a problem; Homebrew formulae similarly have a url at the top level as being the same as stable do; url ...; end and I think livechecks could do the same thing (when moved into Homebrew/brew and formulae). What I'm proposing is that as new features are added to the DSL they be readable by people who have no idea what livecheck is or what it assumes.
For example, instead of having the versions decided by a lambda: make it be stable_version_if (or something) that clarifies that the logic is reflective of finding what stable versions are.

You would seemingly have to develop a way of identifying prerelease/unstable versions, to separate them from stable versions and weird versions that aren't stable but also aren't prerelease/unstable, and this can vary between formulae (if it's feasible at all).

Yes, I'd think so. We have brew audits to do this: different versioning schemes per-package. That's another reason I'm dubious about "ignore versions above this number"; that's not going to survive a major version bump.

I haven't seen anyone else bring up this issue yet (I don't imagine livecheck is used much beyond us), so right now it would be a lot of work without much benefit. It may be something to revisit in the future (once livecheck is in better shape) but I don't think we should bother with it now. The modifications that only relate to this particular feature are minor and could always be done in the future.

I don't think brew livecheck and the tap itself need to change in any of the directions above. The reason I mention this is I do think there will need to be changes like this for the migration to include them in Homebrew/brew and formulae.
I'd suggest that we save this issue for discussion of that goal (moving code into Homebrew/brew and Homebrew/homebrew-core) and you can do whatever is desired in Homebrew/livecheck. I would bear in mind, though, that the DSL will be reviewed and tweaked as it goes into formulae so be careful that you don't add too many new features there that are going to cause issues in their migration.
Thanks for all your work on this @samford!

The primary intention of this feature was to restrict the major (and minor) version for versioned formulae [without modifying the regex], so livecheck doesn't erroneously list higher versions as the newest when it's inappropriate. For example, gtkmm only deals with v2 releases and there's gtkmm3 for v3 releases but livecheck reports v3 releases as newest for gtkmm. I don't currently imagine any other applications for this, so we wouldn't need to worry about the check becoming incorrect after a version bump.
Filtering out certain types of unstable versions was simply another potential application if the API was flexible. Looking to a future where livecheck deals with both stable and unstable versions, it makes sense to avoid using the version filtering feature for this and to handle it differently down the road.
With the previous discussions in mind and since we only need to restrict major and minor for versioned formulae, this could be simplified down to something like:
class PhpAT73
  livecheck :major => 7,
            :minor => 3
end

This basically reads as, "Filter out matches where v.tokens[0].value != 7 && v.tokens[1].value != 3". I think this should be understandable to people unfamiliar with livecheckables (especially compared to the previous implementations of this feature).
However, this only works if we have access to a version's first and second tokens in livecheck in some way. We can't access Version#tokens in livecheck and it wasn't desirable to make them public (#7340), which is understandable. It was suggested earlier that #major and #minor methods (returning tokens[0] and tokens[1] respectively) could be handled in a subclass of Version in livecheck but I don't imagine this is possible either if livecheck can't access #tokens.
If #major and #minor methods aren't desirable inclusions in Homebrew/brew (as an alternative to making #tokens public), then I'm fine with shelving this feature until the situation changes in the future (e.g., livecheck is integrated into Homebrew/brew). I think there's value in the feature (keeping regexes simple, to allow for the possibility of using built-in, standardized regexes) but I can understand if it's technically infeasible for now or simply not seen as necessary enough to warrant inclusion in the end.

This basically reads as, "Filter out matches where v.tokens[0].value != 7 && v.tokens[1].value != 3". I think this should be understandable to people unfamiliar with livecheckables (especially compared to the previous implementations of this feature).

Yup, that makes sense 👍.

If #major and #minor methods aren't desirable inclusions in Homebrew/brew (as an alternative to making #tokens public), then I'm fine with shelving this feature until the situation changes in the future (e.g., livecheck is integrated into Homebrew/brew).

I would suggest subclassing in brew livecheck where you can add these methods. If this can't work: yeh, it may be worth waiting until the Homebrew/brew migration is further along.

I would suggest subclassing in brew livecheck where you can add these methods. If this can't work: yeh, it may be worth waiting until the Homebrew/brew migration is further along.

Yep, sounds like it can't work for now, so let's shelve it and I'll make sure to revisit it in the future when subclassing in livecheck is feasible.
Thanks for the feedback, everyone! I'll comment again when the next feature's ready for review.

Closing this issue as the Livecheck DSL has been implemented and brew livecheck has been migrated to Homebrew/brew in #8180. Thanks everyone!

The main purpose of this PR was to establish a livecheck DSL and this was accomplished in #7179 (and has since been extended to include a strategy method in #8156). Any related discussion (e.g., further additions to the livecheck DSL) can simply be handled in individual issues/PRs.
There are still some areas of the DSL that I outlined here but haven't implemented yet (e.g., alterations, regex prefix/suffix, built-in regexes, etc.). They remain on my to-do list and I plan to take care of them as time permits.


plugin_not_installed

Trying to use the plugin, but when running on android device it says that the plugin is not installed
Your Environment

Plugin version: 3.0.1
Platform: Android
OS version: 8.0.0
Device manufacturer and model:

Running in Simulator:
Cordova version (cordova -v):
Cordova platform version (cordova platform ls):
Plugin configuration options:
Link to your project:

Context

Building a Ionic 3 application, I have followed the steps to use the plugin:
ionic cordova plugin add cordova-plugin-mauron85-background-geolocation@alpha
npm install --save @ionic-native/background-geolocation@4
and when trying to run the app in an Android device, message error plugin_not_installed is shown
Expected Behavior

Show an alert with the BackgroundGeolocationResponse
Actual Behavior

plugin_not_intalled error message
Possible Fix

I am trying other versions of the plugins and the new plugin @mauron85/cordova-plugin-background-geolocation
Steps to Reproduce

Context

Debug logs

this.platform.ready().then(() => { logic plugin code}

You are installing version alpha, install https://github.com/mauron85/cordova-plugin-background-geolocation

@anRoswell  yeah, i'm following the docs from ionic framework, https://ionicframework.com/docs/v3/native/background-geolocation/

You have to install this package https://www.npmjs.com/package/@mauron85/cordova-plugin-background-geolocation
This one https://www.npmjs.com/package/cordova-plugin-mauron85-background-geolocation is deprecated

Try installing exact version with:
cordova plugin add<EMAIL_ADDRESS>

Also I believe there is still issue with scoped packages in cordova. Meaning that when you create platform with cordova platform add it may install also non scoped package cordova-plugin-background-geolocation. It's different package, so beware.
The best way to avoid this, is to add plugin in config.xml with spec declaration. Check https://github.com/mauron85/cordova-plugin-background-geolocation-example/blob/96b0497013f56515c2ab84a16eacceeb4826c31a/config.xml#L50

Thank you @mauron85 for your answer, i see there are an actualization about the plugin 3.1.0, i going to try you suggestion

Yes, the problem seems to be the scoped package, thanks for pointing that out @mauron85 . It didn't work for me to add it in config.xml because the plugin in package.json kept changing back to cordova-plugin-background-geolocation insteado of @mauron85/cordova-.... So what finally worked for me was adding the plugin directly in package.json so the correct plugin gets installed when adding or preparing the platform.
// package.json
{
  "@mauron85/cordova-plugin-background-geolocation": "^3.1.0",
  "cordova": {
    "plugins": {
      "@mauron85/cordova-plugin-background-geolocation": {}
    }
  }
}

After adding a platform, the old package keeps being added to the package.json but that doesn't seem to interfere so far.


Re-enable warning for missing pyyaml

The PR has been merged under commit 84606d2. Thank you for sharing your valuable feedback and making Path4GMNS better!


Integration Test - Release - 2022/04/06 08:07:33

Auto create for integration test. AnotherAuthorUpdatedArticleFlow

@prmerger-dev-1 : Thanks for your contribution! The author(s) have been notified to review your proposed change.


Iss 110 candidate

closes #110 . Addresses items from this earlier PR which I closed to keep this one cleaner.
https://github.com/codeforboston/jobhopper/pull/188

@alexjball , I think this is ready for another look.

react-scripts test --watchAll --verbose

console.log src/services/api/FakeApi.ts:11
FakeApi.getTransitions { socCode: '12345' }
console.log src/services/api/FakeApi.ts:19
FakeApi.getOccupations
PASS src/ducks/tests/store.ts
Store
√ can be constructed (9ms)
States
√ updates the list of states (8ms)
√ handles update errors (2ms)
Occupations
√ updates the list of occupations (1ms)
√ handles update error (2ms)
Transitions
√ updates the list of transitions (4ms)
√ handles update error (2ms)
PASS src/services/api/tests/datahelper.ts
Testing Data Helper Functions
√ transforms transitions (588ms)
console.log src/services/api/FakeApi.ts:15
FakeApi.getStates
console.log src/services/api/FakeApi.ts:11
FakeApi.getTransitions { socCode: '12345' }
PASS src/services/api/tests/api.ts
Fake API
√ Fake can be constructed (5ms)
√ is provided by default
√ retrieves occupations (519ms)
√ retrieves states (543ms)
√ retrieves transitions (509ms)
PASS src/ui/App/App.test.tsx (8.482s)
√ renders learn react link (43ms)
PASS src/Main.test.tsx (8.484s)
√ renders learn react link (51ms)
√ renders percentage as expected (2ms)
Test Suites: 5 passed, 5 total
Tests:       16 passed, 16 total
Snapshots:   0 total
Time:        10.449s
Ran all test suites.

@alexjball , I think we dicussed on Tuesday and this is still waiting for your review


R2dbc support

I've recently decided to try R2DBC and noticed that TestContainers is currently only compatible with JDBC. I've decided to add the support for it, however there are lots of impactful decisions to be made so I'd like some feedback on the assumptions I've made. I've currently implemented a very basic support for Postgresql containers. Before I go any further I'd like to discuss the following topics:

Packaging. So far it seems possible to reuse GenericContainer, so I've decided to work with that and created a new package equivalent to the org.testcontainers.jdbc. The existing container implementations are tightly coupled with JDBC, so it would be best to have a separate class for each R2DBC container. The container implementations would be prefixed with R2dbc  and reside in the packages next to original JDBC versions.

JDBC URI. As @mp911de has mentioned here and also here it would be desirable to not reimplement DataSource discovery based on string parsing. For the time being, I suggest a following approach:

A database container can only be created explicitly in a test
A container should expose an R2dbc object so that a test client can easily get access to the DB without having to create it on their own. We could also expose just the ConnectionFactory, but so far I see no benefit in this.
A container should also expose all the connection params (as a POJO or getters for specific properties) so that if there is a requirement to create a JDBC connection to the DB, they can do so.

When ConnectionFactory discovery is merged maybe we could use that.

Code reuse. Current implementation of R2dbcDatabaseContainer is mostly copied from JdbcDatabaseContainer, and all concrete implementation would contain lots of duplicate code as well. To solve this we could either:

create another abstraction layer between R2dbcDatabaseContainer and GenericDatabaseContainer
pull as much code up into GenericDatabaseContainer  as possible add R2dbc compatibility at that level (seems unlikely)
maybe create some helper classes for the common code

Looking forward to your opinions, I'm happy to be devastated in a review

I believe I've resolved all the conflicts and done all requested changes. I've also added a sample doc section. Anything else left to improve?

@bsideup @kiview I'd appreciate some feedback / action, so that this PR doesn't hang here forever. Thanks alot

FYI https://github.com/r2dbc/r2dbc-spi/issues/37 was merged and now we can finally continue with the R2DBC support 🎉

R2DBC 0.8.0.M8 is released and so URL formats are stable.
R2DBC Javadoc: https://r2dbc.io/spec/0.8.0.M8/api/

@mp911de, @kiview and I met and discussed a couple of weeks ago. Sorry for the late writeup, but what we discussed was a plan to:

With the new milestone release of R2DBC, introduce a String getR2dbcUrl() method for the existing database container classes. As a simple string getter, we'd have no binary dependency on R2DBC, so this would be straightforward to add to our existing modules. (@bsideup I know you have some thoughts on this - let's discuss)

Later, we'll do a separate piece of work to implement a proxying driver implementation, similar to our ContainerDatabaseDriver class and conforming to a R2DBC SPI.

The latter would have a binary dependency on R2DBC and thus belongs in a new module.
As such, I think this particular PR can be trimmed a lot or replaced, unfortunately @Humblehound. Thank you for sparking the conversations, and sorry it's taken a long time so far.

No problem, I'm glad to have the problem resolved one way or another.

hello, so what about testing r2dbc spring-boot apps with postgres in test-container?


Users not finding contact under actions menu

Who is your user?
Someone trying to reach a Collective, like a financial contributor or expense submitter
What are they trying to achieve?
Find the contact button
How are they currently doing this?
Contacting main OC support, because that's the visible contact option, who then has to direct them to the Collective, delaying an answer and using up support resources

@alanna : I am not exactly understanding the problem here. We currently have a contact button under actions menu.

Do you mean that users have trouble finding it? One thing I can suggest is adding the contact link (with a nice icon) near the social icons on the collective page.

Here after the website link we could add a mail button linking to the contact form which makes it obvious that it's a contact form.
@Betree : Let me know if you agree with this one; I can implement. 😄

Status: 💚
@aminakazi's done the design, just need to be implemented. Should be able to put this in the current sprint.

You can find the final designs here.
We've added an email icon in the header and a standalone button in the about section to help the user find the contact options better. The contact form has also been redesigned as a pop-up instead of the way it is working right now.

If there's no objections, I can take this one after we are done with the host reports and the refunds balance issue. 😄

I commented in the Figma, but will also comment here: "Actions" is not an intuitive call to action to any user. "Contact" is also such a basic call to action, it should be elevated from the menu imho. The planet icon is cute, and after thinking about it for a while I get it—but icons are supposed to be immediately intuitive, and not require users to think. I still don't know what that specifically means, or if it is a CTA for existing or future contributors to a project.
All that said, the watercolor backgrounds in Figma are off the hook beautiful, and I love!

From Figma:

The word "Actions" is not an intuitive or cognitively-resonant CTA to draw users in. Additionally, you are speaking to different user types in this menu—which is an anti-pattern. And, "Submit Expense" is redundant with the button to the left of "Actions."
I would recommend two different menus—one "For Contributors" and another "For Admins," and have the latter only visible to administrators.
It is just not clear who can add funds, manage a collective, or what "Contribute" means. "Contact" is the only obvious CTA there, and needs to be elevated from the menu.

Production Ready file is linked here

Plan for this week:

[ ] check against initial problem/get feedback from with @alanna and @alinamanko

Why is this being closed if nothing was solved, and no user testing appears to have been done to learn about the problem or efficacy of a solution?
The problem is unlikely to be not users not noticing it within the Actions menu, but users not thinking to check a menu labeled "Actions," as that is a highly un-discoverable location for a "Contact" item. Attempting to solve the problem with just an icon (especially one that is so light and thin) is also not adding a very discoverable visual feature to a page. Icons alone are an accessibility anti-pattern, and anybody with vision issues will also struggle with the faint outlines and lack of contrast with the icons you're using.
The page is covered with tons of Open Collective specific nomenclature and concepts, but little that is relatable to people not embedded in Open Collective mental models. That's a really crappy experience, and alienates more than it engages.
Also, it's unclear what the proposed solution is, after being linked to a Figma file with many screens.
Guys, people depend upon this product to get paid and to fundraise. It's not cool to burden users with your own ideals of utopian concepts that negligibly translate, in a utility people depend upon for practical needs. You can imbibe values into your products, without burdening people with them as hindrances to basic usability via relatable mental models. For a global product, no less.

It's being closed because a new design surfacing the contact action in
multiple other more visible places on the page has been implemented. It may
take a little more time to get into production, but we close issues when
the related pull request is completed.

It's being closed because a new design surfacing the contact action in multiple other more visible places on the page has been implemented. It may take a little more time to get into production, but we close issues when the related pull request is completed.

Just to note this is in production now. 😄 👍🏽

@SudharakaP Right—but how do you know the problem has been solved for, if there was no user testing—or if there was user testing, why were findings not posted with the issue? I just looked in production, and did see a great and clearly visible "Contact" button, but it also looks nothing like the much less visible solutions offered in the Figma wireframes, above.
All of this is to say, that you guys do a pretty horrible job with surfacing good UX practices. Which is a huge bummer, considering your visibility in open source—and how awful UX practices are understood (and practiced), in open source.
Please do user testing. Please communicate findings from user testing. You guys have the resources. Many projects do not. Arguably, most do not. We need better lead-by-example setters in our space. Thanks.

@ninavizz Thank you for your concern. You are correct that we should do more user testing. And right now, it's being appropriately addressed.
I don't get your contributions to our issues, though. If you want to get closer and collaborate with any team, you should approach with a little more finesse and not with incendiary comments no one will react to. It's easy for all of us to just go to PayPal, or Wise, or whatever and start a flamewar about what we think is not right or what we believe best practices are.
The truth is, thousands of real people use the platform, and we keep an open eye and ear to hear what actual users are requesting. The bottom line here is that especially in open-source, I recommend you start adopting nicer ways to collaborate with people you don't know :)
BTW my invitation for you to get properly in touch and set some collaboration is still open, let me know if that interests you

@Memo-Es Why would I want to get in touch with you to offer my work for free—or, why would you assume I'm here desiring collaboration? I'm not. I'm stuck using your product because a team I work with uses it to solicit contributions, and then uses those contributions to pay contractors overseas. It meets the need of working across borders in paying international contributors, very well. I never wanted to get in touch with you properly "collaborate." I volunteered the option for you to get in touch with me, if you have questions—and I did what good open source contributors do, which is file issues if things are broken. I stopped short of volunteering to fix those, however, because I don't have that bandwidth.
If you detect hostility in my comments, it's because you're assuming an awful lot—your entire team is, and has been.
Following what "actual users are requesting" and then never testing those things, is not how you make usable products. That is not up for debate. In open source, user-centric product development is not well understood, and that is only because very few projects are acclimated to working with UX.
User testing, in combination with listening to all of your users (not just the project leads), is how you create a usable product. Things like privacy/visibility things, need user testing to get done right. Yes, I am pissed that you've created a payments platform that is approached as some utopian community sharing somethingerother—and not as a platform where users have their PII, post invoices, etc. That mandates a lot more respect for clearer communication about privacy, than has been invested in the product. OC takes the utopian/post-capitalism brand-spirit part of your product very seriously, and user privacy is treated like a joke. Businesses are not "collectives," they're businesses. Workers pay our bills from money we earn. I'm an anarchist, and all for collaborative stuff—but that's not what your platform facilitates, as a practical matter.
Anyhow, I'm not here to "collaborate." You can begin, by not assuming that. :)


[BUG] Missing Switch Type Configuration for Tuya Switch Module TS0001 - TZ3000_46t1rvdu

Bug description
"Switch Type" is missing in the config field within ZHA/HA for TS0001 - TZ3000_46t1rvdu.
Despite the standard quirk with this module this config option doesn't show up.
Steps to reproduce
Installing the module. Reading the following data.
Expected behavior
Getting config option for "switch type".
Screenshots/Video

Device signature
Device signature
         "0xe001": {
            "endpoint_attribute": "tuya_external_switch_type",
            "attributes": {
              "0xd030": {
                "attribute_name": "external_switch_type",
                "value": 2
              }
            },
            "unsupported_attributes": {}
          }

This information ist present in the logs.
So ZHA is recognizing this cluster/field and can call this cluster/field.
What is the problem here to show that up in HA?
Diagnostic information
No response
Logs
No response
Additional information
https://github.com/zigpy/zha-device-handlers/blob/cf06668b2e99345296e9b2a1708c3d19cb1ae12b/zhaquirks/tuya/init.py#L1121
https://github.com/zigpy/zha-device-handlers/blob/cf06668b2e99345296e9b2a1708c3d19cb1ae12b/zhaquirks/tuya/init.py#L1113

Maybe there is a mismatch between the "value": 2 and "ExternalSwitchtype.Momentary".
So the values 0, 1, 2 and "ExternalSwitchType."+ "Toggle", "State" or "Momentary".


Package the functions into a Library

Currently the functions are simply defined in .py files and are not a proper library that can be imported and installed via binary, pip, or anaconda. It would be helpful it the functions were properly packaged.
For a starting point on tackling this see: https://code.tutsplus.com/tutorials/how-to-write-package-and-distribute-a-library-in-python--cms-28693

Also, this won't be complete until the plugins are using the library.

I had another look at the url @JMante1 provided on how to package a library and the following questions arose:

They recommend providing testing for the functions which I think is something @tramyn wanted to have a look at. Would we put out a very early alpha stage without testing, should we wait until Tramy has had time for that?
For the metadata of the resulting library, I wanted to double check what we were thinking for the name (excel2sbol potentially?), the versioning system, the license, author and author email. The example provided in the url is as follows:

      name='pathology',
      version='0.1',
      url='https://github.com/the-gigi/pathology',
      license='MIT',
      author='Gigi Sayfan',
<EMAIL_ADDRESS>      description='Add static script_dir() method to Path',
      packages=find_packages(exclude=['tests']),
      long_description=open('README.md').read(),
      zip_safe=False

I think it's OK to put it out effectively without testing, since at this stage we're really just doing so to enable our own internal integration.  The tests will follow soon.
On metadata:

excel2sbol sounds like a fine name.
For versioning system, we should use SemVer
Licensing is BSD 3-clause: https://github.com/SynBioDex/Excel-to-SBOL/blob/master/LICENSE
For author & email, can we leave it blank and just let it be explained by the contributors here on github?

Hi @IsabelMarleen , my response to your first question is to set the field to exclude testing for now. I think it will be useful to get the structure of your python package up and running first and I can go in and add testing later on. However, if your setup is blocking you from finishing up this issue, then commit what you have onto a repo as a separate branch. I can insert a placeholder testing environment to move you forward with this issue.

They recommend providing testing for the functions which I think is something @tramyn wanted to have a look at. Would we put out a very early alpha stage without testing, should we wait until Tramy has had time for that?

I ran the setup.py file as you suggested and the following warning occurred: check: missing meta-data: either (author and author_email) or (maintainer and maintainer_email) must be supplied. I know we discussed that authorship was difficult to specify, but I assume @tramyn could be entered as maintainer. What do you think about this and which email address should we supply?

@IsabelMarleen I think that's a fine plan.

@IsabelMarleen<EMAIL_ADDRESS>is my preferred e-mail address.

The library is now live at https://pypi.org/project/excel2sbol/
@tramyn if you create an account at pypi and let me know your username, I can add you as maintainer of the project and grant you access. I downloaded the library in a virtual environment and it worked after I manually installed the dependency xlrd. I've added that to the setup.py file now, but I was unsure how to process the versioning with the new release. Would this be 0.1.1-alpha? I'll have another go tomorrow. I also updated Excel.py and Composition_reading.py to use the new library.

@IsabelMarleen tramy.nguyen is my pypi username.
I would call this release "1.0.0-alpha".

I just released 1.0.0-alpha and I think that fixed the issue. I also sent Tramy an invite to collaborate. Then I added excel2sbol as a requirement to the two plugins and changed the app.py files accordingly. Once @JMante1 is happy that all of that works fine, I would recommend removing Excel.py, Composition_reading.py and 'col_to_excel.py` from the respective plugin repositories to complete the refactoring.

The refactor for packaging functions broke the code. I am working through the new structure to fix these errors.


Custom metric not visible on mobile

I have custom metric integrated with Lambstatus, it is visible on desktop, but not on mobile...
Is this a bug?

Unfortunately the custom metric is not visible on mobile so far. It’s because the graph is not shown very well on mobile. The PR is welcome.


Utah directory being written to blocks parent directory listing.

I have mounted utahfs in my home directory, under $HOME/Utah to experiment with. When copying a large file in, the parent directory becomes blocked from listing. Attempting to Ctrl+C ls does work, but takes a long time.
Steps to reproduce
~$ # Terminal 1
~$ utahfs-client -conf ~/utahfs.yaml -mnt ~/Utah
~$ cp largefile Utah/largefile

~$ # Terminal 2
~$ ls
<blocked>

Eventually the operation can be killed, the following log is printed:
2020/06/23 12:00:02 filesystem.go:661: context canceled
fuse: 2020/06/23 12:00:02 connection.go:486: *fuseops.GetInodeAttributesOp error: app: transaction not active

This sounds like it might be caused by mutex starvation. Which version of Go are you using?

$ go version
go version go1.14.4 linux/amd64

I installed through via go get.


Blocks off some UWPs

Whenever i launch twitter UWP i/m unable to click on anything, its like  there's a transparent overlay... it happens randomly...

@sammykip can you attach a screenshot please to help us understand the issue better?

we'll a screenshot wont really show you, but it happens randomly, and i have to close the entire app, i cant click on anything on UWPs, i can only scroll, since te first release ive had this issue, mostly after cold boot

Well when i reboot my computer, and i have power toys auto launch on reboot, when i open twitter app or sometimes movies & TV or MSN news it will function well then sometimes, mostly on twitter i cant click on anything unless u close powertoys,same to movies and MSN News...

@sammykip can we setup a teams call and let me see behavior with a screenshare?  email me.
<EMAIL_ADDRESS>

sure i will first try record the behavior with Xbox game recorder since it happens randomly then give you a call and try reproduce the same

Is this still happening with 0.19?


ci: remove skip check

GitHub Actions now support them out of the box: https://github.blog/changelog/2021-02-08-github-actions-skip-pull-request-and-push-workflows-with-skip-ci/.

Codecov Report

Merging #2857 (de525a6) into master (b6d67c3) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master    #2857   +/-   ##
=======================================
  Coverage   93.92%   93.92%
=======================================
  Files          87       87
  Lines        6326     6326
  Branches     1308     1308
=======================================
  Hits         5942     5942
  Misses        354      354
  Partials       30       30

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update b6d67c3...de525a6. Read the comment docs.


Fix README.md

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 83.779%

Totals

Change from base Build<PHONE_NUMBER>:
0.0%

Covered Lines:
5103

Relevant Lines:
6091

💛  - Coveralls


aeson 0.10

aeson-<IP_ADDRESS> depended on by:

aeson-schema-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.10). Mateusz Kowalczyk<EMAIL_ADDRESS>@Fuuzetsu. @Fuuzetsu
aeson-utils-<IP_ADDRESS> (>=0.6 && < 0.10). Silk<EMAIL_ADDRESS>@bergmark @hesselink
arbtt-<IP_ADDRESS> (>=0.6 && < 0.10). Joachim Breitner<EMAIL_ADDRESS>binary-orphans-<IP_ADDRESS> (>=0.8 && < 0.10). Oleg Grenrus<EMAIL_ADDRESS>@phadej. @phadej
binary-tagged-<IP_ADDRESS> (>=0.8 && < 0.10). Oleg Grenrus<EMAIL_ADDRESS>@phadej. @phadej
bloodhound-<IP_ADDRESS> (>=0.7 && < 0.10). Chris Allen<EMAIL_ADDRESS>@bitemyapp
country-codes-0.1.2 (>=0.5 && < 0.10). Felipe Lessa<EMAIL_ADDRESS>@meteficha. @meteficha
datadog-<IP_ADDRESS> (>=0.8 && < 0.10). Ian Duncan<EMAIL_ADDRESS>@iand675. @iand675
digestive-functors-aeson-1.1.16 (>=0.7 && < 0.10). Oliver Charles<EMAIL_ADDRESS>@ocharles
engine-io-1.2.10 (>=0.7 && < 0.10). Oliver Charles<EMAIL_ADDRESS>@ocharles
fay-<IP_ADDRESS> (>0.6 && < 0.10). Adam Bergmark<EMAIL_ADDRESS>@bergmark
fb-1.0.11 (>=0.5 && < 0.10). Felipe Lessa<EMAIL_ADDRESS>@meteficha. @meteficha
flowdock-<IP_ADDRESS> (>=0.8 && < 0.10). Ian Duncan<EMAIL_ADDRESS>@iand675. @brewtown
generic-aeson-<IP_ADDRESS> (>=0.6 && < 0.10). Silk<EMAIL_ADDRESS>@bergmark @hesselink
gipeda-<IP_ADDRESS> (>=0.7 && < 0.10). Joachim Breitner<EMAIL_ADDRESS>@nomeata
hasql-postgres-0.10.5 (>=0.7 && < 0.10). Nikita Volkov<EMAIL_ADDRESS>@nikita-volkov
heist-<IP_ADDRESS> (>=0.6 && < 0.10). No maintainer. @mightybyte
hoauth2-0.4.8 (>=0.7 && < 0.10). No maintainer. @freizl
ide-backend-0.10.0 (>=0.6.2 && < 0.10). FP Complete<EMAIL_ADDRESS>@snoyberg.
ide-backend-common-0.10.0 (>=0.6.2 && < 0.10). No maintainer.
ihaskell-<IP_ADDRESS> (>=0.7 && < 0.10). Andrew Gibiansky<EMAIL_ADDRESS>@gibiansky. @gibiansky
ipython-kernel-<IP_ADDRESS> (>=0.6 && < 0.10). Andrew Gibiansky<EMAIL_ADDRESS>@gibiansky. @gibiansky
json-autotype-1.0.7 (>=0.7 && < 0.10). Michal J. Gajda<EMAIL_ADDRESS>@mgajda
json-schema-<IP_ADDRESS> (>=0.7 && < 0.10). Silk<EMAIL_ADDRESS>@bergmark @hesselink
mandrill-<IP_ADDRESS> (>=<IP_ADDRESS> && < <IP_ADDRESS>). Alfredo Di Napoli<EMAIL_ADDRESS>@adinapoli
mangopay-1.11.4 (>=0.7 && < 0.10). Felipe Lessa<EMAIL_ADDRESS>@meteficha. @meteficha
pandoc-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.10). No maintainer. @jgm
pandoc-citeproc-<IP_ADDRESS> (>=0.7 && < 0.10). No maintainer. @jgm
pandoc-types-<IP_ADDRESS> (>=0.6.2 && < 0.10). No maintainer. @jgm
persistent-template-<IP_ADDRESS> (>=0.7 && < 0.10). Michael Snoyman<EMAIL_ADDRESS>@snoyberg. @snoyberg
pipes-aeson-<IP_ADDRESS> (>=0.6.1 && < 0.10). Renzo Carbonara<EMAIL_ADDRESS>@k0001. @k0001
purescript-<IP_ADDRESS> (>=0.8 && < 0.10). Oleg Grenrus<EMAIL_ADDRESS>@phadej. @purescript
rest-core-<IP_ADDRESS> (>=0.7 && < 0.10). Silk<EMAIL_ADDRESS>@bergmark @hesselink
rest-gen-<IP_ADDRESS> (>=0.7 && < 0.10). Silk<EMAIL_ADDRESS>@bergmark @hesselink
rest-stringmap-<IP_ADDRESS> (>=0.7 && < 0.10). Silk<EMAIL_ADDRESS>rest-types-<IP_ADDRESS> (>=0.7 && < 0.10). Silk<EMAIL_ADDRESS>@bergmark @hesselink
rethinkdb-<IP_ADDRESS> (>=0.7 && < 0.10). Brandon Martin<EMAIL_ADDRESS>@codedmart. @atnnn
servant-server-<IP_ADDRESS> (>=0.7 && < 0.10). Haskell Servant<EMAIL_ADDRESS>@jkarni. @haskell-servant
smoothie-0.4.1 (>=0.8 && < 0.10). Dimitri Sabadie<EMAIL_ADDRESS>@phaazon
snap-<IP_ADDRESS> (==0.6.* || >=<IP_ADDRESS> && < 0.10). No maintainer. @mightybyte
snaplet-fay-<IP_ADDRESS> (>=0.6 && < 0.10). Adam Bergmark<EMAIL_ADDRESS>@bergmark
socket-io-1.3.3 (>=0.7 && < 0.10). Oliver Charles<EMAIL_ADDRESS>@ocharles
stylish-haskell-<IP_ADDRESS> (>=0.6 && < 0.10). Jasper Van der Jeugt. @jaspervdj
tttool-<IP_ADDRESS> (>=0.7 && < 0.10). Joachim Breitner<EMAIL_ADDRESS>@entropia
twitter-feed-<IP_ADDRESS> (>=0.8 && < 0.10). Juan Pedro Villa Isaza<EMAIL_ADDRESS>@jpvillaisaza. @stackbuilders
wai-routes-0.8.1 (>=0.8 && < 0.10). Anupam Jain<EMAIL_ADDRESS>@ajnsit
webdriver-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.10). FP Complete<EMAIL_ADDRESS>@snoyberg. @kallisti-dev
yesod-auth-oauth2-0.1.3 (>=0.6 && < 0.10). Paul Harper<EMAIL_ADDRESS>@thoughtbot
yesod-mangopay-1.11.4 (>=0.6 && < 0.10). Felipe Lessa<EMAIL_ADDRESS>@meteficha. @meteficha

new revisions of binary-orphans and binary-tagged published

Also, be aware of really breaking change https://github.com/bos/aeson/issues/287

twitter-feed is updated in Hackage now.

Mandrill <IP_ADDRESS> is updated & on Hackage now, fixed by: https://github.com/adinapoli/mandrill/pull/22

json-autotype is fixed too.
On Sunday, September 20, 2015, Alfredo Di Napoli<EMAIL_ADDRESS>wrote:

Mandrill <IP_ADDRESS> is updated & on Hackage now, fixed by:
adinapoli/mandrill#22 https://github.com/adinapoli/mandrill/pull/22
—
Reply to this email directly or view it on GitHub
https://github.com/fpco/stackage/issues/845#issuecomment-141772707.

--
Pozdrawiam
Michał

Is there actually any point in everyone reporting here that a fix is uploaded? I would presume that the stackage admins would see what’s fixed and what not based on their tool, and use that to check it. At least until there are only two or three packages left which everyone is waiting on, in which case a “I’m finally done message” might be useful.

smoothie fixed.

"hasql-postgres" is fixed, but I would love to see the maintainers' input on @nomeata's comment.

Is there actually any point in everyone reporting here that a fix is uploaded? I would presume that the stackage admins would see what’s fixed and what not based on their tool, and use that to check it. At least until there are only two or three packages left which everyone is waiting on, in which case a “I’m finally done message” might be useful.

When it's just a few packages: telling us to try again is very helpful. With lots of packages: everyone needs to do the mental checklist of "did everyone else already fix this?" In which case: it's less useful. The curator team will regularly check on open issues and see if they're still relevant, so that seems like the right way to address these big issues.
How's this for a policy: if there are five or more package maintainers pinged in an issue, don't bother writing back.
@bergmark @DanBurton @juhp Any thoughts on that?

Whatever helps You best Michael!
Thanks for making awesome contribution to the commuńity, and making sure
that our packages are all together compatibleand up to date.
On Monday, September 21, 2015, Michael Snoyman<EMAIL_ADDRESS>wrote:

Is there actually any point in everyone reporting here that a fix is
uploaded? I would presume that the stackage admins would see what’s fixed
and what not based on their tool, and use that to check it. At least until
there are only two or three packages left which everyone is waiting on, in
which case a “I’m finally done message” might be useful.
When it's just a few packages: telling us to try again is very helpful.
With lots of packages: everyone needs to do the mental checklist of "did
everyone else already fix this?" In which case: it's less useful. The
curator team will regularly check on open issues and see if they're still
relevant, so that seems like the right way to address these big issues.
How's this for a policy: if there are five or more package maintainers
pinged in an issue, don't bother writing back.
@bergmark https://github.com/bergmark @DanBurton
https://github.com/DanBurton @juhp https://github.com/juhp Any
thoughts on that?
—
Reply to this email directly or view it on GitHub
https://github.com/fpco/stackage/issues/845#issuecomment-141868023.

--
Pozdrawiam
Michał

:+1:

Due to the breaking change that got released mentioned by @phadej, I would recommend that nobody use aeson 0.10 for now. There is already a pull request to attempt to fix this situation: https://github.com/bos/aeson/pull/288

@gregwebs yeah I've been holding out to see how that clears up.

For webdriver, my solution was to simply add a new operator to Test.WebDriver.JSON and use it in place of .:?
(.:??) :: FromJSON a => Object -> Text -> Parser (Maybe a)
o .:?? k = fmap join (o .:? k)

I believe this should have the same result regardless of which aeson version you use.

I used Adam's solution in json-autotype too, and it passed all unit tests
without issues.
On Monday, 28 September 2015, Adam Curtis<EMAIL_ADDRESS>wrote:

For webdriver, my solution was to simply add a new operator to
Test.WebDriver.JSON and use it in place of .:?
(.:??) :: FromJSON a => Object -> Text -> Parser (Maybe a)
o .:?? k = fmap join (o .:? k)

--
Pozdrawiam
Michał

I was a bit impatient so I made Data.Aeson.Compat

Fixed in http://hackage.haskell.org/package/bloodhound-<IP_ADDRESS> but I had to bring the lower-bound up.

Update:
aeson-<IP_ADDRESS> depended on by:

aeson-schema-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.10). Mateusz Kowalczyk<EMAIL_ADDRESS>@Fuuzetsu. @Fuuzetsu
datadog-<IP_ADDRESS> (>=0.8 && < 0.10). Ian Duncan<EMAIL_ADDRESS>@iand675. @iand675
eventstore-<IP_ADDRESS> (>=0.8 && < 0.10). Yorick Laupa<EMAIL_ADDRESS>@YoEight. @YoEight
flowdock-<IP_ADDRESS> (>=0.8 && < 0.10). Ian Duncan<EMAIL_ADDRESS>@iand675. @brewtown
hoauth2-0.4.8 (>=0.7 && < 0.10). No maintainer. @freizl
ide-backend-0.10.0 (>=0.6.2 && < 0.10). FP Complete<EMAIL_ADDRESS>@snoyberg.
ide-backend-common-<IP_ADDRESS> (>=0.6.2 && < 0.10). No maintainer.
pipes-aeson-<IP_ADDRESS> (>=0.6.1 && < 0.10). Renzo Carbonara<EMAIL_ADDRESS>@k0001. @k0001
rethinkdb-<IP_ADDRESS> (>=0.7 && < 0.10). Brandon Martin<EMAIL_ADDRESS>@codedmart. @atnnn
riak-<IP_ADDRESS> (>=0.8 && < 0.10). Tim McGilchrist<EMAIL_ADDRESS>@tmcgilchrist. @markhibberd
snaplet-fay-<IP_ADDRESS> (>=0.6 && < 0.10). Adam Bergmark<EMAIL_ADDRESS>@bergmark
stack-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.10). Emanuel Borsboom<EMAIL_ADDRESS>@borsboom. @commercialhaskell
swagger2-0.4.1 (< 0.10). Nickolay Kudasov<EMAIL_ADDRESS>@fizruk. @GetShopTV
yesod-auth-oauth2-0.1.5 (>=0.6 && < 0.10). Paul Harper<EMAIL_ADDRESS>@thoughtbot

postgresql-binary-<IP_ADDRESS> depended on by:

hasql-postgres-0.10.6 (==0.5.*). Nikita Volkov<EMAIL_ADDRESS>@nikita-volkov

Fixed eventstore https://hackage.haskell.org/package/eventstore-<IP_ADDRESS>

I've bumped stack's upper bound for aeson on Hackage (after checking that all tests still pass).

The package "hasql-postgres" has been deprecated, can you exclude it please?

Hi Nikita,
Since the packages are usually included by either maintainer or dependency
to other package, it will probably happen when the respective person
submits a pull request to GitHub.
On Wednesday, 16 December 2015, Nikita Volkov<EMAIL_ADDRESS>wrote:

The package "hasql-postgres" has been deprecated, can you exclude it
please?
—
Reply to this email directly or view it on GitHub
https://github.com/fpco/stackage/issues/845#issuecomment-165033054.

--
Pozdrawiam
Michał

@mgajda the hasql-postgres is under nikita's name. Made a PR. https://github.com/fpco/stackage/pull/1044

This is blocking at least bloodhound from being upgraded.

aeson-schema-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.10). Mateusz Kowalczyk<EMAIL_ADDRESS>@Fuuzetsu. @Fuuzetsu
datadog-<IP_ADDRESS> (>=0.8 && < 0.10). Ian Duncan<EMAIL_ADDRESS>@iand675. @iand675
flowdock-<IP_ADDRESS> (>=0.8 && < 0.10). Ian Duncan<EMAIL_ADDRESS>@iand675. @brewtown
hoauth2-0.4.8 (>=0.7 && < 0.10). No maintainer. @freizl
ide-backend-0.10.0 (>=0.6.2 && < 0.10). FP Complete<EMAIL_ADDRESS>@snoyberg.
ide-backend-common-<IP_ADDRESS> (>=0.6.2 && < 0.10). No maintainer.
pipes-aeson-<IP_ADDRESS> (>=0.6.1 && < 0.10). Renzo Carbonara<EMAIL_ADDRESS>@k0001. @k0001
rethinkdb-<IP_ADDRESS> (>=0.7 && < 0.10). Brandon Martin<EMAIL_ADDRESS>@codedmart. @atnnn
riak-<IP_ADDRESS> (>=0.8 && < 0.10). Tim McGilchrist<EMAIL_ADDRESS>@tmcgilchrist. @markhibberd
swagger2-0.4.1 (< 0.10). Nickolay Kudasov<EMAIL_ADDRESS>@fizruk. @GetShopTV
yesod-auth-oauth2-0.1.5 (>=0.6 && < 0.10). Paul Harper<EMAIL_ADDRESS>@thoughtbot

We decided to start the new year and the upcoming LTS 4 with a clean slate. We'll be disabling most packages that block upgrades. Please send a pull request to re-enable your package if you update it. If you do so before new years you'll be included in LTS 4.

I fixed this by using aeson-extra which provides a stable interface

@bergmark okay, so this means Bloodhound is disabled then? I asked you if I should restore compatibility with Aeson <0.10 --- some notice on this decision would've been appreciated.
In the meantime: @gregwebs - can I use aeson-extra to avoid CPP here?

@bitemyapp no sir! the latest version of bloodhound will be included in the next nightly.

@bergmark so the libraries that don't support Aeson 0.10 are the ones getting dropped? Sorry for being a bit slow, I'm still catching up on my coffee for the day at 1849.

@bitemyapp that's correct, we're trying to make sure as much as possible is the latest and greatest before releasing LTS 4.

@bergmark I understand now - thank you. Seems the right idea to me. The Aeson 0.10 upgrade for Bloodhound was a titch traumatic. Users seem fine but I wish I'd maintained cross-lacuna compatibility.

Yep, the CPP is already in aeson-extra

@gregwebs well the Stackage goats have already been sacrificed so I'm holding off unless a user of the library speaks up and wants it. Thank you very much for writing it!

Here's another package that was already disabled (for other reasons that have been resolved now)
aeson-<IP_ADDRESS> depended on by:

pagerduty-0.0.4 (>=0.8 && < 0.10). Brendan Hay<EMAIL_ADDRESS>@brendanhay. @brendanhay

New pagerduty release with relaxed upper bound also dependent upon bos/aeson#293.

Thanks for letting us know!
I asked for a patch release here: https://github.com/bos/aeson/issues/328

There is a mistake in handling cases for Maybe in aeson I wonder why no one noticed it :); therefore there is no breaking changes
https://github.com/bos/aeson/pull/326
please do not create -extra packages to fix them.

Fixed riak https://hackage.haskell.org/package/riak-<IP_ADDRESS>

I am afraid I will have to disable riak again:
vector-<IP_ADDRESS> depended on by:

riak-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.11). Tim McGilchrist<EMAIL_ADDRESS>@tmcgilchrist. @markhibberd

@bergmark Released new rethinkdb-<IP_ADDRESS> with aeson 0.10 support.

Released new swagger2-1.0 with aeson-<IP_ADDRESS> support. Corresponding PR — #1066.

If we bump that dependency is there anything else that's a problem?
We have a build setup against stack nightly (nightly-2015-12-20) is there
something else I can point to to check proactively whether we've got an
issue?
On Mon, Dec 21, 2015 at 3:38 PM, Jens Petersen<EMAIL_ADDRESS>wrote:

I am afraid I will have to disable riak again:
vector-<IP_ADDRESS> depended on by:

riak-<IP_ADDRESS> (>=<IP_ADDRESS> && < 0.11). Tim McGilchrist
<EMAIL_ADDRESS>@tmcgilchrist https://github.com/tmcgilchrist.
@markhibberd https://github.com/markhibberd

—
Reply to this email directly or view it on GitHub
https://github.com/fpco/stackage/issues/845#issuecomment-166201604.

@tmcgilchrist I am not sure what you're asking. You can check the Travis build for v1.0.
The only problem I had with aeson-<IP_ADDRESS> was a couple of bugs fixed in HEAD but not released (e.g. bos/aeson#293 and the misplaced MINIMAL pragma). To fix compatibility with aeson-<IP_ADDRESS> I just replaced TH with Generic-based code.

@tmcgilchrist Do you mean if there are other dependencies you might need to bump? If you can build against the latest nightly and  cabal sandbox init && cabal install --dry-run | grep latest doesn't give any results you should be good. Possibly also pass --enable-tests --enable-benchmarks if you have those

Whenever your code handle Maybe via Value type it will be broken in aeson saying that there is a patch for that. So Basically if you use .:? or anything Value -> Maybe a there will be issues as current version is not working correctly there. So if you have Maybe in your type there is no way to write correct fromJson.

@bergmark Yes that's what I meant. Thanks.
New version of riak up on package <IP_ADDRESS>. @juhp

@tmcgilchrist okay thanks - looks good now

houath is bump to 0.5.0 in order to work with aeson-0.10

In my opinion, it was premature to add aeson 0.10 to stackage.  The problem noted in bos/aeson#296 essentially makes it impossible to compile pandoc-types on a machine with 2 GB RAM or less.  Once pandoc enters lts-4.0, many people are going to find that stack install pandoc simply dies with an unexplained "ExitFailure 127."
As long as aeson 0.10 is in stackage, I'm faced with the choice between putting an upper bound on aeson in pandoc-types, thus excluding the pandoc ecosystem from stackage, or having many users complain that they are unable to install pandoc.  (Yes, I know there are workarounds; you can specify a constraint on aeson; but most people aren't going to know that, and there's no way to figure out that that's what is needed.)
See my comment on haskell reddit.

@jgm the problem is with the generics-based default implementation, right? As an implementation workaround, how about using Data.Aeson.TH instead?

@DanBurton I've studiously avoided using any TH in pandoc, due to problems I had in the past with TH + Windows (I can't recall the details now).  Does Data.Aeson.TH generate the same JSON mapping as the generics implementation?

@jgm it does, IIRC there are actual tests for that in aeson

Even if there is a workaround, I think my point stands.  The aim of stackage is to provide a consistent set of packages that make building Haskell projects unproblematic.  If one of the packages makes projects (not just mine, but anything that derives generic instances for complex product or sum types) unbuildable on machines without lots of RAM, that strikes me as a reason not to include it until the relevant bugs are fixed.

@jgm but that sounds as a bug in GHC, not the package itself.

The aeson-0.10 release was a bit of a cluster fart (looking at https://github.com/bos/aeson/issues/293) and there hasn't been visible movement getting the fixes that are in HEAD released.
Being dependent upon aeson I would of (pre-stackage) just added an upper bound < 0.10, but since the broken package is in stackage nightly I'm forced to somehow play along, or my package versions get put in naughty section of stackage's build_constraints.yaml. Seems there should be provision for omitting or rolling back the offender from stackage (nightly at least), rather than forcing downstream dependencies through acts of choreographed contortion.

Can anyone get in touch with @bos? I think the community can benefit from having additional official maintainers of aeson that are focused on shouldering the burden of stability, releases, and responsiveness.

@phadej Well, it seems to be an interaction between changes specific to aeson 0.10 and ghc 7.10.3.  Earlier version of aeson do not have the same problem, even with ghc 7.10.3.  So the hope is that something can be fixed in aeson, even independent of fixes to ghc.  Someone just commented on the aeson issue that removing the INLINE pragmas might do it.

We stackage curators have been considering our options. We didn't properly take the regressions of aeson-0.10 into consideration when we selected it for LTS-4; we intend to take action over the next few days to remedy the situation.

I'd like to help with aeson situation, if only there is something I can help with.

heads up: aeson just merged a PR to revert some of the 0.10 nullability change:
https://github.com/bos/aeson/pull/288

Also, we're exploring a workaround to aeson-0.10's generic-deriving issues here.

@brendanhay (and @ all) as of tonight's nightly, aeson 0.9 is back, and you're no longer in the naughty section for failing to support aeson <IP_ADDRESS>. lts-5 will be released (planned for 2016-01-24) with aeson 0.9.
See #1155 for details.

Thank you!

I've opened a pull request to aeson to address the compilation slowdowns related to GHC generics.

To whomever is still listening to this thread, aeson-0.11 is being prepared for release. If you can clone the repo and test it out with your package, it would be much appreciated.

I will!

At least aeson's master suffers from QuickCheck instances https://travis-ci.org/phadej/aeson/jobs/104043153

also aeson-compat (with small adoptions) and aeson-extra tests pass with nightly-2016-01-22  amended with aeson's master


Updated matrix

Hi, could you please provide the  'conf/conf_mat_updated_8.npy' used in the evaluation scripts? I saw that they got updated in the training notebooks, but I am encountering problems with apex in my system and could not run the training. Since you provided the weights/checkpoints already, could you please provide the updated matrix?

Hi, thank you for your interest in our work. Unfortunately, we do not retain the weights for the updated matrix. However, feel free to share the issues you're encountering with Apex, and maybe we can discuss possible solutions.


Buffer for a decoder

Is it possible to set output yuv buffers for a decoder ?
There is no need to copy yuv to my own buffer if I can set the yuv buffer.
BTW, Why the openh264 preserve 3 yuv buffers for a decoder ?
thank you.

Not sure if I completely got what you mean.
 However, it is not supported.
A copy operation is must whatever in outer decoder (current implementation) or inner decoder (your assumption?).
for 3 yuv buffers, it was designed that YUV components can be independently stored, and also it is reserved for possible RGB or other color space formats.

Thank you for answer.


multiple checks at one go!!

Hi,
I would like to get multiple checks if I click one check box.

What do you mean by multiple checks?

it means that i want to check one item..which should simultaneously check
other dependent items(not all but some)
[image: image001]
Regards,
Himanshu Chaudhary
Contact : +91-9035662923 |<EMAIL_ADDRESS><EMAIL_ADDRESS>Hewlett-Packard Company | 39/40 | Digital park | Hp Avenue | EC-1 |
Bangalore | Karnataka | India
On Fri, Feb 12, 2016 at 9:57 PM, Adrian Ber<EMAIL_ADDRESS>wrote:

What do you mean by multiple checks?
—
Reply to this email directly or view it on GitHub
https://github.com/vitalets/checklist-model/issues/125#issuecomment-183397993
.

If you want to this then you need to either use checklistChange or scope.$watch on the model.