convert() and fieldmap() swallow errors silently by default

This has led to a lot of difficult-to-debug scenarios in my experience.
As you're developing an ETL pipeline, you're often iterating on your conversion functions quickly, which often leads to exceptions being raised. Currently petl eats those errors silently, making it far more difficult to navigate this process.
In the regular operation of an ETL pipeline, it's also undesirable for new, unexpected errors to pass silently.
The current workaround is to set failonerror=True in every call to fieldmap() and the like. I suggest you make failonerror=True be the default. This might be something that would require bumping to version 2.0 since it will likely cause a lot of exceptions to be raised that were previously ignored (for better or for worse!).

Thanks @thatneat, I have contemplated this many times myself for the same reason. I would be happy to make this change.
Regarding version number, strictly speaking this probably should be a major version bump. However I don't have any other major changes planned, so maybe this could be just a minor release with some very clear documentation about the change in behaviour.

In addition, it seems that that flag isn't available on rowgroupmap, but I am not sure why. Is there an implementation reason why it wouldn't be possible?

That is an inconsistency, would be good to review all transform functions
and consistify error handling.
On Monday, January 18, 2016, pmariani<EMAIL_ADDRESS>wrote:

In addition, it seems that that flag isn't available on
rowgroupmap, but I am not sure why. Is there an implementation reason why
it wouldn't be possible?

—
Reply to this email directly or view it on GitHub
https://github.com/alimanfoo/petl/issues/365#issuecomment-172632518.

--
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health http://cggh.org
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Web: http://purl.org/net/aliman
Email<EMAIL_ADDRESS><EMAIL_ADDRESS>Tel: +44 (0)1865 287721

Your call of course, but I'd still lean towards the major version bump, so nobody gets unwanted surprises. Plus being on version 2.0 would help give the appearance of maturity that this project deserves ;)
I could probably come up with a few more breaking changes if you'd like.

I would also prefer a more strict error behavior by default.
convert is the one I have run into problems with custom functions.
TBH I considered this swallow-exceptions-by-default the only major issue with PETL but thought it is such a big change for backward compatibility that I dared not hope it would ever change.
Because this is a big user visible change, that will break a lot of existing code, it must be a major version bump and I agree with @thatneat on "version 2.0 would help give the appearance of maturity that this project deserves" :)

Thanks @thatneat, @krisztianfekete, I would be very happy to include these changes in a 2.0 release. I'll write an email to the list as well asking if anyone has any other backwards-incompatible changes they'd like to propose for 2.0.

FWIW, I agree with the proposal to have errors thrown by default and to bump the version to 2.0.
You may want to do an intermediate release that throws warnings when the errors are swallowed letting devs know that the 2.0 release will be releasing a breaking change in those instances.  Let that simmer for 3-6 months?

On Wednesday, January 20, 2016, Randy Syring<EMAIL_ADDRESS>wrote:

FWIW, I agree with the proposal to have errors thrown by default and to
bump the version to 2.0.

Thanks Randy.

You may want to do an intermediate release that throws warnings when the
errors are swallowed letting devs know that the 2.0 release will be
releasing a breaking change in those instances. Let that simmer for 3-6
months?

Worth considering, but FWIW I think I'd be inclined to go straight for a
2.0 release and ensure behaviour changes are very clear in documentation
and release announcements, would be simpler to implement at least.

—
Reply to this email directly or view it on GitHub
https://github.com/alimanfoo/petl/issues/365#issuecomment-173363943.

--
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health http://cggh.org
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Web: http://purl.org/net/aliman
Email<EMAIL_ADDRESS><EMAIL_ADDRESS>Tel: +44 (0)1865 287721

Yes, please:

the default behaviour should be changed such that any errors are propagated by default instead.
make it happen in a new major release (2.0).

And, once again, thank you for this wonderful tool.

Thanks Brad.
On Monday, 25 January 2016, bmaggard<EMAIL_ADDRESS>wrote:

Yes, please:

the default behaviour should be changed such that any errors are
propagated by default instead.
make it happen in a new major release (2.0).

And, once again, thank you for this wonderful tool.
—
Reply to this email directly or view it on GitHub
https://github.com/alimanfoo/petl/issues/365#issuecomment-174505742.

--
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health http://cggh.org
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Web: http://purl.org/net/aliman
Email<EMAIL_ADDRESS><EMAIL_ADDRESS>Tel: +44 (0)1865 287721

Can you give an ETA for this @alimanfoo?

Hi @thatneat, I won't be able to get to this for at least a couple of
months, I would suggest to go ahead and insert the failonerror=True
statements.
On Friday, April 8, 2016, thatneat<EMAIL_ADDRESS>wrote:

Can you give an ETA for this @alimanfoo https://github.com/alimanfoo?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
https://github.com/alimanfoo/petl/issues/365#issuecomment-207532509

--
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health http://cggh.org
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Email<EMAIL_ADDRESS><EMAIL_ADDRESS>Web: http://purl.org/net/aliman
Twitter: https://twitter.com/alimanfoo
Tel: +44 (0)1865 287721

If failonerror=False, petl doesn't eat errors silently, it returns 'errorvalue'.
So, how about also adding a 'returnexceptions' option, akin to:
In [1]: import petl as etl

In [2]: t1 = etl.wrap([ ['foo'], ['A'], [1] ]).convert('foo', 'lower', errorvalue=Exception)

In [3]: t1
Out[3]:
+---------------------+
| foo                 |
+=====================+
| 'a'                 |
+---------------------+
| <class 'Exception'> |
+---------------------+

... but to have the original exceptions returned instead?  This would enable categorizing failures directly.

Nice idea.
On Tuesday, January 10, 2017, Brad Maggard<EMAIL_ADDRESS>wrote:

If failonerror=False, petl doesn't eat errors silently, it returns
'errorvalue'.
So, how about also adding a 'returnexceptions' option, akin to:
In [1]: import petl as etl
In [2]: t1 = etl.wrap([ ['foo'], ['A'], [1] ]).convert('foo', 'lower', errorvalue=Exception)
In [3]: t1
Out[3]:
+---------------------+
| foo                 |
+=====================+
| 'a'                 |
+---------------------+
| <class 'Exception'> |
+---------------------+
... but to have the original exceptions returned instead? This would
enable categorizing failures directly.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/alimanfoo/petl/issues/365#issuecomment-271693111, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AAq8Qg5FiogPHk3I0cV5mU96Fn8f-Lbuks5rQ-5JgaJpZM4HHI-N
.

--
Alistair Miles
Head of Epidemiological Informatics
Centre for Genomics and Global Health http://cggh.org
The Wellcome Trust Centre for Human Genetics
Roosevelt Drive
Oxford
OX3 7BN
United Kingdom
Email<EMAIL_ADDRESS>Web: http://purl.org/net/aliman
Twitter: https://twitter.com/alimanfoo
Tel: +44 (0)1865 287721

Hmm, @alimanfoo just to be clear, it looks to me like #460 explicitly does not solve the named issue in this ticket, right? It looks like the default is still failonerror=False. Is there still a plan to change that in a future release?

Apologies, I closed this issue too hastily. Personally I'd be happy to change the default to failonerror=True but that would need a major version bump. I need to make a quick release for a downstream project, and so was taking the opportunity to merge in some PRs, but didn't want to introduce any changes that might surprise people.
How about we do a quick release 1.3 which includes the new petl.config.failonerror config option, set to False by default (current behaviour). Then follow up with a 2.0 release shortly after which switches the default to True, but precede that with a message on the mailing list to see if anyone has any objections to the change.

@alimanfoo convert lambda doesn't accept a a function for example what i want is
table1 = etl.convert(table, 'id', lambda v: v + nrows(table_a)) , but the result  is  nrows(table_a) not added to id value
instead this works table1 = etl.convert(table, 'id', lambda v: v + 3993) i know it's related to how lambda is working, but can you please suggest any other way to solve it ???

Hi @BouchaaraAdil, can you explain a bit more what you are trying to do? Do you just want to append the same fixed number to every ID value? Or something different?

i'm migrating some data from different tables into one table, and want to avoid user ids conflicts.. so i want to append number of rows or last id from a table X to the table ids, is a fixed number

Ah OK. So can you just do something like:
suffix = nrows(table_a)
table1 = etl.convert(table, 'id', lambda v: v + suffix)

i already did, didn't work :/

i did a trick, stupid  thing but it worked,
employee_user_count = etl.tail(employee_user, 1)['id'][0]
added a temp column to the table with employee_user_count value, then
users_1 = etl.addfields(users, [('employee_user_count', employee_user_count)])
users_2 = etl.cutout(
 etl.convert(users_1, 'id', lambda v, row: v + row.users_employees_count,
pass_row=True), 'employee_user_count'
)


Fix invalid fileSize atom in header

What / Why
According to the TAR specification, the fileSize header atom should be stored as 11 ascii-encoded octals plus a null terminator.
For some reason a ' ' whitespace is being added by this package, which was creating unusable TAR archives for me (using other libraries to decode).
I'm not sure why this space was added, possibly someone saw the null terminator and assumed that there must be a space at the end.  Either way, removing it makes readable archives.

No response, closing.  If this module creates tarballs that some extant tar implementation can't parse, then I'm happy to revisit it.


Decimal format works incorrectly, when the number is close to a power of 10

Steps to reproduce:
new DecimalFormat("#,##0.0").format(999.999)

Expected:
1,000.0
Got:
:00.0
: code is 58, which is 0 + 10, so there seems to be an issue with the way TDecimalFormat splits the number into digits.

What TeaVM version did you try?

0.7.0-dev-1113
Yeah, sorry for missing that.

Please, try the latest one

Yeah, doesn't seem to be an issue for 0.7.0-dev-1185, thanks!

1185 is not the latest version, BTW

Oh, it seems that we take it from the Bintray repository (where 1185 is the latest available) and should use https://teavm.org/maven/repository, right.


Minimise import for testing?

Description
Would it be possible to add an option to specify how many rows to import from a CSV?
Currently, I maintain two versions of my CSV file: a full export and a smaller, trimmed version. It would be much more convenient to manage just one file and have the importer allow me to define how many rows to import—for example, the first X rows or even specific rows.
This feature would be especially helpful as my import includes various types of data, with some rows containing fields that others do not. Having this flexibility would streamline the process.

Great idea!
However, I'm cautious of adding too many features that aren't needed by the majority of developers using the importer. For that reason, I'm going to close this issue.
We may reconsider this feature in the future if others need it.


[FLINK-4521][web frontend] Fix "Submit new Job" panel in development mode

Submit panel was completely empty in the development mode. This happened because client-side code sent AJAX requests to a development HTTP server (localhost:3000) and not to the the JobServer (localhost:8081) and development server rejected all AJAX requests.
To fix the issue I did the following:

Updated server.js to pass both GET and POST requests to the JobServer (localhost:8081)
Changed all AJAX requests to use development HTTP server
Added a helper function to build a correct URL for both dev and prod modes

Also, it seems that CoffeeScript and JavaScript code were out of sync in the master, so this PR contains a significant number of changes to index.js.

[x] General

The pull request references the related JIRA issue ("[FLINK-XXX] Jira title text")
The pull request addresses only one issue
Each commit in the PR has a meaningful commit message (including the JIRA id)

[x] Documentation

Documentation has been added for new functionality
Old documentation affected by the pull request has been updated
JavaDoc for public methods has been added

[x] Tests & Build

Functionality added by the pull request is covered by tests
mvn clean verify has been executed successfully locally or a Travis build has passed

@rmetzger and @iampeter
What do you think about this pull request? Does this fit with the current Web UI efforts?

Hi @iampeter
Thank you for your review and sorry for the long delay.
I'll update the code in the next few days.

Hi @iampeter
I've fixed the code according to your review.
Could you please review it again?
Best regards,
Ivan.

@mushketyk looks good, thanks.

Hi @iampeter
Thank you for your review.
Do I need to rebase this PR to merge it?

@mushketyk I suppose so, right @rmetzger ?

@rmetzger @iampeter  I've rebased my PR on top of mainline.

Sorry for the long delay with this one. The jar request URL has been fixed in another PR by now. This leaves the refactoring with the util instead of manually concatenating the URL. Should we change this @iampeter? I like it, but we would have to rebase this yet again as new requests have been added in the mean time. Sorry again for the long delay.

Hi @uce,
No worries about the long delay. Just write here if you decide to merge it and I'll rebase it.
Best regards,
Ivan.

Hi @uce
I tried the latest version in the master branch and it seems that submit does not work there.

I rebased my PR on top and tested it and it seems to work fine.
Are you sure that all issues with submit were fixed in current version?

We have a new web dashboard now.
I'm closing this as "Abandoned", since there is no more activity and the code base has moved on quite a bit. Please re-open this if you feel otherwise and work should continue.


ARROW-11699: [R] Implement dplyr::across() for mutate()

This PR introduces a partial implementation for dplyr::across() when called within dplyr::mutate().
arrow_table(iris) %>%
  mutate(across(starts_with("Sepal"), list(round, sqrt)))
#> Table (query)
#> Sepal.Length: double
#> Sepal.Width: double
#> Petal.Length: double
#> Petal.Width: double
#> Species: dictionary<values=string, indices=int8>
#> Sepal.Length_1: double (round(Sepal.Length, {ndigits=0, round_mode=HALF_TO_EVEN}))
#> Sepal.Length_2: double (sqrt_checked(Sepal.Length))
#> Sepal.Width_1: double (round(Sepal.Width, {ndigits=0, round_mode=HALF_TO_EVEN}))
#> Sepal.Width_2: double (sqrt_checked(Sepal.Width))
#>
#> See $.data for the source Arrow object

I've opened a number of follow-up tickets for the tasks needed to be done to provide a more complete implementation:

ARROW-17362: [R] Implement dplyr::across() inside summarise()
ARROW-17387: [R] Implement dplyr::across() inside filter()
~ARROW-17364: [R] Implement .names argument inside across()~ (now done in this PR, will close it once this is merged)
ARROW-17366: [R] Support purrr-style lambda functions in .fns argument to across()

Will come back to this once I'm back from holiday, but as a reminder to my future self, where I was up to:

I have a minimal working example that runs successfully in mutate()
I wanted to incorporate the .names argument from across() and so had refactor the column naming code out into its own function, to pass that into later
I tried to implement across() in summarise() - this broke things with the .groups argument and I don't yet understand what/why, other than that .groups works differently with Datasets versus Tables - there may be some context in https://github.com/apache/arrow/issues/11184
it's also worth considering whether it's worth just focussing on across() for mutate() as a first step, and extending to summarize() later

I've opened https://issues.apache.org/jira/browse/ARROW-17362 as a follow-up to implement across() for summarise() too. It doesn't quite work as-is, and I don't want to overcomplicate this PR for now.

This PR now also implements the .names argument.

This is so close! I'm definitely in favor of deferring filter() and summarise() to another PR (although it will be confusing if they are not implemented before the next release). I really like how you split up the functions in dplyr-across.R...made it easy to review!
The one thing I think this needs is a set of tests directly on expand_across(). I think you can very easily convert the existing tests to something like:
library(rlang, warn.conflicts = FALSE)
library(arrow, warn.conflicts = FALSE)
#> Some features are not enabled in this build of Arrow. Run `arrow_info()` for more information.

testthat::expect_identical(
  arrow:::expand_across(
    data.frame(dbl = double(), dbl2 = double()),
    rlang::quos(across(c(dbl, dbl2), round))
  ),
  list(
    dbl = quo(round(dbl)),
    dbl2 = quo(round(dbl2))
  )
)

Created on 2022-08-26 by the reprex package (v2.0.1)
(With apologies for not suggesting that in my first review! I think those tests would help to communicate exactly what expand_across() is doing and highlight the fact that you did a really good job separating the implementation into highly testable pieces.)

@paleolimbot Thanks for the review!  Quick question for you; I want to keep in the tests which have across() inside of mutate() as it makes it a lot easier to see "how are we matching dplyr's functionality" which I imagine will come up if when there are bug reports about this.  Any tips as to which functionality to include in the expand_across() tests vs. the mutate(across(...) tests? Or is this going to be a pain, and I should just swap them all over?

I think you nailed it - the mutate() tests are about matching dplyr's functionality (e.g., they will fail if dplyr changes something and makes this code out of sync); the expand_across() tests are about expand_across(); and we need both.
I would personally just copy/paste all the places where across() appears in your mutate tests but expect_identical(expand_across(quo(across(...)), ...) instead. If you feel this is out of scope for this PR I am OK with that, too (but I think it's worth trying).

I absolutely agree with the argument that this is a big function and it'd be worth having tests for it, though also wondering if copying and pasting every single test might be a bit excessive - we don't necessarily test intermediate functions in some of our other dplyr verbs (though perhaps we should).  I might just check with @nealrichardson or @jonkeane to see if they see any middle ground in what needs to be tested - if there isn't, I'm happy to copy and paste everything and approach it that way.

Haven't looked at the PR lately but, as I understand it, expand_across() takes a list of quosures and returns a list of quosures. So you should be able to unit-test all of its behavior in isolation, and then you want one or two tests in mutate() that it does the right thing in practice (integration tests). Of course, doing everything in mutate() has its conveniences; you could make it easier to write the unit tests with a test helper, something like:
expect_across_equal <- function(actual, expected) {
  expect_identical(expand_across(actual), expected)
}

expect_across_equal(
  quos(lgl, across(is.numeric, ~ . * 5)),
  quos(lgl, int * 5, dbl * 5)
)

@paleolimbot I've updated the tests as per your and Neals' useful suggestions. Mind giving it another review now?


Create functions project

Demo project for using Cloud Functions from App.  Includes an echo server.
This pull request is primarily to reproduce an issue with Cloud Functions.  However, with a little cleanup, this would be very useful in it's own right.

Hi @jpoag, thanks for taking the time to make this. I see it has some conflicts with the existing sample app that would need to be resolved. I'm guessing when you started creating this sample app, the other one didn't exist yet? Then is this pull request obsolete?

No longer needed.  Issue #32 has been fixed in C++ SDK release 5.3.0 https://firebase.google.com/support/release-notes/cpp-relnotes


Fixed #29983 -- Replaced os.path with pathlib.Path.

https://code.djangoproject.com/ticket/29983

Thanks for the review! I've made the suggested simplifications and included a note in the release notes. If you have suggestions on the wording, let me know.

Makes sense. I'll try to identify all settings that could use a Path and either check if a test exists or add one. I'll report back once complete.

I'll report back once complete.

Hey @jdufresne — Thanks for all the related PRs. Great stuff. More incoming?

Hi Carlton, I have a few more settings to verify before rebasing this PR. Just one example is LOCALE_PATHS, but there may be others. I'll comment here when I believe all are covered.

All settings now have a test in PR #12042.

Awesome work!


Scala Kinesis Enrich: move resolver out of config.hocon into dedicated CLI argument

This is because the Iglu resolver is not part of the topology definition, but rather something at the business level.

Duplicate of #1534


Expose Action to MapAzureSignalR

What things on HubConfiguration would break the Azure SignalR support? Is there anything that would?

EnableJSONP is one we don't support.


Where can i find tensorflow 0,12 version?

It seems that i can't find tensorflow versio 0,12.. Can anyone help me to find it?

If you mean the source code, it's under branch r0.12: https://github.com/tensorflow/tensorflow/tree/r0.12

https://github.com/tensorflow/tensorflow/releases/tag/v0.12.0

We do not support documentation for that version on the site, but those are the best resources if you must use it.


WWW cleanup

See https://trello.com/c/HZ3WNrmX/19-move-vault-lookup-to-the-inventory. Following the first round of testing, this PR performs a few cleanup actions:

renames a few remaining places as www instead of www-dev
switches back to the http://github.com/openmicroscopy/www.openmicroscopy.org repository for deploying the website
removes the vault lookup step from the public playbook
removes the deprecated Jinja template superseded by the usage of openmicroscopy.nginx-proxy

👍 - works, tested this branch against ome-www-dev


Implement a automated DNS solution based on the Hetzner DNS

Interesting to check how the setup of subdomain delegation can be done here: https://veducate.co.uk/dns-delegation-route53/

Used Cloudflare DNS


Fix the automated updates (another try)

Need to set the correct path to create the PR from..
See https://github.com/NixOS/nixpkgs-check-by-name/actions/runs/8383286032/job/22958707099

No worries, I appreciate you clicking the buttons!
And yeah GitHub Actions are pretty terrible to bootstrap, though I should've really tested on a separate branch here haha :sweat_smile:


Allow app installation without Bluetooth Low Energy from Google App Store

Avoid duplicates

[X] This enhancement request has not already been raised before
[X] Enhancement request is specific for Android only, for general issues / questions that apply to iOS and Android please raise them in CWA-Wishlist
[ ] If you are proposing a new feature, please do so in CWA-Wishlist

Current Implementation
CWA Android targets distribution through the Google Play Store only (see https://www.coronawarn.app/en/faq/#apk) and if a user goes directly to https://play.google.com/store/apps/details?id=de.rki.coronawarnapp from a device without Bluetooth Low Energy (BLE) support, they will see the following:

The Android manifest declares android.hardware.bluetooth_le to be required, which causes the app to be blocked on Google Play Store if bluetooth_le is not available on the device.
https://github.com/corona-warn-app/cwa-app-android/blob/9ec222a431340917834e831cc60edd14fa062d27/Corona-Warn-App/src/main/AndroidManifest.xml#L9-L11
Suggested Enhancement
Change Android manifest to set bluetooth_le to not required:
    <uses-feature
        android:name="android.hardware.bluetooth_le"
        android:required="false" />

According to https://developer.android.com/guide/topics/manifest/uses-feature-element
"When you declare android:required="false" for a feature, it means that the application prefers to use the feature if present on the device, but that it is designed to function without the specified feature, if necessary."
Expected Benefits
Although early versions of the app relied heavily on the availability of Bluetooth Low Energy (BLE) on the device, many functional enhancement to the app, which have been added over the lifetime of the app, do not need BLE to work:

PCR Test registration
Rapid Antigen Test registration
Rapid Antigen Test profile
Contact Journal
Statistics
Event QR code creation
Event check-in
Exposure notification from event

Removing the hard restriction for BLE would allow users of devices without BLE support to be able to use the other functions of the app. Older devices are the ones which may fall into this category. Also in conjunction with this enhancement request, the enhancement for support of Android 5.0 (Lollipop) https://github.com/corona-warn-app/cwa-app-android/issues/1799, could benefit from the relaxation of the hard BLE requirement in currently released versions.
Starting from the Android V2.2.1 release, the app itself performs Bluetooth Low Energy compatibility checks and informs the user about any issues. It is therefore no longer necessary to block installation if BLE support is missing on the device. The user will always be informed if any functionality is not available if the app is installed on a device without full or partial BLE support.
Also, according to https://developer.android.com/guide/topics/connectivity/bluetooth-le
"Devices running Android 4.3 (API level 18) and higher have built-in platform support for Bluetooth Low Energy (BLE) in the central role."
CWA currently supports devices from API 23 onwards. If https://github.com/corona-warn-app/cwa-app-android/issues/1799 is implemented, then API 21 would be supported. Both API levels are later than API 18 quoted above.

Related to

Internal Tracking-ID: EXPOSUREAPP-6359.
Internal Tracking-ID: EXPOSUREAPP-6594.

Related issues:

https://github.com/corona-warn-app/cwa-wishlist/issues/443
https://github.com/corona-warn-app/cwa-wishlist/issues/464

@Ein-Tim
Thanks for the links! I will add Internal Tracking-ID: EXPOSUREAPP-6594 to the main description.

On reflection, and based on https://developer.android.com/guide/topics/connectivity/bluetooth-le
"Devices running Android 4.3 (API level 18) and higher have built-in platform support for Bluetooth Low Energy (BLE) in the central role."
it probably has no benefit to implement this request on its own. So I defer to the other two requests which @Ein-Tim listed.
If there are any good arguments for keeping this as a separate request, let me know and I will re-open.


Now status messages include config values

Now if a status message referes to a configuration option the
syntax configname=configvalue will be used to specify the
configuration option's name as well as its value.
Also deleted the item at the DATA_LIMIT_MESSAGE_INDEX index
in the MESSAGE_TABLE since it is no longer needed.

PTAL


Tasks

[ ] Publish crate
[ ] implement builder for curl::easy::Transfer
[ ] Handling errors with error-chain

This thread will have things which are needed to be done, but I only have little necessity to put them individually.


Fix Heatmap Bug in View Config

This fixes a bug that Nils pointed out where you cannot share the state of your Heatmap controller settings.  The changes are:

Update schemas to handle heatmapControls
Add auto-initialization settings for heatmap controller

I intended for GENE_EXPRESSION_COLORMAP_RANGE coordination type to be used for this, just never got around to hooking it up to any of the controls. Ideally I think we would want the gene expression colormap slider changes to be reflected in all components, not just the heatmap, for consistency. It just happens that right now, the slider lives within the heatmap( and it is more complicated to update the colors of the cells in the scatterplot/spatial since they are computed in JS rather than directly in a shader)

Ok @keller-mark - could we hook it up for now to the heatmap controls?  I don't see why not since the change would only be in what component uses it i.e it is independent of the validity of the schema.

Yes I think we should hook up GENE_EXPRESSION_COLORMAP_RANGE to the heatmap controls


orderBy on find is broken

Describe the bug
orderBy on find documentation is incorrect and causes an error in the code as it does not appear to accept that type.
Screenshots

To Reproduce
Steps to reproduce the behavior:

Try adding an order by to any find as documented

Expected behavior
A clear and concise description of what you expected to happen:
For it not to error and order correctly
Serverpod versions

_CLI Version: 1.2.0
_Library version: 1.2.0

In the new api under OpNoteField.db the orderBy param is now a callback that gives you the table, exactly like the where param.
You need to change the code to the following:
var fields = await OpNoteField.db.find(
  session,
  where: (t) => t.opNoteSectionId.equals(section.first.id),
  orderBy: (t) => t.order,
);

)

In that case the documentation is wrong

Alright! Will take a look at that, thank you!


Update to support auto-refreshing OAuth2 tokens

Fix for issue #23
Not completely backward-compatible since it removes the refresh_token parameter from Harvest() and replaces it with a client_secret paramter.

Hi @bpabel, I'm new around here and am attempting to implement support for Harvest API v2. Your pull request is a little on the aged side but I suspect the concept of auto-refreshing OAuth2 tokens is a good feature to support. Since I am breaking backwards compatibility on a large scale there is freedom to make some deep changes. I have only used the personal access token method and am not terribly familiar with the OAuth2 mechanism. I'm wondering if you still need that functionality and if so, do you mind helping me through implementing the change in the support API v2?
Cheers

Hi @Brad-eki , I'm not using Harvest anymore so I probably won't have time to convert the changes over to v2 of the API.  Since the PR was never accepted anyway, feel free to ignore it for your update.  Feel free to ping me with any questions and I'll try to answer as best I can though.  Cheers!

@bpabel, the v2 API changes I've made have only moved the data model to fit the new Harvest paradigm. The auth stuff has been left alone. To me the auth stuff, and URL management, are the things that this library has offered over "going it alone".  All that considered I suspect your changes have a good chance of still being valid and almost certainly required. Last action I can see is @dlo marked your PR as "help wanted". I can't seem to find anything as to why your PR hasn't been accepted.
I'd like to have the Harvest API v2 beta tested with the following:

Support for Harvest API v2 (get, update, delete for all entities, including uploading receipt image files to expenses)
all the auth methodologies working in full (eg; OAuth2 with refresh)
rate limiting

Currently OAuth2 refresh is the only one outstanding.
I'm happy to do work towards accepting your PR. Currently I have no experience in OAuth2 so my learning curve will be steep.
Another consideration is my contributor status has been negotiated based on input on supporting the API v2 in a dev branch. So I would prefer to work with your code outside master unless I get further input from @dlo.
Hoping we can get your code in,
Cheers.


Add more information on the error when calling an instance method from a class

Summary or solution description
The error was The following operation is not supported: 'Example.some_method', now it is The following operation is not supported: Calling instance method from class: 'Example.some_method'
How to Reproduce
https://github.com/CityOfZion/neo3-boa/blob/6687308eadbb2cf20caef508da8d357b978da5a8/boa3_test/test_sc/class_test/UserClassWithInstanceMethodFromClass.py#L1-L12
Tests
https://github.com/CityOfZion/neo3-boa/blob/6687308eadbb2cf20caef508da8d357b978da5a8/boa3_test/tests/compiler_tests/test_class.py#L476-L479
Platform:

OS: Windows 10 x64
Python version: Python 3.7

Task linked: CU-86a0b3t9e Add more information on the error when calling an instance method from a class

coverage: 91.863% (-0.09%) from 91.954% when pulling 6687308eadbb2cf20caef508da8d357b978da5a8 on CU-86a0b3t9e into 3bd6bed56f84099b1256798fabe975c143ed1466 on development.


Does the visualizer need to depend on JAX?

I went through installing dependencies manually, but my overall question for this project would be: couldn't the visualizer run with only MeshCat as dependency?

(Verbose user feedback on!) After installing dependencies manually, I ran into the following import error:
RuntimeError: jaxlib is version 0.3.2, but this version of jax requires version >= 0.3.14.

I upgraded jaxlib, but then run into the following:
RuntimeError: jaxlib version 0.3.20 is newer than and incompatible with jax version 0.3.15. Please update your jax and/or jaxlib packages.

So I guess it will be jaxlib==0.3.15 for now. But given other constraints it won't be able to stay at that precise version for long.

I went through installing dependencies manually, but my overall question for this project would be: couldn't the visualizer run with only MeshCat as dependency?

Short answer: yes. Long answer here below.
The visualizer currently needs two piece of information from jaxsim:

The structure of the kinematic tree. This is necessary to build the node tree used by meshcat. Again, I used the jaxsim resource since it was already done, but I plan to move this functionality to ami-iit/rod.
Computing forward kinematics. This is necessary to compute the transforms of all links, and again I used jaxsim because the algorithm was already implemented there. I agree with you that it would be nice porting the FK algo here in meshcat-viz so that the jaxsim dependency could be dropped for good. Also this is planned, even if with lower priority.

For what regards the dependency of JAX, I'm still debating. I'm not sure how what could be the performance that could be achieved with pure numpy, and having a JIT-compiled FK algorithm in JAX (even just CPU) could be quite interesting.
Regarding the versioning problems of jax/jaxlib, it's a consequence of what described in https://github.com/ami-iit/jaxsim/issues/12. Sadly JAX deprecated and removed some experimental module that jaxsim was using, and I still need to finalize the activity to switch to other resources as documented in https://github.com/ami-iit/jaxsim/issues/12.

I made some progress but got stuck again (https://github.com/ami-iit/jaxsim/pull/23). I guess it's time to migrate the jaxsim parser to rod.

Btw, I reproduced the problem of jax/jaxlib versions in https://github.com/ami-iit/jaxsim/actions/runs/3183397287/jobs/5190569189

I isolated the continuous integration part of https://github.com/ami-iit/jaxsim/pull/23 in https://github.com/ami-iit/jaxsim/pull/24. This PR also fixes the jaxlib version problem described in https://github.com/ami-iit/meshcat-viz-python/issues/4#issuecomment-1267101159.

In https://github.com/ami-iit/meshcat-viz-python/pull/7, I removed the direct dependency from jaxsim. We use external libraries to compute FK, and now it can be done either with jaxsim or idyntree (check meshcat_viz.fk).


Add LICENSE file

There is no LICENSE file in this repository. This makes it difficult for other to re-use the contents of this repository. See https://choosealicense.com/no-permission/ on why this is a problem.

https://github.com/it-at-m/startercode-opendata-munich/commit/f32614e1e01ad787fb2931733610fc8519efe0f5


feat(#178): Refactor XmlInstruction

Refactor XmlInstruction#equals() method.
Closes: #178.

History:

feat(#178): add more tests before changing the logic of comparision method
feat(#178): add more fine grained method for comparision
feat(#178): working comparision code
feat(#178): simplify elements comparision
feat(#178): add javadocs for some methods
feat(#178): hasSameAttributes method
feat(#178): add javadocs
feat(#178): refactor XmlInstructionTest
feat(#178): finish with refactoring of XmlInstruction comparision

PR-Codex overview
This PR focuses on improving the code in the ImprovementDistilledObjects and XmlInstruction classes.
Detailed summary

Added a new method replaceArguments in the ImprovementDistilledObjects class to replace arguments in methods.
Refactored the XmlInstruction class to improve code readability and maintainability.
Added new test cases in the XmlInstructionTest class to cover different scenarios.

✨ Ask PR-Codex anything about this PR by commenting with /codex {your question}

@rultor merge

@rultor merge

@volodya-lombrozo OK, I'll try to merge now. You can check the progress of the merge here

@rultor merge

@volodya-lombrozo Done! FYI, the full log is here (took me 3min)


Remove unnecessary mut from queue_shadows

Objective
Fixes #14952

Thanks for the contribution, looks good to me!

I found some similar errors, all in RenderSet::QueueMeshes. I plan to merge them into this PR. Is that okay?

Since this is already marked for final review I would make a follow-up, but I don't think there's a wrong answer here 😊

Since this is already marked for final review I would make a follow-up, but I don't think there's a wrong answer here 😊

Okay, I have submitted the changes!


.style.setAttribute() Does every browser support it? Sometimes it's hard to know it it's not...

.style.setAttribute() Does every browser support it? Sometimes it's hard to know it it's not there because every browser supports it or because it just hasn't been suggested.

This issue was imported from Google Moderator
Moderator votes: +3
Issue added by knod on 2015-04-24
To vote this issue up or down, simply include +1 or -1 in your comment.

+1

Duplicate of #2378 ?
There's no setAttribute in the spec, only setProperty.

Now available at https://caniuse.com/#feat=mdn-api_cssstyledeclaration_setproperty


Implement Grafeas support

Grafeas is a recently announced standard for producing and consuming metadata about components in the
software supply chain. Grafeas defines data types and an API spec that security auditing programs like DevAudit can use to fetch data about vulnerabilities in components like operating system and development packages, and to report occurrences of such vulnerabilities in deployed applications and containers. Supporting the Grafeas spec means that DevAudit will be able to interoperate with other security and auditing tools and services.
As far as I can tell there are two main aspects of supporting Grafeas:
A. Use a Grafeas-compatible server API as a data source. A Grafeas "vulnerability scanning provider" is roughly analogous to a DevAudit datasource, so we already have the basic framework for supporting this.
The Grafeas API spec is defined in Swagger so we need to:

1. Generate the C# client stubs for the Grafeas API using NSwag
[ ] 1. Generate C# classes using the Grafeas Protocol Buffers spec.
[ ] 2. Fill out the implementation for a Grafeas client querying the API for data on supported package managers like NuGet, NPM, Debian epkg et.al
[ ] 3. Implement IDataSource for the Grapheas data source.

B. Report audit results using the Grapheas metadata format and publish the results to a Graphea server. Again the basic framework support is there and all we need to do is implement an AuditReporter to do this.
So far I haven't been able to generate any code for the Grafeas Swagger spec so I'm waiting on https://github.com/Grafeas/Grafeas/issues/63 to continue.

Cleanup old issues till we can plan future development in more detail.


Password reset via console is not working.

Describe the issue:
Password reset via console application is not working properly in following 2 use-cases.

When password policies are enabled the password reset is not working.

https://user-images.githubusercontent.com/41188245/213606234-99bdf5e9-8af1-479f-9247-5d73a994eb0d.mov

When user resets both error and success messages will appear.(But the password will update in this scenario.)

https://user-images.githubusercontent.com/41188245/213611320-86153bda-278e-4dcf-8422-78a5c27e303a.mov
How to reproduce:

Create a new user
Log in to management console
Enable password pattern and password history
Log in to console and try to change password of the created user

Expected behavior:

Environment information (Please complete the following information; remove  any unnecessary fields) :

Product Version:  IS 6.1.0
OS:  Mac
Database: H2

Case 1: When password policies are enabled the password reset is not working

Password policy is not getting validated from the FE level.

Case 2: When user resets both error and success messages will appear.(But the password will update in this scenario.)

Fixed with: https://github.com/wso2/identity-apps/pull/3746

Tested
https://user-images.githubusercontent.com/47152272/214764208-4f513225-1d14-4bc3-8c72-dc7356120b92.mov


Can not  load model  in paddle project

thanks for your nice work!
i have some trouble which  happen in load_model module  when run paddle code of finetuning process.
For example, i saved 10 mid-models during training. there maybe 8 models cannot be load.
waitting for your reply!

Hello, do you solve the problem? I met with same problem. @Brin333


fixfix be enable http auth, some request link never return.

What problem does this PR solve?
Problem Summary:
if you enable_all_http_auth = true in be.conf, then restart be, and keep using curl   -u  "xxxx:xxxx" http://<IP_ADDRESS>:8040/api/health while be is starting. You may encounter a situation where the link does not return.
Reason:
When be is still starting, there is no information about fe master. When you make an api request to be http port, be needs to request authentication information from fe, which will cause it to request a machine with empty ip and port 0. This rpc call will definitely fail (this is not equivalent to a password error). After receiving this failure, be does not send_reply to the api requester, so this api request cannot be returned.
Release note
None
Check List (For Author)

Test

[ ] Regression test
[ ] Unit Test
[ ] Manual test (add detailed scripts or steps below)
[x] No need to test or manual test. Explain why:

[ ] This is a refactor/code format and no logic has been changed.
[ ] Previous test can cover this change.
[ ] No code files have been changed.
[ ] Other reason

Behavior changed:

[x] No.
[ ] Yes.

Does this need documentation?

[x] No.
[ ] Yes.

Check List (For Reviewer who merge this PR)

[ ] Confirm the release note
[ ] Confirm test cases
[ ] Confirm document
[ ] Add branch pick label

run buildall

LGTM


Misplaced oocdocs generate unhelpful errors

/// test
"meep" println()

results in this oh-so-familiar error:
./test.ooc:1:0 ERROR Expected include, import, statement or declaration

/// test

Perhaps something like "Unexpected oocdoc comment." would be more appropriate?

Both in top-level (Module) and function bodies. Might not be so in scopes though.. good candidate for a follow up some time in the future when I feel like touching that again.

Also, the token positions are.. funky, to say the least. Might be related to #898? Hard to tell.


Pandoc has problem to export on pictures in Obsidian: .png :withBinaryFile: does not exist (No such file or directory)

Hi,

Installed the plugin in Obsidian
Installed locally: https://pandoc.org/installing.html
In the settings of the plugin filled the next following path:''Pandoc Plugin'' Pandoc path ''C:\Program Files\Pandoc\pandoc.exe''
In the settings of the plugin filled the next following path:''Pandoc Plugin'' Export Folder ''C:\temp''

The settings of the plugin are correctly filled in.
I'll get an error message when exporting the note to Word (.docx) (Ctrl + P ) --> ''Pandoc Plugin: Export as Word Document (docx)
Error message:
''
Pandoc export failed: pandoc.exe: xxxxxx\Pasted image<PHONE_NUMBER>2627.png: with binary file: does not exist (No such file or directory)
''
I think it is because the picture files are in a sub-directory, and the plugin does not pass the correct path to Pandoc.
With a blanc, normal note and paste the picture and trying to convert the note (.md) to a Word document it works.
But when I put that same .png picture in a subfolder, then it doesn't work anymore... So how to fix this issue?

Hi,
I had a similar issue here is the solution I found :)
To fix this issue, you can create a Python script that acts as a Pandoc filter to modify the image URLs in your notes to include the correct relative file paths.
Here's a step-by-step guide on how to use the Python script with Pandoc in Obsidian:

Save the following Python script (which we created based on your folder structure) in your Obsidian notes folder (e.g., "Sync/Notes/image_filter.py"). The "Sync" folder is my parent folder for my vault, and all my notes are in the "Notes/" folder.
import os
import urllib.parse
from panflute import Image

# Set a default value for the image directory
DEFAULT_IMAGE_DIR = "../Files"

def action(elem, doc):
    if isinstance(elem, Image):
        url = elem.url.strip()

        if os.path.isabs(url):
            url = '/'.join(url.split(':')[1:]).replace('\\', '/')

        url = urllib.parse.unquote(url)
        abs_path = os.path.abspath(os.path.join(os.getcwd(), url.replace('/', '\\')))
        elem.url = os.path.join(os.path.dirname(abs_path), DEFAULT_IMAGE_DIR, os.path.basename(abs_path))

        return elem

if __name__ == '__main__':
    from panflute import run_filter
    run_filter(action)

Ensure that you have the Pandoc plugin installed in Obsidian.

In the Pandoc plugin settings, add the following argument to the "Pandoc Arguments" field:
--filter=image_filter.py

This tells Pandoc to use the Python script we created as a filter when converting your notes.

Convert your notes using the Pandoc plugin (Ctrl + P > Export as Word Document). The images should now be rendered correctly in the converted documents, even if they are located in a sub-directory.

The filter script ensures that the image paths in your notes are modified to include the correct relative file paths to the sub-directory where the images are stored. This way, Pandoc can locate and render the images properly in the converted documents.
Let me know if that works for you!

@qrospars I tried using your script. The following error message still appears:

Surprising. On my side it works as expected. I am on a windows machine, maybe that's why? @Comprehensive-Jason  Are you on a Mac or a Linux?

I'm on Windows. I believe the problem is with the way I store images in my vault. I store all my notes in a 'Resources' folder, and all my images in a separate 'attachments' folder. I think the script fails because 'attachments' is not a subdirectory of 'Resources'.

Vault:

Resources
attachments

Thanks for this!
@Comprehensive-Jason Neither worked for me until I changed
DEFAULT_IMAGE_DIR = "../Files" to DEFAULT_IMAGE_DIR = "Files"

After the change, the script works on most of my files. I did get this error when testing one of my longer documents with more images though:

This happens because the image is encoded in 64 bit, instead of linking to a file. I didn't think of this case since I never past my images in this format :/
Try changing the code to this (and change the DEFAULT_IMAGE_DIR back to the value that worked for you):
import os
import urllib.parse
from panflute import Image
import re

# Set a default value for the image directory
DEFAULT_IMAGE_DIR = "../Files"

def action(elem, doc):
    if isinstance(elem, Image):
        url = elem.url.strip()

        # Check if the URL is a base64 image
        if re.match(r'^data:image\/[a-zA-Z]*;base64', url):
            # It's a base64 image, leave it as is
            return elem

        if os.path.isabs(url):
            url = '/'.join(url.split(':')[1:]).replace('\\', '/')

        url = urllib.parse.unquote(url)
        abs_path = os.path.abspath(os.path.join(os.getcwd(), url.replace('/', '\\')))
        elem.url = os.path.join(os.path.dirname(abs_path), DEFAULT_IMAGE_DIR, os.path.basename(abs_path))

        return elem

if __name__ == '__main__':
    from panflute import run_filter
    run_filter(action)

@qrospars Thanks so much! I can export images without errors now.
Unfortunately, I keep running into persistent issues exporting equations using the plugin. In HTML mode, equations simply don't show up. In Markdown mode, equations often have the wrong formatting. I've decided to switch to the Obsidian Enhancing Export plugin, which does the same thing but is currently being updated and has fewer issues. Can I adapt your script for that plugin?

For those users who aren't very experienced, I'll add some details:
I use relative paths with a subfolder for attachments. So my attachments folder is located at the path:
P:\Sync\Obsidian\Edu\Tech\BD\Practic\attachments

So what I specified:
DEFAULT_IMAGE_DIR = "Edu/Tech/BD/Practic/attachments"

I also failed to specify simply:
--filter=image_filter.py

So I specified:
--filter=P:\Sync\Obsidian\image_filter.py

Of course, you had to install Python. But apart from that, you will need to install the library used in the script:
pip install panflute

After that I managed to get a doc with pictures on the output.
@qrospars Thanks for your script, now I can write everything in obsidian with pictures and convert it to the formats I want to export.

For those users who aren't very experienced, I'll add some details: I use relative paths with a subfolder for attachments. So my attachments folder is located at the path:
P:\Sync\Obsidian\Edu\Tech\BD\Practic\attachments

So what I specified:
DEFAULT_IMAGE_DIR = "Edu/Tech/BD/Practic/attachments"

I also failed to specify simply:
--filter=image_filter.py

So I specified:
--filter=P:\Sync\Obsidian\image_filter.py

Of course, you had to install Python. But apart from that, you will need to install the library used in the script:
pip install panflute

After that I managed to get a doc with pictures on the output. @qrospars Thanks for your script, now I can write everything in obsidian with pictures and convert it to the formats I want to export.

Sorry from the late response.
It can make a file docxs. When I make a screenshot with Greenshot and paste it in Obsidian and that the .png file not listed in a directory. This step is crucial, otherwise it cannot find the .png file.
So when the .png file in the root (non folder listed) then I CAN export a .DOCXS file, but when place the .png file in a subfolder (pngfolder) then it gives the Pandoc Export error = failed: pandoc.exe
C:\Users\AygulKorkmazlar\Documents\Github\Workvault\Pasted Image 20230xxxxxx.png:WithBinaryfile: does not exist (No such file or directory)
So this is clearly that Pandoc plugin can not find the picture when the picture (.pngfile) in a subfolder.
Also I checked in the settings of the plugin Pandoc Plugin no such any kind of settings that it needs to search to a specific image in a nestled folder, instead of ;looking for a picture in the root folder.
Hope this make sense and can fix this bug/issue.
Still not being fixed.
What to do to fix this issue that the plugin search below in directory's?

Hi everyone,
I refactored the script to search for all files inside the Vault instead of a specific folder. This new version should resolve the issues some users were having with files in nested directories not being found.
The script now traverses the entire Vault directory to find the images, regardless of where they're located. To make it work for your specific Vault, simply set the VAULT_NAME at the top of the script to match the name of your Vault.
The script uses Python's os module to handle file paths in a platform-independent manner, which means it should work correctly on macOS, Linux, and Windows. However, please note that the VAULT_NAME needs to match exactly, including any uppercase and lowercase letters, for the script to find your files correctly.
Here is the final version of the script:
import os
import re
from panflute import Image

# Set your vault name here
VAULT_NAME = "YourVaultName"

def find_file(filename, search_path):
    """
    Given a filename and a directory search path, this function will return the
    relative path of the file if it exists within the search path, otherwise None.

    Args:
        filename (str): The name of the file.
        search_path (str): The directory path where the search will be performed.

    Returns:
        str: The relative path of the file if found, else None.
    """
    for root, dirs, files in os.walk(search_path):
        if filename in files:
            return os.path.relpath(os.path.join(root, filename), search_path)
    return None

def action(elem, doc):
    if isinstance(elem, Image):
        url = elem.url.strip()

        # Check if the URL is a base64 image
        if re.match(r'^data:image\/[a-zA-Z]*;base64', url):
            # It's a base64 image, leave it as is
            return elem

        # Get the absolute path to the current directory
        current_dir = os.path.abspath(os.getcwd())

        # Find the index of the vault name in the path
        vault_index = current_dir.index(VAULT_NAME)

        # Construct the vault path
        vault_path = current_dir[:vault_index + len(VAULT_NAME)]

        # Search for the image file in the vault
        image_path = find_file(os.path.basename(url), vault_path)

        if image_path is not None:
            # Replace the URL of the image with the new path
            elem.url = image_path

        return elem

if __name__ == '__main__':
    from panflute import run_filter
    run_filter(action)

Please test it out and let me know if there are any issues. As always, make sure to back up your data before testing new scripts. Thanks for your patience and your help in improving this script!

Hi qrospars,
Many thanks for your work.
I just tried the final version of your script. Unfortunately it didn't seems to work for me.
I just get the same pandoc error.
.
I've installed Python and panflute package

added the argument in Pandoc plugin

I've set the vault folder

I've put the script on the folder : "...\Leader du changement\3. 🗃 Notes Permanentes" which is one of my main folder (but I use several folders for notes).
I tried to export a note that were in this folder "...\Leader du changement\3. 🗃 Notes Permanentes", and the image is in the same folder. But pandoc still want to look in the root folder (Leader du changement).
I would be really happy to have a solution that work if you can help me.
All the best.
P.S Thanks also StrikS for giving more explanations for "users who aren't very experienced". I'm one of them 😁 I even had to learn how to use "pip install panflute"

Hi everyone,
I refactored the script to search for all files inside the Vault instead of a specific folder. This new version should resolve the issues @Hot12345 was having with files in nested directories not being found.
The script now traverses the entire Vault directory to find the images, regardless of where they're located. To make it work for your specific Vault, simply set the VAULT_NAME at the top of the script to match the name of your Vault.
The script uses Python's os module to handle file paths in a platform-independent manner, which means it should work correctly on macOS, Linux, and Windows. However, please note that the VAULT_NAME needs to match exactly, including any uppercase and lowercase letters, for the script to find your files correctly.
Here is the final version of the script:
import os
import re
from panflute import Image

# Set your vault name here
VAULT_NAME = "YourVaultName"

def find_file(filename, search_path):
    """
    Given a filename and a directory search path, this function will return the
    relative path of the file if it exists within the search path, otherwise None.

    Args:
        filename (str): The name of the file.
        search_path (str): The directory path where the search will be performed.

    Returns:
        str: The relative path of the file if found, else None.
    """
    for root, dirs, files in os.walk(search_path):
        if filename in files:
            return os.path.relpath(os.path.join(root, filename), search_path)
    return None

def action(elem, doc):
    if isinstance(elem, Image):
        url = elem.url.strip()

        # Check if the URL is a base64 image
        if re.match(r'^data:image\/[a-zA-Z]*;base64', url):
            # It's a base64 image, leave it as is
            return elem

        # Get the absolute path to the current directory
        current_dir = os.path.abspath(os.getcwd())

        # Find the index of the vault name in the path
        vault_index = current_dir.index(VAULT_NAME)

        # Construct the vault path
        vault_path = current_dir[:vault_index + len(VAULT_NAME)]

        # Search for the image file in the vault
        image_path = find_file(os.path.basename(url), vault_path)

        if image_path is not None:
            # Replace the URL of the image with the new path
            elem.url = image_path

        return elem

if __name__ == '__main__':
    from panflute import run_filter
    run_filter(action)

Please test it out and let me know if there are any issues. As always, make sure to back up your data before testing new scripts. Thanks for your patience and your help in improving this script!

Still not working.....
As @Oskiator mentioned, I get the same result. It seems that pictures in a nested folder having a trouble with...
And yes I'm also one of them that are not experienced with Python.
But get the same error as @Oskiator mentioned.

If your images are in subfolder relative to the file, you can use this code which is slightly modified.
In my case my subfolder name is attachments.
Change the sub_image_folder for you case.
`import os
import urllib.parse
from panflute import Image
import re
sub_image_folder = 'attachments'
current_dir = os.getcwd()
DEFAULT_IMAGE_DIR = os.path.join(current_dir, sub_image_folder)
def action(elem, doc):
if isinstance(elem, Image):
url = elem.url.strip()
    # Check if the URL is a base64 image
    if re.match(r'^data:image\/[a-zA-Z]*;base64', url):
        # It's a base64 image, leave it as is
        return elem

    if os.path.isabs(url):
        url = '/'.join(url.split(':')[1:]).replace('\\', '/')

    url = urllib.parse.unquote(url)
    abs_path = os.path.abspath(os.path.join(os.getcwd(), url.replace('/', '\\')))
    elem.url = os.path.join(os.path.dirname(abs_path), DEFAULT_IMAGE_DIR, os.path.basename(abs_path))

    return elem

if name == 'main':
from panflute import run_filter
run_filter(action)`

import os
import re
from panflute import Image
Set your vault name here
VAULT_NAME = "Vault"
def find_file(filename, search_path):
"""
Given a filename and a directory search path, this function will return the
relative path of the file if it exists within the search path, otherwise None.
Args:
    filename (str): The name of the file.
    search_path (str): The directory path where the search will be performed.

Returns:
    str: The relative path of the file if found, else None.
"""
# Find the index of the vault name in the path
vault_index = search_path.index(VAULT_NAME)

# Construct the vault path
vault_path = search_path[:vault_index + len(VAULT_NAME)]

for root, dirs, files in os.walk(vault_path):
    if filename in files:
        return os.path.relpath(os.path.join(root, filename), search_path)
return None

def action(elem, doc):
if isinstance(elem, Image):
url = elem.url.strip()
    # Check if the URL is a base64 image
    if re.match(r'^data:image\/[a-zA-Z]*;base64', url):
        # It's a base64 image, leave it as is
        return elem

    # Get the absolute path to the current directory
    current_dir = os.path.abspath(os.getcwd())

    # Search for the image file in the vault
    # image_path = find_file(os.path.basename(url), vault_path)
    image_path = find_file(os.path.basename(url), current_dir)

    if image_path is not None:
        # Replace the URL of the image with the new path
        elem.url = image_path

    return elem

if name == 'main':
from panflute import run_filter
run_filter(action)

My Obsidian is setup to have images placed in subfolders under the current folder:
MyDoc.md
attachments/image.png

Since I use a lot of folders in Obsidian and want to use Pandoc wherever the document may live, I put the filter in my pandoc default folder at ~/.local/share/pandoc/filters and add the Obsidian Pandoc Extra Arguments --filter image_filter.py
To make the filter that @qrospars created in https://github.com/OliverBalfour/obsidian-pandoc/issues/183#issuecomment-1595792931 above to work in my setup I need use the full path for the images.
Just add this line of code in image_filter.py before elem.url = image_path:
image_path = os.path.join(vault_path, image_path)

I gived up on this... there is no solution to the first post.

What I have right now what works is that all the pictures I took and paste in the root folder. So the pictures are not in any sub folders etc.

I gived up on this... there is no solution to the first post.

Yeah, I've given it up som many times but I stumbled upon this thread the other day and decided to give it just one more try and finally got it working in my configuration.
I also had problems with errors like  with binary file: does not exist (No such file or directory) and got rid of them when I changed the Export Folder in the Obsidian Pandoc Settings to the default "same as target".

I found a solution, without the py filter.
install plugin link converter
in link converter options set "converted link format" to Relative path.
then either use the plugin to convert entire vault links to markdown or find your export note, mark all and convert editor selection link to markdown

大家好，
我重构了脚本，以搜索 Vault 内的所有文件，而不是特定文件夹。此新版本应该可以解决这些问题@Hot12345无法找到嵌套目录中的文件。
脚本现在遍历整个 Vault 目录来查找图像，无论它们位于何处。要使其适用于您的特定 Vault，只需将VAULT_NAME脚本顶部的设置为与您的 Vault 名称匹配即可。
该脚本使用 Python 的os模块以独立于平台的方式处理文件路径，这意味着它应该可以在 macOS、Linux 和 Windows 上正常工作。但是，请注意，需要VAULT_NAME完全匹配，包括任何大写和小写字母，以便脚本正确找到您的文件。
以下是脚本的最终版本：
import os
import re
from panflute import Image

# Set your vault name here
VAULT_NAME = "YourVaultName"

def find_file(filename, search_path):
    """
    Given a filename and a directory search path, this function will return the
    relative path of the file if it exists within the search path, otherwise None.

    Args:
        filename (str): The name of the file.
        search_path (str): The directory path where the search will be performed.

    Returns:
        str: The relative path of the file if found, else None.
    """
    for root, dirs, files in os.walk(search_path):
        if filename in files:
            return os.path.relpath(os.path.join(root, filename), search_path)
    return None

def action(elem, doc):
    if isinstance(elem, Image):
        url = elem.url.strip()

        # Check if the URL is a base64 image
        if re.match(r'^data:image\/[a-zA-Z]*;base64', url):
            # It's a base64 image, leave it as is
            return elem

        # Get the absolute path to the current directory
        current_dir = os.path.abspath(os.getcwd())

        # Find the index of the vault name in the path
        vault_index = current_dir.index(VAULT_NAME)

        # Construct the vault path
        vault_path = current_dir[:vault_index + len(VAULT_NAME)]

        # Search for the image file in the vault
        image_path = find_file(os.path.basename(url), vault_path)

        if image_path is not None:
            # Replace the URL of the image with the new path
            elem.url = image_path

        return elem

if __name__ == '__main__':
    from panflute import run_filter
    run_filter(action)

请测试一下，如果有任何问题请告诉我。与往常一样，在测试新脚本之前请务必备份数据。感谢您的耐心和对改进此脚本的帮助！

Thank you for the update! You think the same way I do.
When I first try the 1.0 version, I found that the relative paths can't work in my vault, so I change it into abs_path. Then I found that the abs_path can't work with all folders, so I add a function to make it search all "Attachments" folders, and that works. I share it right here and hope it will help someone else.
By the way, my path structure is:

BASE_DIR

Note folder 1

Attachments

xxxx.png

Note1.md

Note folder 2

Attachments

xxx.png

Node2.md

Note folder 3
...

import os
import urllib.parse
from panflute import Image

# Base directory to start search
BASE_DIR = r"D:\OneDrive\文档\Vault of Obsidian"

def find_image_in_attachments(image_name):
    for root, dirs, files in os.walk(BASE_DIR):
        if 'Attachments' in dirs:
            attachments_dir = os.path.join(root, 'Attachments')
            if image_name in os.listdir(attachments_dir):
                return os.path.join(attachments_dir, image_name)
    return None

def action(elem, doc):
    if isinstance(elem, Image):
        url = elem.url.strip()
        url = urllib.parse.unquote(url)
        image_name = os.path.basename(url)

        image_path = find_image_in_attachments(image_name)
        if image_path:
            elem.url = image_path

        return elem

if __name__ == '__main__':
    from panflute import run_filter
    run_filter(action)


Added favicon

[x] Modifed html files with favicon

Thank you! @sonalirao14 👍
for your contribution!
@GisOpenSourceOrg


Not respecting setting to add bookmark to top versus bottom of folder

This is a Bug Report
Description
The setting to add new bookmarks to the top of the default bookmark folder appears to be treated as enabled regardless of the toggle switch status.
Steps to reproduce the problem

Select a default bookmark folder.
Disable adding new bookmarks to the top of the folder.
Add a new bookmark.
Observe that the bookmark is at the top of the folder, not the bottom.

Expected behavior
Adding the bookmark to the top of a folder should only happen if that setting is enabled.
Specifications

Version of the add-on: 3.0.0
Firefox edition: Firefox (regular edition)
Firefox version: 84.0.2
Firefox language: English
Platform/OS: Linux
Platform/OS version: Kubuntu 20.04

Additional context
This is a very recent change, though I'm not sure what the last properly working version was.

Thank you for reporting this issue! I very recently released version 3.1.0 that does contain a fix around this setting. Would it be possible for you to confirm the issue is still happening with this version? Thanks! 🙂

Thank you for reporting this issue! I very recently released version 3.1.0 that does contain a fix around this setting. Would it be possible for you to confirm the issue is still happening with this version? Thanks! 🙂

New version 3.1.0 seems to fix it!

New version 3.1.0 seems to fix it!


Added the ability for a Post Authenticate Callback functionality

This functionality is helpful if you are using a different session handler in Laravel then your default php session.

Sorry for coming back to you so late...
I agree that your contribution is great, the only little change I would make is to change to name of the callback function in the config file. For now it's fraudPostAuthenticateCallback. Just postAuthenticateCallback would be better and wouldn't sound dangerous, as fraud does...
Could you make this little change and ask for a pull request again ?
Thanks.
Xavier

Good call, that was left over from what I was working on. I removed the default so that it wouldn't call a function that didn't exist.


AWS Not Unsupported after Govee app update

What issue do you have? Please be as thorough and explicit as possible.
I am trying to set up scenes in the advanced tab in lighting I have set up a Tap-to-run  with Snapshot I did follow the directions previously had no issues setting up 3 other tap-to-run and restarting HB and I could see them show up in the log when HB reboots I was able to implement them into custom scene modes. After the Govee update and recently there was firmware update I have lost the ability to do this. After it reboots the logs say unsupported
Details of your setup.

Do you use (1) Homebridge UI-X (2) Homebridge CLI or (3) HOOBS?
Homebridge on truenas

Which version of Homebridge/HOOBS do you have?
v18.18.2

Which platform do you run Homebridge/HOOBS on (e.g. Raspberry Pi/Windows/HOOBS Box)? Please also mention your version of Node.js/NPM if known.
Hombridge on truenas

Which version of this plugin (homebridge-govee) do you have? Has the issue started since upgrading from a previous version?
Installed: v10.2.0

Which Govee devices do you have that are causing issues? Please include product models if applicable.
H70C1

Please paste any relevant logs below.

[Govee] [LAN] client enabled and found 2 device(s).
[12/16/2023, 3:38:37 PM] [Govee] [E5:A7:D9:32:39:36:50:88] initialising with options {"adaptiveLightingShift":0,"aws":"unsupported","ble":"unsupported","brightnessStep":1,"colourSafeMode":true,"lan":"enabled"}.
[12/16/2023, 3:38:37 PM] [Govee] [E5:A7:D9:32:39:36:50:88] initialised with id [E5:A7:D9:32:39:36:50:88] [H705B].
[12/16/2023, 3:38:37 PM] [Govee] [Holiday String Light] initialising with options {"adaptiveLightingShift":0,"aws":"unsupported","ble":"unsupported","brightnessStep":1,"colourSafeMode":true,"lan":"unsupported"}.
[12/16/2023, 3:38:37 PM] [Govee] [Holiday String Light] initialised with id [1D:1F:F9:79:B7:A0:4B:E0] [H70C1].
[12/16/2023, 3:38:37 PM] [Govee] ✓ Setup complete. This plugin has been made with ♥ by bwp91, please consider a ☆ on GitHub if you are finding it useful!

This isn't related to the repo - I author the other govee plugin and getting the same issues.

Fair enough I’ll spend more time on my devices Thank youChris Searle On Dec 16, 2023, at 20:18, Teagan Glenn @.***> wrote:
This isn't related to the repo - I author the other govee plugin and getting the same issues.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: @.***>


Biodata

Scaffolding for tests for the biodata endpoints, specifically biosamples for now. Need tests for get by ID, searching readgroupsets, and callsets.

Everything works, but the biodata in test_data is now contained in a directory structure, something we agreed upon at the beginning of this compliance project that we weren't going to do.
Let's discuss a consistent solution to this before going forward.

+1 modulo flattening out the biosample and individual json files in test_data.

@macieksmuga The directory is flat again and the test files are named for their types. https://github.com/ga4gh/compliance/pull/179/commits/fb831810d2e6aba0f007f0c165dd7b74feb7554a#diff-140b3c2404155d50dab3b256bd9396d4R1

+1

Closing in favor of https://github.com/ga4gh/compliance/pull/195


Add mlstacks compatibility check to CI

I added a check for the zenml release workflow which checks that the current package (editable install on the release branch) can be installed in the same environment as mlstacks.
Pre-requisites
Please ensure you have done the following:

[x] I have read the CONTRIBUTING.md document.
[ ] If my change requires a change to docs, I have updated the documentation accordingly.
[ ] If I have added an integration, I have updated the integrations table and the corresponding website section.
[ ] I have added tests to cover my changes.

Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[x] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)
[ ] Other (add details above)

How critically does zenml rely on mlstacks? Do we want this CI check just to inform us or actually it's a hard requirement before releasing

That's a good question. If you can't install the two in the same environment then you can't use zenml stack deploy... etc, so I'd say if mlstacks is a core part of the ZenML flow / functionality then it should be a hard requirement.
But open to other opinions...

@avishniakov yeah can do. It doesn't add much in terms of time. I'll update.


Flags moved for velocity to achieve the global default flags so that other theme can also use these default flags

Description

Default flags can be added in the public/flags folder and registered in the Locale model.
In this way, other themes can also use these flags, and no need to mess with the velocity path.

How to use?

Just call the model and get the image_url attribute,
$locale = Webkul\Core\Models\Locale::find(2);

echo $locale->image_url;

@devansh-webkul it could be useful to mention this on the devdocs

Yeah, it will be good but before writing docs, we need to shift some paths for the locale's storage also as currently, it is also going in the velocity folder. We will do it later when writing the docs.
I noted all the points.


Program permissions should restrict indicator data view in Sites module

When a user visits the Sites module (by clicking Browse > Sites at the top navbar) they should see:

[ ] All sites associated with any country in which they have access to at least one program. (Note: Sites are country-specific, not program-specific)
[ ] Each site entry contains a link labeled "Indicator data", which produces a new screen when clicked.
[ ] The Indicator data screen for each site is populated only with indicators associated with program(s) to which the user has access. If a user clicks on a site's Indicator data popup, and that site only contains results from a program to which that user does not have access, the user should see nothing in the Indicator data popup.

Example of an an indicator data seen for a user who has access to the VISTA program in the United States:

To test, you could associate a TolaData Demo result with the Catholic Charities site, restrict user permissions to each program, and test whether they see indicators/results for programs they don't have access to.

@RickieT Hey, when you have a chance, can you let us know if this one needs work? (Trying to figure out if this is already done by Parthenon or if we need to do some work ourselves.)

@jennyhavah I was planning on working in this yesterday and got distracted by a lot of other tickets, so I'll put this one at the top of my list right now

QA - Failures and Testing Notes

FAILURE: For user<EMAIL_ADDRESS>I gave permissions to every program in Afghanistan. I then went to Sites, and then clicked indicator data for Afghanistan, and the page was blank.
PASS - The user only saw sites information for the countries in which they had program access, so in this case, Afghanistan / USA
I noticed with program access to USA/AFGH - China is listed as a site in there, not sure why

@jennyhavah I'm gonna close this ticket, as  https://github.com/mercycorps/TolaActivity/issues/1383 covers the issue found in this test verification ticket


Please use white-red-white version of belarusian flag

Hi! I'm from Belarus https://en.wikipedia.org/wiki/Belarus. You may know that we have big troubles with our government and former president Lukashenko. Already 2 months passed after falsified elections but people still hope that we can get freedom. One of the symbols of freedom is our national white-red-white flag. And the official red-green flag is associated with president Lukashenko who is called "Europe's last dictator" and the soviet past.
Please, support the people of Belarus, use our national flag in your library.
https://www.euronews.com/2020/08/26/telegram-changes-emoji-of-belarus-flag-in-solidarity-with-protesters-thecube

The library should render whatever is prescribed by the ISO standard, because the library users will rely on that.

Not my call my friend.. sorry!


HI Please make the option to click exacly 12 cps

Hey! The program already has an option to give 12 clicks per second, but there are several limitations and variations from computer to computer, system to system, and even Windows, which means that it is not possible for the CPS option selected to be exactly the amount of final CPS obtained. This happens more often at selected higher CPSs.


Add a monthly/quarterly report for Digital Production

I'm going to merge this so I can sort out some other bits in this stack; it's been fine for months and I want to get all my open PRs merged.


chore: replacing biohub logo in footer

simple update to biohub logo in footer

Thanks so much for the review and help @mmmmmmaya 😄


Add option for manage uwsgi socket permissions

I've set up Graphite Web with uwsgi using default user/group.
But when I added nginx I got error: permission denied.
uwsgi has some options which can help in this case:
    -C|--chmod-socket                    chmod-socket
    -C|--chmod                           chmod-socket
    --chown-socket                       chown unix sockets

Will create pull request soon

there are already few PRs


gh provider - authenticate as app

It would be nice if the operator could avoid using PATs and instead auth as an gh app.
PATs have wide access and are tied to users, and limited API quota compared to apps which are more secure, have narrow permissions, and increased quota.
Here is a controller which wraps this: https://github.com/evryfs/github-actions-runner-operator/blob/master/controllers/githubapi/runnerapi.go#L39
so that one can use app auth or token auth.
Slightly related to #111 (removing username/pw mechanism), while this issue adds another option.

@sabre1041 @raffaelespazzoli wdyt? I can roll a PR for it if you think it is a good idea.

As farad I know, the PAT is used only to perform the PR needed to release
to OperatorHub. If there is a better way to do it, I'm open to proposals.
On Thu, Aug 12, 2021 at 5:14 PM David J. M. Karlsen <
@.***> wrote:

@sabre1041 https://github.com/sabre1041 @raffaelespazzoli
https://github.com/raffaelespazzoli wdyt? I can roll a PR for it if you
think it is a good idea.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/redhat-cop/group-sync-operator/issues/112#issuecomment-897972377,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABPERXEZKJAGYRUXIKVIPCDT4Q2SPANCNFSM5BV5TFYQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

--
ciao/bye
Raffaele

Are we on the same page here? I am talking about this package: https://github.com/redhat-cop/group-sync-operator/blob/master/pkg/syncer/github.go

As farad I know, the PAT is used only to perform the PR needed to release to OperatorHub. If there is a better way to do it, I'm open to proposals.
…
On Thu, Aug 12, 2021 at 5:14 PM David J. M. Karlsen < @.***> wrote: @sabre1041 <@sabre1041> @raffaelespazzoli <@raffaelespazzoli> wdyt? I can roll a PR for it if you think it is a good idea. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#112 (comment)>, or unsubscribe <github.com/notifications/unsubscribe-auth/ABPERXEZKJAGYRUXIKVIPCDT4Q2SPANCNFSM5BV5TFYQ> . Triage notifications on the go with GitHub Mobile for iOS <apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android <play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email> .
-- ciao/bye Raffaele

sorry not on the same page.
On Thu, Aug 12, 2021 at 6:00 PM David J. M. Karlsen <
@.***> wrote:

Are we on the same page here? I am talking about this package:
https://github.com/redhat-cop/group-sync-operator/blob/master/pkg/syncer/github.go
As farad I know, the PAT is used only to perform the PR needed to release
to OperatorHub. If there is a better way to do it, I'm open to proposals.
… <#m_-2030386324196414751_>
On Thu, Aug 12, 2021 at 5:14 PM David J. M. Karlsen < @.> wrote:
@sabre1041 https://github.com/sabre1041 @.
https://github.com/sabre1041> @raffaelespazzoli
https://github.com/raffaelespazzoli @.***
https://github.com/raffaelespazzoli> wdyt? I can roll a PR for it if
you think it is a good idea. — You are receiving this because you were
mentioned. Reply to this email directly, view it on GitHub <#112 (comment)
https://github.com/redhat-cop/group-sync-operator/issues/112#issuecomment-897972377>,
or unsubscribe <
github.com/notifications/unsubscribe-auth/ABPERXEZKJAGYRUXIKVIPCDT4Q2SPANCNFSM5BV5TFYQ>
. Triage notifications on the go with GitHub Mobile for iOS <
apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675>
or Android <
play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email>
.
-- ciao/bye Raffaele
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/redhat-cop/group-sync-operator/issues/112#issuecomment-897995595,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABPERXBC2LWV6Y45OGBZLGDT4Q77NANCNFSM5BV5TFYQ
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&utm_campaign=notification-email
.

--
ciao/bye
Raffaele