Example of sending an email with attachments using the Gmail API

I use Google translation so sorry if my English is not clear.
Hello everyone
For a long time I was looking for an example of a code that sends an email with attachments via Gmail and I did not find, after much effort I was able to compose a number of codes that do the job, I share them here for the benefit of all.
Note: Some of the code may be redundant or will work better after some modifications, I do not know, much of this code I copied as is and I do not understand what it does.
Attached below is an example of an Aceess file and an example of an Excel file for sending email using Gmail Api.
credits:
Thanks to @timhall for this great library!
And thanks to @RichardWein for sharing his code (in this answer) that keeps the refresh token reusable, using this code is enough to perform authentication google only for the first time, and next times to use the refresh token.
Instructions:

Update the CLIENT_ID constant and the CLIENT_SECRET constant located in the OtherStuff module.
Use the SendEmail function in the Gmail module to send mail
The function accepts the following arguments:

SenderName: The name of the sender as displayed to the recipient of the message
SenderAddress: Sender's Address - Attention! If the address you pass to the function is incorrect, the email will be sent but the sender's name will display the email address instead of the sender's name
Recipients: Recipients - You can move multiple recipients separated by a comma ","
RecipientType: Recipient type - Visible recipients or hidden recipients
Subject: Message subject, optional
MessageText: The content of the message, optional
Files: Attachments - Optional, you can move multiple separated files with the question mark "?"
ReplyToAddress: Reply Address - Optional, if you want the reply address to be different from the sender's address
ReplyToName: Name Of Reply Address - Optional, only relevant when using the ReplyToAddress argument
bAsync: This argument was provided along with some of the code I copied from somewhere, I have no idea what it does (optional)

The Test function serves as an example of sending an email
To reset the refresh token stored on your computer, use the ClearGoogleAuthentication function

Note: The SendEmail function uses the ToBase64 function, it is designed to correctly display the sender name, subject, and mail name for reply in non-English languages, in English it is not consumed, although it also does not interfere with leaving it as is.
Send Email - Gmail Api With Access.zip
Send Email - Gmail Api With Excel.zip

I used the Access version and it help greatly.  But I am having an issue and getting the message :
Error 400: invalid_request
You can't sign in to this app because it doesn't comply with Google's OAuth 2.0 policy for keeping apps secure.
If you’re the app developer, make sure that these request details comply with Google policies.
redirect_uri: urn:ietf:wg:oauth:2.0:oob
Any help ASAP would be great since they are shutting off the old access Monday.

Hi, I'm glad you've used this,
This line:
Private Const LOGIN_REDIRECT_URI As String = "urn:ietf:wg:oauth:2.0:oob"
Located in the "GoogleApiShared" module, it tells Google that after the user verifies his account the response will be a verification code for copying and pasting manually in your app, this option is only available for an app under review, you probably submitted your app for posting (which is great because you get a valid refresh token Forever, compared to a test app that gets a refresh token that is only valid for a week or two) and so you run into this error.
You need to use a reference to uri which basically means a reference to the URL of your private site, then after the user verifies his Google account it will be transferred to your site and the verification code will appear in the "code" parameter. (From there you copy it and paste it into your VBA app).
In order for you to be redirected to your URL you need to create a certificate for a "Web application" type and then add your URL under the heading "Authorized redirect URIs".
Then replace the line for redirection in your code, replace this line:
Private Const LOGIN_REDIRECT_URI As String = "urn:ietf:wg:oauth:2.0:oob"
And put this code in its place:
Private Const LOGIN_REDIRECT_URI As String = "https://your-site.com/your-project/authorization-code"
Additionally, you must add these two lines in the verification request: (it is in the "GetLoginUrl" function)
auth_Request.AddQuerystringParam "access_type", "offline"
auth_Request.AddQuerystringParam "prompt", "consent"
This will ensure that in response to the verification code you will also receive a refresh token and not just an access token.

I opened a personal website in wix for the authorization code, Google requires you to prove ownership of the site, and in wix there is a built-in tool to prove ownership, it went really easily.
You can also make a code on your site that will extract the "code" parameter from the address bar and put it in a text box so that your users will know what to do, I also added a "copy code" button.
It requires very minimal knowledge in JavaScript, wix has their own code words and it is really easy to find the codes in their forum.

Thank you for this information.  I am going to see if I can make this
happen although I'm not 100% sure I understand.  One other question, It is
working in test mode, is there an issue with leaving it in test mode?
Chris Davis
Tangled in Tradition
https://tangledintradition.com
Like us on Facebook
Facebook https://www.facebook.com/tangledintradition
Follow us on Instagram
Instagram https://www.instagram.com/tangledintradition/
On Sun, May 29, 2022 at 5:20 AM hnsywnwtsly @.***>
wrote:

Hi, I'm glad you've used this,
This line:
Private Const LOGIN_REDIRECT_URI As String = "urn:ietf:wg:oauth:2.0:oob"
Located in the "GoogleApiShared" module, it tells Google that after the
user verifies his account the response will be a verification code for
copying and pasting manually in your app, this option is only available for
an app under review, you probably submitted your app for posting (which is
great because you get a valid refresh token Forever, compared to a test app
that gets a refresh token that is only valid for a week or two) and so you
run into this error.
You need to use a reference to uri which basically means a reference to
the URL of your private site, then after the user verifies his Google
account it will be transferred to your site and the verification code will
appear in the "code" parameter. (From there you copy it and paste it into
your VBA app).
In order for you to be redirected to your URL you need to create a
certificate for a "Web application" type and then add your URL under the
heading "Authorized redirect URIs".
Then replace the line for redirection in your code, replace this line:
Private Const LOGIN_REDIRECT_URI As String = "urn:ietf:wg:oauth:2.0:oob"
And put this code in its place:
Private Const LOGIN_REDIRECT_URI As String = "
https://your-site.com/your-project/authorization-code"
Additionally, you must add these two lines in the verification request:
(it is in the "GetLoginUrl" function)
auth_Request.AddQuerystringParam "access_type", "offline"
auth_Request.AddQuerystringParam "prompt", "consent"
This will ensure that in response to the verification code you will also
receive a refresh token and not just an access token.
—
Reply to this email directly, view it on GitHub
https://github.com/VBA-tools/VBA-Web/issues/472#issuecomment-1140409952,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AXA43XLJE6M33BZEDTTMFJ3VMMZELANCNFSM5SCZ2SZA
.
You are receiving this because you commented.Message ID:
@.***>

For me this solution works great
But you can leave it in test mode, it has one significant drawback: the refresh token you receive is valid for one or two weeks only, then once a week or two your user will be required to verify his account again.
To get a refresh token that is valid forever you need to use the app that was submitted for publication, but beware! You do not have to complete the advertising process, it is enough that you submit it for advertising and you do not proceed with the following steps and it already works with a refresh token that is valid forever. The only thing is that your user will see a warning that the app has not been verified.

Thanks for this information,  @hnsywnwtsly. I didn't realise that Google's option to copy and paste the authorisation code was only available in test mode. I've always left my applications in test mode so far. My users don't seem to mind having to re-authorise every week out two.

this was really good information. I am also now trying to get a refresh token that lasts forever..
i however had a few stumbling blocks... hope you can help..
the wix website made my website like shajvarg.wixsite.com/mysite.  so I although I can verify shajvarg.wixsite.com/mysite, I cannot verifiy shajvarg.wixsite.com in google using the SEO tools.. how did you manage to do this?
can we use any youtube video showing gmail api stuff? will they object later? I feel making a video showing all scope usage is a bit too much ..
shaju

hi,
sorry i was using an example
the actual site I made is https://shajvarg.wixsite.com/accura
so I have verified https://shajvarg.wixsite.com/accura in google search console
but I cannot verifiy https://shajvarg.wixsite.com the domain which is a requirement during publishing..
this is my issue

HTML tag.
I just verified the home page address, that's enough, no need to verify the domain.
It should work for you now

shajvarg.wixsite.com/accura
Authorised domain 2 *
Invalid domain: must be a top private domain
this is the error I get during publishing
and shajvarg.wixsite.com only doesn't work

What do you have registered in "Authorized Domain 1" ?

i tried another domain also. which I could verify.
but even if I remove domain 1, the error is still the same..

I do not know,
I entered the address of the home page in the "App Home": https://00187588.wixsite.com/my-site
And in "Authorized Domain 1" I wrote:
00187588.wixsite.com

thanks, i managed to get authorization code with wixsite
now I am not getting refreshtoken
it says invalid client
    ' Make an "authorization code" request (using authorization code to get a new refresh token)
    Set auth_Request = New WebRequest
    auth_Request.Method = WebMethod.HttpPost
    auth_Request.RequestFormat = WebFormat.FormUrlEncoded
    auth_Request.ResponseFormat = WebFormat.JSON
    Set auth_Body = New Dictionary
    auth_Body.Add "client_id", CLIENT_ID
    auth_Body.Add "client_secret", CLIENT_SECRET
    auth_Body.Add "grant_type", "authorization_code"
    auth_Body.Add "code", authorizationCode
    auth_Body.Add "code_verifier", codeVerifier
    auth_Body.Add "redirect_uri", LOGIN_REDIRECT_URI
    Set auth_Request.Body = auth_Body
    Set auth_Response = ExecuteRequest2(auth_Request, auth_TokenClient)

anything different needs to be done?
LOGIN_REDIRECT_URI is the same wixsite one?
shaju

I compared this piece of code with mine, it's exactly the same.
There can be several problems:
Is the client ID you created for a "network application" app?
Did you add your URL in the "Redirect Authorized URI"?
If you were able to get an authorization code then I guess these are not the issues, but there could be another thing:
Did you add the following lines in the verification request? (Found in "GetLoginUrl" procedure)
   auth_Request.AddQuerystringParam "access_type", "offline"
    auth_Request.AddQuerystringParam "prompt", "consent"

client ID is for web application with all URLs pointing to wix site.
getloginurl also is as you said
Private Function GetLoginUrl(ByVal codeChallenge As String, ByVal challengeMethod As String) As String
' Get the login url, forming the string from various elements
' Create a temporary Request, so that we can use Url helpers
Dim auth_Request As New WebRequest
auth_Request.resource = LOGIN_ENDPOINT
auth_Request.AddQuerystringParam "client_id", CLIENT_ID
auth_Request.AddQuerystringParam "redirect_uri", LOGIN_REDIRECT_URI
auth_Request.AddQuerystringParam "response_type", "code"
auth_Request.AddQuerystringParam "code_challenge", codeChallenge
auth_Request.AddQuerystringParam "code_challenge_method", challengeMethod

auth_Request.AddQuerystringParam "scope", OAUTH_SCOPES
auth_Request.AddQuerystringParam "access_type", "offline"
auth_Request.AddQuerystringParam "prompt", "consent"
GetLoginUrl = auth_Request.FormattedResource

End Function

Your code looks just fine
If you want I can try to connect with you remotely, maybe I'll get over the problem
Send me an email
<EMAIL_ADDRESS>

hi,
it appears I have to wait for approval / verification to use gmail.send and gmail.readonly scopes when I publish the app?
i cannot use it immediately like in testing mode

No, you do not have to wait for the app to be approved because these perimeters are considered sensitive and not restricted perimeters, I started using immediately after submitting the publication request.
I read all their material, as long as your app has not been verified, you can use it and it is limited to 100 users, in addition you will be shown a warning that the app is not verified.
I suggest you upload your code here, maybe we'll see what the problem is

thanks for all the help..
I tried the entire procedure with another of my gmail accounts. And it worked..
So all good :-)

Hi
I'm glad to hear
Successfully

Hi,
Thank you for all of this, though I am struggling to get it working.
We are an accounting firm that uses a lot of VBA macro to automate work. We use Gmail to send error/status messages where there are problems with the automation. With Google now shutting down Unsecure apps for business users, I have started looking into replacing my old VBA code for sending emails that just used the username/password.
I have only used the Google APIs and Services pages infrequently, so I am not 100% confident on those pages. I created a project, enabled the Gmail API and created an OAuth credential of type Desktop. I downloaded your sample XLSM file and entered the client ID and secret into the relevant spot on the OtherStuff sheet. When I t to send a test email, it tries to authorise, but I get this error:
Error 400: invalid_request
The out-of-band (OOB) flow has been blocked in order to keep users secure. Follow the Out-of-Band (OOB) flow migration guide linked in the developer docs below to migrate your app to an alternative method.
Request details: redirect_uri=urn:ietf:wg:oauth:2.0:oob
I am at a loss as to where I am going wrong. Would you know?

Hi Joe. I'm not up-to-date with Google's latest security changes, but perhaps I can help a bit.
This is all about how the Google authorisation server can send an authorisation code to your app (your macro) in a secure way. The VBA code that you're using for authorisation sets the redirect_url parameter to use the OOB option. In other words, it tells Google to display the authorisation code in the user's browser, from where they can copy-paste it over to your macro. This option used to be available to apps in 'Testing' mode only. Much of the discussion in this thread has been about how to do the authorisation another way (not using OOB) because the posters didn't want to use Testing mode.
It seems likely that Google is no longer allowing the OOB option, even in Testing mode. However, to make sure that's the case, I suggest you check that your app is using Testing mode. In your Google APIs and Services page, look under 'OAuth consent screen' > 'Publishing status'. If that says Testing, then the OOB option is no longer available, and you will have to use an alternative. If it doesn't say Testing, try using Testing mode, as that will be the easiest option.
The alternative discussed above is to create your own website, tell Google to send the authorisation code to that website, and then make the website display the authorisation code in the user's browser. The posters above created their websites using Wix. I haven't tried doing this myself. Obviously it's going to be more effort than the OOB method.
The option recommended by Google (in general, not specifically for VBA) is to have your app listen for the authorisation code on a web socket. This would avoid having to set up a website, and avoid the user having to manually copy-paste the code. I've just been googling for information on how to do this in VBA. It looks like it might be possible, but it won't be easy to get it working. At some point I might try it, as I might need it in the future.

It's good news that you were able to get past the OOB problem. That seems to mean that OOB still works in Testing mode. If you're OK with continuing in Testing mode, there's no need to use a website.
At the moment the only thing I can suggest for the new error is to double check that the Client ID in your VBA code matches the Client ID in your Google APIs and Services console (under 'Credentials').

I just saw your message, it goes to an email address that I rarely open.
I can't tell what exactly the problem is, but the first thing that jumped out at me is that you wrote that you created a Desktop OAuth certificate, my solution talks about a "Web application" type certificate, even though the application is indeed a desktop application, I create the certificate of the "Web application" type .
It's been a long time so I don't remember exactly what the differences between the types of apps are, I can only say that I work with it on a daily basis and it works great with no problems.
Another thing, for an app in test mode, it has a significant disadvantage in that you will have to re-verify every week, whereas if you submit the application for publication (and stop at this stage, you do not need to progress beyond changing it to a status of submitted for publication) you do not need to verify more than once One, it has been working for me for almost two years.
For example, I work with an application that sends and reads emails not only as a response to customer interaction, in this case if its refresh token is not valid the application is stuck, so it is very important to change the application to publishing mode.
I'd be happy to help if you have any more questions.

Thanks @hnsywnwtsly appreciate your response. I will explain what I have done so far:
Saved your XLSM file to my PC.
Created a new project for "VBA Gmail" in Google Cloud/APIs and Services.
I have added the Gmail API. I have confirmed that it is enabled.
Created an OAuth consent screen, with User type = External, currently in Testing mode. I have added my gmail address as a test user. NB: My business domain is managed by Gmail, so I have entered it, not an @gmail.com address.
Created credentials of type "Desktop"
I have entered the Client ID and Secret from Google into the public const on the OtherStuff module.
I then try and run the Test sub in the Gmail module. I update this to use my email address as the Sender Address and a To Address of my personal email address.
When the process gets to Set auth_Response = ExecuteRequest2(auth_Request, auth_TokenClient), auth_Response.StatusCode returns as 401. The full error message is HTTP code: 401, Unauthorized error: invalid_client, error_description: Unauthorized
If I try and set the credential type to be "Web application", however I need to enter an Authorized redirect URI. I thought that might be urn:ietf:wg:oauth:2.0:oob which is mentioned in the VBA code, but it would not accept that.
Unfortunately I am not a developer, but I am usually useful enough to be able to adapt other peoples efforts and make them work. I read the earlier comments about Wixsite, but that seems beyond my capabilities.
If you cannot provide any further help or assistance, no worries and thank you for your time.

@joebob99
You will have to enter your site's URI and create a "Web application" certificate, you cannot skip this step.
The whole idea of wix (and of other hosting sites, you don't just need wix) is that you don't need to be a developer to set up a site with them, you'll be surprised how easy it is to set up a site with them, and it's also free.
In fact, I think it's easier than manipulating VBA code.
I honestly don't mind giving you my website address, which you enter under "Authorized redirect URI", but if I were you I wouldn't accept the offer, I wouldn't want to depend on someone's website that I don't know and it can stop working without warning.

Just as a sidebar comment, OAuth in general is kind of a pain to do in VBA (if you restrict yourself to VBA and widely available MS DLLs) because of the lack of a decent HttpClient class. For my app, I found using the Device Code flow is usually the most suited to VBA apps. This is the kind of flow that you go through when authenticating your TV apps on your phone.
Once you have your JWT, everything is smooth. It's just that first part where VBA is showing its age.
Good luck!

@zgrose
I didn't understand your meaning "using the Device Code"

There are several ways to authenticate in OAuth called flows. One is called Device Code where a code is displayed in your app and the user enters that code on web page.
I'm not recommending you rewrite your app, I was just mentioning it for future readers to consider. It may not even be available for your Google mail scenario.

first of all, a BIG thank you to @hnsywnwtsly for sharing this code.
After a couple of tries, I successfully ran the test of sending an email. GREAT !
Now I'm struggling with the creation of a draft email (that I simply want to keep in my gmail box, pending on a manual click on the send button in Gmail)
I've tried to update the source code, changing this line:
Auth.AddScope "https://www.googleapis.com/auth/gmail.send"
into:
Auth.AddScope "https://www.googleapis.com/auth/gmail.compose"
and adapting the HTTP request. BUT unfortunately, whenever I try to connect my HTTP request ends up with error code 403 - "Request had insufficient authentication scopes"
Do you any idea of what I'm missing to create a draft email ?
Here is my DraftEmail function source code (in Excel VBA):
Public Function DraftEmail(SenderName As String, _
SenderAddress As String, _
Recipients As String, _
RecipientType As RecipientType, _
Optional Subject As String, _
Optional MessageText As String, _
Optional Files As String, _
Optional ReplyToAddress As String, _
Optional ReplyToName As String, _
Optional ByVal bAsync As Boolean) As String
Dim pGmailClient As New WebClient
pGmailClient.BaseUrl = "https://www.googleapis.com/gmail/v1/"

Dim Auth As New GoogleAuthenticator
Auth.Setup CLIENT_ID, CLIENT_SECRET
Auth.AddScope "https://www.googleapis.com/auth/gmail.compose"
Auth.AddScope "https://www.googleapis.com/auth/gmail.modify"
Auth.Token = MyGetToken(pGmailClient)

Recipients = Replace(Recipients, ",", ", ")
If ReplyToAddress = "" Then
    ReplyTo = ""
Else
    If ReplyToName = "" Then
        ReplyTo = "Reply-To: " & ReplyToAddress & vbCrLf
    Else
        ReplyTo = "Reply-To: " & "=?utf-8?B?" & ToBase64(ReplyToName) & "?=" & " <" & ReplyToAddress & ">" & vbCrLf
    End If
End If

' Construct the email headers and body
Dim StartPart As String
StartPart = "Content-Type: multipart/mixed; boundary=" & STR_BOUNDARY & vbCrLf & _
            "MIME-Version: 1.0" & vbCrLf & _
            GetRecipientTypeStr(RecipientType) & ": " & Recipients & vbCrLf & _
            ReplyTo & _
            "From: =?utf-8?B?" & ToBase64(SenderName) & "?= <" & SenderAddress & ">" & vbCrLf & _
            "Subject: =?utf-8?B?" & ToBase64(Subject) & "?=" & vbCrLf & vbCrLf & _
            "--" & STR_BOUNDARY & vbCrLf & _
            "Content-Type: text/plain; charset=utf-8" & vbCrLf & _
            "Content-Disposition: inline" & vbCrLf & _
            "Content-Transfer-Encoding: base64" & vbCrLf & vbCrLf & _
            ToBase64(MessageText) & vbCrLf & vbCrLf

Dim MiddlePart As String
MiddlePart = GetFilesPart(Files)

Dim EndPart As String
EndPart = "--" & STR_BOUNDARY & "--"

Dim RawMessage As String
RawMessage = StartPart & MiddlePart & EndPart

' Encode the raw message in base64 and ensure it's URL-safe
Dim Base64Message As String
Base64Message = EncodeBase64UrlSafe(RawMessage)

' Construct the JSON payload
Dim JsonPayload As String
JsonPayload = "{""message"":{""raw"":""" & Base64Message & """}}"

' Define the endpoint URL for creating a draft
Dim sUrl As String
sUrl = pGmailClient.BaseUrl & "users/me/drafts"

' Create the HTTP request to create the draft
With CreateObject("Microsoft.XMLHTTP")
    .Open "POST", sUrl, bAsync
    .SetRequestHeader "Host", "www.googleapis.com"
    .SetRequestHeader "Content-Type", "application/json"
    .SetRequestHeader "Authorization", "Bearer " & Auth.Token
    .Send JsonPayload
    If Not bAsync Then
        DraftEmail = .ResponseText
        Debug.Print .Status & ": " & .ResponseText
    End If
End With

End Function
' Helper function to encode data in URL-safe Base64
Public Function EncodeBase64UrlSafe(data As String) As String
Dim base64 As String
base64 = ToBase64(data)
base64 = Replace(base64, "+", "-")
base64 = Replace(base64, "/", "_")
base64 = Replace(base64, "=", "")
EncodeBase64UrlSafe = base64
End Function

Sorry for the late response, I just now saw your response, I receive the notifications in an email address that I rarely open.
Basically, the permission for the scopes you need is granted when you authenticate the account, so you need to update the required scopes in the initial authorization code request, the token you save now is only given for the scope "https://www.googleapis.com/auth/gmail.send" and so You cannot make requests of other scopes with it.
I don't remember exactly how the whole code works (I didn't write it all either) but it seems to me that you need to change this line:
Private Const OAUTH_SCOPES As String = "https://www.googleapis.com/auth/gmail.send"
This is in the GoogleApiShared module.
I'd love to hear if it helped.


Create a manual for using Google Tag Manager

What
Create a document to show the team how to use Google Tag Manager (GTM).
Why
We are using Google Tag Manager to implement Google Analytics - in order to ensure the knowledge for how to use this is not siloed, a manual needs to be created to ensure all devs can pick up cards when they come up for GTM analytics work.
Who needs to know about this
All team
Done when

[ ] Create the manual
[ ] Send it to the team (or take through it if there's a suitable time).

https://docs.google.com/presentation/d/1D09zA2Qe2RJYBArWVzqxGY5G1uzDfp7FlamImjRLXHg/edit?usp=sharing
Got a start on it here


Implement KeepLatestValidVersions compaction policy

This policy should validate checksums, before deciding what files to keep. StateVersion should have lazy method validating checksums.

Right now this is the default behaviour of compacter. One integral file is kept, the rest of versions are removed. #49


Tool should ignore hidden guides.

The tool will select guidelines hidden by a macro. It should not do this.

I've changed my mind on this. I find it useful to be able to marquee to select hidden guides.


Use debug for log about skipping addition of cert

What does this PR do?
This entry can flood the logs if a cert is duplicated like wildcard certs can be.
Motivation
Backport https://github.com/containous/traefik/pull/5641 in v1.7 tree.
More

[ ] Added/updated tests
[ ] Added/updated documentation

Additional Notes

FYI, this is what happened when we deployed the v2.0 patched version:

Hello,
I recommend to use the level WARN instead of INFO.

Hi

I recommend to use the level WARN (the default level) instead of INFO in your configuration.

But if I do this I would lose access logs which use the INFO level, wouldn't I ?

The log level is for Traefik log.
The access logs don't have a level.

As much as this change made sense in the v2 because of the management of the TLS on the routers, as much on the v1 that does not seem to me relevant.
So we will declined.
Thanks anyway.


docs(internal): add empty description

:heavy_check_mark: Checklist:

[x] Have only one commit (if not, squash them into one commit)
[x] Tests for the changes have been added (for features / bug fixes / refactoring)

Codecov Report

:exclamation: No coverage uploaded for pull request base (main@d3afc9a). Click here to learn what that means.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #249   +/-   ##
=======================================
  Coverage        ?   73.19%
=======================================
  Files           ?       16
  Lines           ?      582
  Branches        ?        0
=======================================
  Hits            ?      426
  Misses          ?      120
  Partials        ?       36

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update d3afc9a...67d1b8b. Read the comment docs.


Update docker image

Baserow is currently on version 1.0.1

but the last build of this repo is:

I can't find a new image on docker hub. Do you need to manually push it?

Doesn't seem to be building properly. Will need to put out a revision release shortly. First day back from a 2 week holiday, Might be a day or two.

any luck @tiredofit ?

Not quite. I just switched workstatiosn to a Ryzen based system and started "fresh" for the first time in 13 years. One of the side effects from this, is my Docker (local) builds are throwing some very strange errors for users other than root. It seems when I "ADD" my install folder, whatever folders are in that folder reset the images permissions to 700 which limits the ability to use sudo and change uid, gid.  That and dealing with a meltdown of a data centre, a hardware raid controller flipping bits.
Long story short, no, but soon when I have all ducks in a row.

As long as your data center is not this one you can count yourself lucky 😁

I'm affected by it in our enterprise.

@tiredofit any update? (Have you lost some data or did you had backups?)

We lost about 45 minutes of DB transactions on some not so busy services - One of the fallout was recovering our in house Gitlab installation. That failed and it became a near 2 week process to rebuild by 4 people. I just received word everything is good to go and I'm feeling good now. We didn't lose any data persay, just access to the CI Secrets and Variables we had defined.
We opted for a brand new install as ours was old and tired after 5 years of usage. Exporting from the old frozen system to the new one just took time and team.
So with all that - We have a new Docker baserow image this morning. tiredofit/baserow:0.5.0 or latest should be built now and give you access to Baserow 1.0.1.
There's a new variable and data volume (/data) and MEDIA_HOSTNAME to watch out for - Readme has been updated to support it.
Let me know how this works.
FWIW the reason why your PR wouldn't compile is something to do with the Python cryptography module requiring Rust/Cargo now. Should be smooth sailing for their next release.

Wow, sounds like you had a couple of stressful days. Glad the fallout was minimal. Will test the new image right away. Thanks for the effort 👍

Hold off for a few minutes 0.5.1 is being uploaded as we speak. It's just a smaller version of 0.5.0 as I found 300mb of cache that needed to be removed.

I was setting up a new instance with basserow and only added the media domain/file location to my existing configuration. But when I try to create a new account I get a CORS error. I'm not sure if I can fix that in my trafik config or if that needs to be fixed in the containers nginx?
Error:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://baserow-api.MYDOMAIN.TLD/api/user/. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).

Config:
    baserow-app:
      image: tiredofit/baserow:0.5.1
      container_name: baserow-app
      restart: always
      networks:
        baserow_app:
        traefik_proxy:
            aliases:
                - baserow-api.${DOMAINNAME}
                - baserow-media.${DOMAINNAME}
                - baserow.${DOMAINNAME}
      environment:
        - ZABBIX_HOSTNAME=baserow-app

        - TIMEZONE=Europe/Berlin
        - DEBUG_MODE=FALSE
        - CONTAINER_LOG_LEVEL=INFO

        - API_HOSTNAME=baserow-api.${DOMAINNAME}
        - APPLICATION_HOSTNAME=baserow.${DOMAINNAME}
        - MEDIA_HOSTNAME=baserow-media.${DOMAINNAME}

        - DB_HOST=baserow-db
        - DB_PORT=5432
        - DB_USER=baserow
        - DB_PASS=baserow
        - DB_NAME=baserow

        - LOG_LEVEL=info

        - APP_DEBUG=False
        - BACKEND_WORKERS=5
        - LANGUAGE=en-us
        - PASSWORD_RESET_MAX_AGE=24
      volumes:
        - ${USERDIR}/docker/baserow/media:/data
      depends_on:
        - baserow-db
      labels:
        - traefik.enable=true
        - traefik.http.routers.baserow.rule=Host(`baserow.${DOMAINNAME}`) || Host(`baserow-api.${DOMAINNAME}`) || Host(`baserow-media.${DOMAINNAME}`)
        - traefik.http.services.baserow.loadbalancer.server.port=80

Yuck, CORS issues :) I'm willing to bet its something I've missed, and being set via the Python Baserow code.
You can try in the short term just to set it at your Proxy level while we figure out what has changed with these labels:
      - traefik.http.middlewares.baserow-cors.headers.accessControlAllowOriginList=*
      - traefik.http.middlewares.baserow-cors.headers.accessControlAllowCredentials=true
      - traefik.http.middlewares.baserow-cors.headers.accessControlAllowMethods=GET,POST,PUT,PATCH,DELETE
      - traefik.http.middlewares.baserow-cors.headers.accessControlAllowHeaders=User-Agent,Keep-Alive,Content-Type,Authorization,x-jwt-token,x-api-host
      - traefik.http.middlewares.baserow-cors.headers.accessControlMaxAge=600
      - traefik.http.middlewares.baserow-cors.headers.addVaryHeader=true
      - traefik.http.routers.baserow.middlewares=baserow-cors@docker

CORS fixed but now after login this turns up:

Oof now a websocket :) How fun. OK, There's some big changes that have occurred it seems going from the beta versions to the 1.x series. I'm going to have to review the code and see whats going on here. There's ways around all of this, I just need to reverse engineer it a bit more. I should have some time today and then also next week as my entire org is off on a break for 2 weeks.

O.K., sorry for the constant negative feedback :weary:

Looks like #2 also talks about this. I will review the settings too :)
Not to worry. I built this as a Proof of Concept a while back and admittedly haven't kept up to date with the changes as the product has matured. I'll invest some more energy into it.

In my case, this issue was solved when using http in the domain name. If I using https the issue still appears.


Allow only while using the app in android 10 returns GeolocationStatus.denied

🐛 Bug Report
when selecting Allow only while using the app the new option in Android 10  Geolocator().checkGeolocationPermissionStatus() returns  GeolocationStatus.denied

Version: 5.1.4
Platform:
:robot: Android

Could you tell me more about the code that returns GeolocationStatus.denied?

I can reproduce it with the sample app.

Does the emulator solve the GeolocationStatus.denied error?

I'm not sure I understand what you mean

Could you tell me some details about the code and what you are trying to create? I am trying to reproduce your issue, but I need more information.

try the demo app on an android emulator with android 10

I reproduced this issue and came to the conclusion that the location settings from the device were turned off. To fix this issue go to Settings>Location>Use Location>On.

the location was turned on I checked it, hence when I choose allow all the time it recognized as granted.

The output of the app says you are using FusedLocationProvider.  It's recommended to use Google Play services version 11.6.0 or higher, which includes bug fixes for this class.

Its a completely new permission in the new Android 10. it should neither give Allowed or Denied since it requires different checks.
https://developer.android.com/reference/android/Manifest.permission.html#ACCESS_BACKGROUND_LOCATION

@humazed  What are you trying to achieve with the Geolocator().checkGeolocationPermissionStatus() method? I don't get any errors if I set the location settings to Allowed all the time.

I reproduced the demo app in the new Android 10 and with the Geolocator().checkGeolocationPermissionStatus() method and retrieved 0 errors when I choose to allow only while using the app. Could you send your code so I could take a look at it?

I just cloned the repo and ran it without any changes

I even created a new emulator in case my current one has a problem or something but getting the same behavior

I think the issue would depend more on the settings from the emulator. The app states that there are issues with Google Play Services. Are you using a network or are you at a location that does not support the Google Play Services?

I have tried with a fresh emulator but the problem is the same
The app works fine for any version lower than android10
In the gifs I provided I'm showing the version of Google play services and the the location is enabled

I have this same issue if using version 5.1.4+1 and compiling with SDK 28.  If I compile on SDK 29 it works.  Running this on a Pixel 3a running Android 10.

I have this same issue if using version 5.1.4+1 and compiling with SDK 28. If I compile on SDK 29 it works. Running this on a Pixel 3a running Android 10.

I can confirm that setting compileSdkVersion and targetSdkVersion to 29 resolved this issue.

I have the same issue.

I see that with iOS too. getCurrentPosition() shows dialog box to enable location always or enable location while using the app or deny location access. Selecting "Enable while using the app" results in an exception with an error "PERMISISON_DENIED"
I am using geolocation 5.1.3

A similar error is reproduced on Pixel 4.

Same issue.
Samsung Galaxy S10 Plus, when click on "Keep while-in-use access" get PermissionStatus.denied

We are currently working on a new version of the location permissions plugin. This new version includes respected while-in-use access and no longer returns PermissionStatus.denied.

https://github.com/Baseflow/flutter-permission-plugins/issues/21

Urgh - I just discovered that I am experiencing the same issue for a production app.  I had wrongly assumed that when a user gives in-use access permissions, this method would return PermissionStatus.granted - so I can no longer rely on this it seems.

"I can confirm that setting compileSdkVersion and targetSdkVersion to 29 resolved this issue." solved the issue until I added this in the AndroidManifest.xml
<uses-permission android:name="android.permission.ACCESS_BACKGROUND_LOCATION"/>

By making location_permissions respect LocationPermissionLevel in the checkPermissionStatus method, I prevented Geolocator.getCurrentLocation from calling requestPermissions after the user has granted when in use permission.
https://github.com/JBBx2016/flutter-permission-plugins/commit/29a1124cf805f549112a3534970fc762ca61a6c2

Any update?

any update?

Any Update

Not working on Android 10 devices. :|

As a temporary workaround, you can use my fork
branch Master
where I moved permission stuff from obviously broken location_permissions to another plugin permission_handler

My app just stop working after Google Play service PopUp, what ever i answer "Deny" , "Authorize", etc.
Android 10 with emulator.

still don't have solution zzzzz

Geolocator locator = Geolocator();
    locator
        .checkGeolocationPermissionStatus(
            locationPermission: GeolocationPermission.locationWhenInUse)

Need to pass permission in params.
GeolocationPermission.locationWhenInUse

Fixed in version 6.0.0


Vulnerability roundup 114: asterisk-18.5.1: 6 advisories [9.8]

search, files

[ ] CVE-2021-37706 CVSSv3=9.8 (nixos-21.11)
[ ] CVE-2022-23608 CVSSv3=9.8 (nixos-21.11)
[ ] CVE-2022-26651 CVSSv3=9.8 (nixos-21.11)
[ ] CVE-2022-21723 CVSSv3=9.1 (nixos-21.11)
[ ] CVE-2022-26499 CVSSv3=9.1 (nixos-21.11)
[ ] CVE-2022-26498 CVSSv3=7.5 (nixos-21.11)

CVE details
CVE-2021-37706
PJSIP is a free and open source multimedia communication library written in C language implementing standard based protocols such as SIP, SDP, RTP, STUN, TURN, and ICE. In affected versions if the incoming STUN message contains an ERROR-CODE attribute, the header length is not checked before performing a subtraction operation, potentially resulting in an integer underflow scenario. This issue affects all users that use STUN. A malicious actor located within the victim’s network may forge and send a specially crafted UDP (STUN) message that could remotely execute arbitrary code on the victim’s machine. Users are advised to upgrade as soon as possible. There are no known workarounds.
CVE-2022-23608
PJSIP is a free and open source multimedia communication library written in C language implementing standard based protocols such as SIP, SDP, RTP, STUN, TURN, and ICE. In versions up to and including 2.11.1 when in a dialog set (or forking) scenario, a hash key shared by multiple UAC dialogs can potentially be prematurely freed when one of the dialogs is destroyed . The issue may cause a dialog set to be registered in the hash table multiple times (with different hash keys) leading to undefined behavior such as dialog list collision which eventually leading to endless loop. A patch is available in commit db3235953baa56d2fb0e276ca510fefca751643f which will be included in the next release. There are no known workarounds for this issue.
CVE-2022-26651
An issue was discovered in Asterisk through 19.x and Certified Asterisk through 16.8-cert13. The func_odbc module provides possibly inadequate escaping functionality for backslash characters in SQL queries, resulting in user-provided data creating a broken SQL query or possibly a SQL injection. This is fixed in 16.25.2, 18.11.2, and 19.3.2, and 16.8-cert14.
CVE-2022-21723
PJSIP is a free and open source multimedia communication library written in C language implementing standard based protocols such as SIP, SDP, RTP, STUN, TURN, and ICE. In versions 2.11.1 and prior, parsing an incoming SIP message that contains a malformed multipart can potentially cause out-of-bound read access. This issue affects all PJSIP users that accept SIP multipart. The patch is available as commit in the master branch. There are no known workarounds.
CVE-2022-26499
An SSRF issue was discovered in Asterisk through 19.x. When using STIR/SHAKEN, it's possible to send arbitrary requests (such as GET) to interfaces such as localhost by using the Identity header. This is fixed in 16.25.2, 18.11.2, and 19.3.2.
CVE-2022-26498
An issue was discovered in Asterisk through 19.x. When using STIR/SHAKEN, it is possible to download files that are not certificates. These files could be much larger than what one would expect to download, leading to Resource Exhaustion. This is fixed in 16.25.2, 18.11.2, and 19.3.2.

Scanned versions: nixos-21.11: 3c5ae9be1f1.
Cc @DerTim1
Cc @auntieNeo
Cc @yorickvp

We no longer support nixos-21.11, but this has been fixed in the currently supported NixOS versions:

nixos-22.05: asterisk-16.26.1, asterisk-19.4.1, asterisk-18.12.1
nixos-unstable: asterisk-16.26.1, asterisk-19.4.1, asterisk-18.12.1


Image Resizer settings page needs additional blurb

When we agreed to not do additional settings for for ReactJS, we said we would have verbiage to point people to the settings panel.  It is currently a MIA string.

The fix is now available in 0.16 https://github.com/microsoft/PowerToys/releases


Revoke Dialog Incorrectly Hides Duplicate Labels

I happened to have some 14 OAuth Tokens with the same label on my account, and went to mass-revoke them.  The UI only showed one token in the list of things to revoke:

It did correctly revoke all 14 tokens.

${label} (xN)


[SPARK-5361]python tuple not supported while converting PythonRDD back to JavaRDD

Existing SerDeUtil.pythonToJava implementation does not count in tuple cases: Pyrolite python tuple => java Object[].
So with the following data:
[
(u'2', {u'director': u'David Lean', u'genres': (u'Adventure', u'Biography', u'Drama'), u'title': u'Lawrence of Arabia', u'year': 1962}),
(u'7', {u'director': u'Andrew Dominik', u'genres': (u'Biography', u'Crime', u'Drama'), u'title': u'The Assassination of Jesse James by the Coward Robert Ford', u'year': 2007})
]

Exceptions happen at the genres part:
15/01/16 10:28:31 ERROR Executor: Exception in task 0.0 in stage 3.0 (TID 7)
java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to java.util.ArrayList
	at org.apache.spark.api.python.SerDeUtil$$anonfun$pythonToJava$1$$anonfun$apply$1.apply(SerDeUtil.scala:157)
	at org.apache.spark.api.python.SerDeUtil$$anonfun$pythonToJava$1$$anonfun$apply$1.apply(SerDeUtil.scala:153)
	at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:371)
	at scala.collection.Iterator$$anon$10.hasNext(Iterator.scala:308)

This pull request adds in tuple handling both in SerDeUtil.pythonToJava and JavaToWritableConverter.convertToWritable.

Do you mind opening a JIRA issue for this and updating your pull request title to reference that issue (e.g. [SPARK-XXX] My PR title…)?

Can one of the admins verify this patch?

@JoshRosen Updated. Thanks

Jenkins, this is ok to test.

Test build #25925 has started for   PR 4146 at commit 9f1a097.

This patch merges cleanly.

Test build #25925 has finished for   PR 4146 at commit 9f1a097.

This patch fails Spark unit tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25925/
Test FAILed.

checking

Oh, this looks like a spurious failure due to some Jenkins flakiness (which I'm going to investigate separately).

yeah, build #25927 is down to only 1 unrelated failed test case.
I am watching #25928 to see if it goes away itself.

Can you add a regression test for this?  See python/pyspark/tests.py.  Make sure to reference the JIRA number in the test case name.

@JoshRosen Sure thing. I did not see any similar regression test case in this file though.
Is there any convention to follow? Or will I just create a test class called Regression_5361 or Spark_5361?
Thanks

If it was me, I'd probably put it as a new case in the SerializationTestCase suite.  I think our convention has been to give the tests a descriptive name and just leave a comment next to it linking to the JIRA.  For example: https://github.com/apache/spark/blob/master/python/pyspark/tests.py#L216

great! I will follow it then.

Found a good way to reproduce it:
from pyspark.rdd import RDD

dl = [
    (u'2', {u'director': u'David Lean'}),
    (u'7', {u'director': u'Andrew Dominik'})
]

dl_rdd = sc.parallelize(dl)
tmp = dl_rdd._to_java_object_rdd()
tmp2 = sc._jvm.SerDe.javaToPython(tmp)
t = RDD(tmp2, sc)
t.count()

tmp = t._to_java_object_rdd()
tmp2 = sc._jvm.SerDe.javaToPython(tmp)
t = RDD(tmp2, sc)
t.count() # it blows up here during the 2nd time of conversion

I am going to make a test case from this example.

Test build #25977 has started for   PR 4146 at commit 4cf1187.

This patch merges cleanly.

Test build #25977 has finished for   PR 4146 at commit 4cf1187.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25977/
Test FAILed.

@JoshRosen Added in a test case. Thanks

Test build #25978 has started for   PR 4146 at commit 126be6b.

This patch merges cleanly.

Test build #25978 has finished for   PR 4146 at commit 126be6b.

This patch fails Python style tests.
This patch merges cleanly.
This patch adds no public classes.

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25978/
Test FAILed.

Test build #25981 has started for   PR 4146 at commit 5d90a83.

This patch merges cleanly.

Test build #25981 has finished for   PR 4146 at commit 5d90a83.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25981/
Test PASSed.

@JoshRosen Anything else I can do here?

This seems good to me overall; I left one minor comment regarding a small performance optimization, but aside from that this seems good to me.  Sorry for the delay in review.

Test build #26123 has started for   PR 4146 at commit 903df7d.

This patch merges cleanly.

@JoshRosen updated to toSeq. Seq has a better consistency with line ln no 161. Thanks

Test build #26123 has finished for   PR 4146 at commit 903df7d.

This patch passes all tests.
This patch merges cleanly.
This patch adds no public classes.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/26123/
Test PASSed.

@JoshRosen Can we have this in the next release? We will have to use our own fork if it's not in. Thanks

Let me take one final look to see if I can pull this in for 1.2.1 (since we're cutting a new RC tonight).  In general, this looks safe since only adds new code paths in cases where we'd otherwise throw exception, as opposed to changing the behavior of existing code paths.  If things check out, I'll pull it in for both 1.3.0 and 1.2.1.

@wingchen Actually, just to be clear here, is this problem related to tuple handling, or is the actual issue related to multiple Java <-> Python conversions not working correctly?  If there's nothing tuple-specific about this, do you mind editing the PR title, description, and JIRA to reflect this?

@JoshRosen I found this one reading RDD from sc.newAPIHadoopRDD and writing it back using rdd.saveAsNewAPIHadoopFile. It's only throwing exceptions at the tuple part. That's why I though tuple was the reason.
But it seems that multiple Java <-> Python conversions not working correctly was the real reason for it to fail. I am updating PR title, description, and JIRA accordingly. Thanks

@JoshRosen updated. thanks :)

@wingchen Thanks for updating this.  The new description + code both look good to me, so I'm going to merge this into master for inclusion in 1.3.0 and tag it for a backport into 1.2.2.

@JoshRosen Thanks a lot for your help.

I've cherry-picked this into branch-1.2 (1.2.2).


example in remove bullets

Given the same example in 'remove bullets' section which was in unordered list

[x] I have read freeCodeCamp's contribution guidelines.
[x] My pull request has a descriptive title (not a vague title like Update index.md)
[x] My pull request targets the master branch of freeCodeCamp.
[x] None of my changes are plagiarized from another source without proper attribution.
[x] My article does not contain shortened URLs or affiliate links.

If your pull request closes a GitHub issue, replace the XXXXX below with the issue number.
Closes #XXXXX

Hi @drawingvishal
It seems that similar changes have already been accepted earlier for this article you're editing, sorry about that. 😓
If you feel you have more to add, please feel free to open up a new PR.
Thanks again! 😊

If you have any questions, feel free to reach out through Gitter or by commenting below. 💬


CVE-2022-45473: world-readable logfile

Hello,
when drachtio-server starts, creates /var/log/drachtio with mode 777
This leads to a disclosure because a local user can retrieve sensitive data (like IP and so on).
Here is the details:
drachtio1 ~ # systemctl stop drachtio
drachtio1 ~ # rm -fr /var/log/drachtio
drachtio1 ~ # systemctl start drachtio
drachtio1 ~ # ls -la /var/log/drachtio/
total 12
drwxrwxrwx  3 root root 4096 Nov 18 16:01 .
drwxr-xr-x 15 root root 4096 Nov 18 16:01 ..
drwxrwxrwx  2 root root 4096 Nov 18 16:01 archive
-rw-rw-rw-  1 root root    0 Nov 18 16:01 drachtio.log

To fix this issue, /var/log/drachtio should be created with mode 770

fixed in v0.8.19-rc12

CVE-2022-45473 as been assigned to this issue.

could you please update the various entries you made out in the world to indicate this is fixed

I don't know what stays for "entries you made out in the world"
The CVE was requested when the issue was unfixed. When it was fixed I sent an update, but the update on nvd.nist.gov does not depend on me.


fixed/Module not found: Can't resolve 'react-onclickoutside' #4122

fixed/Module not found: Can't resolve 'react-onclickoutside' #4122
A dependency react-onclickoutside was missing.

Not sure why this should solve the issue you're referring to, but bumping the version is fine.


Generate Poetry's requirements.txt

Currently on the master branch the requirements.txt is empty, this is the file generated by running poetry export --dev -f requirements.txt > requirements.txt

Hey we're changing from MIT to GPLv3 because some of our projects are licensed with that, do you approve? If you do not approve we will be forced to delete your code and rewrite it 😢
By approving of this change, you are agreeing to re-license the code you wrote under GPLv3.

Hey we're changing from MIT to GPLv3 because some of our projects are licensed with that, do you approve? If you do not approve we will be forced to delete your code and rewrite it 😢
By approving of this change, you are agreeing to re-license the code you wrote under GPLv3.

I approve!


ATX validity

The Motivation
ATX validation should be divided into 2:

"syntactic" validity -- determines whether it should be propagated.
"contextual" validity -- determines whether the source of the ATX is "playing by the rules".

The Requirement
Under syntactic validity we have all the conditions that could be determined by the ATX itself (and its view).
Everything else, like conflicting sequence numbers or appropriate amount of ticks (if we add this in the future), is under contextual validity.
A syntactically ATX is considered "Valid", and should be propagated.
If an ATX is not contextually valid, the flow is as follows:

The miner becomes invalid -- all of her votes (in previous and in future blocks) are considered 0; other miners vote against her future blocks (we may, in the future, won't let her publish ATX/blocks, i.e. won't propagate them).
An evidence of the fault should be put on the mesh: if the miner is currently valid (no evidence on the chain) we publish the conflicting ATX* (put it in a block). If the miner is invalid, we do nothing more.

if there are several (conflicting) ATXs at the same time, we sort them and take the lower one to be the evidence.

finalise this once https://github.com/spacemeshos/research/issues/31 is agreed upon

Already implemented.


Ability to watch replay with fog

Post game the fog is lifted for replays, but it would be nice to watch a replay from any player's perspective.

fixed in https://github.com/spqr-game/spqr/pull/727


Archive takes to long to finish

Hi, this is not a issue a think, is more like a doubt.
After install firebasex using ionic 3, the size of my platform ios ( after build), increase like too must, duplice the size and the archive now takes almost a hour to finish. This is a regular behavior using this plugin?

Build times are very much dependent on the power of your dev machine: my machine takes about 40 seconds to build the example project for this plugin using Xcode 11.5.
The long build time is because this plugin includes most of the various components of the Firebase SDK and each of those includes external dependencies and build tasks. The Firebase Auth and Firestore components seem to include a particularly large number hence why v9 of this plugin sees an increase in build times since it pulls in these new SDK components.
There is nothing that can be done to reduce the build time except to reduce the dependencies and build tasks by removing Firebase SDK components you don't use (as outlined in #291) but until such time as this plugin is broken down into modular components, if you choose to use it as is, you will either have to live with the build times or fork it and remove the components you don't use yourself.

My machine takes almost 30 minutes :(

We are facing the same issue. With the update from v7 to v9-cli the build times increased by ~10 minutes.


Elements definitions

Since a few lists of symbols are present in the figure_defs module, and some comments about choosing good colors are given in the docstring, it might be good to provide some defaults for setting symbols and colors as well. I made a quick sketch of how it might be implemented in this PR, but before merging this should (at least) be addressed:

[x] Clean up/remove old code (global symbol list definitions)
[ ] Complete implementation of color schemes
[ ] Bump cosmoplots version

See example usage here.

Shall I merge the PR and update the package on Pypi or shall we wait for the other bullet points to be addressed?

It works as it is now, but I'm not sure how useful it is when it is still not complete.
If putting in the rest if the colors is quick to do (i.e., someone has a clear idea of what colors they wanted to use), then maybe it is nice to include it in the same PR. But I'm not sure what the intended colors are based on the comments in the script, which I suppose was made by @audunth in the old uit_scripts repo?
But if there atm. are no colors that were the intended colors to use (meaning that, to finish the implementation someone would have to make up some palette to use), then merging is fine with me and the rest of the colors can be tracked in a new issue instead.
What do you think?

...which I suppose was made by @audunth in the old uit_scripts repo?

Yes, I just copied that part from the uit_scripts repo.
I personally don't have any strong opinions about the color scheme, I guess we can wait with merging until someone has made a decision.

Might also be a useful resource: https://doi.org/10.1038/s41467-020-19160-7.


BUG: Fix ITK_DIR for cxx workflows

Resolves issue where ITK_DIR was not updated for Windows shortened path lengths changes.
Issue observed at https://github.com/KitwareMedical/ITKMorphologicalContourInterpolation/actions/runs/3634013547/jobs/6131669519

Out of caution I'll wait for https://github.com/KitwareMedical/ITKMorphologicalContourInterpolation/actions/runs/3634196177/jobs/6132042226 to complete successfully before merging.

Seeing failures in https://github.com/KitwareMedical/ITKMorphologicalContourInterpolation/actions/runs/3634196177/jobs/6132042226, it seems that ITK is being built twice. Investigating.

Output from https://github.com/KitwareMedical/ITKMorphologicalContourInterpolation/actions/runs/3640706239/jobs/6148482086 shows that Linux and MacOS builds are running as expected but Windows is failing. It is expected that ITK is built first, then the ITK module is built and tested with any warnings or errors reported to CDash. In the Windows case ITK builds first, but then for some reason ITK builds again and its tests are run as if it were the module under test, with ITK's warnings reported to CDash.  Currently unclear why this happens.
The sole difference in CDash configuration that I can find between previous and updated Windows behavior is that CTEST_CHECKOUT_COMMAND is being implicitly populated in only the Windows version.
CTEST_CHECKOUT_COMMAND=["D:/a/_temp/-111744671/cmake-3.22.2-windows-x86_64/bin/cmake.exe" -P "/d/a/im/../im-init.cmake"]

I've opened https://github.com/InsightSoftwareConsortium/ITKRemoteModuleBuildTestPackageAction/pull/25 to revert the incomplete changes for shortening path names that are currently breaking main. Will use this branch and PR to track continued efforts to fix that procedure.


Integrating the Swift Forum

Apple just moved their swift evolution mailing list to the new forum: https://forums.swift.org/c/evolution. Maybe we can fetch those conversations and show them under appropriate proposals?

 So it is possible to follow the documentation in: https://docs.discourse.org


are iot CRDs intented only for open-shift?

When trying to deploy a IotConfig on a k8 cluster (not oc) I get the error
Configuration Error: Implicit inter-service certificate configuration can only be used on OpenShift 4

I see that the iot crds include interServiceCertificates.
Is it intended that iot only be supported on openshift?

When using plain Kubernetes, you need to manually manage the inter-service certificates:
https://github.com/EnMasseProject/enmasse/blob/22adf763814d6e6b08af7707144ad1ab2c272d84/templates/iot/examples/iot-infra-k8s.yaml#L7-L16


Dashboard: ephemeral certificate is invalid in MacOS Catalina (10.15) & iOS 13

Apple now enforces its requirements for trusted certificates: Requirements for trusted certificates in iOS 13 and macOS 10.15
As such macOS users are unable to access the Flynn Dashboard with a "Your connection is not private" error in Chrome, or untrusted certificate in Safari.
For example: NET::ERR_CERT_REVOKED

So it's really time for the LetsEncrypt automation which was promised a long time ago.

Oh, I thought Flynn was abandoned.
I was just putting this here so people would not spend a day working out what was going wrong given the error is quite deceptive.

Oh, I thought Flynn was abandoned.
I was just putting this here so people would not spend a day working out what was going wrong given the error is quite deceptive.

So, Flynn is dead? do you know any other alternative ?

the trend seems to point to Knative

the trend seems to point to Knative

I don't think their site works?
Try https://cloud.google.com/knative/

is flynn still active or not?

@raj457036 It is not

Is there any similar tool which is easy to use? I don't want to use the scary kubernets at all...

Have you looked at https://www.cloudfoundry.org/

Flynn is still active. I use a fairly recent version of it. Also I had this issue, and fixed it, but I can't quite recall the solution. I am trying to locate so that I can add it here for others.


Generating Random Error Generators

Is your feature request related to a problem? Please describe.
We often like to test new protocols or features with random noise to see how it performs in simulation. It is fairly straightforward to generate realistic random noise for full/TP parameterized operations, using the depolarize, rotate, or kick functions. However, we currently don't have a good tool for generating random CPTP operations that are parameterized by error generators.
Describe the solution you'd like
A function (maybe in lindbladtools?) which can generate valid random CPTP maps parameterized by error generators. Ideal features include:

being able to specify noise strengths for H, S, C, A sectors (CA may have to be tied together, and even SCA being tied together would be a start)
what kind of distribution they are drawn from (normal makes sense for H, uniform makes more sense for SCA)
different noise strengths based on weight or qubit support (maybe weight-2 should be smaller than weight-1, maybe one qubit is particularly bad)

Of course, the challenge is being able to provide these sorts of tunable parameters and ensuring that the SCA matrix is PSD such that the map is CPTP overall...
Describe alternatives you've considered
We've tried variants of this in the past, normally by sampling some matrices and using<EMAIL_ADDRESS>tricks to enforce the PSD constraint. Let's do this once correctly and put it in pyGSTi so we can stop rediscovering this every few months. :)

Just a heads-up to avoid working at cross purposes, I've started working on an implementation of this as part of a soon to be published branch. The currently envisioned features are (thanks to @kevincyoung for letting me pick his brain):

Specification of error generator sectors
Setting a target generator infidelity (if none is specified this wouldn't be fixed)
Relative weights on the error generator sectors (with regards to some norm)
Specification of maximum (pauli) weights for each of the error generator sectors.

This overlaps a good deal with your list of desiderata, with the possible exception of:

different noise strengths based on weight or qubit support (maybe weight-2 should be smaller than weight-1, maybe one qubit is particularly bad)

I hadn't considered this, but it should be doable. This option is definitely something I can envision someone wanting down the road, so implementing it now is definitely in the spirit of 'do it once so we don't need to do this again.'
Follow up on your parenthetical:

(CA may have to be tied together, and even SCA being tied together would be a start)

I hadn't considered this. My current target was for the output of this function to return a dictionary with ElementaryErrorgenLabel objects as keys and rates as values. I.e. the dictionary format used by the from_elementary_errorgens constructor for LindbladErrorgen, or as used in the various set_errorgen_coefficients methods. Such dictionaries are in general agnostic to the implementation details at the LindbladCoefficientBlock level though. What would currently happen if I passed the constructor a dictionary with just, say, S and C terms? Would it allocate parameters for the A terms and just set them to zero?

What would currently happen if I passed the constructor a dictionary with just, say, S and C terms? Would it allocate parameters for the A terms and just set them to zero?

In this case, the LindbladCoefficientBlock that is allocated would be the full SCA one, i.e. yes you would have A terms present in the parameterization and they would be set to 0. It is only possible to have the following sets of blocks:

H (block_type = "ham")
S (block_type = "other_diagonal")
SCA (block_type = "other")

I see. This definitely a scope creepy sort of question, but how much of a lift would it be to decouple the C and A terms? E.g. just having an SC or SA block?  This is something we've mentioned wanting to be able to do before for finer tuned reduce model construction. It is also a parallel issue to this thread, however.

Medium lift IMO. It's luckily mostly localized to LindbladCoefficientBlock, but I wouldn't be surprised if there are inherent assumptions in other places in the code that if you have C, you have A, or vice versa. We'd probably want to build off of the param_mode = "elements", but that is significantly less used/tested compared to param_mode="cholesky", so wouldn't be surprised if we have to do some fixing there too. Plus I have this intuition that the Cholesky decomp is critical to enforcing the PSD constraint, so I'm missing a piece in understanding how we are enforcing CPTP with "elements" parameterization.
To avoid scope creep, I'd be inclined to keep them tied together for this PR, and then when we decouple later, we update the random generation as well. Really, nothing needs to happen in the random generation code if you are describing via the dictionaries and we are automagically detecting block structure, just under the hood we would allow an SC block instead of an SCA block in your example.


Coingecko check fails

I'm submitting a ...
[x] bug report
[ ] feature request
[ ] question about the decisions made in the repository
[ ] question about how to use this project

Summary
Created two PR with new tokens both of them fails on coingecko check. Endpoint are correct.
https://github.com/solana-labs/token-list/pull/9760
https://github.com/solana-labs/token-list/pull/9056

Other information (e.g. detailed explanation, stack traces, related issues, suggestions how to fix, links for us to have context, eg. StackOverflow, personal fork, etc.)

weird, seems like some sort of cloudflare issue
merged #9760
for #9056 see PR

merged


Integrate Pathways summary widget and detail view using with GraphQL API

Pathway data is now available in the new GraphQL API (sample query for IL2RA).

Please implement the summary widget and detail views with the following changes noted below.
Summary widget

Display total number of pathways

Detail view

Remove the "Top-level parent pathways" section as it appears to act as a filter with clickable elements but is not interactive
Change the "View diagram" column heading to "View target (ESR1) and pathway". Within the column, remove the PNG and SVG links and instead construct a link to the Reactome pathway browser page https://reactome.org/PathwayBrowser/#/PATHWAY_ID&FLG=UNIPROT_ID (example)
In the "Top-level parent pathway" column in the data table, please display the top-level parent pathway(s) which are noted with isRoot: True (e.g. in the case below, both Signal Transduction and Immune System would be visible in the column)

I've talked @andrewhercules about this task and started looking into the implementation.

I've talked to @andrewhercules about this task and started looking into the implementation.

Most of the work is ready in #158—I'll push a patch to replace the PNG links soon.

Closing this ticket as PNG link and Reactome link label will be captured in target profile page polish work - see epic #1073


Need to PluginInstall each time to use Black

Using:
Plugin 'ambv/black', {'rtp': 'vim'}
Black command not recognized unless I execute PluginInstall
The plugin is here:
https://github.com/ambv/black/blob/master/plugin/black.vim
Is this a bug in the Plugin?

The problem was with my .vimrc
Just installing from:
Plugin 'ambv/black'

solved it.


Dropdown component - need an accessibility example of  combo box with multiselect with filter

when an team was reviewing the Accessibility guidance at
https://carbondesignsystem.com/components/dropdown/accessibility
the example correctly include:

Multiselect
Combo box

but missing is a combination of both a Combo box with multiselect with filter

[ ] add an example

This combined example would need to explain how to best handle the input cursor where the user is typing into the filter, while also managing the up/down arrow keys to view the items to possibly select, and if  Spacebar would enter a space at the insert cursor or select the item being reviewed with focus.
Many feel it would be best to select the item with the Enter key, while the Spacebar would input a "space" at the insert cursor.


Fix imports to support Components v 2.0.1

💥 Regression Report
The build is not passing with components v 2.0.1 https://github.com/HospitalRun/hospitalrun-frontend/pull/2387/checks?check_run_id=1094941301 because we changed how components' scss files are exported.
To Reproduce
npm run build
Expected behavior
Build without throwing.

Relates to https://github.com/HospitalRun/components/issues/583 and https://github.com/HospitalRun/components/pull/584

Can I pick this to follow up with my pull-request in @hospitalrun/components https://github.com/HospitalRun/components/pull/593#issue-486944957
Please note: now that there's no need for .scss references in the front-end, the main.scss reference in the front-end must be removed as follows.

   ...
   import { Provider } from 'react-redux'
 - import<EMAIL_ADDRESS>  // ***to be removed
 - import './index.css'                               // ***include after App import
    import App from './App'
 // ***must import after App to override component styles
 + import './index.css'
  ...

Closed via: 3844cd3322f68e12318e90c54161324f1509c8df


Fix attributeComparator default/primary attribute valueType

Looks like "primary" and "default" valueTypes are doing the same thing, but attributeComparator ignores "default" valueType. Fixes #135

Thanks!


TreeView.SelectedNodes causes System.AccessViolationException

Accessing a TreeView's SelectedNodes or SelectedNodes.Count property can catastrophically fail bringing down the app and even the debugger. The TreeView has no root nodes and SelectionMode=Single when the crash happens.
System.AccessViolationException
HResult=0x80004003
Message=Attempted to read or write protected memory. This is often an indication that other memory is corrupt.
Source=
StackTrace:

This is why WPF is so much more powerful. It is implemented in managed code.

Hey @robloo, thanks for reporting the issue. We are looking into it.

@jevansaks - wasn't this bug already fixed by Kai. If yes, we can close now.

Agree, this is fixed. If not please open an issue against https://github.com/Microsoft/microsoft-ui-xaml.


Propcore error, can't spawn any props

My e2 simply reads:
Barrel = propSpawn( "models/props_borealis/bluebarrel001.mdl", entity():toWorld( vec( 0, 0, 24 )), 1 )
and I get this console error:
sv: Expression 2 (generic): entities/gmod_wire_expression2/core/custom/prop.lua:36: bad argument #1 to 'IsValidProp' (string expected, got nil)
I get this error with every single prop I try.

my apologies, it was antcore extension causing a conflict. i disabled antcore and reloaded wire and it works now.

My apologies, it was antcore extension causing a conflict. I disabled antcore and reloaded wire and it works now.


Added parse definition

Added definition for Parse JS SDK.
Some less used modules are still not defined (and marked as TODO).

@ryyppy Thank you for checking the code. Can you point me to the documentation about the namespacing? Is namespacing the exported module (Parse) enough? Thanks!

Hey @mmazzarolo no worries, namespaces are a flow-typed specific thing, because we don't want to have collisions with global userspace types (which will eventually happen without this weird $npm$[library] prefix.
For more information please check our CONTRIBUTING.md

I just realized that you declared almost all type declarations etc. in the toplevel scope of the declaration file.. that would mean you would be required to namespace every single type (which is kinda hard to read and also tedious to change)... could you try this:
declare module '@@parse' {
  // declare all the stuff you need
  declare type Parse = {...};
  declare module.exports: Parse;
}

declare module 'parse/node' {
  import Parse from '@@parse';
  declare module.exports: typeof Parse;
}

// same for 'parse/react-native' etc.

I didn't test it like this, but I think this should work to prevent global scope pollution

Tried it but it seems it doesn't work.
I even tried playing with the import itself but with no results:
declare module 'parse/browser' {
  import type Parse from '@@parse'
  declare module.exports: Parse
}

@ryyppy I just namespaced everything :)

Oh snap, probably the import inside declare module is not supported yet... maybe in the next feature version?
Anyways, something seems to be broken during the renaming process :-(

Thank you for your contribution :-)

@ryyppy no worries, thank you for the super-support.
I'll try to keep it updated, we use Parse everyday here.


meta.json

meta.txt

{
"name": "Salvator Satoshi",
"inscription_icon": "307cc34a6f13d18838c058781738cc1f131ba2514894287ad492f791eafef1a5i0",
"supply": "100",
"slug": "salvator-satoshi",
"description": "One of Bitcoin's 1st collections to use an artist's vision in collaboration with AI. Inspired by some of the world's most famous art pieces, the Salvator Satoshi collection illustrates the legend of Satoshi Nakamoto through the eyes of Malikaii Mask.",
"twitter_link": "https://twitter.com/MalikaiiM",
"discord_link": "",
"website_link": "https://linktr.ee/MalikaiiM"
}

meta.txt\


ImportError: cannot import name 'prune_state_dict'

🐛 Bug
After git commit 569c024ae6ee3e8c37c3b9dca975a3df50f7a03
The issue was not there before git commit d3890e593398c485f6593ab8512ac51d37dedc9c

import fairseq
Traceback (most recent call last):
File "", line 1, in
File "./fairseq/fairseq/init.py", line 32, in
import fairseq.criterions  # noqa
File "./fairseq/fairseq/criterions/init.py", line 36, in
importlib.import_module("fairseq.criterions." + file_name)
File "/usr/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "./fairseq/fairseq/criterions/label_smoothed_cross_entropy_latency_augmented.py", line 6, in
from examples.simultaneous_translation.utils.latency import LatencyTraining
File ./fairseq/examples/simultaneous_translation/init.py", line 6, in
from . import models  # noqa
File "./fairseq/examples/simultaneous_translation/models/init.py", line 14, in
"examples.simultaneous_translation.models." + model_name
File "/usr/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "./fairseq/examples/simultaneous_translation/models/convtransformer_simul_trans.py", line 8, in
from fairseq import checkpoint_utils
File "./fairseq/fairseq/checkpoint_utils.py", line 23, in
from fairseq.models import FairseqDecoder, FairseqEncoder
File "./fairseq/fairseq/models/init.py", line 20, in
from .fairseq_model import (
File "./fairseq/fairseq/models/fairseq_model.py", line 17, in
from fairseq.checkpoint_utils import prune_state_dict
ImportError: cannot import name 'prune_state_dict'

To Reproduce
Steps to reproduce the behavior (always include the command you ran):

Run cmd 'python3` and import fairseq
See error

There is already a PR for it. #3286

great, you can close it now


[Doc] Gitea in sub-path config document is incompatible with path escaping

Description

Correct: https://docs.gitea.io/en-us/reverse-proxies/#nginx

Buggy: https://docs.gitea.io/en-us/reverse-proxies/#nginx-with-a-sub-path

The ending slash shouldn't be used, because it causes nginx to unescape the path encodings, it breaks %2f in the path, breaks some Gitea git refs/paths and some Gitea package names/paths.
More details:

https://github.com/go-gitea/gitea/issues/20890#issuecomment-1483950304

Thanks to @ghnp5

Note that the "Buggy" section has a slash so that it will work for the sub-path.
If you remove the slash, all pages on Gitea will get a 404, because the "git/" sub-path is being passed to all URLs to Gitea.
To fix that, this hack is needed:
	location /git/ {
		# this hack prevents URL encoding, which can break the Packages area, for package names with slashes
		rewrite ^ $request_uri;
		rewrite ^/git(/.*) $1 break;
		return 400; # if the second rewrite won't match
		proxy_pass http://<IP_ADDRESS>:1234$uri;


Problem with deploying contract with EIP712 (block.chainid) via hardhat

There is a problem when deploying a hardhat contract with @openzeppelin/contracts/utils/cryptography/draft-EIP712.sol inheritance. Inside constructor EIP712 the block.chainid request on which the error occurs. When deploying to test networks Rinkeby, Kovan, Ropsten, etc. no such problem is observed.
besu/v22.1.0-RC4/windows-x86_64/oracle-java-17

hi @PavelZavadskiy - to clarify, you see the problem only when deploying to mainnet?

to clarify, you see the problem only when deploying to mainnet

I have a problem when deploying in a besu. I run the besu like --network=dev.
Thanks

@PavelZavadskiy can you share a minimal repo with the contract that you are deploying - that would make it a lot easier for us to reproduce.

Also please include how are you deploying the contract?

https://github.com/PavelZavadskiy/ERC712_Test
npm i

@PavelZavadskiy just wanted to confirm that you've tried deploying the same contract with other clients and only Besu has this issue?

just wanted to confirm that you've tried deploying the same contract with other clients and only Besu has this issue?

Yes. The problem is only with the Besu.

Which other clients have you tried?
On Thu, 24 Feb 2022 at 7:47 pm, Pavel @.***> wrote:

just wanted to confirm that you've tried deploying the same contract with
other clients and only Besu has this issue?
Yes. The problem is only with the Besu.
—
Reply to this email directly, view it on GitHub
https://github.com/hyperledger/besu/issues/3417#issuecomment-1049672112,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AJMWR43F5X2KFZ2IO6GZR7DU4X5ETANCNFSM5OJKNC6Q
.
You are receiving this because you were assigned.Message ID:
@.***>

this could be fixed by setting "muirglacierblock": 0, in your genesis file
https://besu.hyperledger.org/en/stable/HowTo/Configure/Genesis-File/

@PavelZavadskiy a solution is given above - reopen if you're still having issues

@TrejGun which genesis file? I tried in IBFT.genesis.json and still have same error.

I believe you have to create one and pass it to the container
  besu:
    container_name: besu
    user: root
    image: hyperledger/besu:latest
    environment:
      BESU_GENESIS_FILE: /var/lib/besu/genesis.json
      .....
    volumes:
      - ${BASE_PATH}/besu:/var/lib/besu
      - ${BASE_PATH}/gen-besu.json:/var/lib/besu/genesis.json


get error on earlier Julia versions when assertions are enabled

On Julia versions before 1.9 and Enzyme main (d7a68c407872b5d5a63a740150efedd902bd9e8b) it appears there is an error with get when LLVM assertions are enabled:
using Enzyme, Test
f12(x, k) = get(Dict(1 => 1.0, 2 => x, 3 => 3.0), k, 1.0)
@test autodiff(Reverse, f12, Active, Active(2.0),  Const(2))[1] == (1.0, nothing)
@test autodiff(Forward, f12, Duplicated(2.0, 1.0), Const(2))    == (1.0,)
@test autodiff(Reverse, f12, Active, Active(2.0),  Const(3))[1] == (0.0, nothing)
@test autodiff(Forward, f12, Duplicated(2.0, 1.0), Const(3))    == (0.0,)
@test autodiff(Reverse, f12, Active, Active(2.0),  Const(4))[1] == (0.0, nothing)
@test autodiff(Forward, f12, Duplicated(2.0, 1.0), Const(4))    == (0.0,)

Error is at https://github.com/EnzymeAD/Enzyme.jl/actions/runs/5808215994/job/15744567711?pr=969:
julia: /workspace/srcdir/llvm-project/llvm/lib/Analysis/ScalarEvolution.cpp:8150: llvm::ScalarEvolution::ExitLimit llvm::ScalarEvolution::computeExitLimitFromSingleExitSwitch(const llvm::Loop*, llvm::SwitchInst*, llvm::BasicBlock*, bool): Assertion `L->contains(Switch->getDefaultDest()) && "Default case must not exit the loop!"' failed.

signal (6): Aborted
in expression starting at /home/runner/work/Enzyme.jl/Enzyme.jl/test/runtests.jl:258
gsignal at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
abort at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
unknown function (ip: 0x7f8d404d8728)
__assert_fail at /lib/x86_64-linux-gnu/libc.so.6 (unknown line)
_ZN4llvm15ScalarEvolution36computeExitLimitFromSingleExitSwitchEPKNS_4LoopEPNS_10SwitchInstEPNS_10BasicBlockEb at /home/runner/work/Enzyme.jl/Enzyme.jl/julia/usr/bin/../lib/libLLVM-13jl.so (unknown line)
computeExitLimit at /workspace/srcdir/Enzyme/enzyme/Enzyme/MustExitScalarEvolution.cpp:110
getContext at /workspace/srcdir/Enzyme/enzyme/Enzyme/CacheUtility.cpp:560
forceAugmentedReturns at /workspace/srcdir/Enzyme/enzyme/Enzyme/GradientUtils.cpp:8195
CreateAugmentedPrimal at /workspace/srcdir/Enzyme/enzyme/Enzyme/EnzymeLogic.cpp:2311
recursivelyHandleSubfunction at /workspace/srcdir/Enzyme/enzyme/Enzyme/AdjointGenerator.h:7380
visitCallInst at /workspace/srcdir/Enzyme/enzyme/Enzyme/AdjointGenerator.h:10326
delegateCallInst at /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/llvm/IR/InstVisitor.h:302 [inlined]
visitCall at /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/llvm/IR/Instruction.def:209 [inlined]
visit at /opt/x86_64-linux-gnu/x86_64-linux-gnu/sys-root/usr/local/include/llvm/IR/Instruction.def:209
   @ Pkg.Operations ~/work/Enzyme.jl/Enzyme.jl/julia/usr/share/julia/stdlib/v1.8/Pkg/src/Operations.jl:1813
 [3] test(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; coverage::Bool, test_fn::Nothing, julia_args::Vector{String}, test_args::Cmd, force_latest_compatible_version::Bool, allow_earlier_backwards_compatible_versions::Bool, allow_reresolve::Bool, kwargs::Base.Pairs{Symbol, IOContext{Base.PipeEndpoint}, Tuple{Symbol}, NamedTuple{(:io,), Tuple{IOContext{Base.PipeEndpoint}}}})
   @ Pkg.API ~/work/Enzyme.jl/Enzyme.jl/julia/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:434
 [4] test(pkgs::Vector{Pkg.Types.PackageSpec}; io::IOContext{Base.PipeEndpoint}, kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:coverage, :julia_args, :force_latest_compatible_version), Tuple{Bool, Vector{String}, Bool}}})
   @ Pkg.API ~/work/Enzyme.jl/Enzyme.jl/julia/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:156
 [5] test(; name::Nothing, uuid::Nothing, version::Nothing, url::Nothing, rev::Nothing, path::Nothing, mode::Pkg.Types.PackageMode, subdir::Nothing, kwargs::Base.Pairs{Symbol, Any, Tuple{Symbol, Symbol, Symbol}, NamedTuple{(:coverage, :julia_args, :force_latest_compatible_version), Tuple{Bool, Vector{String}, Bool}}})
   @ Pkg.API ~/work/Enzyme.jl/Enzyme.jl/julia/usr/share/julia/stdlib/v1.8/Pkg/src/API.jl:171
 [6] top-level scope
   @ ~/work/_actions/julia-actions/julia-runtest/v1/test_harness.jl:15
 [7] include(fname::String)
   @ Base.MainInclude ./client.jl:476
 [8] top-level scope
   @ none:1
in expression starting at /home/runner/work/_actions/julia-actions/julia-runtest/v1/test_harness.jl:7
Error: Process completed with exit code 1.

So after investigating that assertion error is unnecessary here. Nevertheless is being fixed by https://github.com/EnzymeAD/Enzyme/pull/1468 (but will require a jllbump) since it'll make it harder to find actual assertion errors otherwises.

Should have jll bump landed already.


fix: remove redundancy node process clean

Types

[x] 🐛 Bug Fixes

Background or solution
close #229
Changelog
fix: remove redundancy node process clean

Codecov Report

Merging #230 (9dc2437) into main (f988b61) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main     #230   +/-   ##
=======================================
  Coverage   59.28%   59.28%
=======================================
  Files        1181     1181
  Lines       72666    72666
  Branches    15061    15061
=======================================
  Hits        43078    43078
  Misses      26965    26965
  Partials     2623     2623

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f988b61...9dc2437. Read the comment docs.

@yantze 链接的 PR 错了吧？

@yantze 链接的 PR 错了吧？

好了


Document limitations

Related to https://github.com/AcalaNetwork/chopsticks/issues/823 it would be really good to have it clearly documented right in the README what the limitations of the tools are. E.g. what is mocked/what is real? On what can you rely on what not? What are things that could work with chopsticks, but still fail in production?

There are many limitations and I am not sure if it make sense to try to list them all.
Generally, anything that involves something more than onchain STF new_state = f(old_state) are not guaranteed to work.
In practice, here is an incomplete list that I can think of:

mocked tx pool
no real block finalization
mocked inherents
simulated XCM channels


Package extensions for EvoTrees and LaplaceRedux

Closes #301. Accordingly with that issue:

Created package extensions for EvoTrees and LaplaceRedux models.
Removed the NearestNeighborModels dependency: it wasn't used as a dependency for any actual functionality in the package, so I assume it's a leftover from some past functionality that isn't used anymore and can safely be removed from the package.
Since the MLJDecisionTreeInterface.jl and DecisionTree.jl packages are pretty light and form a core part of the Feature Tweak algorithm, which we don't want to move out of the main package, these remain dependencies of the core package.

Great! Just one question: do we do usual tests (model predictions and also counterfactual generation) for these somewhere?

I checked this now and turns out that the tests for all generators aside from the Feature Tweak tests (which are run for DecisionTrees and RandomForests) use Models.standard_models_catalogue, which only contains LinearModel, FluxModel, and FluxEnsemble. Model predictions are tested for EvoTrees, but not for LaplaceRedux. From the time of the software project, I faintly remember us discussing that the LaplaceRedux models are missing something from being fully differentiable and that they weren't ready for counterfactual generation yet because of that, which is also why methods like train() haven't been implemented for it, though please correct me if I'm wrong. As for EvoTrees, I think we got it working with Growing Spheres during the software project, but it doesn't seem like counterfactual generation tests are being run for it at the moment.

As for EvoTrees, I think we got it working with Growing Spheres during the software project, but it doesn't seem like counterfactual generation tests are being run for it at the moment.

ok, thanks for checking. Would it be straightforward enough to add that here?

As for EvoTrees, I think we got it working with Growing Spheres during the software project, but it doesn't seem like counterfactual generation tests are being run for it at the moment.

ok, thanks for checking. Would it be straightforward enough to add that here?

Sorry, forgot about this PR for a while. I added this test, but an error came up somewhere inside the growing_spheres_generation() method. I haven't worked on Growing Spheres, so I think it's better if I resolve this together with Mariusz - I'll create an issue for this. In the meantime, I think it's fine to merge this PR, since the initialization of EvoTrees as an extension worked well and the error is coming from the counterfactual generation process which is unrelated to this PR, but @pat-alt let me know if you have a different opinion on this.

As for EvoTrees, I think we got it working with Growing Spheres during the software project, but it doesn't seem like counterfactual generation tests are being run for it at the moment.

ok, thanks for checking. Would it be straightforward enough to add that here?

Sorry, forgot about this PR for a while. I added this test, but an error came up somewhere inside the growing_spheres_generation() method. I haven't worked on Growing Spheres, so I think it's better if I resolve this together with Mariusz - I'll create an issue for this. In the meantime, I think it's fine to merge this PR, since the initialization of EvoTrees as an extension worked well and the error is coming from the counterfactual generation process which is unrelated to this PR, but @pat-alt let me know if you have a different opinion on this.

Happy new year 🥳  agree this should be merged

Happy new year! After fixing the conflicts, I'm now getting the error DataDeps.NoValidPathError("No writable path exists to save the data.") for a few tests on Julia 1 and Julia 1.10. I'm a bit confused about this, I don't think my PR is influencing anything about how DataDeps works. @pat-alt any idea why I'm getting this?

Happy new year! After fixing the conflicts, I'm now getting the error DataDeps.NoValidPathError("No writable path exists to save the data.") for a few tests on Julia 1 and Julia 1.10. I'm a bit confused about this, I don't think my PR is influencing anything about how DataDeps works. @pat-alt any idea why I'm getting this?

I was going to say that it may have to do with artifacts and serialization, but it seems to be passing now, so I think this can be merged (not sure why coverage has gone down a bit here)

I was going to say that it may have to do with artifacts and serialization, but it seems to be passing now, so I think this can be merged (not sure why coverage has gone down a bit here)

Seems like it was just a faulty run then, the change that I committed after my last comment was completely trivial. I think the reason coverage went down is just that I uncommented the final two lines of laplace_redux.jl when moving the code to the extension folder. I think those lines should remain uncommented, so once you approve, this is ready for merge.


[Tokenizer] update tokenizer versions

Description
Bump up huggingface tokenizer and also sentencepiece tokenizer

Codecov Report
Base: 72.08% // Head: 71.31% // Decreases project coverage by -0.77% :warning:

Coverage data is based on head (9344549) compared to base (bb5073f).
Patch coverage: 71.54% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff              @@
##             master    #2127      +/-   ##
============================================
- Coverage     72.08%   71.31%   -0.78%
- Complexity     5126     6285    +1159
============================================
  Files           473      624     +151
  Lines         21970    27847    +5877
  Branches       2351     3004     +653
============================================
+ Hits          15838    19858    +4020
- Misses         4925     6529    +1604
- Partials       1207     1460     +253

Impacted Files
Coverage Δ

api/src/main/java/ai/djl/modality/cv/Image.java
69.23% <ø> (-4.11%)
:arrow_down:

...rc/main/java/ai/djl/modality/cv/MultiBoxPrior.java
76.00% <ø> (ø)

...rc/main/java/ai/djl/modality/cv/output/Joints.java
71.42% <ø> (ø)

.../main/java/ai/djl/modality/cv/output/Landmark.java
100.00% <ø> (ø)

...main/java/ai/djl/modality/cv/output/Rectangle.java
72.41% <0.00%> (ø)

...i/djl/modality/cv/translator/BigGANTranslator.java
21.42% <0.00%> (-5.24%)
:arrow_down:

.../modality/cv/translator/ImageFeatureExtractor.java
0.00% <0.00%> (ø)

.../ai/djl/modality/cv/translator/YoloTranslator.java
27.77% <0.00%> (+18.95%)
:arrow_up:

...modality/cv/translator/wrapper/FileTranslator.java
44.44% <ø> (ø)

...y/cv/translator/wrapper/InputStreamTranslator.java
44.44% <ø> (ø)

... and 558 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Parse AAAA records as well

https://github.com/diasdavid/node-ipfs-mdns/blob/master/src/index.js#L100

Blocking on a bug in the dependency, mafintosh/multicast-dns#21.

Oops, didn't realize you had forked your own copy. Should this be done on your branch or upstream?

I believe I made a fork because there were some bugs in the original and while it wasn't merged, I used my fork, but now should be ok to use the one published in npm by Mathias :)

AAAA records are now supported.


Configuration adjustments

Adjusts database and laminas developer tool configs. Adds missing composer packages. Deletes composer.lock and adds it to gitignore.

Thanks @CreativeNative


[Slider] Feat: add a scale prop to Slider

Closes #16574
This pr adds a scale prop to Slider.

[x] I have followed (at least) the PR section of the contributing guide.

Sweet, do you think that you could add a new demo, at the end that demonstrates how to use it? basically: https://github.com/mui-org/material-ui/issues/16574#issuecomment-510625169? And maybe an explanation of how it works :)


A couple of fixes to get_willow_image

Ready for review

Just investigating some ResourceWarnings that are now popping up...

ResourceWarning fixed. Now ready for review!

Hi @timheap, do you have time to review this?
Thanks!

:+1: Looks good to me

Thanks @timheap!


exit status when the amount of input data is too small

The chloroExtractor terminates with exit status 1 if the input dataset is too small which terminates the whole nextflow pipeline if the errors are not set to be ignored.

Interesting. So your nextflow pipeline would be able to deal with the situation that chloroExtractor failed due to lack of input data? We could consider terminating because of this reason as a valid outcome with exit status 0. So far I considered it a kind of failure but I see the point in changing it. @greatfireball do you have an opinion on this subject?

We only need to change one character here :smiley:
https://github.com/chloroExtractorTeam/chloroExtractor/blob/ce5ffbdd202be9a21c28df576bae1bd8c41f7d88/bin/scale_reads.pl#L383

I already told @MarvinMcFly :)
Nevertheless, due to it changes the behavior, we should both agree to that change. My suggestion would be to add another option (die-without-error-state), which will exit with a code of 0 instead of 1. As default, we are able to still exit with 1

Fine for me


5.0: Make bcrypt optional

Since the hash provider can be configured, we should make crypt an optional dependency.

We should consider defaulting to Argon2 also in the future.


Improve custom generator class discovery in load_generators method

Checklist:

[ ] Added tests for changed code where applicable.
[ ] Documentation reflects the changes where applicable.
[x] Updated the CHANGELOG.md file with your changes.
[x] My PR is ready to review.

Resolves: #137
Description of the changes being introduced by the pull request:

Codecov Report

Merging #138 (00e81af) into master (4c93d8e) will not change coverage.
The diff coverage is 100.00%.

@@            Coverage Diff            @@
##            master      #138   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           40        40
  Lines          801       810    +9
=========================================
+ Hits           801       810    +9

Impacted Files
Coverage Δ

fastapi_mvc/__init__.py
100.00% <100.00%> (ø)

fastapi_mvc/borg.py
100.00% <100.00%> (ø)

fastapi_mvc/commands/__init__.py
100.00% <100.00%> (ø)

fastapi_mvc/generators/__init__.py
100.00% <100.00%> (ø)

fastapi_mvc/generators/controller/__init__.py
100.00% <100.00%> (ø)

fastapi_mvc/generators/generator/__init__.py
100.00% <100.00%> (ø)

fastapi_mvc/generators/project/__init__.py
100.00% <100.00%> (ø)

...okiecutter.package_name}}/app/controllers/ready.py
100.00% <100.00%> (ø)

...}/{{cookiecutter.package_name}}/config/__init__.py
100.00% <100.00%> (ø)

fastapi_mvc/parsers/__init__.py
100.00% <100.00%> (ø)

... and 2 more

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more