Fix languge unique chars

This PR fixed 1 bug and introduces(unveils?) one bug
Bug # 1:
In [7]: search_dates("july 2018 - december 2020", add_detected_language=True)
Out[7]:
[('july 2018', datetime.datetime(2018, 7, 13, 0, 0), 'en'),
 ('december 2020', datetime.datetime(2020, 12, 13, 0, 0), 'en')]

In [8]: search_dates("july 2018 - december 2020", add_detected_language=True, languages=['en','es'])
Out[8]:
[('july 2018', datetime.datetime(2018, 7, 13, 0, 0), 'en'),
 ('december 2020', datetime.datetime(2020, 12, 13, 0, 0), 'en')]

In [9]: search_dates("july 2018 - december 2020", add_detected_language=True, languages=['en','fr'])
Out[9]:
[('2018', datetime.datetime(2018, 5, 13, 0, 0), 'fr'),
 ('2020', datetime.datetime(2020, 5, 13, 0, 0), 'fr')]

As you can see, including some languages, like French (in addition to English) causes issues with finding dates. This is caused by the character_check / get_unique_characters methods, some languages had punctuations in their "unique characters", for example "-" for French or '.' for Thai.
Due to this reason the parser thinks that the text July 2018 - December 2020 is in French and ignores the months.
This is fixed by making sure that punctuations don't get into the unique chars set.
Bug # 2
In [11]: search_dates("01.09 – 03.09.2017", add_detected_language=True)
Out[11]:
[('01.09', datetime.datetime(2020, 1, 9, 0, 0), 'th'),
 ('03.09.2017', datetime.datetime(2017, 3, 9, 0, 0), 'th')]

After I fixed this issue I found another issue that was "hidden" before, the following test: https://github.com/scrapinghub/dateparser/blob/master/tests/test_search.py#L670
Only ever worked because of the previous bug, 01.09 – 03.09.2017 contains dots, so the parser thought it was in Thai, and used the specific Thai word splitter https://github.com/scrapinghub/dateparser/blob/master/dateparser/languages/locale.py#L256
Now that the parser correctly recognizes that the text is not in Thai (so English), it uses the English splitter that splits on dots, which screws up the date.
I tried fixing it as well but I wasn't successful, hopefully you can get it working with your experience with the codebase.
Obviously, due to the second bug, 1 test is failing

I wonder how hard it would be to change the French and Thai language detection to only trigger if there are no spaces surrounding their respective characters.

I wonder how hard it would be to change the French and Thai language detection to only trigger if there are no spaces surrounding their respective characters.

What would that achieve?

I imagine the goal of including those characters in language detection was to detect dates in the formats common for those locales. That’s what we loose with this patch. Maybe we can figure out a way to keep the old indented behavior while still fixing the issue.


Support blocks that return extended types as output.

All tests run and passed.
Resolves issue 3981.

 Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.:x: matthewtebbs sign nowYou have signed the CLA already but the status is still pending? Let us recheck it.

Looks like it failed a lint check.
linting...
pxtblocks/blocklyloader.ts[784, 21]: misplaced opening brace

Sorry, I didn't realize that lint was a separate script. I can fix.


No Comms on 2016 protocol w/ 2017 robot

With all settings configured, I am getting no Comms on a new router and roboRio.

Hi, I completely missed your issue report. Sorry for that. Can you try to install the latest version (from the Play Store) and check if your issue is still present? Also, try to input your robot's static IP if you still don't get comms, mDNS is not supported on all Android devices AFAIK...

I was able to get our 2019 bot to connect for a second, very intermittently, but it only moved when the joystick was all the way forward, no in-between


New Component: SIP / WebRTC client

For example using: https://sipjs.com/

FYI I looks like that we've received an okay to add a SIP interface to one of our projects, so I think I can start work on this in the coming weeks.

FYI I looks like that we've received an okay to add a SIP interface to one of our projects, so I think I can start work on this in the coming weeks.

OK I assigned it to you.

OK I assigned it to you.

Seems there is a delay / shift in priority at our client.. but they want this at some point

@jepsar do you have an update here :) ?

Wondering if this is still the ideal future chat component after these years since the website demo seems broken?
Go to: https://sipjs.com/
Type "Hello Bob!" on Alice view, and click send, nothing seems happened on web UI, and on console, I can see SIP/2.0 404 Not Found
Before sent:

After sent:

Probably not unless its just their infrastructure is down.  Although last commit and release was 2 years ago.

Yeah, it appears the last release from SIP.js https://github.com/onsip/SIP.js/releases/tag/0.21.2 was two years ago, with no updates since then. I am curious about the long-term for this component, epecially since the related request in PF Core https://github.com/primefaces/primefaces/issues/8516 was redirected to this ticket.
Do we want to build a message component like vaadin? https://vaadin.com/docs/latest/components/message-input

this one is more modern and updated 6 months ago: https://jssip.net/
I think we just need to find the right library, with the RIGHT open source license ,that is still active.

Let's see what's the thought from @jepsar as well since he's the intial author of ticket.

I'm not bound to a certain implementation. We recently started using Vonage. So far I haven't had the need to create a component for it.


Adding angular variation of clumping index

Two extra variables were added (clump_a and clump_b) in or to allow clumping index to vary with Sun zenith angle following Pinty et al. (2006).

bors r+


Aggregate mobiledata transactions by day for featurization, read data format from yml, fix parquet data reading, initial data-checker

Looks good! -- Emily


[Workload]. Reduce the amount of clutter in your image

Title: [Workload] microservices test: uncluttered_image
Is your workload test idea related to a problem? Please describe.

Some images have inappropriate data and binaries installed on them

Describe the solution you'd like

inappropriate binaries to search for: e.g. netstat, nmap
Size reduction: To reduce the size of your image, install only what is strictly needed inside it. It might be tempting to install extra packages, and then remove them at a later step. However, this approach isn't sufficient. Because each instruction of the Dockerfile creates a layer, removing data from the image in a later step than the step that created it doesn't reduce the size of the overall image (the data is still there, just hidden in a deeper layer).  https://cloud.google.com/architecture/best-practices-for-building-containers#reduce_the_amount_of_clutter_in_your_image

Test Category Name

Microservices

Documentation tasks:

[ ] Update installation instructions if needed
[ ] Update Test Categories md if needed
[ ] Update USAGE md if needed

[ ] How to run
[ ] Description and details

[ ] What the best practice is
[ ] Why are we testing this

[ ] Remediation steps if test does not pass

QA tasks
Dev Review:

[ ] walk through A/C
[ ] do you get the expected result?
[ ] if yes,

[ ] move to Needs Peer Review column
[ ] create Pull Request and follow check list
[ ] Assign 1 or more people for peer review

[ ] if no, document what additional tasks will be needed

Peer review:

[ ] walk through A/C
[ ] do you get the expected result?
[ ] if yes,

[ ] move to Reviewer Approved column
[ ] Approve pull request

[ ] if no,

[ ] document what did not go as expected, including error messages and screenshots (if possible)
[ ] Add comment to pull request
[ ] request changes to pull request

@taylor is this issue still relevant for CNTI?


Getting Deadlocks when deploying multiple sites report to 1 report server

Hi,
We have written a PS script utilising ReportingServiceTool (latest) to deploy our reports via Azure Devops.
The script does the following

Deletes the current report folder with all the reports in it
Recreates the folder
Assigns the permissions
Creates the Data Sources
Deploys the reports

Our scenario is that we have 5+ report folders (1 for each site we deploy) per report server.
We deploy around 800+ reports per site.
We deploy the sites simultaniously.
Our reports servers are SQL 2019 and SSRS 2019.
We run the scripts with Powershell 5.1 on the Windows 2016 servers.
However, when we do this we get errors on different steps each time but all resulting in the same final error
"System.Data.SqlClient.SqlException: Transaction (Process ID 58) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction."
We have increased the time out as per recommendation by Microsoft support, recreated the database, even tested it on SQL 2016 / SSRS 2016 with the same outcome.
https://docs.microsoft.com/en-us/answers/questions/276827/ssrs-2019-amp-powershell-reporting-tools-getting-d.html
[error]Exception calling "ListChildren" with "2" argument(s):
"System.Web.Services.Protocols.SoapException: An error occurred within the report server database. This may be due to a connection failure, timeout or low disk condition within the database. ---> Microsoft.ReportingServices.Diagnostics.Utilities.ReportServerStorageException: An error occurred within the report server database
[error]Exception occurred while deleting catalog item '/Report1_SSRS'! Exception calling "DeleteItem" with "1" argument(s):
"System.Web.Services.Protocols.SoapException: An error occurred within the report server database.
[error]Exception occurred while deleting catalog item '/Reports_1_SSRS'! Exception calling "DeleteItem" with "1" argument(s): "System.Web.Services.Protocols.SoapException: An error occurred within the report server database.  This may be due to a connection failure, timeout or low disk condition within the database. ---> Microsoft.ReportingServices.Diagnostics.Utilities.ReportServerStorageException: An error occurred within the report server database.  This may be due to a connection failure, timeout or low disk condition within the database. ---> System.Data.SqlClient.SqlException: Transaction (Process ID 68) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
[error]Failed to create item {ReportName}.rdl : Failed to create catalog item {UploadPathtoReport}.rdl : Exception calling "CreateCatalogItem" with "7" argument(s): "System.Web.Services.Protocols.SoapException: An error occurred within the report server database. This may be due to a connection failure, timeout or low disk condition within the database. ---> Microsoft.ReportingServices.Diagnostics.Utilities.ReportServerStorageException: An error occurred within the report server database. This may be due to a connection failure, timeout or low disk condition within the database. ---> System.Data.SqlClient.SqlException: Transaction (Process ID 282) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

MORE DETAILS TO SOME ERRORS -
Exception calling "ListChildren" with "2" argument(s):
"System.Web.Services.Protocols.SoapException: An error occurred within the report server database. This may be due to a connection failure, timeout or low disk condition within the database. ---> Microsoft.ReportingServices.Diagnostics.Utilities.ReportServerStorageException: An error occurred within the
report server database.  This may be due to a connection failure, timeout or low disk condition within the database. ---> System.Data.SqlClient.SqlException: Transaction (Process ID 58) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.

at Microsoft.ReportingServices.Library.ReportingService2005Impl.ListChildren
(String Item, Boolean Recursive, CatalogItemList& CatalogItems)
at Microsoft.ReportingServices.WebServer.ReportingService2010.ListChildren(String ItemPath, Boolean Recursive, CatalogItem[]& CatalogItems)"
At C:\Users{hidden for post}\Documents\WindowsPowerShell\Modules\ReportingServicesTools\ReportingServicesTools\Functions\CatalogItems\Get-RsFolderContent.ps1:85
char:17

            $Proxy.ListChildren($Item, $Recurse)

            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : NotSpecified: (:) [], ParentContainsErrorRecordE
xception
FullyQualifiedErrorId : SoapException

Exception occurred while deleting catalog item '/Report1_SSRS'! Exception calling "DeleteItem" with "1" argument(s):
"System.Web.Services.Protocols.SoapException: An error occurred within the report server database.  This may be due to a connection failure, timeout or
low disk condition within the database. ---> Microsoft.ReportingServices.Diagnostics.Utilities.ReportServerStorageException: An error occurred within the
report server database.  This may be due to a connection failure, timeout or low disk condition within the database. --->
System.Data.SqlClient.SqlException: Transaction (Process ID 61) was deadlocked on lock resources with another process and has been chosen as the deadlock
victim. Rerun the transaction.
at Microsoft.ReportingServices.Library.ReportingService2005Impl.DeleteItem(String Item, Guid batchId)
at Microsoft.ReportingServices.Library.ReportingService2005Impl.DeleteItem(String Item)
at Microsoft.ReportingServices.WebServer.ReportingService2010.DeleteItem(String ItemPath)"
At C:\Users{hidden for post}\Documents\WindowsPowerShell\Modules\ReportingServicesTools\ReportingServicesTools\Functions\CatalogItems\Remove-RsCatalogItem.ps1:90
char:21

...             throw (New-Object System.Exception("Exception occurred wh ...

            ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

CategoryInfo          : OperationStopped: (:) [], Exception
FullyQualifiedErrorId : Exception occurred while deleting catalog item '/Report1_SSRS'! Exception calling "DeleteItem" with "1" argument(s):
"System.Web.Services.Protocols.SoapException: An error occurred within the report server database.  This may be due to a connection failure, timeout or low disk condition within the database. ---> Microsoft.ReportingServices.Diagnostics.Utilities.ReportServerStorageException: An error occurred within the report server database.  This may be due to a connection failure, timeout or low disk condition within the database. ---> System.Data.SqlClient.SqlException: Transaction (Process ID 61) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
at Microsoft.ReportingServices.Library.ReportingService2005Impl.DeleteItem(String Item, Guid batchId)
at Microsoft.ReportingServices.Library.ReportingService2005Impl.DeleteItem(String Item)
at Microsoft.ReportingServices.WebServer.ReportingService2010.DeleteItem(String ItemPath)"

Hoping you can give us some advise or guidance.
Thank you.

Just as added info herewith the 1222 trace we ran while deploying the reports.
02/19/2021 12:20:46,spid23s,Unknown,waiter id=process195e1037468 mode=S requestType=wait
02/19/2021 12:20:46,spid23s,Unknown,waiter-list
02/19/2021 12:20:46,spid23s,Unknown,owner id=process195e106f468 mode=X
02/19/2021 12:20:46,spid23s,Unknown,owner-list
02/19/2021 12:20:46,spid23s,Unknown,keylock hobtid=72057594052280320 dbid=5 objectname=Reports.dbo.Catalog indexname=IX_Policy id=lock195c438e080 mode=X associatedObjectId=72057594052280320
02/19/2021 12:20:46,spid23s,Unknown,waiter id=process195e106f468 mode=U requestType=wait
02/19/2021 12:20:46,spid23s,Unknown,waiter-list
02/19/2021 12:20:46,spid23s,Unknown,owner id=process195e1037468 mode=X
02/19/2021 12:20:46,spid23s,Unknown,owner-list
02/19/2021 12:20:46,spid23s,Unknown,keylock hobtid=72057594047299584 dbid=5 objectname=Reports.dbo.SnapshotData indexname=PK_SnapshotData id=lock195d11c6280 mode=X associatedObjectId=72057594047299584
02/19/2021 12:20:46,spid23s,Unknown,resource-list
02/19/2021 12:20:46,spid23s,Unknown,Proc [Database Id = 5 Object Id =<PHONE_NUMBER>]
02/19/2021 12:20:46,spid23s,Unknown,inputbuf
02/19/2021 12:20:46,spid23s,Unknown,EXEC CleanOrphanedPolicie
02/19/2021 12:20:46,spid23s,Unknown,frame procname=Reports.dbo.DeleteObject line=194 stmtstart=10482 stmtend=10532 sqlhandle=0x03000500960d3b7369c60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:46,spid23s,Unknown,NOT EXISTS (SELECT ItemID FROM [Catalog] WHERE [Catalog].[PolicyID] = [Policies].[PolicyID]
02/19/2021 12:20:46,spid23s,Unknown,AND
02/19/2021 12:20:46,spid23s,Unknown,[Policies].[PolicyFlag] = 0
02/19/2021 12:20:46,spid23s,Unknown,WHERE
02/19/2021 12:20:46,spid23s,Unknown,[Policies]
02/19/2021 12:20:46,spid23s,Unknown,DELETE
02/19/2021 12:20:46,spid23s,Unknown,frame procname=Reports.dbo.CleanOrphanedPolicies line=5 stmtstart=196 stmtend=524 sqlhandle=0x030005001f6768460dc60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:46,spid23s,Unknown,executionStack
02/19/2021 12:20:46,spid23s,Unknown,process id=process195e1037468 taskpriority=0 logused=23524 waitresource=KEY: 5:72057594052280320 (2b682d85aa08) waittime=2373 ownerId=6759604 transactionname=user_transaction lasttranstarted=2021-02-19T12:20:44.180 XDES=0x195e017c490 lockMode=S schedulerid=4 kpid=74580 status=suspended spid=51 sbid=0 ecid=0 priority=0 trancount=2 lastbatchstarted=2021-02-19T12:20:44.190 lastbatchcompleted=2021-02-19T12:20:44.190 lastattention=1900-01-01T00:00:00.190 clientapp=Report Server hostname={Hidden} hostpid=74372 loginname=ReportUser isolationlevel=read committed (2) xactid=6759604 currentdb=5 lockTimeout=4294967295 clientoption1=671088672 clientoption2=128056
02/19/2021 12:20:46,spid23s,Unknown,Proc [Database Id = 5 Object Id =<PHONE_NUMBER>]
02/19/2021 12:20:46,spid23s,Unknown,inputbuf
02/19/2021 12:20:46,spid23s,Unknown,(R.Path = @Path OR R.Path LIKE @Prefix ESCAPE '*'
02/19/2021 12:20:46,spid23s,Unknown,WHERE
02/19/2021 12:20:46,spid23s,Unknown,INNER JOIN [SnapshotData] AS SD ON R.Intermediate = SD.SnapshotDataID
02/19/2021 12:20:46,spid23s,Unknown,Catalog AS R WITH (XLOCK)
02/19/2021 12:20:46,spid23s,Unknown,FROM
02/19/2021 12:20:46,spid23s,Unknown,TransientRefcount = TransientRefcount + case when R.Type = 8 then 1 ELSE 0 END
02/19/2021 12:20:46,spid23s,Unknown,ExpirationDate = case when R.Type = 8 then DATEADD(d<c/> 14<c/> GETDATE()) ELSE ExpirationDate END<c/>
02/19/2021 12:20:46,spid23s,Unknown,-- to fix VSTS 384486 keep shared dataset compiled definition for 14 days
02/19/2021 12:20:46,spid23s,Unknown,SET PermanentRefcount = PermanentRefcount - 1<c/>
02/19/2021 12:20:46,spid23s,Unknown,UPDATE SnapshotData
02/19/2021 12:20:46,spid23s,Unknown,frame procname=Reports.dbo.DeleteObject line=54 stmtstart=2638 stmtend=3638 sqlhandle=0x03000500960d3b7369c60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:46,spid23s,Unknown,executionStack
02/19/2021 12:20:46,spid23s,Unknown,process id=process195e106f468 taskpriority=0 logused=0 waitresource=KEY: 5:72057594047299584 (7166b320963a) waittime=2382 ownerId=6759842 transactionname=user_transaction lasttranstarted=2021-02-19T12:20:44.210 XDES=0x195d5adc490 lockMode=U schedulerid=11 kpid=55224 status=suspended spid=55 sbid=0 ecid=0 priority=0 trancount=2 lastbatchstarted=2021-02-19T12:20:44.213 lastbatchcompleted=2021-02-19T12:20:44.213 lastattention=1900-01-01T00:00:00.213 clientapp=Report Server hostname={Hidden} hostpid=74372 loginname=ReportUser isolationlevel=read committed (2) xactid=6759842 currentdb=5 lockTimeout=4294967295 clientoption1=671088672 clientoption2=128056
02/19/2021 12:20:46,spid23s,Unknown,process-list
02/19/2021 12:20:46,spid23s,Unknown,deadlock victim=process195e106f468
02/19/2021 12:20:46,spid23s,Unknown,deadlock-list
02/19/2021 12:20:44,spid40s,Unknown,waiter id=process195e1037468 mode=S requestType=wait
02/19/2021 12:20:44,spid40s,Unknown,waiter-list
02/19/2021 12:20:44,spid40s,Unknown,owner id=process195e105eca8 mode=X
02/19/2021 12:20:44,spid40s,Unknown,owner-list
02/19/2021 12:20:44,spid40s,Unknown,keylock hobtid=72057594052280320 dbid=5 objectname=Reports.dbo.Catalog indexname=IX_Policy id=lock195ab386a80 mode=X associatedObjectId=72057594052280320
02/19/2021 12:20:44,spid40s,Unknown,waiter id=process195e105eca8 mode=U requestType=wait
02/19/2021 12:20:44,spid40s,Unknown,waiter-list
02/19/2021 12:20:44,spid40s,Unknown,owner id=process195e1037468 mode=X
02/19/2021 12:20:44,spid40s,Unknown,owner-list
02/19/2021 12:20:44,spid40s,Unknown,keylock hobtid=72057594047299584 dbid=5 objectname=Reports.dbo.SnapshotData indexname=PK_SnapshotData id=lock195b5e2ee00 mode=X associatedObjectId=72057594047299584
02/19/2021 12:20:44,spid40s,Unknown,resource-list
02/19/2021 12:20:44,spid40s,Unknown,Proc [Database Id = 5 Object Id =<PHONE_NUMBER>]
02/19/2021 12:20:44,spid40s,Unknown,inputbuf
02/19/2021 12:20:44,spid40s,Unknown,EXEC CleanOrphanedPolicie
02/19/2021 12:20:44,spid40s,Unknown,frame procname=Reports.dbo.DeleteObject line=194 stmtstart=10482 stmtend=10532 sqlhandle=0x03000500960d3b7369c60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:44,spid40s,Unknown,NOT EXISTS (SELECT ItemID FROM [Catalog] WHERE [Catalog].[PolicyID] = [Policies].[PolicyID]
02/19/2021 12:20:44,spid40s,Unknown,AND
02/19/2021 12:20:44,spid40s,Unknown,[Policies].[PolicyFlag] = 0
02/19/2021 12:20:44,spid40s,Unknown,WHERE
02/19/2021 12:20:44,spid40s,Unknown,[Policies]
02/19/2021 12:20:44,spid40s,Unknown,DELETE
02/19/2021 12:20:44,spid40s,Unknown,frame procname=Reports.dbo.CleanOrphanedPolicies line=5 stmtstart=196 stmtend=524 sqlhandle=0x030005001f6768460dc60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:44,spid40s,Unknown,executionStack
02/19/2021 12:20:44,spid40s,Unknown,process id=process195e1037468 taskpriority=0 logused=28364 waitresource=KEY: 5:72057594052280320 (ed76754105b8) waittime=4998 ownerId=6758288 transactionname=user_transaction lasttranstarted=2021-02-19T12:20:36.750 XDES=0x195e017c490 lockMode=S schedulerid=4 kpid=74580 status=suspended spid=51 sbid=0 ecid=0 priority=0 trancount=2 lastbatchstarted=2021-02-19T12:20:36.757 lastbatchcompleted=2021-02-19T12:20:36.753 lastattention=1900-01-01T00:00:00.753 clientapp=Report Server hostname={Hidden} hostpid=74372 loginname=ReportUser isolationlevel=read committed (2) xactid=6758288 currentdb=5 lockTimeout=4294967295 clientoption1=671088672 clientoption2=128056
02/19/2021 12:20:44,spid40s,Unknown,Proc [Database Id = 5 Object Id =<PHONE_NUMBER>]
02/19/2021 12:20:44,spid40s,Unknown,inputbuf
02/19/2021 12:20:44,spid40s,Unknown,(R.Path = @Path OR R.Path LIKE @Prefix ESCAPE '*'
02/19/2021 12:20:44,spid40s,Unknown,WHERE
02/19/2021 12:20:44,spid40s,Unknown,INNER JOIN [SnapshotData] AS SD ON R.Intermediate = SD.SnapshotDataID
02/19/2021 12:20:44,spid40s,Unknown,Catalog AS R WITH (XLOCK)
02/19/2021 12:20:44,spid40s,Unknown,FROM
02/19/2021 12:20:44,spid40s,Unknown,TransientRefcount = TransientRefcount + case when R.Type = 8 then 1 ELSE 0 END
02/19/2021 12:20:44,spid40s,Unknown,ExpirationDate = case when R.Type = 8 then DATEADD(d<c/> 14<c/> GETDATE()) ELSE ExpirationDate END<c/>
02/19/2021 12:20:44,spid40s,Unknown,-- to fix VSTS 384486 keep shared dataset compiled definition for 14 days
02/19/2021 12:20:44,spid40s,Unknown,SET PermanentRefcount = PermanentRefcount - 1<c/>
02/19/2021 12:20:44,spid40s,Unknown,UPDATE SnapshotData
02/19/2021 12:20:44,spid40s,Unknown,frame procname=Reports.dbo.DeleteObject line=54 stmtstart=2638 stmtend=3638 sqlhandle=0x03000500960d3b7369c60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:44,spid40s,Unknown,executionStack
02/19/2021 12:20:44,spid40s,Unknown,process id=process195e105eca8 taskpriority=0 logused=0 waitresource=KEY: 5:72057594047299584 (a7101ea26f8a) waittime=4998 ownerId=6758993 transactionname=user_transaction lasttranstarted=2021-02-19T12:20:38.020 XDES=0x195cacac490 lockMode=U schedulerid=9 kpid=76660 status=suspended spid=60 sbid=0 ecid=0 priority=0 trancount=2 lastbatchstarted=2021-02-19T12:20:38.020 lastbatchcompleted=2021-02-19T12:20:38.020 lastattention=1900-01-01T00:00:00.020 clientapp=Report Server hostname={Hidden} hostpid=74372 loginname=ReportUser isolationlevel=read committed (2) xactid=6758993 currentdb=5 lockTimeout=4294967295 clientoption1=671088672 clientoption2=128056
02/19/2021 12:20:44,spid40s,Unknown,process-list
02/19/2021 12:20:44,spid40s,Unknown,deadlock victim=process195e105eca8
02/19/2021 12:20:44,spid40s,Unknown,deadlock-list
02/19/2021 12:20:39,spid26s,Unknown,waiter id=process195e1037468 mode=S requestType=wait
02/19/2021 12:20:39,spid26s,Unknown,waiter-list
02/19/2021 12:20:39,spid26s,Unknown,owner id=process195e106f468 mode=X
02/19/2021 12:20:39,spid26s,Unknown,owner-list
02/19/2021 12:20:39,spid26s,Unknown,keylock hobtid=72057594052280320 dbid=5 objectname=Reports.dbo.Catalog indexname=IX_Policy id=lock195aabfe180 mode=X associatedObjectId=72057594052280320
02/19/2021 12:20:39,spid26s,Unknown,waiter id=process195e106f468 mode=U requestType=wait
02/19/2021 12:20:39,spid26s,Unknown,waiter-list
02/19/2021 12:20:39,spid26s,Unknown,owner id=process195e1037468 mode=X
02/19/2021 12:20:39,spid26s,Unknown,owner-list
02/19/2021 12:20:39,spid26s,Unknown,keylock hobtid=72057594047299584 dbid=5 objectname=Reports.dbo.SnapshotData indexname=PK_SnapshotData id=lock195b5e2ee00 mode=X associatedObjectId=72057594047299584
02/19/2021 12:20:39,spid26s,Unknown,resource-list
02/19/2021 12:20:39,spid26s,Unknown,Proc [Database Id = 5 Object Id =<PHONE_NUMBER>]
02/19/2021 12:20:39,spid26s,Unknown,inputbuf
02/19/2021 12:20:39,spid26s,Unknown,EXEC CleanOrphanedPolicie
02/19/2021 12:20:39,spid26s,Unknown,frame procname=Reports.dbo.DeleteObject line=194 stmtstart=10482 stmtend=10532 sqlhandle=0x03000500960d3b7369c60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:39,spid26s,Unknown,NOT EXISTS (SELECT ItemID FROM [Catalog] WHERE [Catalog].[PolicyID] = [Policies].[PolicyID]
02/19/2021 12:20:39,spid26s,Unknown,AND
02/19/2021 12:20:39,spid26s,Unknown,[Policies].[PolicyFlag] = 0
02/19/2021 12:20:39,spid26s,Unknown,WHERE
02/19/2021 12:20:39,spid26s,Unknown,[Policies]
02/19/2021 12:20:39,spid26s,Unknown,DELETE
02/19/2021 12:20:39,spid26s,Unknown,frame procname=Reports.dbo.CleanOrphanedPolicies line=5 stmtstart=196 stmtend=524 sqlhandle=0x030005001f6768460dc60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:39,spid26s,Unknown,executionStack
02/19/2021 12:20:39,spid26s,Unknown,process id=process195e1037468 taskpriority=0 logused=28364 waitresource=KEY: 5:72057594052280320 (d8f7d57a33e8) waittime=2187 ownerId=6758288 transactionname=user_transaction lasttranstarted=2021-02-19T12:20:36.750 XDES=0x195e017c490 lockMode=S schedulerid=4 kpid=74580 status=suspended spid=51 sbid=0 ecid=0 priority=0 trancount=2 lastbatchstarted=2021-02-19T12:20:36.757 lastbatchcompleted=2021-02-19T12:20:36.753 lastattention=1900-01-01T00:00:00.753 clientapp=Report Server hostname={Hidden} hostpid=74372 loginname=ReportUser isolationlevel=read committed (2) xactid=6758288 currentdb=5 lockTimeout=4294967295 clientoption1=671088672 clientoption2=128056
02/19/2021 12:20:39,spid26s,Unknown,Proc [Database Id = 5 Object Id =<PHONE_NUMBER>]
02/19/2021 12:20:39,spid26s,Unknown,inputbuf
02/19/2021 12:20:39,spid26s,Unknown,(R.Path = @Path OR R.Path LIKE @Prefix ESCAPE '*'
02/19/2021 12:20:39,spid26s,Unknown,WHERE
02/19/2021 12:20:39,spid26s,Unknown,INNER JOIN [SnapshotData] AS SD ON R.Intermediate = SD.SnapshotDataID
02/19/2021 12:20:39,spid26s,Unknown,Catalog AS R WITH (XLOCK)
02/19/2021 12:20:39,spid26s,Unknown,FROM
02/19/2021 12:20:39,spid26s,Unknown,TransientRefcount = TransientRefcount + case when R.Type = 8 then 1 ELSE 0 END
02/19/2021 12:20:39,spid26s,Unknown,ExpirationDate = case when R.Type = 8 then DATEADD(d<c/> 14<c/> GETDATE()) ELSE ExpirationDate END<c/>
02/19/2021 12:20:39,spid26s,Unknown,-- to fix VSTS 384486 keep shared dataset compiled definition for 14 days
02/19/2021 12:20:39,spid26s,Unknown,SET PermanentRefcount = PermanentRefcount - 1<c/>
02/19/2021 12:20:39,spid26s,Unknown,UPDATE SnapshotData
02/19/2021 12:20:39,spid26s,Unknown,frame procname=Reports.dbo.DeleteObject line=54 stmtstart=2638 stmtend=3638 sqlhandle=0x03000500960d3b7369c60401d1ac000001000000000000000000000000000000000000000000000000000000
02/19/2021 12:20:39,spid26s,Unknown,executionStack
02/19/2021 12:20:39,spid26s,Unknown,process id=process195e106f468 taskpriority=0 logused=0 waitresource=KEY: 5:72057594047299584 (a7101ea26f8a) waittime=2253 ownerId=6758401 transactionname=user_transaction lasttranstarted=2021-02-19T12:20:36.783 XDES=0x195d5adc490 lockMode=U schedulerid=11 kpid=55224 status=suspended spid=55 sbid=0 ecid=0 priority=0 trancount=2 lastbatchstarted=2021-02-19T12:20:36.783 lastbatchcompleted=2021-02-19T12:20:36.783 lastattention=1900-01-01T00:00:00.783 clientapp=Report Server hostname={Hidden} hostpid=74372 loginname=ReportUser isolationlevel=read committed (2) xactid=6758401 currentdb=5 lockTimeout=4294967295 clientoption1=671088672 clientoption2=128056
02/19/2021 12:20:39,spid26s,Unknown,process-list
02/19/2021 12:20:39,spid26s,Unknown,deadlock victim=process195e106f468
02/19/2021 12:20:39,spid26s,Unknown,deadlock-list
02/19/2021 12:17:27,spid58,Unknown,DBCC TRACEON 1222<c/> server process ID (SPID) 58. This is an informational message only; no user action is required.

And herewith the SQL Profiler Output
xml_report
<deadlock>
    <victim-list> <victimProcess id="process195e105f468" /> </victim-list>
    <process-list>
        <process
            id="process195e105f468"
            taskpriority="0"
            logused="508"
            waitresource="KEY: 5:72057594045136896 (0a1f1722cb01)"
            waittime="691"
            ownerId="14764658"
            transactionname="user_transaction"
            lasttranstarted="2021-02-22T08:42:24.593"
            XDES="0x195d5b28490"
            lockMode="U"
            schedulerid="9"
            kpid="36768"
            status="suspended"
            spid="68"
            sbid="0"
            ecid="0"
            priority="0"
            trancount="2"
            lastbatchstarted="2021-02-22T08:42:24.597"
            lastbatchcompleted="2021-02-22T08:42:24.597"
            lastattention="1900-01-01T00:00:00.597"
            clientapp="Report Server"
            hostname="{HOSTNAME HIDDEN}"
            hostpid="66240"
            loginname="{REPORTS USER HIDDEN}"
            isolationlevel="read committed (2)"
            xactid="14764658"
            currentdb="5"
            lockTimeout="4294967295"
            clientoption1="671088672"
            clientoption2="128056"
        >
            <executionStack>
                <frame procname="Reports.dbo.DeleteObject" line="92" stmtstart="5106" stmtend="5602" sqlhandle="0x03000500960d3b7369c60401d1ac000001000000000000000000000000000000000000000000000000000000">
                    UPDATE [DataSource] SET [Flags] = [Flags] & 0x7FFFFFFD, -- broken link [Link] = NULL FROM [Catalog] AS C INNER JOIN [DataSource] AS DS ON C.[ItemID] = DS.[Link] WHERE (C.Path = @Path OR C.Path LIKE @Prefix ESCAPE
                    '*'
                </frame>
            </executionStack>
            <inputbuf> Proc [Database Id = 5 Object Id =<PHONE_NUMBER>] </inputbuf>
        </process>
        <process
            id="process195e1067088"
            taskpriority="0"
            logused="32272"
            waitresource="KEY: 5:72057594052280320 (ec7e3c594ed4)"
            waittime="689"
            ownerId="14764660"
            transactionname="user_transaction"
            lasttranstarted="2021-02-22T08:42:24.593"
            XDES="0x195d9c00490"
            lockMode="S"
            schedulerid="10"
            kpid="38608"
            status="suspended"
            spid="69"
            sbid="0"
            ecid="0"
            priority="0"
            trancount="2"
            lastbatchstarted="2021-02-22T08:42:24.597"
            lastbatchcompleted="2021-02-22T08:42:24.597"
            lastattention="1900-01-01T00:00:00.597"
            clientapp="Report Server"
            hostname="{HOSTNAME HIDDEN}"
            hostpid="66240"
            loginname="{REPORTS USER HIDDEN}"
            isolationlevel="read committed (2)"
            xactid="14764660"
            currentdb="5"
            lockTimeout="4294967295"
            clientoption1="671088672"
            clientoption2="128056"
        >
            <executionStack>
                <frame procname="Reports.dbo.CleanOrphanedPolicies" line="5" stmtstart="196" stmtend="524" sqlhandle="0x030005001f6768460dc60401d1ac000001000000000000000000000000000000000000000000000000000000">
                    DELETE [Policies] WHERE [Policies].[PolicyFlag] = 0 AND NOT EXISTS (SELECT ItemID FROM [Catalog] WHERE [Catalog].[PolicyID] = [Policies].[PolicyID]
                </frame>
                <frame procname="Reports.dbo.DeleteObject" line="194" stmtstart="10482" stmtend="10532" sqlhandle="0x03000500960d3b7369c60401d1ac000001000000000000000000000000000000000000000000000000000000">
                    EXEC CleanOrphanedPolicie
                </frame>
            </executionStack>
            <inputbuf> Proc [Database Id = 5 Object Id =<PHONE_NUMBER>] </inputbuf>
        </process>
    </process-list>
    <resource-list>
        <keylock hobtid="72057594045136896" dbid="5" objectname="Reports.dbo.DataSource" indexname="PK_DataSource" id="lock195bdb35180" mode="X" associatedObjectId="72057594045136896">
            <owner-list> <owner id="process195e1067088" mode="X" /> </owner-list> <waiter-list> <waiter id="process195e105f468" mode="U" requestType="wait" /> </waiter-list>
        </keylock>
        <keylock hobtid="72057594052280320" dbid="5" objectname="Reports.dbo.Catalog" indexname="IX_Policy" id="lock195c444a200" mode="X" associatedObjectId="72057594052280320">
            <owner-list> <owner id="process195e105f468" mode="X" /> </owner-list> <waiter-list> <waiter id="process195e1067088" mode="S" requestType="wait" /> </waiter-list>
        </keylock>
    </resource-list>
</deadlock>

This project handles only the PowerShell functions,
For the product issues I would recommend to contact Microsoft support.


Rendering broken in Opera 11.60 on Windows 7 (64 Bit)

Most pages of the viewer's test document are rendered completely white. Only figures, e.g. on page 2 and 6, are renderered but with seemingly random glyphs instead of the correct annotations.
Does this happen on other Windows 7 systems, too? Or do I have a local flaw?

If webfonts are disabled via opera:config, the page is actually rendered. Of course, the fonts are not correct in this case.

I too experience this, Opera 11.6 on a Windows 7 machine.
I get an error 'Unable to use webfont' and a link to a font data URL. I've found no articles or documentation discussing this error?

Could you please provide the error message here? It might be helpful since I do not get this one.

That is the "error message", 'cause there isn't really any error messages, only an info message in log "Unable to use webfont"

From Opera bug-tracker land:

The font passes our validation, but FreeType returns a "broken table" error (presumably other platform implementations does too). Looks like the 'name' table is broken (offset size == 0).

blocked by opera issue DSK-358789

Also, name table does not contain windows (unicode) name. Looking into sanitizing those.


Fix unit tests

Unit tests were not getting ran in CI and so they haven't been getting updated for a while. This PR updates the tests for recent changes that they are now passing.
Proposed Changes

Update all packages to 0.27.8 so that they are in sync with k8s.io/kubernetes 1.27.8 regardless of CVE fixes
Add a second definition for k8s.io/cri-api so that v1alpha2 can still be used
Fix method definitions and imports for k8s.io/kubernetes updates
Update seccomp strings to use the seccomd structure from https://github.com/Mirantis/cri-dockerd/pull/223
Remove stra method calls from stats test after https://github.com/Mirantis/cri-dockerd/pull/233

@nwneisen - on Mac I see the following with and without the PR:
~/gopath/src/github.com/Mirantis/cri-dockerd/core>  go test
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] Unable to create pod sandbox due to conflict. Attempting to remove sandbox. Container containerid
ERRO[0000] Failed to remove the conflicting container (containerid): container not stopped
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] Unable to create pod sandbox due to conflict. Attempting to remove sandbox. Container containerid
ERRO[0000] Failed to remove the conflicting container (containerid): random error
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] Unable to create pod sandbox due to conflict. Attempting to remove sandbox. Container containerid
ERRO[0000] Failed to remove the conflicting container (containerid): Error response from daemon: No such container: containerid
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] updateCreateConfig is unsupported in this build
INFO[0000] getSecurityOpts is unsupported in this build
INFO[0000] Pulling the image without credentials. Image: gcr.io/test/image
INFO[0000] Pulling the image without credentials. Image: gcr.io/test/image
INFO[0000] Unknown propagation mode for hostPath /mnt/7
INFO[0000] Pulling the image without credentials. Image: registry.k8s.io/pause:3.6
INFO[0000] getSandBoxSecurityOpts is unsupported in this build
INFO[0000] getSandBoxSecurityOpts is unsupported in this build
INFO[0000] getSandBoxSecurityOpts is unsupported in this build
INFO[0000] Pulling the image without credentials. Image: registry.k8s.io/pause:3.6
INFO[0000] getSandBoxSecurityOpts is unsupported in this build
INFO[0000] determinePodIPBySandboxID is unsupported in this build
INFO[0000] Failed to read pod IP from plugin/docker: Couldn't find network status for bar/foo through plugin: invalid network status for
INFO[0000] determinePodIPBySandboxID is unsupported in this build
INFO[0000] getSandBoxSecurityOpts is unsupported in this build
--- FAIL: TestSandboxHasLeastPrivilegesConfig (0.00s)
    sandbox_helpers_test.go:196:
        	Error Trace:	/Users/vikramharinau/gopath/src/github.com/Mirantis/cri-dockerd/core/sandbox_helpers_test.go:196
        	Error:      	Not equal:
        	            	expected: 0
        	            	actual  : 1
        	Test:       	TestSandboxHasLeastPrivilegesConfig
        	Messages:   	sandbox should use runtime/default
panic: runtime error: index out of range [0] with length 0 [recovered]
	panic: runtime error: index out of range [0] with length 0

goroutine 2167 [running]:
testing.tRunner.func1.2({0x2414f60, 0xc00013be90})
	/usr/local/opt/go/libexec/src/testing/testing.go:1545 +0x238
testing.tRunner.func1()
	/usr/local/opt/go/libexec/src/testing/testing.go:1548 +0x397
panic({0x2414f60?, 0xc00013be90?})
	/usr/local/opt/go/libexec/src/runtime/panic.go:914 +0x21f
github.com/Mirantis/cri-dockerd/core.TestSandboxHasLeastPrivilegesConfig(0xc000683380?)
	/Users/vikramharinau/gopath/src/github.com/Mirantis/cri-dockerd/core/sandbox_helpers_test.go:205 +0x1f9
testing.tRunner(0xc000683ba0, 0x2634af8)
	/usr/local/opt/go/libexec/src/testing/testing.go:1595 +0xff
created by testing.(*T).Run in goroutine 1
	/usr/local/opt/go/libexec/src/testing/testing.go:1648 +0x3ad
exit status 2
FAIL	github.com/Mirantis/cri-dockerd/core	0.444s

~/gopath/src/github.com/Mirantis/cri-dockerd/core>  go test
# k8s.io/kubernetes/pkg/kubelet/container
../vendor/k8s.io/kubernetes/pkg/kubelet/container/helpers.go:42:81: undefined: v1.Handler
FAIL	github.com/Mirantis/cri-dockerd/core [build failed]

Given Mac is oftentimes used for dev work, I found it worthwhile to mention here in case someone runs into this.

@vikramhh Mac falls into the unsupported category. Especially since nobody us running a Mac cloud.
If someone with a Mac is concerned about it, then they will have to step up and take ownership of supporting Mac.


pytorch version dependency issue

Bug description
Please see issue here.
https://github.com/pypa/pip/issues/11524#issuecomment-1283381523
Is lightning depending on 1.9.* somewhere as mentioned in the issue there?
How to reproduce the bug
See https://github.com/pypa/pip/issues/11524#issuecomment-1283381523

Error messages and logs
see https://github.com/pypa/pip/issues/11524#issuecomment-1283381523
Environment
see https://github.com/pypa/pip/issues/11524#issuecomment-1283381523
More info
No response

Does this fix work for you @min-xu-ai? #15184
I'm not sure I understand fully what needs to be done, but let's try this.

Closing. Resolved in https://github.com/pypa/pip/issues/11524


enclave-tls: support to verify the quote with user-defined callback

For example, mrenclave and mrsigner from peer side cannot be authenticated by enclave-tls itself. How to check the validation of them is the responsibility of enclave application.

add user callback function which will be passed into verify callback as a check machnism.
For example, add singer identity verification and enclave's meta info(SVN and ISVProdID)

user can verify mrenclave and mrsigner on user client side as  initial implementation.


Run tests using LocalProvider

What is the expected enhancement?
Now that we have LocalProvider, we should make that the default environment for running tests in the CI. Specifically, https://github.com/Qiskit-Extensions/quantum-serverless/blob/main/.github/workflows/docker-build.yaml could be updated to just run the notebooks (rather than docker build, etc.)
In theory, this means we could also drop docker compose. Longer-term though, we still need full e2e testing on Kubernetes (xref https://github.com/Qiskit-Extensions/quantum-serverless/issues/117#issuecomment-1594718083).

In the interim, will note that docs/getting_started/basic/01_running_program.ipynb works fine using LocalProvider

All notebook samples are using the ServerlessProvider now.  Should they be updated to use the LocalProvider in the docs or just modified during the tests on the fly?


Solves the issue with chandra missing object_id

This PR solves issue #98 by adding manually an object id at the catalog level. Also fixed bug with data loading which was a consequence of not having an object_id column in the dataset.

I double checked, works fine


chore: fix incorrect error log in rewards distribution

When TotalVotesWeight was equal to twoTPlusOne a incorrect error log is logged, for instance on testnet this is often logged: "DistributeRewards - TotalVotesWeight (667) is smaller than two twoTPlusOne (667)
Distribution was actually ok since even when this was logged, bonusVotesWeight had the default value of 0 which is correct value for this case.

prnet cleaned-up successfully


Add icons for buildpacks/stacks

Visual candy
Should ensure we use a sprite image so that we're not loading a tonne of small icons.

Won't do.


why  sampler.get_autocorr_time() gives nans!

trying to estimate 6 paramaters, nwalkers=32, nsteps=1e6
tau    = sampler.get_autocorr_time()
print("taus", tau)
gives: taus [nan nan  1. nan  1.  1.]?

Hard to diagnose from such limited info. Are you sure that you don't have nans in your chains?

no nans, no +-inf in chains! getting these warnings...
autocorr.py:36: RuntimeWarning: invalid value encountered in true_divide   acf /= acf[0]
autocorr.py:41: RuntimeWarning: invalid value encountered in less   m = np.arange(len(taus)) < c * taus
99: RuntimeWarning: invalid value encountered in greater  flag = tol * tau_est > n_t

That suggests that there's some sort of under or overflow happening. Take a
look at your chains to see if any parameters are especially large or small
and the post the specific chain that is failing somewhere and I'll take
a look at it.
On Thu, Apr 16, 2020 at 4:24 PM Sultan Hassan<EMAIL_ADDRESS>wrote:

no nans, no +-inf in chains! getting these warnings...
autocorr.py:36: RuntimeWarning: invalid value encountered in true_divide
acf /= acf[0]
autocorr.py:41: RuntimeWarning: invalid value encountered in less m =
np.arange(len(taus)) < c * taus
99: RuntimeWarning: invalid value encountered in greater flag = tol *
tau_est > n_t
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/dfm/emcee/issues/342#issuecomment-614875947, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AACVQSTH2J66DOY3TDYPYGTRM5SRRANCNFSM4MKAZ5TA
.

--
Dan Foreman-Mackey
Associate Research Scientist
Flatiron Institute
http://dfm.io

thanks Dan, seems like no real exploration of the parameter space is happening:
the range (min , max) of the six parameters from the chain:
0  -0.005010458610130385 -0.004670719405149635
1   38.04524868811062 38.045567812313415
2   7.545941815153315 7.546350085425933
3   0.1797592064878929 0.1802134523596176
4   3.4671571905145857 3.46753115770362
5   0.516954651642297 0.51744226666063
I am trying to understand why this is the case, and don't know why the first parameter is negative while I am specifying prior range from 0.01 to 1 to it. Anyway will explore more, I wanted to ask: I am running nwalkers=1e3, and nsteps=1e5 which should be enough for 6 params, but now getting the warning below, so what is the good number of steps and walkers to use given the number of params? thanks again
WARNING:root:The chain is shorter than 50 times the integrated autocorrelation time for 6 parameter(s). Use this estimate with caution and run a longer chain!
N/50 = 200;
tau: [500.34525389<PHONE_NUMBER>8<PHONE_NUMBER>4<PHONE_NUMBER>7<PHONE_NUMBER>2
646.74954819]

Hi, I'm having the same issue right now--My autocorrelation times are coming out as nans sometimes. I noticed that where my log_prob = -inf, the draw has unreasonable parameters. Specifically, one of my parameters is negative when I've specified it should be 0 < parameter < 3 in the prior. Wondering if there was ever a resolution for this issue? Thank you!

@laldoroty — if you're finding parameters out of bounds in your final chain that normally means that you've initialized walkers outside of the bounds. It's worth adding a check to make sure that you're not!

Yes, I've definitely got parameters initialized out of bounds if I look at the State() object, but I thought the skip_initial_state_check=False argument in run_mcmc() would prevent that?

If you initialize out of bounds, those walkers typically won't ever move into bounds so those chains will cause many problems (this one included :D) so I'm glad that you identified the issue!
The skip_initial_state_check flag bypasses a check for a different common initialization issue and you should only turn that on if you have a really strong argument that you should.


Support CPI API v2

Currently the AliCloud CPI only supports CPI API v1, but v2 is available and many CPIs already support it. Main benefit is, that the Registry becomes obsolete. You can checkout https://bosh.io/docs/v2-migration-guide/ to get an overview of all necessary changes.

HI. @friegger I am sorry for late reply. The latest release v38.0.0 has supported it and please have check.
Please reopen it and let me know if there is any issue.


Improvement for k8s.io/docs/concepts/storage/volume-snapshots/

Maybe its just me, but reading the page first time it is not clear what these volume-snapshots are really good for. Reading it again doesn't help, but it makes one thing clear: Taking a snapshot is just half of the story.
Maybe a practical example (e.g. backup and restore of a mysql database, migration to another cluster) would be an improvement.

Thanks for the suggestion
/language en
/triage accepted
/sig storage
/priority backlog

\assign

/assign


pick changelog fixes from master

did not land in generated changelog as that is generated from stable
(see https://github.com/dlang/phobos/pull/6199 and https://github.com/dlang/phobos/pull/6226)

We should really figure out how to update the generated changelog additively and purge changelog fragments right away when merging. This would also make the pending changelog accurately.

Hmm the error seems unrelated, but I'm without computer for today.

We should really figure out how to update the generated changelog additively and purge changelog fragments right away when merging.

Yeah, this is bothering me too. I will have a look into this next week. It should be as easy as reading the full text blob and dumping it again. An alternative would be to allow the script to read files from e.g. changelog/2.079.0/dmd

I will have a look into this next week. It should be as easy as reading the full text blob and dumping it again.

Thx that would be great. Let's figure out if we can make it parseable. That would also help with branches and backporting at some point.
It's quite structure at least. Alternatively we could save the combined changelog in some structured format (JSON, XML) and generate a .dd from that, but it doesn't seem too necessary.
Ddoc

$(CHANGELOG_NAV_LAST 2.078.3)

$(VERSION ...
  $(BUGSTITLE_TEXT_HEADER
    $(LI ...)
    $(LI ...)
  )
  $(BUGSTITLE_TEXT_HEADER ...
    $(LI ...)
    $(LI ...)
  )
  $(BUGSTITLE_TEXT_HEADER ...
    $(LI ...)
    $(LI ...)
  )
  $(CHANGELOG_SEP_HEADER_TEXT_NONEMPTY)
  $(CHANGELOG_SEP_HEADER_TEXT)
  $(BUGSTITLE_TEXT_BODY
    $(LI ...)
    $(LI ...)
    $(LI ...)
  )
  $(BUGSTITLE_TEXT_BODY
    $(LI ...)
    $(LI ...)
    $(LI ...)
  )
  $(CHANGELOG_SEP_TEXT_BUGZILLA)
  $(BUGSTITLE_BUGZILLA
    $(LI ...)
    $(LI ...)
    $(LI ...)
  )
  $(BUGSTITLE_BUGZILLA
    $(LI ...)
    $(LI ...)
    $(LI ...)
  )
)
$(D_CONTRIBUTORS_HEADER 78)
$(D_CONTRIBUTORS
  $(D_CONTRIBUTOR ...)
  $(D_CONTRIBUTOR ...)
)
$(D_CONTRIBUTORS_FOOTER)
$(CHANGELOG_NAV_LAST 2.078.3)
Macros:
    VER=2.079.0
    TITLE=Change Log: $(VER)

CircleCI is using the wrong base branch.
dmd_access failure is present on stable in general http://dtest.dlang.io/results/833dabe8e414cc828288063d4dd274acf586ff1e/!base/build.log.


pypoetry failing to install mysqlclient

pypoetry keeps on failing to install when executing the make test. This is after following your installation recommendations on an Ubuntu system (22.04.1). The following is the start of the error message that I receive.
`• Installing mysqlclient (2.1.1): Failed
CalledProcessError
Command '['/........./workspace/WiSE/owid/etl/.venv/bin/python', '-m', 'pip', 'install', '--use-pep517', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/........./workspace/WiSE/owid/etl/.venv', '--no-deps', '/......./.cache/pypoetry/artifacts/2d/90/d6/339cd2c929c6d8a54167285d71ca5d4c6996063ebeba2e43a6d5e3f698/mysqlclient-2.1.1.tar.gz']' returned non-zero exit status 1.The latter part of the second section of "this" error message reads:   × Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> [27 lines of output]
/bin/sh: 1: mysql_config: not found
/bin/sh: 1: mariadb_config: not found
/bin/sh: 1: mysql_config: not found
`
Looking around it suggested that the sql c library is absent and causes these problems, however it is available on my system, just not within this environment. Can someone help me to fix this?

@Zoophobus this looks like MySQL client is missing on your system. Could you try installing it with sudo apt install mysql-client (or perhaps sudo apt install libmysqlclient-dev) and then try again?

Hi thanks for responding, unfortunately it's not that simple, below is the response
~$ sudo apt-get install mysql-client [sudo] password for zoo:  Reading package lists... Done Building dependency tree... Done Reading state information... Done mysql-client is already the newest version (8.0.32-1ubuntu22.04). 0 upgraded, 0 newly installed, 0 to remove and 70 not upgraded.

The same for the dev packages!
... install libmysqlclient-dev Reading package lists... Done Building dependency tree... Done Reading state information... Done libmysqlclient-dev is already the newest version (8.0.32-1ubuntu22.04). libmysqlclient-dev set to manually installed. 0 upgraded, 0 newly installed, 0 to remove and 70 not upgraded.

This could be because the mysql executable is not in your system's PATH. I assume which mysql is not giving you any path, right? You'll have to find where is mysql installed and add it to your path with export PATH=$PATH:/path/to/mysql/bin. (Try e.g. sudo find / -name mysql_config to find the path)

Sorry to be a pain again, however the output of which mysql is the /usr/bin/mysql path. I've setup mysql previously for some django projects, so I'm really quite stumped as to what is occurring here, although that it's something stupid I think is now a given

Hi Marigold,
Thanks for the help, miraculously after coming back to this and just re-entering poetry install the mysqlclient for the local pip environment was installed `Installing dependencies from lock file
Package operations: 1 install, 0 updates, 0 removals
• Installing mysqlclient (2.1.1)
Installing the current project: etl (0.1.0)
`
I'm not sure exactly what changed in the meantime, regardless though this is working (ish) so I won't jinx it. (Although now I have a new import error with the libstdc++ library). So while I'll close this issue be warned that I may well be back soon


make sure opencv builds fine with the recent versions of libav

WIP
the patch is based on #2293, but should hopefully pass the tests.
The other notable change - motion jpeg decoder has been temporary disabled because of suspicious reports in the video repositioning tests.

:+1:


ArrayBond.updateListener - replaced object

I would use the replaced object in the updateListener. I actually need it, because I need to copy the viewmodel status to the new viewmodel so my UI doesnt "blink"

That's a great suggestion, thank you. Let's me implement it.

Implemented in v3.3.


Custom layout

More appropriate name for #52 .

@dwightguth I think I got all of the copyrights correct this time. I also ran dos2unix on all files in the repo, there were some with funky line-endings.


Reformats code according to Google Java Style

Files had various formatting differences, as did pull requests. Rather than
create our own style, this inherits and requires the well documented Google
Java Style.

Specifically, I reformatted using Intellij and this file

NetflixOSS » feign » feign-pull-requests #43 SUCCESS
This pull request looks good


refactor(userspace,driver): merge ppm_sc_codes and ppm_tp_codes

What type of PR is this?
/kind cleanup
Any specific area of the project related to this PR?
/area driver-kmod
/area libscap-engine-bpf
/area libscap-engine-kmod
/area libscap-engine-modern-bpf
/area libscap
/area libpman
/area libsinsp
Does this PR require a change in the driver versions?
Yep, but it was already bumped in this release cycle.
What this PR does / why we need it:
sc becomes scap codes (not SysCall).
Ported scap-open and sinsp-example too.
Note: at low level, drivers manage syscalls and tracepoints in a different way,
eg: see scap_kmod ioctls.
But at higher level (ie: sinsp and scap external APIs), tracepoints and syscalls are just ppm_sc codes.
My only concern is about SYS_ENTER and SYS_EXIT; it is a bit too overkill to manage them standalone, at the same time, it feels a bit weird to have them among the ppm_sc codes, since they are actually "filters" above the ppm_sc syscalls.
Which issue(s) this PR fixes:
Fixes #
Special notes for your reviewer:
Does this PR introduce a user-facing change?:

cleanup!: dropped ppm_tp_code and merged them with ppm_sc_code.

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Note: i tested bpf and modern-bpf for now.
Taking my time to double check that kmod is ok before killing my kernel :rofl:

/milestone 0.11.0
/cc @incertum @jasondellaluce @Andreagit97

You'll notice that we now have a super clean external API, but internally (mostly kmod code) we have a little more complexity.
It is a good trade off IMHO and perhaps we could further improve the situation somehow.

/hold

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP, incertum
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP,incertum]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Rebased on top of master ;)

[APPROVALNOTIFIER] This PR is APPROVED
This pull-request has been approved by: FedeDP
The full list of commands accepted by this bot can be found here.
The pull request process is described here

Needs approval from an approver in each of these files:

OWNERS [FedeDP]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

Some more work; 3 interesting sets related tests are failing for now.

Following #916  i am going to assert the size of the new scap_ppm_sc.c private table, and add designated initializers to it.

Opened https://github.com/falcosecurity/libs/tree/maintainers/899-testing for sharing the missing tests more conveniently plus includes comments re clarifications what ground truth should even be for the generic events edge cases.

@incertum i think we should think a bit more whether enforcing SYS_{ENTER,EXIT} is ok or not; i see it's creating a little bit of confusion.
Perhaps we might want to retain ppm_tp_codes internally in scap engines/drivers, and only expose PPM_SC_ codes that map to an event (ie: SYS_{ENTER,EXIT}) won't be exposed anymore in that case.
Let's wait a bit more; in the meantime i might open a PR extracting the libsinsp::events::set.data() fix so that it gets fixed.

There have been some design changes: basically we decided to avoid mapping 1:1 ppm_sc tp codes and real tracepoints;
therefore, in the end i added back the ppm_tp_code that is now just used by 3 scap engines: {bpf,kmod,modern}.
Basically, when receiving the list of PPM_SC codes (or the single ppm_sc code at runtime during scap_set_ppm_sc calls), it is now the engine responsibility to parse the list of ppm_sc codes and find the correct tracepoint set that must be enabled.
This has some advantages:

PPM_SC enums will refers to driver-based events for real (ie: syscalls or tracepoint ones)
when retrieving the sc_set from the event_set, SYS_ENTER/EXIT won't pollute your set, improving UX
engines will take care of attaching/detaching needed tracepoints transparently from rest of userspace application (eg:  no need to remember to instrument scap-open to attach sys-enter anymore!)

Drivers tests are failing for now; still investigating!
@Andreagit97 might be helpful here :1st_place_medal:

Why do we want to expose tracepoints anywhere outside the engine? Before we had engines, the ring buffers were similarly exposed, which would make e.g. gvisor/test_input/source_plugin do a dance pretending they want a ring buffer in the first place (or give up and ignore scap_readbuf entirely). Right now the ringbuffer code is shared between the engines that want it but it's an implementation detail for all of them (scap core and sinsp neither know nor care). I believe this is the right approach for tracepoints too. Let the consumer operate on syscalls/events, who cares how they're implemented underneath.

yeah probably not so evident but this is exactly what the PR does. Let's imagine, libsinsp exposes 3 stable APIs:

Event codes (PPME_...). They are used for inspection during the capture, you can analyze their content by searching for its code:if(PPME_SYSCALL_CLONE3_X)
{
       // do something...
}

As you know they are not stable so they could change between libs version if we bump the schema for example
2. Event names. They could be used by various consumers to ask libsisnp to trace something, consider the Falco rules as an example.
3. PPM_SC codes. This is the method the consumer has to select which logic should be enabled. For example, again in Falco let's imagine the user selected a set of event names to use, we can convert these event names into PPM_SC codes so in this way libsinsp will know what to enable.

So considering point `3`: the PPM_SC code it's just a way to tell libraries to enable/disable an event, this is not strictly related to tracepoints, we are not asking to enable/disable `SYS_ENTER` or `SYS_EXIT` we are saying we want `PPME_PAGE_FAULT_E` event and the way we chose to communicate it to sinsp is `PPM_SC_PAGE_FAULT`

Why do we need PPM_SC_* in the first place? AIUI, the main use case is having an arch-independent number in PPME_GENERIC_* events. But what can we do with these events other than map PPM_SC_* to a name? Nothing AFAIK. So how about we drop these completely and instead provide (easily autogenerated) tables for each arch we support (linux/amd64, linux/arm64, linux/s390x, linux/x86? did I miss something? in any case, we can extract the tables for all architectures from the kernel sources). The user facing API for enabling/disabling syscalls can simply use the native _NR* constants (please note: I'm not 100% following whether we aim to enable/disable syscalls or events).

I would be a super fan of this but there are some issues :/

We can enable and disable syscall through __NR_... codes but what about other events like PPME_SCHEDSWITCH_6_E, PPME_PAGE_FAULT_E and others?

Generic events send to userspace the PPM_SC they referred to, in userspace we perform the translation from code to syscall name and in live capture we could use __NR.. but what about scap-files? if i read a scap-file generated by an x86 machine on a aarch64 what would happen?

Grzegorz brought very good points, and I think Andrea's answers already cover most concerns. Will try to provide my view.
First, I vote for keep having a platform-agnostic and manually-manageable PPM_SC enumeration rather than relying on external compatibility headers with the __NR constants. I find it more convenient and predictable, plus thanks to Fede's work on the syscalls-bumper tool, we can achieve a good degree of automation too.
Second, I think the value of the PPM_SC enum is that it gives a more simplified and condensed view of the events we support for the syscall event source implemented through kernel drivers. The event_table contains every detail about all events, including the legacy versions, the non-driver events (container, meta-events, plugins, etc...), and dummy/unuded events. Instead, the PPM_SC enum can be the right source of truth about the events supported by our kernel instrumentation engines, not having to deal with the ambiguity of multiple event  version, enter and exit pairs, and all. Moreover, the event_table has the cumbersome duty of representing a contract for capture files, whereas PPM_SC is only relevant in the context of live captures. I think the PPM_SC enum is the right piece of information we can pass to the kmod, bpf, and modern_bpf engines to configure them for live event collection. Accordingly, I don't see PPM_SC ever containing thinks like pluginevent or container. I think this is safe to be exported as part of libscap's API, because this is effectively a piece of configuration for a subset of the engines supported by libscap.
Continuing on the second point, I think Fede is going on the right solution on how we should map tracepoint events. I think there is a semantic distinction between the events related to a tracepoint, and the act of attaching a tracepoint. We already have some examples of the first case, such as procexit and switch which can be requested to a driver and can be collected as valid events. I expect events of this kind to grow if we really choose the route of LSM hooks and kprobe-implemented events. Instead, I think that attaching a tracepoint is the act through which one libscap engine understands how to instrument the kernel in order to collect the events selected by users. For this reason, I think the PPM_SC enum can be the right place to host events such as procexit and switch, whereas I agree on not exporting tracepoint details such as sys_enter and sys_exit, and having them internal to the engines only. In fact, the counter-proof is that it's not possible (and makes no sense) to produce events of type sys_enter, but it is indeed possible to produce procexit events.

We can enable and disable syscall through __NR_... codes but what about other events like PPME_SCHEDSWITCH_6_E, PPME_PAGE_FAULT_E and others?

Good point. We'd need our own constants for these.

Generic events send to userspace the PPM_SC they referred to, in userspace we perform the translation from code to syscall name and in live capture we could use __NR.. but what about scap-files? if i read a scap-file generated by an x86 machine on a aarch64 what would happen?

We'd need to know that the capture came from x86. I have a patch for this. It's trivial but it's somewhat deep in my queue :/

First, I vote for keep having a platform-agnostic and manually-manageable PPM_SC enumeration rather than relying on external compatibility headers with the __NR constants. I find it more convenient and predictable, plus thanks to Fede's work on the syscalls-bumper tool, we can achieve a good degree of automation too.
Second, I think the value of the PPM_SC enum is that it gives a more simplified and condensed view of the events we support for the syscall event source implemented through kernel drivers. The event_table contains every detail about all events, including the legacy versions, the non-driver events (container, meta-events, plugins, etc...), and dummy/unuded events. Instead, the PPM_SC enum can be the right source of truth about the events supported by our kernel instrumentation engines, not having to deal with the ambiguity of multiple event version, enter and exit pairs, and all. Moreover, the event_table has the cumbersome duty of representing a contract for capture files, whereas PPM_SC is only relevant in the context of live captures. I think the PPM_SC enum is the right piece of information we can pass to the kmod, bpf, and modern_bpf engines to configure them for live event collection. Accordingly, I don't see PPM_SC ever containing thinks like pluginevent or container. I think this is safe to be exported as part of libscap's API, because this is effectively a piece of configuration for a subset of the engines supported by libscap.

Good point about multiple event versions. Also, since we already have PPM_SC_* in captures, we're stuck with them more or less forever :/ So I guess I need to retract my point about removing them (and I get to disagree about "PPM_SC is only relevant in the context of live captures" ;]).
We have the two--mostly independent--uses for PPM_SC_*:

platform-independent syscall numbers for PPME_GENERIC_*
a label for a set of events we're interested in (PPM_SC_CLONE = {PPME_CLONE_[EX] in all the million variants})

For the first one, we clearly need (and want) syscall numbers only. For the second, I'm not actually sure. I'd go the other way and make sure (verify statically on build?) that every PPME_* (syscall or otherwise) maps exactly to one PPM_SC_*, which then can be used to enable/disable the event.
The funny thing is how little libscap actually cares about the event schema (it's a contract between the engine and libsinsp, effectively), but we're maintainig the schema as a part of libscap. Will think about this and probably come up with some more crazy ideas.

Continuing on the second point, I think Fede is going on the right solution on how we should map tracepoint events. I think there is a semantic distinction between the events related to a tracepoint, and the act of attaching a tracepoint. We already have some examples of the first case, such as procexit and switch which can be requested to a driver and can be collected as valid events. I expect events of this kind to grow if we really choose the route of LSM hooks and kprobe-implemented events. Instead, I think that attaching a tracepoint is the act through which one libscap engine understands how to instrument the kernel in order to collect the events selected by users. For this reason, I think the PPM_SC enum can be the right place to host events such as procexit and switch, whereas I agree on not exporting tracepoint details such as sys_enter and sys_exit, and having them internal to the engines only. In fact, the counter-proof is that it's not possible (and makes no sense) to produce events of type sys_enter, but it is indeed possible to produce procexit events.

I'm not buying events related to a tracepoint. Non-syscall events, sure. But in the end, they're just events like any other. Is instrumenting open via sys_enter fundamentally different from generating the same events via fs/do_sys_open? Would we make open a tracepoint-related event if we switch to the dedicated tracepoint? Why would anybody care?
Also, PPME_GENERIC_* are exactly sys_enter/sys_exit events ;P

Good point. We'd need our own constants for these.

That's why we chose PPM_SC :thinking:

For the first one, we clearly need (and want) syscall numbers only. For the second, I'm not actually sure. I'd go the other way and make sure (verify statically on build?) that every PPME_* (syscall or otherwise) maps exactly to one PPM_SC_*, which then can be used to enable/disable the event.

The initial idea was to use directly events but we saw some issues:

they are not stable between libs versions like PPM_SC. One day you enable "clone" through PPME_SYSCALL_CLONE_20_E and the next version you have to use PPME_SYSCALL_CLONE_23_E...
we cannot handle generic events. If we want to enable just alarm syscall that is generic how could we enable it since we have just PPME_GENERIC_E...
why are using an event to enable a syscall, do we have to pass the enter event or the exit one to the libs API?

I'm not buying events related to a tracepoint. Non-syscall events, sure. But in the end, they're just events like any other. Is instrumenting open via sys_enter fundamentally different from generating the same events via fs/do_sys_open? Would we make open a tracepoint-related event if we switch to the dedicated tracepoint? Why would anybody care?

Yeah, that's exactly the point, this solution allows us to keep the same code even if the underlying instrumentation change. Today we already have a case: PPME_SYSCALL_EXECVE_19_X. This event could be generated by both sys_exit and sched_proc_exec tracepoints and the userspace know nothing about the instrumentation point, it just know the  PPME_SYSCALL_EXECVE_19_X event :)

Kindly asking what the integration roadmap would look like?
It appears that this PR has significantly grown beyond its initial scope, therefore suggesting the following:
Could we split this PR into separate PRs?

Immediate fixes unrelated to refactors
Possible refactor and enhanced coverage of unit tests
Changes to the ppm sc table (desired would be to first discuss pros and cons in an issue and gather feedback from the community as well and roadmap implications wrt to maintaining changes)
Possibility of merging ppm_sc_codes and ppm_tp_codes (same desired would be to first discuss pros and cons in an issue and gather feedback from the community as well and roadmap implications wrt to maintaining changes)

The reasoning behind these suggestion is that recently we implemented many changes fairly fast wrt the ppm sc API without having had a breather to reflect on what worked and what didn't work. This is because, we needed several follow up PRs for bug fixes and/or addressing corner cases which could be a reflection of that going forward perhaps we could benefit from enhanced roadmap planning in order to ensure we are moving towards the direction that is best for the overall project as per our governance and also attempt to accomplish right-sized incremental progress and enhanced stability.
What do you all think?

In the meantime, i'll leave this PR open in a wip state, so that we can track the parent of the new PRs.

just pushed tons of tests here so we don't lose them :)

Thank you.

new table to map each PPME event to all the PPM_SC that generate it

Please also see https://github.com/falcosecurity/libs/pull/915#discussion_r1120334736 for immediate fixes

finally, the ground work for the LSM/Kprobe support that is both needed for a completely clean libsinsp::events:: API going forward, ie: merging tracepoints into PPM_SC enum

It seems we are most worried about integration and operationalization of LSM hooks/kprobes. Would it be worthwhile opening a proposal to provide each stakeholder and the community as a whole with an opportunity to provide their perspective (also from a threat detection perspective as well as production stability and scalability in mind)?

It seems we are most worried about integration and operationalization of LSM hooks/kprobes. Would it be worthwhile opening a proposal to provide each stakeholder and the community as a whole with an opportunity to provide their perspective (also from a threat detection perspective as well as production stability and scalability in mind)?

I mean, it's not like we are worried :) it's that the ppm_sc/ppm_tp merge is a requirement for that too; it is also a requirement for a clean libsinsp::events:: API, since without that, we need a doubled API for tracepoints too; something that is horrible from a ux pov, and is not what we want (i mean, without that patch, we are not able to map a PPME_SCHEDSWITCH to its ppm_sc, therefore all the converting methods between events <-> ppm_sc are a bit buggy.
LSM/kprobes can then come in the future (like Falco 0.36) but this ground work IMHO needs to land asap.
I'd love to avoid another API break for this after Falco 0.35 too :)
Moreover, i actually think that the PR won't be super large after all (i expect like a ~300locs plus tests).
/cc @gnosek @Andreagit97 @jasondellaluce @Molter73

Moreover, i actually think that the PR won't be super large after all (i expect like a ~300locs plus tests).

Ehm i was wrong :rofl:

Everything is merged! We can close this one!
/close

@FedeDP: Closed this PR.

In response to this:

Everything is merged! We can close this one!
/close

Instructions for interacting with me using PR comments are available here.  If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.


fix(deploy): allow fetching C4GH and regular files

Changes

Allows fetching both C4GH and regular files in the resolvers settings. Previously this was always attempting to get C4GH files even if the directory contained regular files.

Good catch! Tests failing though?

It should be intermittent. It happens because each test launches a server on a new port, and sometimes the port is in use.
I'll see if there's a way to make it more consistent.


Token contained 1 grants but none satisfied the request

Describe the bug
Generation a token for http://localhost:4466/management with the following claim
{
  "iat":<PHONE_NUMBER>,
  "exp":<PHONE_NUMBER>,
  "grants": [
    {
      "target": "*/*",
      "action": "*"
    }
  ]
}

works
playground:
query {
  listProjects {
    name
    stage
  }
}

returns this
{
  "data": {
    "listProjects": [
      {
        "name": "soon-backend",
        "stage": "dev"
      }
    ]
  }
}

Trying this without wildcards in the target like this
{
  "iat":<PHONE_NUMBER>,
  "exp":<PHONE_NUMBER>,
  "grants": [
    {
      "target": "soon-backend/dev",
      "action": "*"
    }
  ]
}

returns
{
  "data": null,
  "errors": [
    {
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "listProjects"
      ],
      "code": 3015,
      "message": "Authentication token is invalid: Token contained 1 grants but none satisfied the request.",
      "requestId": "local:management:cjo8pnsd4002c0936954wajkx"
    }
  ]
}

Expected behavior
Should get the same result as with "target": "*/*"
Versions (please complete the following information):

OS: Ubuntu 16.04
prisma CLI: prisma/1.18.1 (linux-x64) node-v9.7.1
Prisma Server: 1.18.1

Hi @Melv thanks for reporting. As far as I can see, this behavior is correct.
The claim to access server wide information (in this case listProjects) must be * on both target and action.
The situation is, that listProjects could theoretically cover more than just the service you provided in target and Prisma is more conservative in this regard, not allowing anything without */*.
If this doesn't make sense to you, let me know and I'll reopen the issue.


Missing documentation for some cirq.linalg.* functions

cirq.linalg.deconstruct_single_qubit_matrix_into_angles in particular.  It is not exported to the top-level cirq.* namespace so maybe the documentation script is missing them?
In this case, deconstruct_single_qubit_matrix_into_angles should probably be added to cirq/__init__.py since the rest of cirq.linalg is there.

Where are you looking at the docs?  Readthedocs?  The build docs script output?

Readthedocs: https://cirq.readthedocs.io/en/stable/api.html#algebra-and-representation

These are the URsL where I might expect it to be:
https://cirq.readthedocs.io/en/stable/generated/cirq.deconstruct_single_qubit_matrix_into_angles.html
https://cirq.readthedocs.io/en/stable/generated/cirq.linalg.deconstruct_single_qubit_matrix_into_angles.html
Latest:
https://cirq.readthedocs.io/en/latest/generated/cirq.deconstruct_single_qubit_matrix_into_angles.html
https://cirq.readthedocs.io/en/latest/generated/cirq.linalg.deconstruct_single_qubit_matrix_into_angles.html

Yeah the doc coverage test only looks at a few namespaces for methods (cirq,cirq.experiments, cirq.testing, and cirq.google), so it would not fail if this were not at top level.  It would get generated by inclusion into the api.rst.


merge package from arlac77/npm-package-template

package.json

chore(package<EMAIL_ADDRESS>

Coverage remained the same at 4.268% when pulling 32fee96fa1a2439bf0020706b56ba299b609e4d5 on template-sync-1 into d7c835a4a4a93a4e11647f04b3bbf03c46627b7c on master.


don't receive events after some uptime

Hi, I have some contracts on loom network and I wrote nodejs application for handling contract events from blockchain. Everything okay for first some hours, but some time later events didn't come. This problem is real only with uptime for some hours. When I emit event in first hour - application handle it and wrote to db, but after some hours - there are no events and I tried to debug it with debug mode and found smth interesting:

for each subscription for each block we make rpc call to the node like

ws-rpc-client Sending RPC msg to wss://extdev-plasma-us1.dappchains.com/queryws, method getevmlogs

after receiving answer like
loom-provider Response payload {
  "id": 42,
  "jsonrpc": "2.0",
  "result": []
}

after which it sends requests for filter
loom-provider Request payload {
  "method": "eth_getLogs",
  "params": [
    {
      "fromBlock": "0x6dd392",
      "toBlock": "0x6dd394",
      "address": "0xd0c1a482d6eb9616c3ee860d87f18bb154e3014c",
      "topics": [
        "0x237d4a1ca852c519fa30ab619571a169a13767b6f3af65886d6ed5b6dd6273b6"
      ]
    }
  ],
  "id": 42,
  "jsonrpc": "2.0"
}

and  don't receive anything. maybe i messed up with logs, but i can see only this picture for latest minute or smth like that (period when transaction was sent and added to blockchain and some more time)
for all blocks i have no handled events on nodejs part (as i can see - there are no responses for filters). and this problem is coming only with time
2. are u sure that for each subscription is good idea to call node? why not better to send information about new block to client and parse it and find topics on client?

We're working on a new version of LoomProvider which leverages our new implementation for JSON RPC, if there's an issue, probably this issue will be sane or at least will be easier to follow. Anyway you can check our new implementation for  JSON RPC today on https://loomx.io/developers/en/json-rpc-methods.html

We made few changes on LoomProvider recently, you can now use JSON RPC to call WebSocket connections directly using the same JSON RPC protocol from Ethereum, or if you prefer you can pass the /eth URL on the read URL, for instance:
const client = new Client('default', 'wss://extdev-plasma-us1.dappchains.com/websocket', 'wss://extdev-plasma-us1.dappchains.com/eth')

Could you please check the /eth approach example above. Thanks


What is the system requirement to run the sample code?

What is the system requirement to run the following sample code?
from transformers import AutoTokenizer
from intel_extension_for_transformers.transformers import AutoModelForCausalLM, WeightOnlyQuantConfig
model_name = "Intel/neural-chat-7b-v1-1"     # Hugging Face model_id or local model
config = WeightOnlyQuantConfig(compute_dtype="int8", weight_dtype="int4")
prompt = "Once upon a time, a little girl"
tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
inputs = tokenizer(prompt, return_tensors="pt").input_ids
model = AutoModelForCausalLM.from_pretrained(model_name, quantization_config=config)
gen_tokens = model.generate(inputs, max_new_tokens=300)
outputs = tokenizer.batch_decode(gen_tokens)

The CPU can support the avx512 instruction set, and we recommend the RAM bigger than 16GB.

Ok.... My CPU does not support avx512. Maybe I should have bought AMD CPU. What can I use?

Don't worry, we just supported AVX2 fp32 (compute_dtype) inference in #493. In addition, if you are using a 12th+ Core™ Processor, stay tune as we are support int8 inference with AVX_VNNI.
BTW, AMD client CPU does not support AVX512 either if you are referring client products.

AVX_VNNI support has been added in #565. You can enable it by setting comput_dtype to int8, which should outperform llama.cpp a lot on 12/13th gen Core CPUs.
@yuchengliu1 are working to squeeze some more performance based on the hybrid architecture.
Feel free to reopen the issue if you have further questions.

Hi @DDEle , I am trying to build the graph. I followed the steps from README. After cmake .. -G Ninja , when I run ninja , getting following error:

/home/datascience/intel-extension-for-transformers/intel_extension_for_transformers/llm/library/jblas/jblas/kernel_avx512_bf16.h:24:32: error: attribute(target("avx512bf16")) is unknown
#pragma GCC target("avx512bf16")

Even tried disabling AVX 512 as, cmake -G Ninja -DNE_AVX512=OFF -DNE_AVX512_VBMI=OFF -DNE_AVX512_VNNI=OFF , still the same results.
I am running Linux on Intel(R) Xeon(R) Platinum 8167M CPU @ 2.00GHz

Hi @amir1m,
Please check https://github.com/intel/intel-extension-for-transformers/issues/726#issuecomment-1820037722 for detailed explanation and solution.


Re #83343. Enable kb for Safari when Fullscreen.

This PR fixes #83343

It would probably still not fix it for Firefox though...

@bpasero yup, I create a new issue to track/test Safari #84144 and this PR will only fix Safari. For the Firefox one, there is nothing we can do but reading the 100 page long discussion on https://bugzilla.mozilla.org/show_bug.cgi?id=380637 ;(. We can generate a list for Firefox and talk with them if we get a chance.


Neon (new theme)

Added a new theme termed as "Neon" with 'Base' and 'Lovepink' variants.
Thank you!
bBSempai

I think you can directly add those colorschemes to the existing Lovelace theme if @adriankarlen agrees

So should I just do that?

Let's wait for adriankarlen reply first

brooooo @adriankarlen

I think you can do it, so open another pull requests and just add and commit the new color schemes in the existing theme

A bit late to the party, but of course that's okay :)


FDL master must check if token was received

After the token was forwarded to the next active station, the FDL master must check whether this station has actually received the token and started making use of it.

Initial check was added in commit bd48eaf476a1 ("fdl: master: Check if token was received after forwarding it") but behavior when the token was not received is not yet correct.

And a testcase for this in commit aed674e9d02e ("fdl: tests: Add (disabled) testcase for token resend").


[Gate] Fix outbound access on pipe buffer for NUCLEO-G4

# Comments for current PR
Copy changes introduced in 8ed03be to NUCLEO-G431

!!! WARNING : DO NOT EDIT THE CHECKLIST BELOW !!!
Checklist : Merge from DEVELOP branch
Code Quality

[ ] Each function has a header (description, inputs, outputs)
[ ] Code is commented (particularly in hard to understand areas)
[ ] There are no new warnings that can be corrected
[ ] Commits policy is respected (constitancy commits, clear commits comments)

Tests Review

[ ]  Manual Tests for new features have been reviewed
[ ]  Automatic Tests for new features have been reviewed

Tests are passed the best as I can

[ ]  Non regression existing tests are passed OK
[ ]  New tests for new features & bug fixes are passed OK

Documentation

[ ] Documentation is up to date
[ ] Changelog for next release is up-to-date : fill here
[ ] If needed, tests are briefly described in test repository

With the 2.2.0 version there is a new serial protocol that fix this issue


CORS data access blocked with URL provided to allow access

Using Chrome extension for Allow-Control-Allow-Origin:
From Vaadin demo Site:
XMLHttpRequest cannot load https://em2db.demo.vaadin.com/db_0e7ab06211/. A wildcard '*' cannot be used in the 'Access-Control-Allow-Origin' header when the credentials flag is true. Origin 'https://wc.demo.vaadin.com' is therefore not allowed access. The credentials mode of an XMLHttpRequest is controlled by the withCredentials attribute.
From polymer serve:
XMLHttpRequest cannot load https://em2db.demo.vaadin.com/db_9e5c96a3b0/. A wildcard '*' cannot be used in the 'Access-Control-Allow-Origin' header when the credentials flag is true. Origin 'http://localhost:8080' is therefore not allowed access. The credentials mode of an XMLHttpRequest is controlled by the withCredentials attribute.

Are you able to use it without the plugin? I am able to access the demo just fine on my end without any plugins.

By plug-in do you mean the chrome extension? After my post, I returned to
both the local & Vaadin site demos and data had been populated. I am now
able to turn off the extension and the data appears to be working (have not
written new records). The error messages below remain with pouchDB follow
on to 404. I have not used pouchDB before, so maybe the initial CORS block
was related, but only a temporary block(?). As an aside what is the
difference between the Vaadin PuchDB element and the polymer Element (
https://elements.polymer-project.org/elements/app-pouchdb)?
Remaining Console Messages after initial data load:
XMLHttpRequest cannot load https://em2db.demo.vaadin.com/db_9e5c96a3b0/. A
wildcard '*' cannot be used in the 'Access-Control-Allow-Origin' header
when the credentials flag is true. Origin 'http://localhost:8080' is
therefore not allowed access. The credentials mode of an XMLHttpRequest is
controlled by the withCredentials attribute.
24overview-page.html:256 update
https://em2db.demo.vaadin.com/db_9e5c96a3b0/_local/twrQMLUlkFCnJo8zB5g.wg%3D%3D?
Failed to load resource: the server responded with a status of 404 (Object
Not Found)
pouchdb.js:927
The above 404 is totally normal. PouchDB is just checking if a remote
checkpoint exists.
26overview-page.html:256 update
https://em2db.demo.vaadin.com/db_9e5c96a3b0/_local/8VqeZV86o4sk_FlVAV.lFA%3D%3D?
Failed to load resource: the server responded with a status of 404 (Object
Not Found)
https://em2db.demo.vaadin.com/db_9e5c96a3b0/_local/twrQMLUlkFCnJo8zB5g.wg%3D%3D?
Failed to load resource: the server responded with a status of 404 (Object
Not Found)
pouchdb.js:927
The above 404 is totally normal. PouchDB is just checking if a remote
checkpoint exists.
https://em2db.demo.vaadin.com/db_9e5c96a3b0/_local/8VqeZV86o4sk_FlVAV.lFA%3D%3D?
Failed to load resource: the server responded with a status of 404 (Object
Not Found)
pouchdb.js:927 The above 404 is totally normal. PouchDB is just checking if
a remote checkpoint exists.
On Thu, Aug 18, 2016 at 11:57 AM, Marcus Hellberg<EMAIL_ADDRESS>wrote:

Are you able to use it without the plugin? I am able to access the demo
just fine on my end without any plugins.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/vaadin/expense-manager-demo/issues/46#issuecomment-240769539,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AFwNql0ntY_c35DC4VUjPB5zCEFcNOb3ks5qhIDdgaJpZM4JnhUY
.

The main difference between the two pouchdb implementations is that vaadin-pouchdb also has a browser local db that is kept in sync.
I'm still not seeing any CORS issues on my end. Not sure what is causing you to see them.


Updated README to build the PostgreSQL cartridge + bug fix

renamed Code/PgSQL/rdkit/README to Code/PgSQL/rdkit/README.md such that markdown can be used
updated Code/PgSQL/rdkit/README.md with updated instructions how to build and install the PostgreSQL cartridge on all supported OSs
fixed an issue with the PgSQL install script on Windows where the path to the update_sql directory was incorrect

Reference Issue

What does this implement/fix? Explain your changes.
Any other comments?

@greglandrum The macOS failure here is due to the usual pySimDivPickers failing and is unrelated to this PR.

@ptosco I'm willing to merge this as is, but if you have thoughts about the "how to restart the server" point, I thought I'd give you some time.

I'm not sure how robust the details of all the commands are in terms of inter-system variability, but this is clearly an expert user topic and there's only so much we can do.

Exactly. This is more intended as examples of things that can go wrong and possible solutions than as a recipe for all needs.


Futures 0.3 + async/await syntax

Hi again! I have a two-part question here.
First, I'm trying to work out how SPA routing could work with wasm-bindgen, where navigating to a new route requires fetching data from an API. Based on this conversation it sounds like the fetch request would need to be called by JS at some point, but I'm not sure what that should look like.
Is this possible with just one #[wasm_bindgen(start)] function, or is more JS required?
Second, I would like to use the new async/await! syntax if possible but I'm running into compiler errors with my local experiments. Here's an example:
#![feature(async_await, futures_api, await_macro, pin)]

use futures::compat::Compat;
use js_sys::Promise;
use wasm_bindgen::prelude::*;
use wasm_bindgen_futures::future_to_promise;

async fn test() -> Result<JsValue, JsValue> {
    Ok(JsValue::from(1))
}

#[wasm_bindgen]
pub fn run() -> Promise {
    future_to_promise(Compat::new(test()))
}

I see errors like this:
error[E0277]: the trait bound `std::future::GenFuture<[static generator@web_client\src\lib.rs:87:45: 89:2 {}]>: std::pin::Unpin` is not satisfied in `impl core::future:
:future::Future`
  --> web_client\src\lib.rs:97:5
   |
97 |     future_to_promise(fut)
   |     ^^^^^^^^^^^^^^^^^ within `impl core::future::future::Future`, the trait `std::pin::Unpin` is not implemented for `std::future::GenFuture<[static generator@web_
client\src\lib.rs:87:45: 89:2 {}]>`
   |
   = help: the following implementations were found:
             <std::future::GenFuture<T> as std::pin::Unpin>
   = note: required because it appears within the type `impl core::future::future::Future`
   = note: required because it appears within the type `impl core::future::future::Future`
   = note: required because of the requirements on the impl of `futures::future::Future` for `futures_util::compat::compat03as01::Compat<impl core::future::future::Futu
re>`
   = note: required by `wasm_bindgen_futures::future_to_promise`

error: aborting due to previous error

For more information about this error, try `rustc --explain E0277`.
error: Could not compile `web_client`.

I'm using futures-preview = { version = "0.3.0-alpha.10", features = ["compat"] } and trying to convert to a 0.1 Future that wasm-bindgen-futures can use and convert to a Promise. I don't have a full grasp on futures/async/await in Rust so there's a good chance I'm doing something obviously wrong.
Bonus question: has the futures 0.3 API stabilized enough that it could be supported? 😃
Thanks for your time!

For the first part I think in theory you can use the conversions between JS promises and Rust futures to define everything in Rust, but you may run into snags to work through!
For the second that looks related to the futures 0.1 and 0.3 shims perhaps, maybe a Box is needed?
For the last part, we're waiting on 0.3 to run on stable Rust, but we'll update to that as soon as we can! In the meantime though having a wasm-bindgen-futures-0.3 crate or something like that is totally plausible to do!

@corbinu Sure, I have a lot of experience with Futures (having ported my Signals crate from 0.1 to 0.3), so I can help.
Personally, I think a nightly feature flag (or similar) would be the way to go, but it's up to @alexcrichton

#[wasm_bindgen]
pub fn run() -> Promise {
    future_to_promise(Compat::new(test()))
}

Bumped into this issue today. The solution is to pin the GenFuture like this
future_to_promise(Compat::new(test().boxed()))

Futures have stabilized in nightly! 🎊

Right, but we have to wait until it actually lands in the Stable compiler before we can use it. That'll probably be sometime in July.

Thanks for the ping on this @steveklabnik, and I actually personally think we should go ahead and get this done. This would be behind an off-by-default feature gate to ensure that we don't regress current users of the wasm-bindgen-futures crate, but users using an appropriate compiler could enable the feature and get access to async/await!
@Pauan would you be ok with that strategy?

@alexcrichton Yeah, that sounds great to me.

Support for this has been implemented in https://github.com/rustwasm/wasm-bindgen/pull/1507!

To use Futures 0.3, first enable the futures_0_3 feature:
[dependencies]
wasm-bindgen-futures = { version = "0.3.20", features = ["futures_0_3"] }

Then import and use the futures_0_3 module:
use wasm_bindgen_futures::futures_0_3::{JsFuture, future_to_promise, spawn_local};

(You also have to wait for the new version of wasm-bindgen-futures to be released)

Any idea on when that release may happen? I'd like to give it a try!

I've posted a new release at https://github.com/rustwasm/wasm-bindgen/pull/1536


iso3166,iso4217: generate lists again after long hiatus 4fbd027

The data feeds are back up so we can use them to generate these lists again.
https://github.com/moov-io/ach/pull/782

Codecov Report

Merging #879 (4fbd027) into master (ca0eaaf) will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #879   +/-   ##
=======================================
  Coverage   92.27%   92.27%
=======================================
  Files          66       66
  Lines        5129     5129
=======================================
  Hits         4733     4733
  Misses        233      233
  Partials      163      163