Python Support

I am to write a gNMI Telemetry receiver in Python. Due to gNMI authentication guidelines, this is using TLS auth and encryption.
OpenConfig provides an example in GO using the older openconfig specs:
https://github.com/openconfig/reference/blob/master/telemetry/collector/cli/main.go
Just wondering, how to do the following in Python:

Set "InsecureSkipVerify" option to not validate the server certificate against root certificate.
Set "ServerHostOverride" attribute, for example in cases where struggling with aliases or DNS
Set cipher-list from python (only found how to do this using environment)
Set logging/debug options from python (only found how to do this using environment)

Any help on this one?

Items 2) and 3) are fixed now:
2. Using channel option "grpc.ssl_target_name_override"
3. Use os.environ["GRPC_SSL_CIPHER_SUITES"] = "xxxxx"
Anyhow, I did not yet found a solution for 1) and 4). Interesting enough, even if I change the environment using "os.environ" gRPC logging is not getting enabled. Python code:
os.environ["GRPC_TRACE"] = "all"
os.environ["GRPC_VERBOSITY"] = "DEBUG"

What I am actually looking for, is an equivalent for the following python code:
import ssl
context = ssl.SSLContext(ssl.PROTOCOL_SSLv23)
context.options |= ssl.OP_NO_SSLv2
context.options |= ssl.OP_NO_SSLv3
context.options |= ssl.OP_NO_COMPRESSION
context.set_ciphers("AES128")

if secure:
  context.verify_mode = ssl.CERT_REQUIRED
  context.check_hostname = True
else:
  context.verify_mode = ssl.CERT_NONE
  context.check_hostname = False

For me it seems, we don't have native access to those SSL/TLS related objects from python, as the python module for grpc is using the native C library as backend. Therefore potentially the answer is, to use the "grpc.socket_mutator" for this. Anyhow, I haven't found any example on how to use the socket_mutator - and even unclear on how to provide GRPC_ARG_POINTER from python.

Beside of the "GO" example mentioned in my original description, the GRPC documentation contains a reference on how to do this in JAVA:

If the issuing certificate authority is not known to the client then a properly configured SslContext or SSLSocketFactory should be provided to the NettyChannelBuilder or OkHttpChannelBuilder, respectively.

In consequence, if there is no ability to provide those SSL options using python today, I would ask to add the following to the next GRPC maintenance release:

Extend Environment Options for GRPC Clients:

GRPC_SSL_CERT_REQUIRED = True | False
GRPC_SSL_CHECK_HOSTNAME = True | False

Extend GRPC Channel Options:

grpc.cert_required = 0 | 1
grpc.check_hostname = 0 | 1
grpc.cipher_suite = "<string>"

Extend GRPC Auth Guide:
Add info, how to set TLS options in python.

Add API to enable GRPC logging from python

Improve logging and error codes
For example if cipher suites do not match between server and client

Setting GRPC_TRACE and GRPC_VERBOSITY should certainly work. Have you tried rather than setting them from inside your python code, setting them when you invoke the Python interpreter?
@jboeuf: what do you think of the security feature requests here?

Sorry @wisotzky but I'm very much not in favor of adding these features which degrade security and provide our users more ways to shoot themselves in the foot. In particular, I will not approve of an option that does "InsecureSkipVerify" as experience has shown me that these types of features are triggered in test environments because it's practical and then shipped to prod because everyone forgets about them.
On the cipher suite bit, gRPC/HTTP2/TLS stack must support the TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 cipher suite as per http://httpwg.org/specs/rfc7540.html#rfc.section.9.2.2.
The env var trick is mostly for users who want better cipher suites which would be preferred than that one but I understand the potential abuse for this which is unfortunate.
Regarding debugging, this is more of a gRPC request than a security one. It looks like the core grpc_tracer_set_enabled function would make it easier to be controllable directly with python (or any wrapped languages). @ctiller do you agree with that?

@jboeuf, you've indicated - having the options (1) not to validate the server certificate against the CA root certificate and (2) not to validate the subjectAltName/CN against the server hostname/ip-address should only be used lab/testing. But I would add "during development" and "for trouble-shooting purposes" as other potential applications.
Saying this, I am NOT convinced that gRPC should dictate how application MUST implement security features, while I am okay, that gRPC enforces security by default. If you want to write an application which enforces security - developers should just use default settings, and they are done. If you want to write a validation/troubleshooting tool or provide those options to a more sophisticated application, I want to have all the options. Please remind, that most web-browser allow you to accept self-signed certificates - so unclear, why this should not be possible for web-applications.

@jboeuf, about tweaking cipher suites - I would not consider this as "abuse". Depending on the server implementation, there are some HTTP/2 over TLS 1.2 stacks - which simply do not support the TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256 cipher suite. Even this is violating rfc7540 there are some environments where this cannot easily be changed (for example: java8 does not support ECDHE cipher suites).

@nathanielmanistaatgoogle @jboeuf @ctiller
I do not like change environment variables from python to manipulate parameters like client cipher-suites and logging/debugging. Even if I change the environment from python it is unclear, if those changes make it to become active. In my early testing, I've had sometimes the feeling that environment variables must be set before importing grpc - else those changes are simply ignored.
The preferred solution is clearly to use the environment to define/overwrite defaults - but allow the actual process to overwrite any of this as needed. Also keep in mind, that sometimes processes run over hours/days and you might want to increase/decrease log-levels or change other settings during run time. This is basically impossible, if those values need to be set before the library is imported.
Also I would prefer if grpc/tls logging would use a wrapper, to use the standard python logger for this. Some people including me prefer to write debug logs into files and influence the log-format. It is rather unclear how to do this with grpc, as everything is just dumped to the terminal session in its predefined format.


Smartthings integrtaion issue

Home Assistant release with the issue:
arch | armv7l, os_name | Linux, python_version | 3.7.3, version | 0.97.2, virtualenv | true
Last working Home Assistant release (if known):
0.96.5.
Operating environment (Hass.io/Docker/Windows/etc.):
virtualenv
Component/platform:
Smartthings
Hello.
I installed the new Raspbian. Reinstalled home assistant. And now I have problems with the Smartthings component.

I entered the personal access token created above and click ‘Submit’. Everything went well.
I authorized the app by pressing ‘Allow’.  Success.
Returned to Home Assistant and click ‘Submit’
And here is the problem. Message appeared - Please ensure you have installed and authorized the Home Assistant SmartApp and try again.

I used the online checker - https://www.digicert.com/help/. Everything is OK with DNS and SSL Certificate.
I run the following command from outside my local network. curl -X POST https://.........
Answer - curl: (7) Failed to connect to ........... port 443: Connection refused.
Please, help me to solve this issue.
Before that, I used version HA 0.96.5. And there were no problems. After installing the latest version, I can’t configure the integration.
My log -
2019-08-16 13:30:01 INFO (SyncWorker_9) [homeassistant.loader] Loaded smartthings from homeassistant.components.smartthings
2019-08-16 13:30:21 DEBUG (MainThread) [homeassistant.components.smartthings.smartapp] Setup endpoint for https://.........
2019-08-16 13:31:22 DEBUG (MainThread) [homeassistant.components.smartthings.smartapp] Updated SmartApp '.......' under parent app '......'
2019-08-16 13:40:29 ERROR (MainThread) [homeassistant.core] Error doing job: SSL handshake failed
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/sslproto.py", line 625, in _on_handshake_complete
raise handshake_exc
File "/usr/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
self._sslobj.do_handshake()
File "/usr/lib/python3.7/ssl.py", line 763, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1056)
2019-08-16 13:40:29 ERROR (MainThread) [homeassistant.core] Error doing job: SSL error in data received
Traceback (most recent call last):
File "/usr/lib/python3.7/asyncio/sslproto.py", line 526, in data_received
ssldata, appdata = self._sslpipe.feed_ssldata(data)
File "/usr/lib/python3.7/asyncio/sslproto.py", line 189, in feed_ssldata
self._sslobj.do_handshake()
File "/usr/lib/python3.7/ssl.py", line 763, in do_handshake
self._sslobj.do_handshake()
ssl.SSLError: [SSL: SSLV3_ALERT_CERTIFICATE_UNKNOWN] sslv3 alert certificate unknown (_ssl.c:1056)
End later
DEBUG (MainThread) [homeassistant.components.smartthings.smartapp] Setup endpoint for https://XXXXXXXX:8123/api/webhook/XXXXXXX......

Did you find solution for your issue. I have it the same?

Yes, i removed integration with script. And all work.

Use this utily
https://pypi.org/project/hass-smartthings-remove/
This Will remove all your integration.


MIssing Documentation for a complete setup trying to use Amazon IAP

Please use the Discussion board if you want to get some help. Please use issues to report bugs.

Description
I am working with a project that alreay have this library installed. Thing is that i am getting getSuscriptions as empty array all the time. I think this is due to missing configurations (build.gradle, AndroidManifest, MainActivity, strings, so on) as those were not provided by previous developer. So i started looking into the documentation and was not able to find any references.
The library that is installed is 9.0.4 which is rather old, so i went ahead and updated it to 12.7.2 but was unable to identify where should i be using the app_public_key (just to name one thing)
Expected Behavior
There should be some documentation stating the configuration (basic at least) that need to be put in place to be able to use this end to end
Screenshots

Environment:

react-native-iap: 9.0.4, 10, 11, 12.7.2
react-native: react-native-tvos@latest
Platforms (iOS, Android, emulator, simulator, device): FireTV Stick 4k

To Reproduce
Steps to reproduce the behavior:

Go to '...'
Press '...'
Error '...' is shown

[Optional] Additional Context

Hi,
I have actually written an article on how to get started with Amazon IAP: https://dev.to/amazonappdev/react-native-iap-one-package-to-rule-them-all-4f0n
Hope this helps!

@anishamalde Amazon pays support only for android only not for iOS right?

@lbusz hey. did you manage to find how do you apply the app_public_key?
did you follow this https://developer.amazon.com/docs/appstore-sdk/integrate-appstore-sdk.html?
Thank you!


Emily Jiang - committer quote

I have been working in the MicroProfile community from the beginning, working on MicroProfile Config and then Fault Tolerance, and other specrifications. I thoroughly enjoyed the interaction with other contributors from different companies and individuals. The experience of helping others and being helped was amazing! I enjoy the technical discussions, project management, presenting at various conferences. It is a great fun! I hope you can join us and make MicroProfile a big success in Cloud Native space. Emily Jiang - MPWG Committee

Nice Emily,
I was glad to review your quote today, it made me smile!  Now, the word count was 77 which surpasses the 60 recommended.  Please see below a 2nd draft, disregard whatever you see not fitting to your thoughts.
On your name, it seems to be more powerful is you sign it:  Emily Jiang - MPWG Committer
The Prospectus uses company and labels, individuals quotes aim at welcoming without bodies, ideally, you speak on YOU- Emily the MicroProfile contributor who became a committer through awesome work. The most powerful signature is that!
See below and if you adjust or approve, then, I will move it to the excel sheet. Also, please add your pic into this ticket. We need your preferred pic!
I have been working with the MicroProfile Community from the beginning. Contributing to MicroProfile Config, Fault Tolerance, and later on other specifications, I have thoroughly enjoyed interacting with independent contributors and different companies. The experience of helping others and being helped on technical discussions, project management, and presenting at various conferences is amazing. I hope you can join us in making MicroProfile a big success in Cloud Native space!

word count 69

Thankk you @aeiras for the updated version! I updated slightly.
I have been working with the MicroProfile Community from the beginning. Contributing to MicroProfile Config, Fault Tolerance, and later on other specifications, I have thoroughly enjoyed interacting with independent contributors and different companies. The experience of helping others and getting help from others on technical discussions, project management, and presenting at various conferences is amazing. I hope you can join us in making MicroProfile a big success in Cloud Native space!
Here is my avatar.

Nice work Emily, Ryan is adding your quote today! :)
closing this ticket!


Change File System Plugin

You should change File System Plugin to https://github.com/Aragas/PCLExt.FileStorage
Since current one is very old and doesn't work on Android KitKat and up

I would recommend working to send a PR down to the original repo as I haven't had any issues with PCL storage at all.

Hi, can somebody plese take care of this plugin https://github.com/dsplaisted/PCLStorage ?
It have many issues and some PR. But nobody to maintain it.
Thank you :heart:

Hi, if you want a .NET Standard Storage plugin, please vote here 👍 👍 👍
https://xamarin.uservoice.com/forums/144858-xamarin-platform-suggestions/suggestions/32307739-create-a-net-standard-storage-plugin-for-xamarin

And on top of the System.IO is in .NET Standard 2.0! No need for PCL Storage


Use a trick to speed up travis builds

Idea from https://blog.esciencecenter.nl/travis-caching-and-incremental-builds-6518b89ee889
and tool from https://github.com/MestreLion/git-tools/blob/master/git-restore-mtime
Evidence: https://travis-ci.org/Agda-zh/PLFA-zh/jobs/560033018

Technically there may cause some issues for files modified between a new change and a previous successful travis build...

Happy to give this a shot.


Issue to start a new environnment

I try to create a new environnment but I have a problem :
Traceback (most recent call last):
File "test.py", line 16, in
env = gym.make('trading-v0')
File "/home/nicolas/.local/lib/python3.5/site-packages/gym/envs/registration.py", line 163, in make
return registry.make(id)
File "/home/nicolas/.local/lib/python3.5/site-packages/gym/envs/registration.py", line 121, in make
patch_deprecated_methods(env)
File "/home/nicolas/.local/lib/python3.5/site-packages/gym/envs/registration.py", line 181, in patch_deprecated_methods
env.seed  = env._seed
AttributeError: 'TradingEnv' object has no attribute '_seed'
Versions :
gym (0.9.6)
Does it requires an older version of gym ?

How can I fix this error?
runfile('C:/Users/jalal/Desktop/RL/gym_trading-master/run.py', wdir='C:/Users/jalal/Desktop/RL/gym_trading-master')
Traceback (most recent call last):
File "", line 1, in
runfile('C:/Users/jalal/Desktop/RL/gym_trading-master/run.py', wdir='C:/Users/jalal/Desktop/RL/gym_trading-master')
File "C:\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 705, in runfile
execfile(filename, namespace)
File "C:\Anaconda3\lib\site-packages\spyder\utils\site\sitecustomize.py", line 102, in execfile
exec(compile(f.read(), filename, 'exec'), namespace)
File "C:/Users/jalal/Desktop/RL/gym_trading-master/run.py", line 2, in
import gym_trading
File "C:\Users\jalal\Desktop\RL\gym_trading-master\gym_trading_init_.py", line 5, in
entry_point='gym_trading.envs:TradingEnv'
File "C:\Anaconda3\lib\site-packages\gym\envs\registration.py", line 164, in register
return registry.register(id, **kwargs)
File "C:\Anaconda3\lib\site-packages\gym\envs\registration.py", line 157, in register
raise error.Error('Cannot re-register id: {}'.format(id))
Error: Cannot re-register id: trading-v0


Investigate / fix streaming issues for non-web optimized content

The Issue
Steps to Reproduce
Non-web optmized Mp4s (and possibly other file types) fail to play on lbry web. Example:lbry://ABoyAndHisDog#278f98e230f0c5965a1682fa007db1e920080c8e
The media could not be loaded, either because the server or network failed or because the format is not supported.
System Configuration

LBRY Daemon version:
LBRY App version:
LBRY Installation ID:
Operating system:

Anything Else

Screenshots

This is now fixed with Andrey's api changes...


docs: rename all mentions of eth2-package and eth-network-package to ethereum-package

Where did you see the issue in our documentation?
https://docs.kurtosis.com/how-to-local-eth-testnet
Issue description
Rename/update all mentions of eth-network-package and eth2-package to ethereum-package, including links:

https://docs.kurtosis.com/how-to-local-eth-testnet
https://docs.kurtosis.com/how-to-compose-your-own-testnet
https://docs.kurtosis.com/how-to-full-mev-with-eth2-package
https://docs.kurtosis.com/quickstart
https://github.com/kurtosis-tech/awesome-kurtosis#kurtosis-web3-starlark-packages

and probably other places

https://github.com/kurtosis-tech/kurtosis/pull/1443 pending review!


Struct args in Spanner queries

This will be available soon in spanner client lib. See for more detail: https://github.com/GoogleCloudPlatform/google-cloud-java/pull/3287
Perhaps wait for GA spanner client lib to add this functionality?

We can add it even before to play and provide some feedback if that.
Only the problem that we can't release our GA without their GA.

FYI, this feature is already available https://github.com/GoogleCloudPlatform/google-cloud-java/releases/tag/v0.52.0

@jabubake
How do people usually use the @Param tags with Struct and array types? For Spring Data query methods I think people use these tags on the right hand side of where-statements. like ... where colA = @valA and colB + colC < @val2  .
Users can't put arrays or Structs on the right sides of those where-statements, correct? If that's the case, then I'm not sure how this new struct-arg feature would fit in to spring data spanner..
@meltsufin FYI

https://cloud.google.com/spanner/docs/data-types
Looks like STRUCTs can be used in WHERE clauses with some caveats, so I think this is worth implementing support in our @Query annotated query methods.
The user can provide:

a Struct typed arg, which would be directly bound to the tag like any other tag and param currently supported.
a POJO, which we will use the same logic as our write-conversion for making mutations to construct a Struct to then bind to the tag.


Document the extraction of data from a k8s resource

See behavior mentionned in https://github.com/argoproj/argo-workflows/issues/6097#issuecomment-856166000
Checklist:

[ ] My organization is added to USERS.md.

Tips:

Your PR needs to pass the required checks before it can be approved. If the check is not required (e.g. E2E tests) it does not need to pass
Sign-off your commits to pass the DCO check: git commit --signoff.
Run make pre-commit -B to fix codegen or lint problems.
Say how how you tested your changes. If you changed the UI, attach screenshots.

why the PR does not merge and still says "a review is required" ?

You need to fix DCO and Sematic checks. You can click the failed checks for details on how to fix them.


Sveltekit + Cloudflare adapter - "PrismaClient is unable to be run in the browser."

Bug description
I'm using the latest version of SvelteKit with the Cloudflare adapter.
I'm following along with the tutorial on deploying to cloudflare workers: Deploying to Cloudflare Workers
I made a simple endpoint: /routes/users/[id].ts
The code looks like this:
import type { RequestHandler } from '@sveltejs/kit';
import { PrismaClient } from '@prisma/client';

const db = new PrismaClient();

export const get: RequestHandler = async ({ params }) => {
	const user = await db.user.findUnique({ where: { id: parseInt(params.id) } });

	if (user) {
		return {
			body: { user }
		};
	}

	return { status: 404 };
};

Running locally in dev works fine, but when I deploy with Cloudflare pages, I always get this error:
"PrismaClient is unable to be run in the browser."
It's definitely not running on the browser, since SvelteKit endpoints always run on the backend.
Since SvelteKit uses Vite instead of webpack, I'm not sure if there are any VITE specific steps needed.
How to reproduce

npm init svelte@next my-app # choose typescript
npm install -D prisma
npx prisma init
PRISMA_CLIENT_ENGINE_TYPE=dataproxy npx prisma generate
Add the above code for a SvelteKit endpoint
Add the file .node-version with content: 17.4
Push the code to a repo and publish via Cloudflare Pages

Expected behavior
I expect it to work the same way as when I run npm run dev which runs the query perfectly.
Prisma information
generator client {
  provider = "prisma-client-js"
  previewFeatures = ["dataProxy"]
}

datasource db {
  provider = "mysql"
  url      = env("DATABASE_URL")
}

model User {
  id        Int      @id @default(autoincrement())
  createdAt DateTime @default(now())
  email     String   @unique
  name      String?
}

Environment & setup

OS: Mac
Database: MySQL
Node.js version: v17.4.0

Prisma Version
prisma                  : 3.9.1
@prisma/client          : 3.9.1
Current platform        : darwin-arm64
Query Engine (Node-API) : libquery-engine bcc2ff906db47790ee902e7bbc76d7ffb1893009 (at<EMAIL_ADDRESS>Migration Engine        : migration-engine-cli bcc2ff906db47790ee902e7bbc76d7ffb1893009 (at node_modules/@prisma/engines/migration-engine-darwin-arm64)
Introspection Engine    : introspection-core bcc2ff906db47790ee902e7bbc76d7ffb1893009 (at node_modules/@prisma/engines/introspection-engine-darwin-arm64)
Format Binary           : prisma-fmt bcc2ff906db47790ee902e7bbc76d7ffb1893009 (at node_modules/@prisma/engines/prisma-fmt-darwin-arm64)
Default Engines Hash    : bcc2ff906db47790ee902e7bbc76d7ffb1893009
Studio                  : 0.457.0

This issue is because Prisma currently requires aliasing when you want to run on Workers (using something like esbuild) to work around the fact that Workers look very much like a browser.
SvelteKit's @sveltejs/adapter-cloudflare runs on Pages Functions which don't current allow a build step. I would instead advise you to move it to an external API that is on Workers (and can therefore be bundled and aliased with esbuild).
This issue should probably be kept open regardless though because in ideal circumstances bundling isn't required when using Cloudflare Workers or Functions. The only reason it is currently is that Prisma throws an error because Workers look so much like a browser.

Hey everyone 👋. We have released some changes a few minutes ago in<EMAIL_ADDRESS>tl;dr;

Instead of PRISMA_CLIENT_ENGINE_TYPE, use prisma generate --data-proxy to enable the Data Proxy
import { PrismaClient } from '@prisma/client/edge' for a Workers compatible Prisma Client
Please be aware that .env aren't bundled into the generated Prisma Client any longer (see docs)
You can also remove any package location aliasing from your build scripts
Warnings around eval are now also fixed

one doesn't use Prisma's data proxy

Unfortunately, no. Prisma's Data Proxy allows connecting to a database via HTTP (Prisma proxies HTTP to a regular TCP database connection).
Cloudflare Workers (or Functions) can't make TCP connections directly, so the only way is to use the Data Proxy to connect via HTTP.

I was too stuck at this point. Prisma is great, but unfortunately having another layer of communication & potential cost (Data Proxy) is not an option for our team 🙏🏼.
Cloudflare has added support for TCP using cloudflare:sockets. Hopefully collaboration between prisma & cloudflare teams will help simplify this setup further (https://github.com/prisma/prisma/issues/19488).

Got a native TCP postgres connection working in workers following the guide - prisma client still thinks it's running in a browser and client/edge won't accept non 'prisma://' urls. Roadblocked.

Hey everyone 👋. We have released some changes a few minutes ago in<EMAIL_ADDRESS>tl;dr;

Instead of PRISMA_CLIENT_ENGINE_TYPE, use prisma generate --data-proxy to enable the Data Proxy
import { PrismaClient } from '@prisma/client/edge' for a Workers compatible Prisma Client
Please be aware that .env aren't bundled into the generated Prisma Client any longer (see docs)
You can also remove any package location aliasing from your build scripts
Warnings around eval are now also fixed

I'm really tired, tried everything even the Prisma AI, Chat GPT. But the prisma just doesn't work with cloudflare in my svletekit project.

Here what the Prisma AI said.

The expected behavior is for it to work the same way as when running npm run dev, which runs the query perfectly. However, the error "PrismaClient is unable to be run in the browser" occurs when deploying with Cloudflare pages.
Unfortunately, the knowledge sources do not provide a solution to this issue. For more information, you can refer to the GitHub issues where these problems were reported: Issue 11712 and Issue 12981.


After upgrade to v2.5.0, Dashboard shows only 10 websites out of 36

Describe the Bug
After upgrade to v2.5.0, Dashboard shows only 10 websites out of total 36. It has been showing all prior upgrade for months.
User: admin
Database
PostgreSQL
Relevant log output
No response
Which browser are you using? (if relevant)
Chrome
How are you deploying your application? (if relevant)
Running in docker container, hosted on AWS ECS

Hello are you able to paginate your websites at the bottom? We also provided a search feature so you can filter your websites.

Hi, unfortunatrly, no pager or anything appears below:

however Edit shows all the sites but not the dashboard:

I tried Reset function also but no luck

Oh you're correct, the pager is missing on the dashboard and only displaying top 10 websites. I'm labelling this as a bug, we will get it fixed.

Fixed in v2.6.1.

Thank you!


Dependence of training accuracy on number of CPU nodes

I am training MNIST classifier using the mnist example provided in MXNET repository.
I came to know that convergence is not guaranteed when using asynchronous update of weights from the documentation at following link: Sync vs. Async
With the multi CPU node cluster that I am training MNIST classifier on, I tried once with async update and the other time with sync update of weights, with 5 nodes of t2.micro instances of AWS.
The training starts with a good accuracy of 80+ while using 'dist_sync' and converges. However, it starts and even arrives at only around 10-12% accuracy at the end of 20 epochs, while using 'dist_async' updation of weights.
On the other side, with lower number of cluster nodes, like 4 CPU nodes, both of them are leading to convergence. I am unable to understand clearly, what could be the reason of dependence of number of CPU nodes in cluster for distributed training convergence.
Can anyone please help me understand about this.
MXNET version: 0.9.3

@qiyuangong
Can you please share your insights on this issue.

Actually, I am not an expert in deep learning (I am still trying to be :) ).  Here are some suggestions:
First, your result seems not good enough, even in sync model. Normally, it should be more than 96% with default parameters. Use export PS_VERBOSE=1; your current command to print more details. (PS_VERBOSE=2 will print more)
Second, you can debug locally on your machine, with 'local' model (But you have to kill process manually when they go wrong). The basic workflow is the same as yarn, but run locally in different processes. Aforementioned export PS_VERBOSE=1 also works in local model.
Also, as mentioned by MXNet members in #5422, distributed training is meaningless in small datasets. It seems that they have not added warnings in docs yet.

Thanks for your insights. Will try your suggestions and get back.

This issue is closed due to lack of activity in the last 90 days. Feel free to reopen if this is still an active issue. Thanks!


add 'esc' close overlay.

Maybe   add 'esc' close overlay  will be better

:+1:
As mentioned, navigating through the bookmarks (with tab or up/down arrows) and then pressing ESC works. Would be nice to close the window without any bookmarks selected though. Current behavior feels almost like a bug.


Save and Load JSON files to a external server of the user choice.

This must be configured by the main application, is not part of the Diplomata role.


Fix undefined res.ext for non native parse

For non native path.parse (node <= 0.10) basename resolve to whateverundefined du to res.ext being undefined.

@doowb were you able to take a look at this already?  I can do it now if not.
@pascalduez thanks for the pr

Yeah, see this comment: https://github.com/jonschlinkert/parse-filepath/issues/3#issuecomment-126008448
This change works fine and can be merged. We still need to make updates for the other issue.

k, thanks

still need to make updates for the other issue.

remind me, I don't see any other issues...?

I did PR #4 to update tests and add a .travis.yml file. Will you enable this repo on travis-ci? The tests were only failing on node v0.10.x because path.dirname returns a different value ('' instead of '.') in some cases on node v0.10.x. This keeps the api similar to path.dirname on 0.10.x so it'll work in that environment without doing anything complicated in the actual code here.

Will you enable this repo on travis-ci?

done! sorry I didn't see this earlier. thanks for the pr!


fix list-object-versions with prefix

Coverage remained the same at 93.392% when pulling e1aa42cf7df129a1bf618e5e599d3e6d30b674a8 on thehesiod:thehesiod-list-object-versions into c9e392a5fef902585de12800031c265460be4934 on spulec:master.

Coverage remained the same at 93.392% when pulling e1aa42cf7df129a1bf618e5e599d3e6d30b674a8 on thehesiod:thehesiod-list-object-versions into c9e392a5fef902585de12800031c265460be4934 on spulec:master.

Coverage remained the same at 93.392% when pulling e1aa42cf7df129a1bf618e5e599d3e6d30b674a8 on thehesiod:thehesiod-list-object-versions into c9e392a5fef902585de12800031c265460be4934 on spulec:master.

@thehesiod This is great! It needs a test on this behavior before we can merge it. Mind making an attempt at adding a test for this and if you get stuck just push what you have and ask for help here?
I recommend starting around here: https://github.com/thehesiod/moto/blob/e1aa42cf7df129a1bf618e5e599d3e6d30b674a8/tests/test_s3/test_s3.py#L749

done

Coverage remained the same at 93.392% when pulling b54c26d6c4063deb018a20c31ac951ff6272fe2b on thehesiod:thehesiod-list-object-versions into c9e392a5fef902585de12800031c265460be4934 on spulec:master.

Nice! Great work. Merging.

It's merged. Thanks for the contribution @thehesiod. Welcome to Moto! :)

I'm working on a scenario where I have a SNS topic that's supposed to trigger Lambdas, but I'm guessing lambda execution is not supported as that would require moto using docker or something under the covers.  It's unfortunate there doesn't seem to be a botocore API to be able to get SNS notifications, but I saw the trick of hooking a SQS queue to get them that way.  thanks for this module!

@thehesiod Please share with the Moto community (in an issue or wherever is convenient) whatever setup you come up with. Everybody is trying to solve very similar problems!

logged https://github.com/spulec/moto/issues/1047


Repetion with Seperator Parsing DSL Method

this.SUBRULE(this.AssignmentExpression)
                this.MANY(() => {
                    this.CONSUME(Comma)
                    this.SUBRULE2(this.AssignmentExpression)
                })

is a bit verbose.
this.MANY_SEP(() => {
    this.SUBRULE1(this.AssignmentExpression)
}, Comma)

is a bit more succinct.

In the final version the separator appears before the action
for example:
this.MANY_SEP(Comma, () => {
    this.SUBRULE1(this.AssignmentExpression)
})


Update Linux distributions setup documentation

This pull request fixes an issue: https://github.com/draffensperger/golp/issues/16

Thanks for the PR!


is.gd

is.gd is not working.
Can you add an alternative please? (or 2-3)

FYI bitly requires an API key that you can generate and enter there, or tinyurl should just work. If you have other providers in mind let me know, happy to take a look.


Sync Mysql schema w.r.t. Oracle

As discussed in #6099, here is the PR. Changes are:

as suggested by @yuyiguo , not null was put back for a few cases
set back primary key for dbsbuffer_file_parent (though I don't get why we would need both, I guess only child would be enough)
synced other constraints from oracle to mysql

I also think we should remove the uniqueness from those 4 columns in dbsbuffer_algo, they make no sense for me (though I kept them here).
@yuyiguo @hufnagel you may want to take a look here

Beginning pylint and unit tests.
See https://cmssdt.cern.ch/jenkins/job/DMWM-PR-test/246/ for details

Summary of pylint changes for pull request 6126:

Score for src/python/WMComponent/DBS3Buffer/MySQL/Create.py decreased from 9.20 to 9.17

src/python/WMComponent/DBS3Buffer/MySQL/Create.py got worse in pylint. Report follows:

Line 13 in Create.init W0613 Unused argument 'dbi'
Line 13 in Create.init W0613 Unused argument 'params'
plus 6 comments on code style

Unit test changes for pull request 6126:

WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testBadFWJR changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testSetLocationByLFN changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testAddCKSumByLFN changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testLoad changed from success to error
WMCore_t.WMSpec_t.StdSpecs_t.TaskChain_t.TaskChainTests:testMultithreadedTasksTaskChain changed from error to success
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testMergeSuccessNoLumi changed from success to error
WMComponent_t.TaskArchiver_t.TaskArchiver_t.TaskArchiverTest:testD_Timing changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testBulkLoad changed from success to error
WMComponent_t.TaskArchiver_t.TaskArchiver_t.TaskArchiverTest:testTaskArchiverPollerAlertsSending_notifyWorkQueue changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testAddChildTransaction changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testCreateDeleteExists changed from success to error
WMCore_t.WMSpec_t.StdSpecs_t.DQMHarvest_t.DQMHarvestTests:testDQMHarvest changed from error to success
WMCore_t.WebTools_t.REST_t.RESTTest:testSanitisePassHTTP changed from failure to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testCreateTransaction changed from success to error
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testParentlessMergeSuccess changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testAddChild changed from success to error
WMComponent_t.TaskArchiver_t.TaskArchiver_t.TaskArchiverTest:testTaskArchiverPollerAlertsSending_killSubscriptions changed from success to error
WMComponent_t.AlertGenerator_t.Pollers_t.Couch_t.CouchTest:testAlertGeneratorCouchPollerBasic changed from failure to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testGetParentStatusDAO changed from success to error
WMCore_t.WMBS_t.Subscription_t.SubscriptionTest:testLoad changed from error to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testCountFilesDAO changed from success to error
WMCore_t.WMBS_t.Subscription_t.SubscriptionTest:testWhitelistBlacklist changed from error to success
WMCore_t.WMBS_t.Workflow_t.WorkflowTest:testCreateDeleteExists changed from error to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testSetLocation changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testGetParentLFNs changed from success to error
WMCore_t.WorkQueue_t.WMBSHelper_t.WMBSHelperTest:testCreateSubscription changed from success to error
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testMergedSkim changed from success to error
WMComponent_t.RetryManager_t.RetryManager_t.RetryManagerTest:testZ_Profile changed from error to success
WMComponent_t.DBS3Buffer_t.DBSUpload_t.DBSUploadTest:testCloseSettingsPerWorkflow changed from success to error
WMCore_t.WMBS_t.JobSplitting_t.WMBSMergeBySize_t.WMBSMergeBySize:testMinMergeSize1 changed from error to success
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testFailedJob changed from success to error
WMCore_t.WebTools_t.REST_t.RESTTest:testDAOBased changed from success to error
WMComponent_t.PhEDExInjector_t.PhEDExInjectorPoller_t.PhEDExInjectorPollerTest:test_CustodialSiteA changed from success to error
WMComponent_t.PhEDExInjector_t.PhEDExInjectorSubscriber_t.PhEDExInjectorSubscriberTest:testNormalModeSubscriptions changed from failure to error
WMComponent_t.AlertGenerator_t.Pollers_t.MySQL_t.MySQLTest:testAlertGeneratorMySQLDbSizePollerCriticalThreshold changed from error to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testGetChildrenDAO changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testAddRunSet changed from success to error
WMCore_t.WebTools_t.RESTFormat_t.RESTFormatTest:testReturnFormat changed from success to error
WMCore_t.WebTools_t.NestedModel_t.NestedModelTest:testOuterFooPass changed from success to error
WMCore_t.WMBS_t.JobSplitting_t.FixedDelay_t.FixedDelayTest:testAllAcquired changed from error to success
WMCore_t.WMBS_t.Workflow_t.WorkflowTest:testOutput changed from error to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testLocationsConstructor changed from success to error
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testMergeSuccessSkippedFiles changed from success to error
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testSplitJobs changed from success to error
WMCore_t.WorkQueue_t.WorkQueueSchema_t.WorkQueueSchemnaTest:testSchemaGeneration changed from success to error
WMCore_t.WMBS_t.JobSplitting_t.EventBased_t.EventBasedTest:testMoreEvents changed from error to success
WMCore_t.WMBS_t.JobSplitting_t.SizeBased_t.SizeBasedTest:testMultipleFiles2500 changed from error to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testAddParents changed from success to error
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testReportReturn changed from success to error
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testNoFileReport changed from success to error
WMCore_t.WorkQueue_t.WMBSHelper_t.WMBSHelperTest:testTruncatedWFInsertion changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testProperties changed from success to error
WMComponent_t.JobSubmitter_t.JobSubmitter_t.JobSubmitterTest:testB_thresholdTest changed from error to success
WMComponent_t.DBSUpload_t.DBSUploadPoller_t.DBSUploadTest:testG_closeSettingsPerWorkflow changed from failure to error
WMComponent_t.TaskArchiver_t.TaskArchiver_t.TaskArchiverTest:testC_Profile changed from success to error
WMCore_t.WMBS_t.JobSplitting_t.ParentlessMergeBySize_t.ParentlessMergeBySizeTest:testDifferentSubscritionIDs changed from error to success
WMCore_t.WMSpec_t.StdSpecs_t.MonteCarloFromGEN_t.MonteCarloFromGENTest:testMonteCarloFromGEN changed from error to success
WMComponent_t.TaskArchiver_t.TaskArchiver_t.TaskArchiverTest:testDQMRecoPerformanceToDashBoard changed from success to error
WMCore_t.WMBS_t.Workflow_t.WorkflowTest:testCreateLoadWithUserAtt changed from error to success
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testDBRollback changed from success to error
WMCore_t.WMBS_t.Subscription_t.SubscriptionTest:testListIncompleteDAO changed from error to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testXSetBlock changed from success to error
WMComponent_t.AlertGenerator_t.Pollers_t.Couch_t.CouchTest:testAlertGeneratorCouchErrorsPollerCriticalThreshold changed from failure to success
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testEmptyFWJR changed from success to error
WMComponent_t.AlertGenerator_t.Pollers_t.Couch_t.CouchTest:testAlertGeneratorCouchCPUPollerCriticalThreshold changed from failure to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testDeleteTransaction changed from success to error
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testMergeSuccess changed from success to error
WMCore_t.WorkQueue_t.WMBSHelper_t.WMBSHelperTest:testKillWorkflow changed from success to error
WMComponent_t.DBS3Buffer_t.DBSBufferDataset_t.DBSBufferDatasetTest:testBasic changed from success to error
WMComponent_t.JobAccountant_t.JobAccountant_t.JobAccountantTest:testZ1_BigHeritage changed from success to error

Beginning pylint and unit tests.
See https://cmssdt.cern.ch/jenkins/job/DMWM-PR-test/248/ for details

Beginning pylint and unit tests.
See https://cmssdt.cern.ch/jenkins/job/DMWM-PR-test/249/ for details

No pylint warnings for pull request 6126:

Unit test changes for pull request 6126:

WMCore_t.WMSpec_t.StdSpecs_t.TaskChain_t.TaskChainTests:testMultithreadedTasksTaskChain changed from error to success
WMComponent_t.DBS3Buffer_t.DBSBufferFile_t.DBSBufferFileTest:testSetLocationTransaction changed from error to success
WMCore_t.WMSpec_t.StdSpecs_t.DQMHarvest_t.DQMHarvestTests:testDQMHarvest changed from error to success
WMComponent_t.AlertGenerator_t.Pollers_t.Couch_t.CouchTest:testAlertGeneratorCouchPollerBasic changed from failure to success
WMCore_t.WMBS_t.Subscription_t.SubscriptionTest:testLoad changed from error to success
WMCore_t.WMBS_t.Subscription_t.SubscriptionTest:testWhitelistBlacklist changed from error to success
WMCore_t.WMBS_t.Workflow_t.WorkflowTest:testCreateDeleteExists changed from error to success
WMComponent_t.RetryManager_t.RetryManager_t.RetryManagerTest:testZ_Profile changed from error to success
WMCore_t.WMBS_t.JobSplitting_t.WMBSMergeBySize_t.WMBSMergeBySize:testMinMergeSize1 changed from error to success
WMComponent_t.AlertGenerator_t.Pollers_t.MySQL_t.MySQLTest:testAlertGeneratorMySQLDbSizePollerCriticalThreshold changed from error to success
WMCore_t.WMBS_t.JobSplitting_t.FixedDelay_t.FixedDelayTest:testAllAcquired changed from error to success
WMCore_t.WMBS_t.Workflow_t.WorkflowTest:testOutput changed from error to success
WMCore_t.WMBS_t.JobSplitting_t.EventBased_t.EventBasedTest:testMoreEvents changed from error to success
WMCore_t.WMBS_t.JobSplitting_t.SizeBased_t.SizeBasedTest:testMultipleFiles2500 changed from error to success
WMComponent_t.JobSubmitter_t.JobSubmitter_t.JobSubmitterTest:testB_thresholdTest changed from error to success
WMCore_t.WMBS_t.JobSplitting_t.ParentlessMergeBySize_t.ParentlessMergeBySizeTest:testDifferentSubscritionIDs changed from error to success
WMCore_t.WMSpec_t.StdSpecs_t.MonteCarloFromGEN_t.MonteCarloFromGENTest:testMonteCarloFromGEN changed from error to success
WMCore_t.WMBS_t.Workflow_t.WorkflowTest:testCreateLoadWithUserAtt changed from error to success
WMCore_t.WMBS_t.Subscription_t.SubscriptionTest:testListIncompleteDAO changed from error to success
WMComponent_t.AlertGenerator_t.Pollers_t.Couch_t.CouchTest:testAlertGeneratorCouchErrorsPollerCriticalThreshold changed from failure to success
WMComponent_t.AlertGenerator_t.Pollers_t.Couch_t.CouchTest:testAlertGeneratorCouchCPUPollerCriticalThreshold changed from failure to success

Alan:
The patch looked OK to me. I don't have the completed two create files to compare, but I am sure you already did  the sync.
self.indexes["01_pk_dbsbuffer_file_parent"] =
"""ALTER TABLE dbsbuffer_file_parent ADD
(CONSTRAINT dbsbuffer_file_parent_pk PRIMARY KEY (child, parent) %s)""" % tablespaceIndex
The above PK is needed, only child can not be used as primary key as a primary key requires uniqueness. This is a map table so only one pair of child-parent should be in the table. That is good to be used as PK and one does not have to define an extra PK.
Did you mean to remove below?
self.indexes["02_pk_dbsbuffer_algo"] =
"""ALTER TABLE dbsbuffer_algo ADD
(CONSTRAINT dbsbuffer_algo_unique UNIQUE (app_name, app_ver,
app_fam, pset_hash) %s)""" % tablespaceIndex
An Algo is defined as a group of app_name, app_ver, app_fam and pset_hash. The uniqueness of the group is make sure there is only one group with the same 4 variables will be in the DB. if it is removed, then the same group may be added in the table multiple times. The unique key will make the table smaller and quick searching.
Yuyi

Thanks for reviewing and explaining these things to me Yuyi :-)
Dirk also taught me a few things this afternoon.
I had a few constraint name issues this afternoon, but after fixing a few of them, I managed to deploy an agent (oracle) and so far so good.
I'll test myssql tomorrow and let you guys know about the outcome.

Everything is fine for mysql flavor as well. From my side, we're good to go.


Highlight end scores better

Currently the end scores in "Letzte Spiele" box is shown just as numbers. It would be nice if the scores are highlighted somehow, to quickly see if it's a win, loss or even.
Highlighting it with a background color, similar to the "St. Pauli" highlighting in the first box, might work. More ideas are very welcome.

Please assign this to me

There you go, have fun :)


The cells do not work on the mobile app.

I have the heat map set up and it works on my desktop but when I go to look at the same heat map on my mobile app, the heat map loads but none of the cells populate with the color they should. They just stay empty as if nothing exists data wise. Does anybody else have this issue or am I doing something wrong?

same over here

Same problem


CODE OUTLINE拓展

Issue Type: Feature Request
请将CODE OUTLINE拓展内置到VSCODE中，去掉右侧可拖动总览划栏，替换成CODE OUTLINE分栏，请参考eclipse
VS Code version: Code 1.22.2 (3aeede733d9a3098f7b4bdc1f66b63b0f48c1ef9, 2018-04-12T16:38:45.278Z)
OS version: Windows_NT x64 6.1.7601

Please post your question in English.
Can you attach a screenshot from Eclipse for reference?

Thank you very much for improving this feature, like your efficiency
============================================================
Best Regards!
黄友强
Job:  福建省福州市趣读网络科技（研发部）
Call:<PHONE_NUMBER>0 Email<EMAIL_ADDRESS>vvvvvvvvvvvvvvvvvvvvvvvvvvvv

------------------ 原始邮件 ------------------
发件人: "vscodebot[bot]"<EMAIL_ADDRESS>发送时间: 2018-04-17 01:27:26
收件人:Microsoft/vscode<EMAIL_ADDRESS>抄送:黄友强<PHONE_NUMBER>0@139.com,Author<EMAIL_ADDRESS>主题: Re: [Microsoft/vscode] CODE OUTLINE拓展 (#47943)
Closed #47943.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub, or mute the thread.


Adding a node breaks microk8s

After adding a node to a MicroK8s cluster, I started getting connection-related errors on each invocation of the microk8s kubectl get command.
Examples run on both nodes:
$ microk8s kubectl get pod
Unable to connect to the server: net/http: TLS handshake timeout

$ microk8s kubectl get node
Get "https://<IP_ADDRESS>:16443/api/v1/nodes?limit=500": dial tcp <IP_ADDRESS>:16443: connect: connection refused - error from a previous attempt: read tcp <IP_ADDRESS>:60964-><IP_ADDRESS>:16443: read: connection reset by peer

It seems to randomly alternate between those two distinguished errors.
But I noticed that all services were working as expected, because I was able to access them via a browser.
Firewall is not enabled on either of machines. And both machines have static public IPs.
My guessed issue was that the master-node's hostname wasn't matching the fully qualified domain name, but fixing that didn't help.
Then I rebooted the machine and that's when services failed to start: I can't access the corresponding web sites anymore.
microk8s status returned microk8s is not running. Use microk8s inspect for a deeper inspection.
Here's the output of microk8s inspect:
Inspecting system
Inspecting Certificates
Inspecting services
  Service snap.microk8s.daemon-cluster-agent is running
  Service snap.microk8s.daemon-containerd is running
  Service snap.microk8s.daemon-kubelite is running
  Service snap.microk8s.daemon-k8s-dqlite is running
  Service snap.microk8s.daemon-apiserver-kicker is running
  Copy service arguments to the final report tarball
Inspecting AppArmor configuration
Gathering system information
  Copy processes list to the final report tarball
  Copy disk usage information to the final report tarball
  Copy memory usage information to the final report tarball
  Copy server uptime to the final report tarball
  Copy openSSL information to the final report tarball
  Copy snap list to the final report tarball
  Copy VM name (or none) to the final report tarball
  Copy current linux distribution to the final report tarball
  Copy network configuration to the final report tarball
Inspecting kubernetes cluster
  Inspect kubernetes cluster
Inspecting dqlite
  Inspect dqlite

WARNING:  Docker is installed.
File "/etc/docker/daemon.json" does not exist.
You should create it and add the following lines:
{
    "insecure-registries" : ["localhost:32000"]
}
and then restart docker with: sudo systemctl restart docker
Building the report tarball
  Report tarball is at /var/snap/microk8s/4595/inspection-report-20230214_091301.tar.gz

I attached the tarball.
inspection-report-20230214_091301.tar.gz
As a note, it happened to me in the past, but then I decided to just nuke the server and recover it from a backup.

Hi @m-ocean-it,
I see the following in the inspection report you sent:
# network/ip-addr
default via <IP_ADDRESS> dev eth0 proto dhcp src <IP_ADDRESS> metric 100
<IP_ADDRESS>/24 dev eth0 proto kernel scope link src <IP_ADDRESS> metric 100
<IP_ADDRESS> dev eth0 proto dhcp scope link src <IP_ADDRESS> metric 100
<IP_ADDRESS> dev eth0 proto dhcp scope link src <IP_ADDRESS> metric 100

# dqlite/cluster.yaml
- Address: <IP_ADDRESS>:19001
  ID:<PHONE_NUMBER>608546238
  Role: 0

When joining a control plane, MicroK8s attempts to rebind dqlite (the etcd-compatible datastore) from <IP_ADDRESS>:19001 to $IP_ADDRESS_USED_IN_MICROK8S_JOIN:19001, so that the dqlite nodes can communicate and form a cluster.
The problem here arises from the fact the public IP is not known on the host (I suppose it's a floating IP), so starting dqlite fails:
Feb 14 09:11:11 kubernetes.bossa-nova-server.website microk8s.daemon-k8s-dqlite[70124]: time="2023-02-14T09:11:11Z" level=fatal msg="Failed to start server: listen to <IP_ADDRESS>:19001: listen tcp <IP_ADDRESS>:19001: bind: cannot assign requested address\n"
Feb 14 09:11:11 kubernetes.bossa-nova-server.website systemd[1]: snap.microk8s.daemon-k8s-dqlite.service: Main process exited, code=exited, status=1/FAILURE
Feb 14 09:11:11 kubernetes.bossa-nova-server.website systemd[1]: snap.microk8s.daemon-k8s-dqlite.service: Failed with result 'exit-code'.
Feb 14 09:11:11 kubernetes.bossa-nova-server.website systemd[1]: snap.microk8s.daemon-k8s-dqlite.service: Consumed 1.558s CPU time.
Feb 14 09:11:12 kubernetes.bossa-nova-server.website systemd[1]: snap.microk8s.daemon-k8s-dqlite.service: Scheduled restart job, restart counter is at 643.

Can the two nodes communicate with each other via the internal network? <IP_ADDRESS>/24? In that case, this is the IP address you should use for joining the cluster.
There are ways to salvage the current cluster, but it might be better to start from a clean setup. Let me know and we can discuss in the issue. See here for a full scope https://microk8s.io/docs/restore-quorum, but it might be simpler in your situation.
You do not see any effect on the services, as any pods/workloads on the cluster will keep running without issues. Only the Kubernetes control plane is affected (basically, the datastore cannot come up).
An immediate work-around would be to join any new nodes as worker-only nodes (microk8s join $URL/$TOKEN --worker), so that dqlite keeps running on the first node only.
If I may ask, what cloud/environment/use case is this? Thanks!
I would be willing to mark this as a bug somehow. MicroK8s should most likely prevent dqlite from re-binding to an IP address if that address is not known to the host.

Thanks for responding!

The problem here arises from the fact the public IP is not known on the host

Which host do you mean? The node that is joining or the one that is being joined? And what do you mean by "known"? Judging by the logs you mentioned, you're talking about the original node (because I assigned a kubernetes subdomain to it).

I suppose it's a floating IP

Both machines have static IPs.

Can the two nodes communicate with each other via the internal network?

Nope, they exist in different networks.

If I may ask, what cloud/environment/use case is this?

I have a one-node cluster running on the cloud and I want to add our local machines to the cluster  in order to reduce spending while allowing more scaling.

An immediate work-around would be to join any new nodes as worker-only nodes

Interestingly, that doesn't work either. As soon as I launch the join command, it breaks the control-plane in the same way.
Another thing I noticed is that the add-node command gives only one IP address as an option. The same IP is in the alternative interface:
From the node you wish to join to this cluster, run the following:
microk8s join <IP_ADDRESS>:25000/...

Use the '--worker' flag to join a node as a worker not running the control plane, eg:
microk8s join <IP_ADDRESS>:25000/... --worker

If the node you are adding is not reachable through the default interface you can use one of the following:
microk8s join <IP_ADDRESS>:25000/...

I can't find any help on adding a node from a different network. It feels like I should somehow enable such interface.

Alright, here's what worked. I created a VPN using TailScale and added just one line to the /etc/hosts file on the master node, and then I was able to easily form a cluster. So, for now, I've worked around the problem of machines being in separate networks.

@m-ocean-it  I am also having the same issue when I try to join two nodes existing in different network over a public IP by having firewall disabled and allowed all port communication between these two nodes.
I was successfully able to join the nodes, but somehow the DNS resolution in the pods running on the second node as worker node are getting failed, the reason is just like how the join command always provides the internal private IP to join, and replacing it with public IP solved the joining part, but somehow the Calico or CoreDNS doesn't able to communicate using public IP between nodes, they are trying via local IP only.
I even tried suggested solution here - https://github.com/canonical/microk8s/issues/2402#issuecomment-2089675284
No help yet.
I tried with Ubuntu 20.04, 22.04, 23.10, 24.04 with Microk8s 1.29/stable, 1.30/stable, 1.30/edge with all possible combination.
I tried by creating two different virtual networks for two different virtualmachines having static public IPs assigned to them.


Insights for lazy loading in Angular 2

Are you see that implementantion?
http://blog.mgechev.com/2015/09/30/lazy-loading-components-routes-services-router-angular-2/

Well, this is quite easy to implement. This basic article is use SystemJS to load modules async (which in our case replaced by webpack), so any difference - is how you will load modules.
For webpack you may need something like '@angularclass/webpack-toolkit' which provide you a toolkit for lazy load. Take a look on _https://github.com/AngularClass/angular2-webpack-starter/blob/master/src/app/app.routes.ts_ this one. You only need to make sure that it will fit in your current setup.
If you think about to lazy load already preload component, than it possible, but for sure NOT reasonable.

The AngularClass/angular2-webpack-starter is a good referencial project. May we put the webpack-toolkit for lazy load in this project at Yeoman generator.
 It is not interesting I guess.

It good option but adding it to template will enforce everyone to use it. This is not quite what everyone needed, just some of them :), so probably may be article of how to do so be enough?

Nice, an article is enough. I missing also, some tutorial to add a Gulp tools or something like that to run some tasks like a concat js and css files.
Thanks!

With webpack you don't actually need for such tool. Just create proper bundle

Thanks @alexsandro-xpt!
I did read that post (http://blog.mgechev.com/2015/09/30/lazy-loading-components-routes-services-router-angular-2/) a long time ago when trying to figure out whether we could do lazy loading in the Angular 2 template. The mechanism described there looks excessively complex and risky to be used in a general-purpose template as this, plus I don't know how well it still applies to current builds of Angular 2, which have changed enormously since September 2015 when that post was written.
Hopefully the final version of Angular 2 will have a supported native method of doing lazy loading!


Merge Scaling For Better Results

Experimental tests for testing scaling with merge against equivalent fully generated data

Added per-test factors (multipliers) for scaling up individual benchmarks; row, static, incremental

Typical run time achieved locally for each benchmark operation is ~8 secs or more

The most interesting class for review would be StandardTestRunner, which conditionally uses DH merge

The vast majority of changed files in this PR have been changed to set scale for individual benchmark operation

SelectTest was failing against 0.24.0 because of https://github.com/deephaven/deephaven-core/pull/3806. Updated "Release" workflow to use 0.24.1 instead.


Ssh options

related to https://github.com/RackHD/on-taskgraph/pull/222
@RackHD/corecommitters  @pscharla
We needed to make the options for the underlying ssh.exec method accessible specifically to ensure that the chef install workflow can function when a pseudo-tty is required

+1


difficulties of  Buddhist era

introduction
you can visit this page to use the most accurate buddhist era in the world
for the details of code, can visit code link on github
this article will introduce the details and difficulties of my developing this calendar, I hope it can provide some idea to other developers.

item
note

time in single day
dawn, noon and sunset is the related to the yogi everyday precept

month named bylunar mansion nakkhatta
this can show the name of month and important day

year named by zodiac
this is srilanka tradition, and the Zodiac order is close to Chinese tradition

dawn and noon time
dawn time
code is very simple but accurate
first ,we give the time and space data to  date_sun_info function, then we can get the information of the sun in that place and that time
grammar
date_sun_info(timestamp,latitude,longitude);
$sun_info=date_sun_info(strtotime("now"),7.738562,80.519675);
$Unix_dawn=($sun_info['civil_twilight_begin']-$sun_info['civil_twilight_begin']+$sun_info['nautical_twilight_begin'])*1000;

this is php code, it also realized in JavaScript on  the page which can be downloaded and used.
if you are no intested in the procedure, pls ignore the following part.
basic knowledge
the dawn can categorized in 3 kinds:

type
angle of sun below the horizon
note

Civil dawn
0°~6°
civil people can see the object without artificial light sourcenormally consider as the start of day time

nautical dawn
6°~12°
civil people cannot see the object clearly without artificial light sourcebut on the sea mariners can see objects

Astronomical dawn
12°~18°
mariners cannot see the object clearly on the seait also show the end/start of Astronomical observing

for the detail can check DAWN in wikipedia

how to calculate 6°
according to pāli text, in the dawn time, there is light in 4 directions, that means the civil twilight.
So at the starting point, the sun is 6° below the horizon in the morning.
That means dawn time is 6° earlier than sunrise.
How long is 6° for the sun pass by?
Normally we will think: the sun go 360° is 24 hours,
pass time=24hr/360°×6°=0.4hr=24mins

But we cannot ignore the height of the sun at midday.
Only if the height=90°, then the time is 24 mins.
That means even the tropical zone, there are only 2  ideal days in one year.
For other day, all will be longer than 24 mins.
That's why some monastery just minus 30 or 40 mins from the sunrise, it simple but inaccurate.
set α as the height of the sun at midday on one single day,
real-period=24min/sinα

It seemed complicated, right?
but in php, we can find a function called date_sun_info(), grammar:
date_sun_info(timestamp,latitude,longitude);

example:
$sun_info=date_sun_info(strtotime("now"),7.738562,80.519675);
$Unix_dawn=$sun_info['civil_twilight_begin']*1000;

then we can get a array contains 6 kinds of twilight:
advance accurate: astronomical refraction
the deviation of light or other electromagnetic wave from a straight line as it passes through the atmosphere due to the variation in air density as a function of height.
Atmospheric refraction of the light from a star is zero in the zenith, less than 1′ (one arc-minute) at 45° apparent altitude, and still only 5.3′ at 10° altitude; it quickly increases as altitude decreases, reaching 9.9′ at 5° altitude, 18.4′ at 2° altitude, and 35.4′ at the horizon; all values are for 10 °C and 1013.25 hPa in the visible part of the spectrum.

35.4′ will cause 3~5 mins earlier.
real-period=35.4‘/60/360×60min/hr×24hr/sinα=2.36min/sinα

Don't worry, even php didn't consider astronomical refraction in civil_twilight_begin,
but the sunrise already consider the astronomical refraction,
$sun_info=date_sun_info(strtotime("now"),7.738562,80.519675);
//6° period
$6_period=$sun_info['civil_twilight_begin']-$sun_info['nautical_twilight_begin']
//astronomical refraction
$a_r=$6_period-($sun_info['sunrise']-$sun_info['civil_twilight_begin'])
$real_dawn=$sun_info['civil_twilight_begin']-$a_r

to simplify and change the time stamp for sec into ms
final code
$sun_info=date_sun_info(strtotime("now"),7.738562,80.519675);
$Unix_dawn=($sun_info['civil_twilight_begin']-$sun_info['civil_twilight_begin']+$sun_info['nautical_twilight_begin'])*1000;

noon time
lunar mansion(nakkhatta) and nakshatra month(māsa)
zodiac name of year
the Chinese zodiac and Buddhist era zodiac is very similar
the difference is the last day of zodiac year.
In Chinese lunar calendar: consider the spring festive eve is the last day and spring festive is the first day.
In Buddhist era : consider the vesākha full moon day is the last, and following day is the first day.
EX

Do you know about Buddhist Sun apk on the play and ios store?
I can maybe update to include calendars as well.


Allow excluding archived repositories

Fixes #224

👋 @natete, Thanks for your PR! This is a great feature for this action. Could you add the example Output in github action workflow file? 😁


Experiencing interrupted exceptions while trying to do getAll() on RMapCache with size 500

Hi,
currently I have like 0.3M records in RMapCache and I need to get all these entries for some processing. I tried to use get(key), it works sometime and fails intermittently with Interrupted Exception. So, tried to list all keys for 0.3M and doing getAll(list).
I'm partitioning my 0.3M list into sublists of size 500 and trying to do getAll() but I'ms till getting interrupted exceptions continuosly.
I don't have any other job trying to access the same cache but it is till failing.
I also see loading of entries into RMapCache was successful and I see all the entries in the cache but when I do get(key)/getAll(list), it is failing intermittently.
org.redisson.client.RedisException: java.lang.InterruptedException
	at org.redisson.command.CommandAsyncService.get(CommandAsyncService.java:113)
	at org.redisson.RedissonObject.get(RedissonObject.java:82)
	at org.redisson.RedissonMap.getAll(RedissonMap.java:571)
	at com.oracle.pic.xray.cache.testCache.lambda$getAll$2(testCache.java:111)
	at java.lang.Iterable.forEach(Iterable.java:75)
	at com.oracle.pic.xray.cache.testCache.getAll(testCache.java:108)

It seems that threads invoking Redisson are got interrupted. You need to check app server threads management.


Page Cache to Stash Cache

I feel there could be some major benefits to moving the page cache to the stash library. There was small discussion on pr #2492
There was some argument that varnish might not be a great fit into the model of stash, but I think it would be fairly trivial to implement a varnish driver which checks to makes sure it's acting on Collection objects. If it isn't, that's fine it just doesn't save to varnish cache, and if it does, it can easily act on the cache appropriately when the cache object is added / removed.
By leveraging stash we not only allow for the flexibility of additional drivers, but also driver pools and tiered caching levels (not to mention we'd already have numerous caching drivers available to us then to use).

Work in progress at #2829


Formatting Vue SFC files error with<EMAIL_ADDRESS>

Your Environment

Prettier version: 3.2.5
node version: 20.11.1
package manager: [npm@7, pnpm@6, yarn@2] <EMAIL_ADDRESS>IDE: [VScode, Webstorm, CLI] vscode

Describe the bug

An error appear with<EMAIL_ADDRESS>works well with<EMAIL_ADDRESS>To Reproduce

I made a respository which can reproduce this.
https://github.com/shalldie/sort-import-err-in-compiler-sfc-2.7.16
And here's the steps (also in README.md):
Steps to reproduce
Case<EMAIL_ADDRESS>Now is<EMAIL_ADDRESS>
pnpm i
format the file src/App.vue
it shows error:

TypeError: Cannot read properties of undefined (reading 'match')

Case<EMAIL_ADDRESS>
Edit package.json to change the version to<EMAIL_ADDRESS>run pnpm i
format the file src/App.vue
It works well.

Contribute to @ianvs/prettier-plugin-sort-imports

[ ] I'm willing to fix this bug 🥇

@shalldie I've released version 4.3.0 which fixes the bug you reported here. Please try it out and let me know if it's working correctly for you now. And thanks for the report with the reproduction, that was very helpful!

4.3.0 does a good work !


Change Location to a multi-select option

In Search, Location should allow multiple options to be selected. Check boxes (like Impacts) or a multi-select drop down list would work, but gets very long once multiple locations are in the DB. Maybe an option to expand the list with check boxes at each level would work and create a better UX.
World

United Kingdom

...

United States

Arizona

...

Ohio

No, a person cannot be in two locations at the same time.  The search already is ancestor-search.   For example, if the hierarchy was:   world-usa-arizona-pima-tucson.    A search for "tucson' would automatically search all parent governments.  A search for Arizona would search world/usa/arizona only, and not search pima or tucson.
Multiple locations would be a major re-write of the application.


DL from radio station's xml-feed leads into errors rather than dowloads

Hi lightpohl,
thank you so much for this great tool! I really do enjoy every update, so cheers again.
I don't use too many different podcasts to download, but I keep running into issues with this one:
https://www.ndr.de/podcast/podcast4336.xml
I get:
(node:5527) UnhandledPromiseRejectionWarning: TypeError [ERR_INVALID_URL]: Invalid URL: undefined     at new NodeError (internal/errors.js:322:7)     at onParseError (internal/url.js:270:9)     at new URL (internal/url.js:346:5)     at /snapshot/dist/podcast-dl-9.3.2.js     at processTicksAndRejections (internal/process/task_queues.js:95:5) (Usepodcast-dl --trace-warnings ...to show where the warning was created) (node:5527) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). To terminate the node process on unhandled promise rejection, use the CLI flag--unhandled-rejections=strict(see https://nodejs.org/api/cli.html#cli_unhandled_rejections_mode). (rejection id: 1) (node:5527) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.
and --trace-warnings is an "unknown option". Hm.
The problem is most likely the human sitting at my keyboard, but he'd also be very grateful for a hint in which direction to go for the xml-feed to work.
Thanks and all the very best!

Thank you for the bug report! The RSS parser couldn't find the feedUrl on its own, but I added a fallback that should get that feed working for you. 👍
The fix is now available in the latest release! https://github.com/lightpohl/podcast-dl/releases/tag/v9.3.3

Thank you so much, lightpohl, it runs smoothly and makes me very happy. :smile:


Making merge_fields actually optional

Not including the parameter merge_fields in create_embedded_template_draft results in error about 'merge field has undefined name'. This appears to be caused by the merge fields being added to the parameter list even when it is not provided.

Looks good!


Cleanup assignments app after the refactoring of the users app

rename 'person' to 'user'
get_participants (with queryset)
see, that the users are sorted
change id to pk in some templates

all done


⚠ Bofrost.lu has degraded performance

In 6b98bda, Bofrost.lu (https://www.bofrost.lu) experienced degraded performance:

HTTP code: 200
Response time: 1259 ms

Resolved: Bofrost.lu performance has improved in e155c3c after 7 minutes.


poetry: update dependencies

Description of changes
Diff: https://github.com/python-poetry/poetry/compare/refs/tags/1.3.2...1.4.0
Changelog: https://github.com/python-poetry/poetry/blob/refs/tags/1.4.0/CHANGELOG.md

Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
23.05 Release Notes (or backporting 22.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Don't you also need to update pkgs/development/python-modules/poetry-core/default.nix to 1.5.1?
Here's my diff:
diff --git a/pkgs/development/python-modules/poetry-core/default.nix b/pkgs/development/python-modules/poetry-core/default.nix
index 82438a69545..e94a68e9dec 100644
--- a/pkgs/development/python-modules/poetry-core/default.nix
+++ b/pkgs/development/python-modules/poetry-core/default.nix
@@ -15,7 +15,7 @@

 buildPythonPackage rec {
   pname = "poetry-core";
-  version = "1.4.0";
+  version = "1.5.1";
   format = "pyproject";

   disabled = pythonOlder "3.7";
@@ -23,8 +23,8 @@ buildPythonPackage rec {
   src = fetchFromGitHub {
     owner = "python-poetry";
     repo = pname;
-    rev = version;
-    hash = "sha256-SCzs2v0LIgx3vBYTavPqc7uwAQdWsdmkbDyHgIjOxrk=";
+    rev = "refs/tags/${version}";
+    hash = "sha256-h3d0h+WCrrNlfPOlUx6Rj0aG6untD6MiunqvPj4yT+0=";
   };

   propagatedBuildInputs = lib.optionals (pythonOlder "3.8") [

Already on 1.5.1 on the target branch, which is staging-next.

Builds on both darwin after rebasing on a more recent staging-next version.

Ah, sorry for the noise. My local staging-next branch was not synced although I thought it already was.


upgrade webpack from 3.10 to 4.6.  update configs to webpack 4.

analyze bundle size when running locally.  takes advantage of webpack 4 changes wrt plugins.  specifies mode (now set to 'production' by default).  webpack-cli split out and must be imported separately.
[ Configuration
You no longer need to use these plugins:
NoEmitOnErrorsPlugin -> optimization.noEmitOnErrors (on by default in production mode)
ModuleConcatenationPlugin -> optimization.concatenateModules (on by default in production mode)
NamedModulesPlugin -> optimization.namedModules (on by default in develoment mode)
CommonsChunkPlugin was removed -> optimization.splitChunks, optimization.runtimeChunk
]  https://github.com/webpack/webpack/wiki/Changelog-WIP

Codecov Report

Merging #140 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #140   +/-   ##
=======================================
  Coverage   61.66%   61.66%
=======================================
  Files          95       95
  Lines        1414     1414
=======================================
  Hits          872      872
  Misses        542      542

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8c4d5dc...11940fe. Read the comment docs.

Coverage remained the same at 57.314% when pulling 11940fe377b4e0418e9c24f2261acfd0748bc91c on RedSquirrelious:webpack46 into 8c4d5dcfeb1ab6f2d5a4705d2c3adbfb5919ec98 on Azure:master.


Cant seem to get your migration OA.Persistence to work

I cant seem to get your migration to work and was wondering if there is a 3rd way for creating the database model for this project?  ty.

hi @bgard68 ,
Did you followed below steps for migration?
https://github.com/Amitpnk/Onion-architecture-ASP.NET-Core#step-6-create-database-sample-is-for-microsoft-sql-server
Could you please share error message?

Yes I tried all those steps but says datasource is unrecognized.
On Sat, Nov 21, 2020, 12:42 AM Amit P Naik<EMAIL_ADDRESS>wrote:

hi @bgard68 https://github.com/bgard68 ,
Did you followed below steps for migration?
https://github.com/Amitpnk/Onion-architecture-ASP.NET-Core#step-6-create-database-sample-is-for-microsoft-sql-server
Could you please share error message?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/Amitpnk/Onion-architecture-ASP.NET-Core/issues/30#issuecomment-731518447,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AHHEI4RCHBUWRO3QQYJ4GC3SQ5ONVANCNFSM4TV5XTAA
.

Refer below image for reference, make sure you are using MS SQL server
https://github.com/Amitpnk/Onion-architecture-ASP.NET-Core/blob/develop/docs/img/Step4.png


Would the Python API be supported?

I’m thinking about creating a generic package builder that supports both PEP 517 and legacy projects. But I want to be able to run the builder “out of band” i.e. the package may be built against a different interpreter than the one running the tool. Both PackageBuilder and IsolatedEnvironment seem to be very well implemented and modulised (much kudos) to make this possible, and my plan is to subclass and override limited parts of them. But would this be a supported usage going forward, or should I expect things to break (like pip’s implementation)?

Yes, it is public API so you can rely on it. This was exactly my goal when designing.

Thanks! I’ll try it out and report back if anything is missing.

Please do :grin:! Just be aware that IsolatedEnvironment is unreleased, although I don't think we will need to change the public API. It currently can't handle environment markers, #61 should fix it.

I am aware, thanks. And install is one of the functions I need to override anyway, so the environment marker issue is not a problem for me 😛


Use HTML parser to scan the file type as it's shown to user

Description

Add HTML files scan in text representation and tables research. Line numbers may be wrong.

How has this been tested?

[x] UnitTests
[x] place a password in table with named header "password" - the password should be found

Thank you for implment html parser feature. And I've checked it and working good.
BTW I think we need some refactoring for DeepContentProvider. It's too huge now and app.py has a lot of if-else chain cry Let's think about refactoring next time.
LGTM +1

https://github.com/Samsung/CredSweeper/pull/270 - some refactoring was made.


Relationship profile line should be solid, not dashed

@jakubklimek Can you please list all types of edges and their required visual? From top of my head we have:

association
generalization
profile or a class / profile
profile of an association

Seems correct.

relationship - full line, arrow at the end
generalization - full line, empty triangle at the end (needs to be changed - there were doubts if possible in the library)
profile of - dashed line, empty triangle at the end
profile of association - full line, arrow at the end

For now this is adressed using CSS for the empty arrows.
Without it we need to introduce custom mark definitions like in demo.
I would pospone this as a part of bigger rewrite of edges together with #548.


Distinguishing Laravel release versions in packages

I've got a package that I'd like to update to be compatible with 5.0. There are small differences with the way packages are registered, but beyond registration, my package will remain unchanged. I suppose I could just make a breaking change and bump the minor version, but my guess is that people will hang on L4 for a while longer, and it'd be cumbersome to maintain two versions where the only difference is package registration.
So, I thought about how to solve it. Parsing composer files is nasty, but Illuminate\Foundation\Application has a VERSION constant. Yay! Except, Illuminate\Foundation is no longer a separate composer package that I can list as a dependency. Boo! (I think it was back in L3 days, but no longer). My next thought is that the application version would get into Illuminate\Support\ServiceProvider since every package would use this class. This requires an extra release step, but then any package could know if it's running in 5.0 or not.
Anyway, I wanted to see if we could get a conversation going around ways that packages can find out which version of Laravel they are running in. It seems like a small, but potentially very useful way to maintain packages without having to explicitly support two releases.

Just bump your major version, example, version 1 is for Laravel 4 and version 2 is for Laravel 5, simple.

One of the reasons I mentioned above for bringing this up is to not have to maintain two releases when the only thing that's different is the package registration or perhaps one or two other small changes (like location of the migrations directory). If I knew the Laravel version in my package, then I can make the small forks in logic required.

While it's simple, the recommended way is to have 2 separate versions, one for each major Laravel version.
If you don't want multiple versions, then you'll need to check the Laravel version on your Service Provider and apply the changes accordingly.

To get the version of Laravel on your Service Provider, do something like:
$app = $this->app;

$version = $app::VERSION;

or
$version = \Illuminate\Foundation\Application::VERSION;

As I mentioned above, doing so requires that my package requires laravel/framework as a dependency since \Illuminate\Foundation\Application is not available otherwise. It seems like overkill when normally a package only requires illuminate/support. It's not that I can't do it in the ways you've mentioned, it's that I'm trying to convey that these two choices -- either maintaining multiple, simultaneous releases or including the entire framework as a dependency of a package -- don't seem like they should be the only choices. Including the version in illuminate/support somewhere would be pretty simple and would give the benefit to the community of package makers of not necessarily having to maintain multiple versions of packages.
The reason this seems like an issue to me is because package registration changed with L5, but not necessarily other things about packages. Many package internals are the same, it's just the registration that has to be updated. This could be an easy solution.
If the answer is "yo, if you want the Laravel version, depend on laravel/framework, otherwise get ready to maintain multiple releases" then so be it. But I thought it was at least worth some discussion of alternatives.

You have to make your own decision. It depends on the package. If it's just the package method, perhaps you can live with that. But most packages also use config (which have changed), depend on interfaces (which are now different contracts), have blade views (new escaping behavior), publish resources (different commands/setup) etc.
I would say, start supporting 5.0 for new features. Keep the old version for bugs/security-fixes. Taylor also suggested to bump the major version (https://twitter.com/taylorotwell/status/512050601115414528)
But this isn't the place for a discussion, better use the forums/IRC


Try-with-Resources Not Closing Database Connections for One Select Query

Facing an issue with try-with-resources as it is not closing the database connections automatically particularly for one select query while working for other queries of the same class within a java application. Please suggest what could be the issue here.
Steps to Reproduce
Execute the problematic select query using the try-with-resources statement.
Monitor the database connections to observe that the connection is not closed.
Execute other queries in the same class to confirm that their connections are closed properly.
Expected Behavior
The database connection should be automatically closed after the try-with-resources block for all queries.
Actual Behavior
The database connection remains open for one specific select query, causing potential resource leaks.

Any comments please??

@baliyanshilpa04 why do you think connection should be closed? Please provide a configuration and test that reproduces behaviour described

@baliyanshilpa04 HikariCP reuses connections. When you call the method close you will return the connection to the pool, but the physical connection to the database will still exist.
Look at com.zaxxer.hikari.pool.ProxyConnection for more details.


Consider UTF-8 as the default HTTP encoding

Use case
Sending unicode data should not corrupt it.
Possible solution
We might consider using UTF-8 by default instead of the current ISO_8859_1.
See https://github.com/apache/jmeter/blob/13793b74d96576731062e83b3c365b2adbe6a8df/src/protocol/http/src/main/java/org/apache/jmeter/protocol/http/sampler/PostWriter.java#L52
See

https://html.spec.whatwg.org/multipage/form-control-infrastructure.html#selecting-a-form-submission-encoding (If candidate encodings is empty, return UTF-8

Possible workarounds
No response
JMeter Version
5.5
Java Version
No response
OS Version
No response

I wonder if following lines (the information about the default encoding) in jmeter.properties are still valid based on this change:
The encoding to be used if none is provided (default ISO-8859-1)
#sampleresult.default.encoding=ISO-8859-1

Good catch. I somehow missed that org.apache.jmeter.samplers.SampleResult#DEFAULT_HTTP_ENCODING was used as a fallback for org.apache.jmeter.samplers.SampleResult#DEFAULT_ENCODING.


Refactor and enhance issue indexer to support both searching, filtering and paging

Fix #24662.
Replace #24822 and #25708 (although it has been merged)
Background
In the past, Gitea supported issue searching with a keyword and conditions in a less efficient way. It worked by searching for issues with the keyword and obtaining limited IDs (as it is heavy to get all) on the indexer (bleve/elasticsearch/meilisearch), and then querying with conditions on the database to find a subset of the found IDs. This is why the results could be incomplete.
To solve this issue, we need to store all fields that could be used as conditions in the indexer and support both keyword and additional conditions when searching with the indexer.
Major changes

Redefine IndexerData to include all fields that could be used as filter conditions.
Refactor Search(ctx context.Context, kw string, repoIDs []int64, limit, start int, state string) to Search(ctx context.Context, options *SearchOptions), so it supports more conditions now.
Change the data type stored in issueIndexerQueue. Use IndexerMetadata instead of IndexerData in case the data has been updated while it is in the queue. This also reduces the storage size of the queue.
Enhance searching with Bleve/Elasticsearch/Meilisearch, make them fully support SearchOptions. Also, update the data versions.
Keep most logic of database indexer, but remove issues.SearchIssueIDsByKeyword in models to avoid confusion where is the entry point to search issues.
Start a Meilisearch instance to test it in unit tests.
Add unit tests with almost full coverage to test Bleve/Elasticsearch/Meilisearch indexer.

I think it's safer to have a separate PR for "fix: LoadAttributes context", to avoid potential conflicts (because other PRs may also need such change)

I think it's safer to have a separate PR for "fix: LoadAttributes context", to avoid potential conflicts (because other PRs may also need such change)

ref #26055

#26128 is ready to review.

@lunny I think it should wait for the author to take a look at these review comments first, before the unnecessary "urgent" wait-merge.

The NoConditionID and sometimes MilestoneIDs = []int64{0} and sometimes MilestoneIDs = []int64{NoConditionID} seems difficult to understand (I have seen it many times), I suspect some of them is not right?

MilestoneIDs = []int64{0} is for indexer searching, and MilestoneIDs = []int64{db.NoConditionID} is for db querying.
I don't want use db.NoConditionID in indexer, just keep it simple for indexer, 0 means = 0, -1 means = -1, and a flag NoLabelOnly(not 0 nor -1) means len(lables) == 0

MilestoneIDs = []int64{0} is for indexer searching, and MilestoneIDs = []int64{db.NoConditionID} is for db querying.
I don't want use db.NoConditionID in indexer, just keep it simple for indexer, 0 means = 0, -1 means = -1, and a flag NoLabelOnly(not 0 nor -1) means len(lables) == 0
I think I'm careful in handling them (I hope I'm careful enough 🥲).

It looks pretty weird to me, but since it works, no block from my side.

blocking so I get a chance to review before it gets merged.

Oops, sorry I dismissed your review (I thought it was for mine .......), I will add a "blocked" manually.


fix: wait until node height bigger than zero

关联的Issues (Related Issues)

involve https://github.com/orgs/filecoin-project/projects/51/views/7
改动 (Proposed Changes)

add wait loop for chain height

附注 (Additional Info)

自查清单 (Checklist)
在你认为本 PR 满足被审阅的标准之前，需要确保 / Before you mark the PR ready for review, please make sure that:

[x] 符合Venus项目管理规范中关于PR的相关标准 / The PR follows the PR standards set out in the Venus project management guidelines
[x] 具有清晰明确的commit message / All commits have a clear commit message.
[x] 包含相关的的测试用例或者不需要新增测试用例 / This PR has tests for new functionality or change in behaviour or not need to add new tests.
[x] 包含相关的的指南以及文档或者不需要新增文档 / This PR has updated usage guidelines and documentation or not need
[ ] 通过必要的检查项 / All checks are green

Codecov Report

Merging #248 (e7ed063) into master (2b3d885) will increase coverage by 3.09%.
The diff coverage is 76.52%.

@@            Coverage Diff             @@
##           master     #248      +/-   ##
==========================================
+ Coverage   50.64%   53.74%   +3.09%
==========================================
  Files          63       63
  Lines        6595     6784     +189
==========================================
+ Hits         3340     3646     +306
+ Misses       2954     2839     -115
+ Partials      301      299       -2

Flag
Coverage Δ

unittests
53.74% <76.52%> (+3.09%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

main.go
0.00% <0.00%> (ø)

gateway/mock_gateway_client.go
69.69% <52.94%> (+1.69%)
:arrow_up:

testhelper/mock_full_node.go
79.72% <79.00%> (+0.52%)
:arrow_up:

service/message_service.go
58.99% <81.81%> (+8.48%)
:arrow_up:

service/message_state_refresh.go
70.88% <82.60%> (+19.12%)
:arrow_up:

integration_test/mock_messager_server.go
73.78% <100.00%> (ø)

models/mysql/message.go
77.31% <100.00%> (ø)

models/sqlite/message.go
80.50% <100.00%> (+0.83%)
:arrow_up:

service/address_service.go
63.26% <100.00%> (-3.74%)
:arrow_down:

testhelper/mock_data.go
85.27% <100.00%> (ø)

... and 14 more


Graphql interfaces

Hi!
First of all, I want to thank for this module/framework/whatever, it is awesome!
For a module that I'm writing I've encountered the need to use graphql interfaces to provide more flexibility in the types that can be resolved by some fields, and I see that Vulcan does not provide support for them (at least for what I saw). I've been doing some simple hacks under the hood and I've managed to accomplish what I was intending, but before doing a PR I wanted to debate with you guys what do you think.
From the problems that I've encountered, there are a few main issues that must be taken into consideration:
1. How interfaces are created
This can be done in raw graphql and passing it to addGraphQLSchema. It has one problem though: it is mandatory to define the __resolveType resolver that provides the type of the document implementing the interface.
import { addGraphQLSchema, addGraphQLResolvers } from 'meteor/vulcan:core';

addGraphQLSchema(`
  interface HierarchicalInterface {
    _id: String
    parentId: String
    parent: HierarchicalInterface
  }
`);

addGraphQLResolvers({
  HierarchicalInterface: {
    // in the examples I've found they determine the types based on the `obj`
    // properties, but that requires to know the types beforehand. Here I'm just
    // returning the parentType name (don't know if will work for all cases)
    __resolveType: (obj, context, info) => info.parentType.name,
  },
});

Maybe a createInterface function could be created to simplify this process?
2. How interfaces are passed to createCollection function
In my implementation I've passed an interfaces property in the  options parameter, so it is at the same level that collectionName, typeName, schema, etc.; and that property is just an array containing the interfaces names.
import { createCollection } from 'meteor/vulcan:core';

// must have the fields with same name and types required by the interface
// it will also provide the extra `name` field
import schema from './schema';

export default createCollection({
    collectionName: 'Categories',
    typeName: 'Category',
    interfaces: ['HierarchicalInterface'],
    schema,
    // ... other options,
});

Maybe the interfaces should be inside the schema object itself?
3. How data is retrieved by fragments
When writing fragments for interfaces it is required to include inline fragments in order to make them really useful, and for now Vulcan does not support them. The problem seems to be that their syntax starts with ... -the same as for subfragments-, and the current code treat everything that starts with ... as subfragments and just breaks when adding an inline-fragment.
import { registerFragment } from 'meteor/vulcan:core';

registerFragment(`
  fragment CategoriesItem on Category {
    _id
    name
    parentId
    parent {
      _id
      ... on Category {
        name
      }
    }
  }
`);

By tweaking the vulcan:lib.lib.modules.fragments.registerFragment function I've managed to fix it and everything seem to work fine.

That's it, I'll fork the repo, commit the changes and create a PR so you can see what disasters I've written to make this work xD
Looking forward for your thoughts.

Thanks for the well thought-out post! By the way, maybe you saw this already but although I've never used interfaces in Vulcan, the Voting package does implement a union:
https://github.com/VulcanJS/Vulcan/blob/master/packages/vulcan-voting/lib/graphql.js#L16-L32
So I'm not sure if we need a new createInterface function at this point? Although I'm open to it if you think it can really make things easier.
For passing the interface to createCollection, can you remind me why that's necessary? Where and how would you access collection.interfaces?
And finally for fragments, yes I agree completely, we should support both inline fragments and subfragments. I also suck at regex so don't worry about that :)
Looking forward to the PR!

Hi Sacha,
Yes I saw the voting package and considered using unions, but the biggest problem I see with them is that you need to know beforehand what types do you want to create the union from (correct me if I'm wrong). This does not comply with the open-closed principle and fails to provide an extensible solution. Of course it still may be a good solution when using them in more closed systems or parts not intended to be reusable.
For example the case of your voting package: with the votable union you're creating a new type-like definition that is the union of two types and that cannot be changed. If you wanted your package to be extended and reused by others, for example to allow others to define more votable types, then interfaces are the way to go. You would define the interface and then implement it in both Post and Comment types, leaving the door open for others to implement their new Votable solutions.
I've found a great article with more insight in this topic: GraphQL Tour: Interfaces and Unions by Clay Allsopp.
Note that the article it also talks about the interface resolver __resolveType problematic:

The GraphQL execution process will ask the same sorts of questions to interfaces as unions (“hey, what type is this?”), but the difference with interfaces is that an interface may not know all the types that implement it.

[...] GraphQL-JS offers an alternative to interface.resolveType. The GraphQL execution process will first traverse all of the types (defined by schema.types) and try to invoke isTypeOf if it is defined.

It seems like GraphQL-JS adresses this issue by allowing a isTypeOf method for each type and calls them for each object in order to determine it.
About why to pass the interfaces in the createCollection, for the moment I'm just accessing them when generating the graphQLSchema in order to set them as type TypeName implements Interface1, InterfaceN.
I'm adding an example package to show how interfaces can be used. For now there is only the simple Categories example of OP, but I can add a more complex where interfaces truly show their potential.

I think those are all great points. Maybe the voting package could be reimplemented using interfaces then? In any case I agree with everything you've said so far, just let me know if you need anything else from me :)

Created PR.


Dataset-based templates

It would be useful to have templates based on a dataset contents.
Let's say we want to access all quads belonging to subjects matching quad template QuadTemplate(null, P, null).
Applying $dataset->copy(QuadTemplate(null, P, null)) is too restrictive as it won't keep other predicates of the subjects matched by the QuadTemplate(null, P, null).
We would need something like:
$matches = $dataset->copy(QuadTemplate(null, P, null));
$dataset->copy(MaskTemplate($matches, MaskTempalte::SUBJECTS))

Can be implemented as:

Helper allowing to extract an array of distinct subject/property/object/graph Terms.
Pass it to a template described in #2

rdfInterface v2 introduces listSubjects()/listPredicates()/listObjects()/listGraphs() methods accepting an optional quads filter so it's now out of scope of this library.