performance issue with security and privacy enabled

code level:
commit e338d6a25865db96c4e6374168473965a2b39054
Merge: 67e2a10 15dcc91
Author: Binh Q Nguyen<EMAIL_ADDRESS>Date:   Mon Feb 15 12:10:54 2016 -0500
Merge pull request #675 from bcbrock/docs

A couple of comments for outside-of-Vagrant builds

security and privacy: enabled
test: invoke a to b, then invoke b to a, then repeat continuously
chaincode can only process 5 invoke transactions per second.
Below is the information I collected during the test.  Detailed log is in the attached file, sec_test0216.txt
sec_test0216.txt

There are many selectNextCert messages through out the log:
15:53:53.548 [CRYPTO.client.lukas] selectNextTCert -> ERRO 03f No TCert in cache found.

an error message when deploy after successfully deploy and creating VM

15:53:54.543 [rest] Deploy -> INFO 044 Successfuly deployed chainCode: bb540edfc1ee2ac0f5e2ec6000677f4cd1c6728046d5e32dede7fea11a42f86a6943b76a8f9154f4792032551ed320871ff7b7076047e4184292e01e3421889c.
15:53:55.627 [container] newDockerClient -> INFO 045 Creating VM with endpoint: unix:///var/run/docker.sock
15:54:03.169 [container] newDockerClient -> INFO 046 Creating VM with endpoint: unix:///var/run/docker.sock
15:54:03.358 [ledger] sendProducerBlockEvent -> ERRO 047 Error unmarshalling deployment transaction for block event: proto: can't skip unknown wire type 7 for protos.ChaincodeDeploymentSpec

it spends 12 seconds on selectNextCert from 15:59:48 to 16:00:00

15:59:48.222 [rest] Invoke -> INFO 1bff REST invoking chaincode...
15:59:48.223 [rest] Invoke -> INFO 1c00 Local user 'lukas' is already logged in. Retrieving login token.
15:59:48.224 [CRYPTO] InitClient -> INFO 1c01 Initializing client [lukas]...
15:59:48.224 [CRYPTO] InitClient -> INFO 1c02 Client already initiliazied [lukas]. Increasing counter from [149]
15:59:48.224 [CRYPTO] InitClient -> INFO 1c01 Initializing client [lukas]...
15:59:48.224 [CRYPTO] InitClient -> INFO 1c02 Client already initiliazied [lukas]. Increasing counter from [149]
15:59:49.114 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c03 No TCert in cache found.
15:59:49.971 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c04 No TCert in cache found.
15:59:50.963 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c05 No TCert in cache found.
15:59:51.816 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c06 No TCert in cache found.
15:59:52.775 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c07 No TCert in cache found.
15:59:53.711 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c08 No TCert in cache found.
15:59:54.595 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c09 No TCert in cache found.
15:59:55.533 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c0a No TCert in cache found.
15:59:56.410 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c0b No TCert in cache found.
15:59:57.344 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c0c No TCert in cache found.
15:59:58.246 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c0d No TCert in cache found.
15:59:59.199 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c0e No TCert in cache found.
16:00:00.078 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c0f No TCert in cache found.
16:00:00.956 [CRYPTO.client.lukas] selectNextTCert -> ERRO 1c10 No TCert in cache found.

Then it creates VM repeatedly

16:00:18.351 [container] newDockerClient -> INFO 1c11 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.485 [container] newDockerClient -> INFO 1c12 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.487 [container] newDockerClient -> INFO 1c13 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.489 [container] newDockerClient -> INFO 1c14 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.490 [container] newDockerClient -> INFO 1c15 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.491 [container] newDockerClient -> INFO 1c16 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.493 [container] newDockerClient -> INFO 1c17 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.495 [container] newDockerClient -> INFO 1c18 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.496 [container] newDockerClient -> INFO 1c19 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.498 [container] newDockerClient -> INFO 1c1a Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.499 [container] newDockerClient -> INFO 1c1b Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.501 [container] newDockerClient -> INFO 1c1c Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.502 [container] newDockerClient -> INFO 1c1d Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.504 [container] newDockerClient -> INFO 1c1e Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.506 [container] newDockerClient -> INFO 1c1f Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.507 [container] newDockerClient -> INFO 1c20 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.509 [container] newDockerClient -> INFO 1c21 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.511 [container] newDockerClient -> INFO 1c22 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.513 [container] newDockerClient -> INFO 1c23 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.514 [container] newDockerClient -> INFO 1c24 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.515 [container] newDockerClient -> INFO 1c25 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.517 [container] newDockerClient -> INFO 1c26 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.519 [container] newDockerClient -> INFO 1c27 Creating VM with endpoint: unix:///var/run/docker.sock
16:00:18.522 [container] newDockerClient -> INFO 1c28 Creating VM with endpoint: unix:///var/run/docker.sock

There are 56 seconds between two invoke requests, see

15:59:48.222 [rest] Invoke -> INFO 1bff REST invoking chaincode...
and
16:00:44.684 [rest] Invoke -> INFO 1c29 REST invoking chaincode...
16:00:44.686 [rest] Invoke -> INFO 1c2a Local user 'lukas' is already logged in. Retrieving login token.
16:00:44.687 [CRYPTO] InitClient -> INFO 1c2b Initializing client [lukas]...

the top 150 cpu users are listed below by using go pprof
(pprof) top 150
7.93s of 8.31s total (95.43%)
Dropped 143 nodes (cum <= 0.04s)
flat  flat%   sum%        cum   cum%
1.94s 23.35% 23.35%      1.94s 23.35%  runtime._ExternalCode
0.82s  9.87% 33.21%      0.82s  9.87%  math/big.divWW
0.60s  7.22% 40.43%      2.76s 33.21%  math/big.nat.divLarge
0.48s  5.78% 46.21%      0.70s  8.42%  runtime.scanobject
0.37s  4.45% 50.66%      1.29s 15.52%  runtime.makeslice
0.35s  4.21% 54.87%      0.35s  4.21%  math/big.addMulVVW
0.29s  3.49% 58.36%      0.86s 10.35%  runtime.mallocgc
0.24s  2.89% 61.25%      0.24s  2.89%  math/big.subVV
0.23s  2.77% 64.02%      0.23s  2.77%  runtime.readvarint
0.21s  2.53% 66.55%      0.61s  7.34%  math/big.basicMul
0.19s  2.29% 68.83%      0.19s  2.29%  math/big.mulAddVWW
0.19s  2.29% 71.12%      0.19s  2.29%  runtime.heapBitsForObject
0.17s  2.05% 73.16%      0.97s 11.67%  runtime.gentraceback
0.16s  1.93% 75.09%      0.31s  3.73%  runtime.scanblock
0.13s  1.56% 76.65%      0.36s  4.33%  runtime.step
0.12s  1.44% 78.10%      0.92s 11.07%  runtime.newarray
0.11s  1.32% 79.42%      0.18s  2.17%  runtime.greyobject
0.10s  1.20% 80.63%      0.10s  1.20%  runtime.memclr
0.09s  1.08% 81.71%      1.12s 13.48%  math/big.nat.mul
0.08s  0.96% 82.67%      0.08s  0.96%  math/big.mulWW
0.08s  0.96% 83.63%      0.08s  0.96%  runtime.mSpan_Sweep.func1
0.07s  0.84% 84.48%      0.07s  0.84%  math/big.nat.norm
0.07s  0.84% 85.32%      0.07s  0.84%  math/big.shlVU
0.07s  0.84% 86.16%      0.07s  0.84%  runtime.futex
0.06s  0.72% 86.88%      0.06s  0.72%  math/big.subVW
0.05s   0.6% 87.48%      1.17s 14.08%  math/big.(*Int).Mul
0.05s   0.6% 88.09%      0.05s   0.6%  math/big.shrVU
0.05s   0.6% 88.69%      0.41s  4.93%  runtime.pcvalue
0.05s   0.6% 89.29%      0.05s   0.6%  runtime.usleep
0.05s   0.6% 89.89%      0.05s   0.6%  syscall.Syscall
0.04s  0.48% 90.37%      1.16s 13.96%  math/big.nat.make
0.04s  0.48% 90.85%      0.08s  0.96%  math/big.nat.shl
0.04s  0.48% 91.34%      0.21s  2.53%  math/big.nat.sub
0.04s  0.48% 91.82%      0.60s  7.22%  runtime.scanframeworker
0.03s  0.36% 92.18%      2.79s 33.57%  math/big.(*Int).Mod
0.03s  0.36% 92.54%      0.15s  1.81%  math/big.nat.add
0.03s  0.36% 92.90%      2.80s 33.69%  math/big.nat.div
0.03s  0.36% 93.26%      0.11s  1.32%  runtime.heapBitsSweepSpan
0.02s  0.24% 93.50%      2.82s 33.94%  math/big.(*Int).QuoRem
0.02s  0.24% 93.74%      0.07s  0.84%  math/big.nat.clear
0.02s  0.24% 93.98%      0.20s  2.41%  runtime.funcspdelta
0.02s  0.24% 94.22%      0.14s  1.68%  runtime.mCentral_CacheSpan
0.01s  0.12% 94.34%      2.66s 32.01%  crypto/elliptic.(*CurveParams).doubleJacobian
0.01s  0.12% 94.46%      0.15s  1.81%  math/big.(*Int).Add
0.01s  0.12% 94.58%      0.23s  2.77%  math/big.(*Int).Sub
0.01s  0.12% 94.71%      0.08s  0.96%  math/big.nat.mulAddWW
0.01s  0.12% 94.83%      0.98s 11.79%  runtime.gcscan_m
0.01s  0.12% 94.95%      0.05s   0.6%  runtime.mHeap_Alloc.func1
0.01s  0.12% 95.07%      0.08s  0.96%  runtime.mProf_Malloc
0.01s  0.12% 95.19%         1s 12.03%  runtime.markroot
0.01s  0.12% 95.31%      0.07s  0.84%  runtime.newobject
0.01s  0.12% 95.43%      0.93s 11.19%  runtime.scanstack
0     0% 95.43%      0.97s 11.67%  crypto/ecdsa.Sign
0     0% 95.43%      2.73s 32.85%  crypto/ecdsa.Verify
0     0% 95.43%      3.22s 38.75%  crypto/elliptic.(*CurveParams).ScalarBaseMult
0     0% 95.43%      4.55s 54.75%  crypto/elliptic.(*CurveParams).ScalarMult
0     0% 95.43%      1.81s 21.78%  crypto/elliptic.(*CurveParams).addJacobian
0     0% 95.43%      0.11s  1.32%  crypto/elliptic.(*CurveParams).affineFromJacobian
0     0% 95.43%      0.96s 11.55%  crypto/x509.(*CertPool).findVerifiedParents
0     0% 95.43%      0.96s 11.55%  crypto/x509.(*Certificate).CheckSignature
0     0% 95.43%      0.96s 11.55%  crypto/x509.(*Certificate).CheckSignatureFrom
0     0% 95.43%      0.96s 11.55%  crypto/x509.(*Certificate).Verify
0     0% 95.43%      0.96s 11.55%  crypto/x509.(*Certificate).buildChains
0     0% 95.43%      0.96s 11.55%  crypto/x509.checkSignature
0     0% 95.43%      3.81s 45.85%  github.com/openblockchain/obc-peer/openchain.(*Devops).Invoke
0     0% 95.43%      3.78s 45.49%  github.com/openblockchain/obc-peer/openchain.(*Devops).createExecTx
0     0% 95.43%      3.81s 45.85%  github.com/openblockchain/obc-peer/openchain.(*Devops).invokeOrQuery
0     0% 95.43%      0.89s 10.71%  github.com/openblockchain/obc-peer/openchain/consensus/helper.(*ConsensusHandler).HandleMessage
0     0% 95.43%      0.89s 10.71%  github.com/openblockchain/obc-peer/openchain/consensus/helper.(*ConsensusHandler).doChainTransaction
0     0% 95.43%      2.77s 33.33%  github.com/openblockchain/obc-peer/openchain/crypto.(*clientImpl).(github.com/openblockchain/obc-peer/openchain/crypto.getTCertsFromTCA)-fm
0     0% 95.43%      3.78s 45.49%  github.com/openblockchain/obc-peer/openchain/crypto.(*clientImpl).NewChaincodeExecute
0     0% 95.43%      2.77s 33.33%  github.com/openblockchain/obc-peer/openchain/crypto.(*clientImpl).getNextTCert
0     0% 95.43%      2.77s 33.33%  github.com/openblockchain/obc-peer/openchain/crypto.(*clientImpl).getTCertsFromTCA
0     0% 95.43%      1.01s 12.15%  github.com/openblockchain/obc-peer/openchain/crypto.(*clientImpl).newChaincodeExecuteUsingTCert
0     0% 95.43%      1.01s 12.15%  github.com/openblockchain/obc-peer/openchain/crypto.(*clientImpl).signUsingTCertDER
0     0% 95.43%         1s 12.03%  github.com/openblockchain/obc-peer/openchain/crypto.(*clientImpl).signUsingTCertX509
0     0% 95.43%      2.77s 33.33%  github.com/openblockchain/obc-peer/openchain/crypto.(*keyStore).GetNextTCert
0     0% 95.43%      0.46s  5.54%  github.com/openblockchain/obc-peer/openchain/crypto.(*nodeImpl).sign
0     0% 95.43%      0.86s 10.35%  github.com/openblockchain/obc-peer/openchain/crypto.(*nodeImpl).verify
0     0% 95.43%      0.86s 10.35%  github.com/openblockchain/obc-peer/openchain/crypto.(*peerImpl).TransactionPreValidation
0     0% 95.43%      0.86s 10.35%  github.com/openblockchain/obc-peer/openchain/crypto.(*validatorImpl).TransactionPreValidation
0     0% 95.43%      0.96s 11.55%  github.com/openblockchain/obc-peer/openchain/crypto/utils.CheckCertAgainRoot
0     0% 95.43%      0.94s 11.31%  github.com/openblockchain/obc-peer/openchain/crypto/utils.ECDSASign
0     0% 95.43%      1.77s 21.30%  github.com/openblockchain/obc-peer/openchain/crypto/utils.ECDSAVerify
0     0% 95.43%      1.39s 16.73%  github.com/openblockchain/obc-peer/openchain/crypto/utils.VerifySignCapability
0     0% 95.43%      0.90s 10.83%  github.com/openblockchain/obc-peer/openchain/peer.(*PeerImpl).Chat
0     0% 95.43%      0.90s 10.83%  github.com/openblockchain/obc-peer/openchain/peer.(*PeerImpl).handleChat
0     0% 95.43%      3.81s 45.85%  github.com/openblockchain/obc-peer/openchain/rest.(*ServerOpenchainREST).Invoke
0     0% 95.43%      3.81s 45.85%  github.com/openblockchain/obc-peer/openchain/rest.(*ServerOpenchainREST).SetOpenchainServer
0     0% 95.43%      3.81s 45.85%  github.com/openblockchain/obc-peer/openchain/rest.(*ServerOpenchainREST).SetResponseType
0     0% 95.43%      0.90s 10.83%  github.com/openblockchain/obc-peer/protos._Peer_Chat_Handler
0     0% 95.43%      3.81s 45.85%  github.com/openblockchain/obc-peer/vendor/github.com/gocraft/web.(*Router).ServeHTTP
0     0% 95.43%      3.81s 45.85%  github.com/openblockchain/obc-peer/vendor/github.com/gocraft/web.(*middlewareHandler).invoke
0     0% 95.43%      3.81s 45.85%  github.com/openblockchain/obc-peer/vendor/github.com/gocraft/web.middlewareStack.func1
0     0% 95.43%      0.90s 10.83%  github.com/openblockchain/obc-peer/vendor/google.golang.org/grpc.(*Server).Serve.func2.1.1
0     0% 95.43%      0.90s 10.83%  github.com/openblockchain/obc-peer/vendor/google.golang.org/grpc.(*Server).handleStream
0     0% 95.43%      0.90s 10.83%  github.com/openblockchain/obc-peer/vendor/google.golang.org/grpc.(*Server).processStreamingRPC
0     0% 95.43%      0.12s  1.44%  math/big.(*Int).GCD
0     0% 95.43%      0.08s  0.96%  math/big.(*Int).Lsh
0     0% 95.43%      0.12s  1.44%  math/big.(*Int).ModInverse
0     0% 95.43%      3.81s 45.85%  net/http.(*conn).serve
0     0% 95.43%      3.81s 45.85%  net/http.serverHandler.ServeHTTP
0     0% 95.43%      3.81s 45.85%  reflect.Value.Call
0     0% 95.43%      3.81s 45.85%  reflect.Value.call
0     0% 95.43%      1.90s 22.86%  runtime._System
0     0% 95.43%         1s 12.03%  runtime.backgroundgc
0     0% 95.43%      3.81s 45.85%  runtime.call32
0     0% 95.43%      3.81s 45.85%  runtime.call64
0     0% 95.43%      0.07s  0.84%  runtime.callers
0     0% 95.43%      0.07s  0.84%  runtime.callers.func1
0     0% 95.43%      0.07s  0.84%  runtime.futexwakeup
0     0% 95.43%         1s 12.03%  runtime.gc
0     0% 95.43%      0.98s 11.79%  runtime.gc.func1
0     0% 95.43%      0.22s  2.65%  runtime.gcAssistAlloc
0     0% 95.43%      0.22s  2.65%  runtime.gcAssistAlloc.func1
0     0% 95.43%      0.48s  5.78%  runtime.gcBgMarkWorker
0     0% 95.43%      0.22s  2.65%  runtime.gcDrainN
0     0% 95.43%      0.48s  5.78%  runtime.gcDrainUntilPreempt
0     0% 95.43%      6.26s 75.33%  runtime.goexit
0     0% 95.43%      0.14s  1.68%  runtime.mCache_Refill
0     0% 95.43%      0.11s  1.32%  runtime.mSpan_Sweep
0     0% 95.43%      0.14s  1.68%  runtime.mallocgc.func2
0     0% 95.43%      0.07s  0.84%  runtime.notewakeup
0     0% 95.43%         1s 12.03%  runtime.parfordo
0     0% 95.43%      0.23s  2.77%  runtime.pcdatavalue
0     0% 95.43%      0.08s  0.96%  runtime.profilealloc
0     0% 95.43%      0.94s 11.31%  runtime.scang
0     0% 95.43%      0.60s  7.22%  runtime.scanstack.func1
0     0% 95.43%      0.07s  0.84%  runtime.schedule
0     0% 95.43%      0.06s  0.72%  runtime.startm
0     0% 95.43%      1.52s 18.29%  runtime.systemstack
0     0% 95.43%      0.05s   0.6%  runtime.wakep
0     0% 95.43%      0.05s   0.6%  syscall.Write
0     0% 95.43%      0.05s   0.6%  syscall.write

The issue concerning "error unmarshalling deployment transaction" is tracked in #686

@adecaro  we slacked about the first part of this log
@muralisrini please look at why we try to recreate the VM repeatedly

I captured logs for two invokes and calculate the time for each step based on the timestamp.
21:09:28.449 [rest] Invoke -> INFO 05f REST invoking chaincode...
21:09:28.453 [rest] Invoke -> INFO 060 Local user 'lukas' is already logged in. Retrieving login token.
21:09:28.455 [CRYPTO] InitClient -> INFO 061 Initializing client [lukas]...
21:09:28.459 [CRYPTO.client.lukas] init -> INFO 062 Init keystore...
21:09:28.463 [CRYPTO.client.lukas] init -> INFO 063 Init keystore...done.
21:09:28.464 [CRYPTO.client.lukas] initCryptoEngine -> INFO 064 Initializing node crypto engine...
21:09:28.525 [CRYPTO.client.lukas] initCryptoEngine -> INFO 065 Initializing node crypto engine...done!
21:09:28.525 [CRYPTO.client.lukas] init -> INFO 066 Initialization...done.
21:09:28.525 [CRYPTO.client.lukas] init -> INFO 067 Init keystore...
21:09:28.525 [CRYPTO.client.lukas] init -> INFO 068 Init keystore...done.
21:09:28.525 [CRYPTO.client.lukas] init -> INFO 069 Initialization...done.
21:09:28.525 [CRYPTO] InitClient -> INFO 06a Initializing client [lukas]...done!
21:09:28.554 [CRYPTO] closeClientInternal -> INFO 06b Closing client [lukas]...
21:09:28.554 [CRYPTO.client.lukas] close -> INFO 06c Closing keystore...
21:09:28.555 [CRYPTO.client.lukas] close -> INFO 06d Closing keystore...done!
21:09:28.555 [CRYPTO] closeClientInternal -> INFO 06e Closing client [lukas]...done! [].
21:09:28.555 [rest] Invoke -> INFO 06f Successfuly invoked chainCode with txuuid (90cd17ed-71c9-41ee-8196-ee4a085d26d7)
total: 555 - 449 = 106 ms
crypto: 555 - 455 = 100 ms
init crypto engine: 525 - 464 = 61 ms
InitClient:  554 - 525 = 31 ms
21:09:38.644 [rest] Invoke -> INFO 070 REST invoking chaincode...
21:09:38.647 [rest] Invoke -> INFO 071 Local user 'lukas' is already logged in. Retrieving login token.
21:09:38.650 [CRYPTO] InitClient -> INFO 072 Initializing client [lukas]...
21:09:38.657 [CRYPTO.client.lukas] init -> INFO 073 Init keystore...
21:09:38.659 [CRYPTO.client.lukas] init -> INFO 074 Init keystore...done.
21:09:38.660 [CRYPTO.client.lukas] initCryptoEngine -> INFO 075 Initializing node crypto engine...
21:09:38.709 [CRYPTO.client.lukas] initCryptoEngine -> INFO 076 Initializing node crypto engine...done!
21:09:38.709 [CRYPTO.client.lukas] init -> INFO 077 Initialization...done.
21:09:38.710 [CRYPTO.client.lukas] init -> INFO 078 Init keystore...
21:09:38.710 [CRYPTO.client.lukas] init -> INFO 079 Init keystore...done.
21:09:38.711 [CRYPTO.client.lukas] init -> INFO 07a Initialization...done.
21:09:38.711 [CRYPTO] InitClient -> INFO 07b Initializing client [lukas]...done!
21:09:38.741 [CRYPTO] closeClientInternal -> INFO 07c Closing client [lukas]...
21:09:38.741 [CRYPTO.client.lukas] close -> INFO 07d Closing keystore...
21:09:38.741 [CRYPTO.client.lukas] close -> INFO 07e Closing keystore...done!
21:09:38.741 [CRYPTO] closeClientInternal -> INFO 07f Closing client [lukas]...done! [].
21:09:38.741 [rest] Invoke -> INFO 080 Successfuly invoked chainCode with txuuid (57e59623-ad27-4d77-9428-b25c66ba375f)
total: 741 - 644 = 97 ms
crypto: 741 - 650 = 91 ms
init crypto engine: 709 - 660 = 49 ms
InitClient: 741 - 711 = 30 ms
In both cases, CRYPTO uses 94% of time, and initCryptoEngine + initClient uses 92% and 87% respectively of total CRYPTO time. It will be worthy to look initCryptoEngine and initClent first.

Worked with @dongmingh on this. Basically requests to the chaincode resulted in encrypt/decrypt errors causing the chaincode to go down. This explains why the vms were getting restarted.
Before we go deeper into actual "performance" issue with security, we need to fix the basic security problems.
We know approximately where the encrypt/decrypt errors are coming from but it may or may not be related to the #686 ... will dig deeper.
@dongmingh : please feel free to add any missing details

@muralisrini recreates the VM crash on his system.  We can fix the encrypt/decrypt first. And see if this is related to the selectNextTCert no TCert in cache found issue.  If the no TCert issue still remains after encrypt/decrypt is resolved, then we have to fix it.  After the two issues resolved, we can have more accurate performance study with security.

Test with security disabled, the VM crash issue is not seen.

There are a couple of security related fixes needed in chaincode

don't store path from deploy tx in  security context as other transaction contexts only use names
resurrect security context from saved deployment transaction (long overdue) if init hasn't been called yet

The second fix can also be used to prevent queries and invokes from starting up container (another long overdue fix).

work with @muralisrini  with a sandbox code with the fixes he mentioned in above comment.  The VM crash is fixed but the error message of "no TCert in cache found" is still seen.  While this error message is shown on the log, the test traffic becomes very slow.  The chaincode can only do 10 query per second, tps = 10.  It seems we are making SQL query for every request for get certificate.
Run another quick test with storing the query cert for the very first time and then reuse it for the subsequent query, the tps improves to 50.  This is just for test purpose.
We will continue to work to improve the performance.

@dongmingh @muralisrini working on a fix for the "no TCert in cache found" issue. I'm in the test face. The PR will be ready soon.

I work with @muralisrini today to pick up the TCertPool fix from #705 and do not see the message of "No TCert in cache found".  But the tps goes down to 0.5 query per second.  Then we did a test of commenting out closeClientInternal(), i.e. not close client, then the tps increases to 30 per second.

Please, @dongmingh, may you give me more information on the tests that you run in such a way I can replicate them and analyze what happens to the crypto layer?
Regarding closeClientInternal(), rather than commenting out that call, please, comment out the call to CloseClient(). Also, I would suggest to call InitClient() right after RegisterCliert(.) and then keep the Client instance somewhere ready to be used in order to avoid repeated call also to InitClient().
Regarding the 30 tps, do you mean transaction creation or transaction validation?

I run it on vagrant using SoupUI to drive the query transaction. The thread is set to 1.  Let me know if you need any information on the test.  I can show you in person.
30 tps means 30 queries per second.

@dongmingh Yeah, actually, it would be useful to have a call on Monday to talk about this.
Btw, may you increase the number of thread to at least 2 and see if the tps increases?

with threads = 20, after 3 min runs, the tps = 20.
Here are some log, see ERROR messages of "Failed inserting TCert":
20:49:06.857 [crypto] InitClient -> INFO 595f Client already initiliazied [lukas]. Increasing counter from [19]
20:49:06.864 [crypto] closeClientInternal -> INFO 5960 Closing client [lukas]...
20:49:06.874 [crypto] InitClient -> INFO 5961 Initializing client [lukas]...
20:49:06.875 [crypto] InitClient -> INFO 5962 Client already initiliazied [lukas]. Increasing counter from [19]
20:49:07.571 [crypto.client.lukas] storeUsedTCert -> ERRO 5963 Failed inserting TCert to UsedTCert: [database is locked].
20:49:07.574 [crypto.client.lukas] storeUsedTCert -> ERRO 5964 Failed inserting TCert to UsedTCert: [database is locked].
20:49:07.629 [crypto] closeClientInternal -> INFO 5965 Closing client [lukas]...
20:49:07.629 [crypto] closeClientInternal -> INFO 5966 Closing client [lukas]...
20:49:07.633 [crypto] InitClient -> INFO 5967 Initializing client [lukas]...
20:49:07.633 [crypto] InitClient -> INFO 5968 Client already initiliazied [lukas]. Increasing counter from [18]
20:49:07.637 [crypto] InitClient -> INFO 5969 Initializing client [lukas]...
20:49:07.638 [crypto] InitClient -> INFO 596a Client already initiliazied [lukas]. Increasing counter from [19]
20:49:07.914 [crypto] closeClientInternal -> INFO 596b Closing client [lukas]...
20:49:07.914 [crypto] closeClientInternal -> INFO 596c Closing client [lukas]...
20:49:07.917 [crypto] closeClientInternal -> INFO 596d Closing client [lukas]...
20:49:07.917 [crypto] InitClient -> INFO 596e Initializing client [lukas]...
20:49:07.918 [crypto] InitClient -> INFO 596f Client already initiliazied [lukas]. Increasing counter from [17]
20:49:07.927 [crypto] InitClient -> INFO 5970 Initializing client [lukas]...
20:49:07.927 [crypto] InitClient -> INFO 5971 Client already initiliazied [lukas]. Increasing counter from [18]
20:49:07.942 [crypto] InitClient -> INFO 5972 Initializing client [lukas]...
20:49:07.943 [crypto] InitClient -> INFO 5973 Client already initiliazied [lukas]. Increasing counter from [19]
20:49:07.957 [crypto] closeClientInternal -> INFO 5974 Closing client [lukas]...
20:49:07.959 [crypto] closeClientInternal -> INFO 5975 Closing client [lukas]...
20:49:07.967 [crypto] closeClientInternal -> INFO 5976 Closing client [lukas]...
Please ping me on slack to talk about this.

Okay, I now see too the message "Failed inserting TCert". It is related to sqlite3. Adding an external lock fix the issue.
Then, how do you count the number of tps?

The SoupUI shows the tps.
If I change the thread to 100, the log shows the other error message:
21:11:19.919 [crypto] InitClient -> INFO 6139 Initializing client [lukas]...
21:11:19.919 [crypto] InitClient -> INFO 613a Client already initiliazied [lukas]. Increasing counter from [100]
21:11:21.023 [crypto] closeClientInternal -> INFO 613b Closing client [lukas]...
21:11:21.027 [crypto] InitClient -> INFO 613c Initializing client [lukas]...
21:11:21.027 [crypto] InitClient -> INFO 613d Client already initiliazied [lukas]. Increasing counter from [100]
21:11:21.605 [crypto.client.lukas] GetNextTCert -> ERRO 613e Failed getting a new TCert. Buffer is empty!
21:11:21.605 [crypto.client.lukas] GetNextTCert -> ERRO 613f Failed getting a new TCert. Buffer is empty!
21:11:21.646 [crypto.client.lukas] GetNextTCert -> ERRO 6140 Failed getting a new TCert. Buffer is empty!
21:11:21.661 [crypto.client.lukas] GetNextTCert -> ERRO 6141 Failed getting a new TCert. Buffer is empty!
21:11:22.017 [crypto] closeClientInternal -> INFO 6142 Closing client [lukas]...
21:11:22.019 [crypto] InitClient -> INFO 6143 Initializing client [lukas]...
21:11:22.020 [crypto] InitClient -> INFO 6144 Client already initiliazied [lukas]. Increasing counter from [100]
21:11:22.843 [crypto.client.lukas] GetNextTCert -> ERRO 6145 Failed getting a new TCert. Buffer is empty!
21:11:23.629 [crypto.client.lukas] GetNextTCert -> ERRO 6146 Failed getting a new TCert. Buffer is empty!
21:11:23.820 [crypto.client.lukas] GetNextTCert -> ERRO 6147 Failed getting a new TCert. Buffer is empty!

Yes, make sense this. I have a better picture now. Let's continue the investigation. Is there a document that describes how to use SoapUI?

with the latest code from @adecaro, it performs 20 tps.

@dongmingh If you have already moved to go1.6, may you pull again and test how tps you get? I have modified a few lines to take advantage of the new optimized elliptic curve provided by go1.6. Thanks.

With the latest code from @adecaro this afternoon, the tps = 235 for query and 130 for invoke with both security and privacy enabled, threads = 100.
However, I observed a crash which is not crypto related, see detail in bug #714

With the latest code from PR #705, crash is not seen and the performance as follow
The setting and results are as follow with your latest PR code.
test setting:
security enabled
privacy enabled
TLS enabled
rootcert: file: "/var/openchain/production/.obcca/tlsca.cert"
logging: peer and crypto : error
results:
batch = 200, tps (query/invoke) = 236/147
batch = 500, tps (query/invoke) = 236/135
where batch is the number of TCerts obtained for at a given time from obcca.
This is an improvement of 30 times from the tps = 4 invoke we observed when the bug was opened.

Thanks, @dongmingh @adecaro ... will merge PR #705

Closing as per Murali and Dongming.


Clean up event emitter in task land

See #38417

@isidorn if you see problems in debug land please ping


WebAssembly support

would be awesome if there was a loader for esbuild to process WASM files, for example for pure-evm library:
import pure_evm from 'https://esm.sh/pure-evm'

rn it throws this as there's no loader:
/* esm.sh - error */
throw new Error("[esm.sh] " + "esbuild: No loader is configured for \".wasm\" files: esm-build-fa59bded8e9da6e6fecf3f6a67f775cb59d73841/node_modules/pure-evm/pure-evm_bg.wasm");
export default null;

works fine in v55:
import * as pure_evm from "https://esm.sh/pure-evm"


Fails to analyse firebase

Hi,
I've tried analysing Firebase and it shows some decoding errors:
swift-package-info git:(main)  .build/debug/swift-package-info --for https://github.com/firebase/firebase-ios-sdk -v 7.6.0 --product FirebaseCrashlytics

Identified Swift Package:
Repository URL: https://github.com/firebase/firebase-ios-sdk
Version: 7.6.0
Product: FirebaseCrashlytics
Error: keyNotFound(CodingKeys(stringValue: "range", intValue: nil), Swift.DecodingError.Context(codingPath: [CodingKeys(stringValue: "dependencies", intValue: nil), _JSONKey(stringValue: "Index 5", intValue: 5), CodingKeys(stringValue: "requirement", intValue: nil)], debugDescription: "No value associated with key CodingKeys(stringValue: \"range\", intValue: nil) (\"range\").", underlyingError: nil))

Am I doing something wrong?

Hi @kikeenrique, you are doing it right. It's a bug on parsing Package.swift contents from https://github.com/firebase/firebase-ios-sdk.
I'll work on a fix.
Thanks for reporting it 😄 !

Hey @kikeenrique, the fix is already available on latest version, and the report for swift-package-info --for https://github.com/firebase/firebase-ios-sdk -v 7.6.0 --product FirebaseCrashlytics should work as expected.
In case you are using mint, running mint install marinofelipe/swift-package-info would update to latest version.
Let me know how it goes, and thanks again for the issue 👍 .

Hi again @marinofelipe,
Thanks a lot for your quick response and fix!
Good news, it doesn't fail anymore at that point, the issue is fixed. And works in a Intel machine.
Bad news, on a M1 machine it's blown up by mem consumption.
Output:
swift-package-info git:(main) ✗  .build/debug/swift-package-info --for https://github.com/firebase/firebase-ios-sdk -v 7.6.0 --product FirebaseCrashlytics

Identified Swift Package:
Repository URL: https://github.com/firebase/firebase-ios-sdk
Version: 7.6.0
Product: FirebaseCrashlytics

[2/7] Generating archive for empty app...[1]
    931 killed     .build/debug/swift-package-info --for  -v 7.6.0 --product FirebaseCrashlytics

And same result but using verbose flag:

Finished Shell command

Cloning empty app

Running Shell command

fatal: destination path 'swift-package-info' already exists and is not an empty directory.

Finished Shell command

Measuring empty app size

xcodebuild archive -project swift-package-info/MeasurementApp.xcodeproj -scheme MeasurementApp -archivePath /var/folders/14/vldz0v6j15s_ctxm_h6r7ctr0000gn/T/archive.xcarchive -derivedDataPath /var/folders/14/vldz0v6j15s_ctxm_h6r7ctr0000gn/T -configuration Release -arch arm64 CODE_SIGNING_REQUIRED=NO CODE_SIGNING_ALLOWED=NO ENABLE_BITCODE=NO

Running Shell command

objc[2632]: Class AMSupportURLConnectionDelegate is implemented in both ?? (0x1f4cf8188) and ?? (0x1189a02b8). One of the two will be used. Which one is undefined.

objc[2632]: Class AMSupportURLSession is implemented in both ?? (0x1f4cf81d8) and ?? (0x1189a0308). One of the two will be used. Which one is undefined.

[1]    2609 killed     .build/debug/swift-package-info --for  -v 7.6.0 --product FirebaseCrashlytics

Hey @kikeenrique, that's very interesting.
Question, which tool do you use check memory/cpu consumption of a process on terminal?
So, swift-package-info uses xcodebuild archive to calculate the binary size, which itself makes pc's fans screen.
I expect that it consumes a lot of memory, and it is used twice for measuring the binary size.
On one of this times it has to resolve all dependencies, in this case for FirebaseCrashlytics, which are many.
If for example I run the dependencies subcommand (that doesn't use the binary size provider):
/usr/bin/time -lp swift run swift-package-info dependencies --for https://github.com/firebase/firebase-ios-sdk -v 7.6.0 --product FirebaseCrashlytics --verbose

The process executes much faster, and seems to use considerably less memory.
real         6.52
user         1.67
sys          1.41
  72237056  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    103862  page reclaims
        77  page faults
         0  swaps
         0  block input operations
         0  block output operations
        20  messages sent
     14925  messages received
         0  signals received
      9117  voluntary context switches
      4526  involuntary context switches

And as another example, here is when xcodebuild archive is run for the app with FirebaseCrashlytics added:
/usr/bin/time -lp xcodebuild archive -scheme MeasurementApp -archivePath Temp/Archive -derivedDataPath Temp  -configuration Release \
        -arch arm64 \
        CODE_SIGNING_REQUIRED=NO \
        CODE_SIGNING_ALLOWED=NO \
        ENABLE_BITCODE=NO

And the results
real       112.88
user        36.81
sys         13.14
 463654912  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
   1138098  page reclaims
      9077  page faults
         0  swaps
         0  block input operations
         0  block output operations
       369  messages sent
     14779  messages received
       137  signals received
     18274  voluntary context switches
     74078  involuntary context switches

As comparison, here is when swift-package-info is used, running the full report (seem to on average eat almost half the memory of when manually running xcodebuild archive):
/usr/bin/time -lp swift run swift-package-info --for https://github.com/firebase/firebase-ios-sdk -v 7.6.0 --product FirebaseCrashlytics --verbose

Output:
real        43.56
user        21.56
sys          9.29
 264790016  maximum resident set size
         0  average shared memory size
         0  average unshared data size
         0  average unshared stack size
    206698  page reclaims
     11905  page faults
         0  swaps
         0  block input operations
         0  block output operations
       465  messages sent
     18746  messages received
         0  signals received
     23517  voluntary context switches
     56810  involuntary context switches

I can check if there is any memory leak, or if's there anything that can be done to make it lighter, but how much of memory xcodebuild archive eats up is really out of the tool's control, and unfortunately I think there's no other easy way of calculating binary size if not by archiving and measuring an app/product size.
Can you to please try adding FirebaseCrashlytics as a package dependency to MeasurementApp.xcodeproj and run xcodebuild archive for it in your side?
It would be nice to double check if the output is similar with your tooling. As you might have seen my benchmarks here were with /usr/bin/time.
This would probably be the command to try out:
xcodebuild archive -scheme MeasurementApp -archivePath Temp/Archive -derivedDataPath Temp  -configuration Release \
        -arch arm64 \
        CODE_SIGNING_REQUIRED=NO \
        CODE_SIGNING_ALLOWED=NO \
        ENABLE_BITCODE=NO

@kikeenrique I'm closing the bug, since it was fixed!
Regarding performance, let me know if you have any other interesting findings, and if you'd like to potentially help on investigating improvements that could be done.
I'll personally get some time to further evaluate potential performance improvements later, here is an issue for that.
Thanks!


custom label / flattening out vectors question

I have an application that exposes the normal kubernetes_namespace and kubernetes_pod_name labels as well as a subject_name label in a gauge. I'd like to create HPA resources based on the sum of values for a specific subject_name. I'm new to looking at HPA and just set up the adapter. I'm using the community helm chart to deply it - this is what my rules looks like:
rules:
  default: false
  custom:
    - seriesQuery: 'subject_message_count{kubernetes_namespace!="",kubernetes_pod_name!=""}'
      resources:
        overrides:
          kubernetes_namespace:
            resource: "namespace"
          kubernetes_pod_name:
            resource: "pod"
      name:
        matches: ""
        as: "subject_message_count"
      metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>}) by (subject_name, <<.GroupBy>>)

A query against the raw API:
kubectl get --raw "/apis/custom.metrics.k8s.io/v1beta1/namespaces/test/pods/*/subject_message_count" |jq
{
  "kind": "MetricValueList",
  "apiVersion": "custom.metrics.k8s.io/v1beta1",
  "metadata": {
    "selfLink": "/apis/custom.metrics.k8s.io/v1beta1/namespaces/test/pods/%2A/subject_message_count"
  },
  "items": [
    {
      "describedObject": {
        "kind": "Pod",
        "namespace": "test",
        "name": "server-66d9766549-vld9c",
        "apiVersion": "/v1"
      },
      "metricName": "subject_message_count",
      "timestamp": "2021-01-26T01:13:25Z",
      "value": "618",
      "selector": null
    }
  ]
}

I wanted to see more items, one for each subject that it finds. Can someone help me out here? or point me towards some examples that are more complex than exposing default k8s object labels?

For your particular use case external metrics would be better suited, custom metrics are meant to be tied to Kubernetes resources which means that only the labels you specified in the resources part of your config will be matched, the others will be dropped. On the contrary, external metrics are designed to autoscale based on metrics coming from outside of Kubernetes. In your case you want to autoscale based on the subject_name label which does not directly describe a Kubernetes resource, so you should use external metrics if you want to configure your HPA based on this data.

For your particular use case external metrics would be better suited, custom metrics are meant to be tied to Kubernetes resources which means that only the labels you specified in the resources part of your config will be matched, the others will be dropped. On the contrary, external metrics are designed to autoscale based on metrics coming from outside of Kubernetes. In your case you want to autoscale based on the subject_name label which does not directly describe a Kubernetes resource, so you should use external metrics if you want to configure your HPA based on this data.

That makes sense. Is there a good example out there to help me understand how to write the rules for external metrics?

That makes sense. Is there a good example out there to help me understand how to write the rules for external metrics?

The is a small example in this manifest: https://github.com/DirectXMan12/k8s-prometheus-adapter/blob/master/deploy/manifests/custom-metrics-config-map.yaml#L99-L117. The syntax is very close to custom metrics, so you will not have to change a lot of things.

The is a small example in this manifest: https://github.com/DirectXMan12/k8s-prometheus-adapter/blob/master/deploy/manifests/custom-metrics-config-map.yaml#L99-L117. The syntax is very close to custom metrics, so you will not have to change a lot of things.

Thanks, sorry for the late response. This is what i ended up with:
rules:
  default: false
  external:
  - metricsQuery: sum(<<.Series>>{<<.LabelMatchers>>}) by (subject_name, <<.GroupBy>>)
    name:
      as: subject_message_count
      matches: ""
    resources:
      overrides:
        kubernetes_namespace:
          resource: namespace
    seriesQuery: subject_message_count{kubernetes_namespace!=""}

This query:  kubectl get --raw "/apis/external.metrics.k8s.io/v1beta1/namespaces/test/subject_message_count?labelSelector=subject_name%3Ddefault" |jq
Produces this output (which i can then use in HPAs):
{
  "kind": "ExternalMetricValueList",
  "apiVersion": "external.metrics.k8s.io/v1beta1",
  "metadata": {},
  "items": [
    {
      "metricName": "subject_message_count",
      "metricLabels": {
        "subject_name": "default"
      },
      "timestamp": "2021-02-16T21:40:26Z",
      "value": "1"
    }
  ]
}


Support using Core without OCMock available

Looks like we missed some imports as a part of splitting out the OCMock stuff into it's own subspec

Good catch, thanks Orta!

Pushed 0.6.2.


Multiple custom learning sites per Tenant

[X] Question
[ ] Bug
[ ] Content

Question / Expected or Desired Behavior
Can you create multiple custom learning sites per tenant?  We already had one site provisioned for Office365 Learning that the organization was using.  We just provisioned another site for non-O365 content, but when we did that it broke permissions on the initial site as far as the home page tile navigation.
Thanks for your contribution! Sharing is caring.

You can only have 1 master learning site per tenant.  That site collection is where content is provisioned and administrated.  All users that you intend to view your content need to have view access to that site but they do not need to use it as their main learning site, although you can.  You can then use the learning pathways viewer web part in any other site collection and configure the web part to show all, or only a subset of content.  Please look for announcements about updates coming next week at Ignite.
Now that you've created a second site collection you've effectively taken control away from the original and it no longer serves up the content.  You can easily set it back but you will need to fix the tenant app property using powershell.
If using PnP Powershell the command is:
Set-PnPStorageEntity -Key MicrosoftCustomLearningSite -Value $clSite -Description "Custom Learning Site Collection"
where $clSite is the url of the site collection you want to be the master.


Problem with bower-asset packages and "minimum-stability": "stable"

I use last version of composer(1.1.0) and composer-asset-plugin.
Example of my composer.json:
{
    "name": "yiisoft/yii2-app-advanced",
    "description": "Yii 2 Advanced Application Template",
    "keywords": ["yii2", "framework", "advanced", "application template"],
    "homepage": "http://www.yiiframework.com/",
    "type": "project",
    "license": "BSD-3-Clause",
    "minimum-stability": "stable",
    "support": {
        "issues": "https://github.com/yiisoft/yii2/issues?state=open",
        "forum": "http://www.yiiframework.com/forum/",
        "wiki": "http://www.yiiframework.com/wiki/",
        "irc": "irc://irc.freenode.net/yii",
        "source": "https://github.com/yiisoft/yii2"
    },
    "require": {
        "php": ">=5.6.0",
        "yiisoft/yii2": "2.0.8",
        "yiisoft/yii2-imagine": "*",
        "yiisoft/yii2-jui": "*",
        "yiidoc/yii2-redactor": "*",

        "bower-asset/marionette": ">=2.2.1",
        "bower-asset/jquery-infinite-scroll": "dev-master#81088b2e8e9fc141f41ab06da3ca86552003e018",
        "bower-asset/animo.js": "*",
        "bower-asset/store.js": ">=1.3.16",
        "bower-asset/uri.js": "*",
        "bower-asset/kendo-ui-core": "2014.2.716",
        "bower-asset/jquery.lazyload": "*",
        "bower-asset/es6-promise": ">=2.0.1",
        "bower-asset/jquery-raty": "*",
        "bower-asset/dotdotdot": ">=1.7.3",
        "bower-asset/jstree": "*",
        "bower-asset/input-bubbles-timeweb": "*",
        "bower-asset/d3": "*",
        "bower-asset/baron": "0.7.14",
        "bower-asset/momentjs": "dev-master",
        "bower-asset/humanize-duration": "^3.3@dev",
        "bower-asset/raven-js": "> 2.1.0",

        "mpdf/mpdf": "> 5.7.3a",
        "hflabs/suggestions-jquery": "dev-master",
        "pear/net_socket": "dev-trunk",
        "kbsali/redmine-api": "~1.0",
        "netcarver/textile": "3.5.*",
        "h4cc/wkhtmltopdf-amd64": "dev-master",
        "raven/raven": "^0.13.0@dev",
        "fiamma06/yii2-zabbix-api": "dev-master",
        "zeroclipboard/zeroclipboard": "dev-master",
        "bazilio/yii2-newrelic": "0.0.6",
        "myclabs/php-enum": "^1.4",
        "justinrainbow/json-schema": "~2.0",
        "phpseclib/phpseclib": "*",
        "guzzlehttp/guzzle": "^6.2"
    },
    "require-dev": {
        "yiisoft/yii2-debug": "*",
        "yiisoft/yii2-gii": "*",
        "yiisoft/yii2-faker": "*",
        "yiisoft/yii2-coding-standards":"dev-master",
        "yiisoft/yii2-codeception": "*",
        "codeception/codeception": "2.1.*@dev",
        "sebastian/phpcpd": "*",
        "phpmd/phpmd" : "@stable"
    },

    "config": {
        "secure-http": false,
        "process-timeout": 1800,
        "discard-changes": true,
        "preferred-install": "dist"
    },
    "extra": {
        "asset-installer-paths": {
            "npm-asset-library": "vendor/npm",
            "bower-asset-library": "vendor/bower"
        }
    },
    "repositories":[
        {
            "type":"git",
            "url":"http://github.com/Timeweb/yii2-coding-standards"
        },
        {
            "type": "package",
            "package": {
                "name": "hflabs/suggestions-jquery",
                "version": "master",
                "source": {
                    "url": "https://github.com/hflabs/suggestions-jquery",
                    "type": "git",
                    "reference": "origin/master"
                }
            }
        }
    ],
    "scripts": {
        "test": "cd common;../vendor/bin/codecept run"
    }
}

and i have some problems after run composer update:
  Problem 1
    - Installation request for bower-asset/uri.js * -> satisfiable by bower-asset/uri.js[dev-master].
    - Removal request for bower-asset/uri.js == 9999999-dev
  Problem 2
    - Installation request for bower-asset/jquery.lazyload * -> satisfiable by bower-asset/jquery.lazyload[dev-master].
    - Removal request for bower-asset/jquery.lazyload == 9999999-dev
  Problem 3
    - Installation request for bower-asset/d3 * -> satisfiable by bower-asset/d3[dev-master].
    - Removal request for bower-asset/d3 == 9999999-dev
  Problem 4
    - Removal request for bower-asset/jquery-raty == <IP_ADDRESS>99999-dev
    - Removal request for bower-asset/jquery-raty == 9999999-dev
    - Installation request for bower-asset/jquery-raty * -> satisfiable by bower-asset/jquery-raty[2.7.0.x-dev, dev-master].

On     "minimum-stability": "dev" all work fine.

Use the last dev version 1.2.0@dev of plugin.
However, there is a stability problem with the bower-asset/input-bubbles-timeweb dependency, because any tags are imported, because there is no defined tag (see https://github.com/Timeweb/inputBubbles/tags).
With bower-asset/input-bubbles-timeweb: @dev, it's ok.

composer global require fxp/composer-asset-plugin:1.2.0@dev get errors:
Your requirements could not be resolved to an installable set of packages.
  Problem 1
    - The requested package fxp/composer-asset-plugin 1.2.0@dev exists as fxp/composer-asset-plugin[dev-master, 1.2.x-dev, v1.0.0, v1.0.0-beta1, v1.0.0-beta2, v1.0.0-beta3, v1.0.0-beta4, v1.0.1, v1.0.2, v1.0.3, v1.1.0, v1.1.1, v1.1.2, v1.1.3, v1.1.4] but these are rejected by your constraint.

composer global require fxp/composer-asset-plugin:1.2.x@dev  work fine:
  - Removing fxp/composer-asset-plugin (v1.1.4)
  - Installing fxp/composer-asset-plugin (dev-master 7f3338b)
    Cloning 7f3338b433ca21ae8e8f56d4b5a82f0146242dce

I fixed bower-asset/input-bubbles-timeweb to @dev and run composer update, and problem not resolved:
Updating dependencies (including require-dev)
Your requirements could not be resolved to an installable set of packages.

  Problem 1
    - Installation request for bower-asset/uri.js * -> satisfiable by bower-asset/uri.js[dev-master].
    - Removal request for bower-asset/uri.js == 9999999-dev
  Problem 2
    - Installation request for bower-asset/jquery.lazyload * -> satisfiable by bower-asset/jquery.lazyload[dev-master].
    - Removal request for bower-asset/jquery.lazyload == 9999999-dev
  Problem 3
    - Installation request for bower-asset/d3 * -> satisfiable by bower-asset/d3[dev-master].
    - Removal request for bower-asset/d3 == 9999999-dev
  Problem 4
    - Removal request for bower-asset/jquery-raty == <IP_ADDRESS>99999-dev
    - Removal request for bower-asset/jquery-raty == 9999999-dev
    - Installation request for bower-asset/jquery-raty * -> satisfiable by bower-asset/jquery-raty[2.7.0.x-dev, dev-master].

Remove your vendor directory and composer.lock.
My test with "bower-asset/input-bubbles-timeweb": "@dev":
$ composer install --prefer-dist
Loading composer repositories with package information
Warning: Accessing github.com over http which is an insecure protocol.
Updating dependencies (including require-dev)
  - Installing yiisoft/yii2-composer (2.0.4)
    Downloading 100%

  - Installing bower-asset/jquery (2.1.1)
    Downloading 100%

  - Installing bower-asset/kendo-ui-core (2014.2.716)
    Downloading 100%

  - Installing bower-asset/baron (0.7.14)
    Downloading 100%

  - Installing bower-asset/momentjs (dev-master d6651c2)
    Downloading 100%

  - Installing hflabs/suggestions-jquery (master origin/master)
    Cloning origin/master

  - Installing pear/pear_exception (v1.0.0)
    Downloading 100%

  - Installing pear/net_socket (dev-trunk bbe6a12)
    Downloading 100%

  - Installing h4cc/wkhtmltopdf-amd64 (dev-master 0c1f180)
    Downloading 100%

  - Installing raven/raven (0.13.0)
    Downloading 100%

  - Installing bower-asset/yii2-pjax (v2.0.6)
    Downloading 100%

  - Installing bower-asset/punycode (v1.3.2)
    Downloading 100%

  - Installing bower-asset/jquery.inputmask (3.2.7)
    Downloading 100%

  - Installing cebe/markdown (1.1.0)
    Downloading 100%

  - Installing ezyang/htmlpurifier (v4.7.0)
    Downloading 100%

  - Installing yiisoft/yii2 (2.0.8)
    Downloading 100%

  - Installing fiamma06/yii2-zabbix-api (dev-master 9d94677)
    Downloading 100%

  - Installing zeroclipboard/zeroclipboard (dev-master d88febe)
    Downloading 100%

  - Installing sobanvuex/php-newrelic (1.2.0)
    Downloading 100%

  - Installing bazilio/yii2-newrelic (0.0.6)
    Downloading 100%

  - Installing imagine/imagine (v0.5.0)
    Downloading 100%

  - Installing yiisoft/yii2-imagine (2.0.3)
    Downloading 100%

  - Installing bower-asset/jquery-ui (1.11.4)
    Downloading 100%

  - Installing yiisoft/yii2-jui (2.0.5)
    Downloading 100%

  - Installing yiidoc/yii2-redactor (2.0.1)
    Downloading 100%

  - Installing bower-asset/underscore (1.8.3)
    Downloading 100%

  - Installing bower-asset/backbone (1.3.3)
    Downloading 100%

  - Installing bower-asset/backbone.wreqr (v1.3.7)
    Downloading 100%

  - Installing bower-asset/backbone.babysitter (v0.1.12)
    Downloading 100%

  - Installing bower-asset/marionette (v2.4.7)
    Downloading 100%

  - Installing bower-asset/jquery-infinite-scroll (dev-master 81088b2)
    Downloading 100%

  - Installing bower-asset/animo.js (1.0.3)
    Downloading 100%

  - Installing bower-asset/store.js (v1.3.20)
    Downloading 100%

  - Installing bower-asset/uri.js (v1.18.1)
    Downloading 100%

  - Installing bower-asset/jquery.lazyload (1.9.7)
    Downloading 100%

  - Installing bower-asset/es6-promise (v3.2.2)
    Downloading 100%

  - Installing bower-asset/jquery-raty (2.7.0)
    Downloading 100%

  - Installing bower-asset/dotdotdot (v1.8.1)
    Downloading 100%

  - Installing bower-asset/jstree (3.3.1)
    Downloading 100%

  - Installing bower-asset/input-bubbles-timeweb (dev-master ab22146)
    Downloading 100%

  - Installing bower-asset/d3 (v3.5.17)
    Downloading 100%

  - Installing bower-asset/humanize-duration (v3.8.0)
    Downloading 100%

  - Installing bower-asset/raven-js (3.0.5)
    Downloading 100%

  - Installing setasign/fpdi (1.6.1)
    Downloading 100%

  - Installing mpdf/mpdf (v6.1.1)
    Downloading 100%

  - Installing kbsali/redmine-api (v1.5.7)
    Downloading 100%

  - Installing netcarver/textile (v3.5.5)
    Downloading 100%

  - Installing myclabs/php-enum (1.4.1)
    Downloading 100%

  - Installing justinrainbow/json-schema (2.0.5)
    Downloading 100%

  - Installing phpseclib/phpseclib (2.0.2)
    Downloading 100%

  - Installing psr/http-message (1.0)
    Downloading 100%

  - Installing guzzlehttp/promises (1.2.0)
    Downloading 100%

  - Installing bower-asset/bootstrap (v3.3.6)
    Downloading 100%

  - Installing yiisoft/yii2-bootstrap (2.0.6)
    Downloading 100%

  - Installing yiisoft/yii2-debug (2.0.6)
    Downloading 100%

  - Installing bower-asset/typeahead.js (v0.11.1)
    Downloading 100%

  - Installing phpspec/php-diff (v1.1.0)
    Downloading 100%

  - Installing yiisoft/yii2-gii (2.0.5)
    Downloading 100%

  - Installing fzaninotto/faker (v1.6.0)
    Downloading 100%

  - Installing yiisoft/yii2-faker (2.0.3)
    Downloading 100%

  - Installing squizlabs/php_codesniffer (2.6.1)
    Downloading 100%

  - Installing yiisoft/yii2-coding-standards (dev-master 3d72053)
    Cloning 3d72053868d28162867a893fb8ac9c3c8aea9a7c

  - Installing yiisoft/yii2-codeception (2.0.5)
    Downloading 100%

  - Installing myclabs/deep-copy (1.5.1)
    Downloading 100%

  - Installing sebastian/version (2.0.0)
    Downloading 100%

  - Installing sebastian/resource-operations (1.0.0)
    Downloading 100%

  - Installing sebastian/recursion-context (1.0.2)
    Downloading 100%

  - Installing sebastian/object-enumerator (1.0.0)
    Downloading 100%

  - Installing sebastian/global-state (1.1.1)
    Downloading 100%

  - Installing sebastian/exporter (1.2.1)
    Downloading 100%

  - Installing sebastian/environment (1.3.7)
    Downloading 100%

  - Installing sebastian/diff (1.4.1)
    Downloading 100%

  - Installing sebastian/comparator (1.2.0)
    Downloading 100%

  - Installing symfony/yaml (v3.1.0)
    Downloading 100%

  - Installing doctrine/instantiator (1.0.5)
    Downloading 100%

  - Installing webmozart/assert (1.0.2)
    Downloading 100%

  - Installing phpdocumentor/reflection-common (1.0)
    Downloading 100%

  - Installing phpdocumentor/type-resolver (0.1.8)
    Downloading 100%

  - Installing phpdocumentor/reflection-docblock (3.0.2)
    Downloading 100%

  - Installing phpspec/prophecy (v1.6.1)
    Downloading 100%

  - Installing phpunit/php-text-template (1.2.1)
    Downloading 100%

  - Installing phpunit/phpunit-mock-objects (3.1.3)
    Downloading 100%

  - Installing phpunit/php-timer (1.0.8)
    Downloading 100%

  - Installing sebastian/code-unit-reverse-lookup (1.0.0)
    Downloading 100%

  - Installing phpunit/php-token-stream (1.4.8)
    Downloading 100%

  - Installing phpunit/php-file-iterator (1.4.1)
    Downloading 100%

  - Installing phpunit/php-code-coverage (3.3.3)
    Downloading 100%

  - Installing phpunit/phpunit (5.3.5)
    Downloading 100%

  - Installing symfony/polyfill-mbstring (v1.2.0)
    Downloading 100%

  - Installing symfony/dom-crawler (v3.1.0)
    Downloading 100%

  - Installing symfony/css-selector (v3.1.0)
    Downloading 100%

  - Installing symfony/browser-kit (v3.1.0)
    Downloading 100%

  - Installing symfony/event-dispatcher (v3.1.0)
    Downloading 100%

  - Installing symfony/console (v3.1.0)
    Downloading 100%

  - Installing symfony/finder (v3.1.0)
    Downloading 100%

  - Installing facebook/webdriver (1.1.1)
    Downloading 100%

  - Installing guzzlehttp/psr7 (1.3.0)
    Downloading 100%

  - Installing guzzlehttp/guzzle (6.2.0)
    Downloading 100%

  - Installing codeception/codeception (2.1.x-dev 58bacde)
    Downloading 100%

  - Installing theseer/fdomdocument (1.6.1)
    Downloading 100%

  - Installing sebastian/finder-facade (1.2.1)
    Downloading 100%

  - Installing sebastian/phpcpd (2.0.4)
    Downloading 100%

  - Installing symfony/filesystem (v3.1.0)
    Downloading 100%

  - Installing symfony/config (v3.1.0)
    Downloading 100%

  - Installing symfony/dependency-injection (v3.1.0)
    Downloading 100%

  - Installing pdepend/pdepend (2.2.4)
    Downloading 100%

  - Installing phpmd/phpmd (2.4.3)
    Downloading 100%

sobanvuex/php-newrelic suggests installing ext-newrelic (Get the New Relic PHP extension from http://newrelic.com)
imagine/imagine suggests installing ext-imagick (to use the Imagick implementation)
imagine/imagine suggests installing ext-gmagick (to use the Gmagick implementation)
setasign/fpdi suggests installing setasign/fpdf (FPDI will extend this class but as it is also possible to use "tecnickcom/tcpdf" as an alternative there's no fixed dependency configured.)
setasign/fpdi suggests installing setasign/fpdi-fpdf (Use this package to automatically evaluate dependencies to FPDF.)
setasign/fpdi suggests installing setasign/fpdi-tcpdf (Use this package to automatically evaluate dependencies to TCPDF.)
phpseclib/phpseclib suggests installing ext-libsodium (SSH2/SFTP can make use of some algorithms provided by the libsodium-php extension.)
sebastian/global-state suggests installing ext-uopz (*)
phpunit/php-code-coverage suggests installing ext-xdebug (>=2.4.0)
phpunit/phpunit suggests installing phpunit/php-invoker (~1.1)
symfony/browser-kit suggests installing symfony/process ()
symfony/event-dispatcher suggests installing symfony/http-kernel ()
symfony/console suggests installing symfony/process ()
symfony/console suggests installing psr/log (For using the console logger)
facebook/webdriver suggests installing phpdocumentor/phpdocumentor (2.*)
codeception/codeception suggests installing codeception/phpbuiltinserver (Extension to start and stop PHP built-in web server for your tests)
codeception/codeception suggests installing codeception/specify (BDD-style code blocks)
codeception/codeception suggests installing codeception/verify (BDD-style assertions)
codeception/codeception suggests installing monolog/monolog (Log test steps)
codeception/codeception suggests installing symfony/phpunit-bridge (For phpunit-bridge support)
symfony/dependency-injection suggests installing symfony/expression-language (For using expressions in service container configuration)
symfony/dependency-injection suggests installing symfony/proxy-manager-bridge (Generate service proxies to lazy load them)
Package raven/raven is abandoned, you should avoid using it. Use sentry/sentry instead.
Writing lock file
Generating autoload files

This worked. Than you.


Xamarin Forms XAML Preview support

When I try to open XAML in Form Previewer "Initializing renderer" runs infinitely long

XAML Previewer is a Prerelease. I don't have any informations about supporting custom renderers in it (which CachedImage is).  If someone has any info about that, please let me know. Meanwhile, you can use Image and then after prototyping replace it with CachedImage which has the same basic properties.

Thanks a lot for answer and the excellent library!
Yes, alfa version Previewer supports custom renderers.
But when I try to use FFImageCache in XAML, for example ffimageloading:CachedImage/  I catch an exception in Previewer
Invalid XAML The type initializer for FFImageLoading.Forms.CachedImage threw an exception.
Unfortunately, logs do not show  any problem. I'm trying to find problem in constructor of CachedImage, but it is very simple an clear.

@vlkam Thanks.
Did you try to check /Users/username/Library/Logs folder? Maybe those logs would give some more informations.

Did you try to check /Users/username/Library/Logs folder?

It's FFImageLoading's folder ? I cannot found it.
I created an empty solution, ran it and made a snapshot of logs from
C:\Users\username\AppData\Local\Xamarin\Logs\14.0
then I added ffimageloading:CachedImage into XAML, caught error and again made the snapshot, compared these snapshots and I found no significant differences
The Previewer uses only PCL part of solution and custom renders from platform specific part. Previewer does not create the AppDelegate. May be problem is that ?
sorry for my english, I'm working on it :-)

Ohh.... I'm so sorry, I don tell you, I work in Visual Studio (windows)

I have checked, double checked and triple checked the CachedImage class code, removed all suspicious fragments, tried to create a very simple instance of it but got an exception every time.
I have no idea what it could be.

Hi @vlkam
If it's Windows, you could use VS help menu, Xamarin > Logs. Open the last .svclog file. There should be some log entry about your issue.

@vlkam Do you have news on this topic?

This is the exception that occurs on a Windows machine:
2017-05-22 11:06:12.3] INFO: : Initializing the previewer with Android API level 25.
Xamarin.Designer|Information|0|
[2017-05-22 11:06:41.2] ERROR: : The renderer encountered an error while rendering this file: System.TypeInitializationException: The type initializer for 'FFImageLoading.Forms.CachedImage' threw an exception.
at MyProject.Pages.User.UserProfile..ctor () in D:\MyProjects\MyProject_git\MyProject\MyProject\Pages\User\UserProfile.xaml.cs:21
at MyProject.Pages.Navigation.UserMenuMasterDetailPage..ctor () in D:\MyProjects\MyProject_git\MyProject\MyProject\Pages\Navigation\UserMenuMasterDetailPage.cs:15
at MyProject.Phone.App..ctor () in D:\MyProjects\test\MyProject\MyProject\App.cs:54
at  at offset 12 in file:line:column :0:0
at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) in :0
and my setup is:
Microsoft Visual Studio Enterprise 2015
Version 14.0.25431.01 Update 3
Microsoft .NET Framework
Version 4.6.01586
Installed Version: Enterprise
Xamarin   <IP_ADDRESS> (3f99c5a)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.
Xamarin Inspector Support   <IP_ADDRESS> (1f3067d)
Adds support for inspecting Xamarin and WPF apps.
Xamarin.Android   <IP_ADDRESS> (b16fb82)
Visual Studio extension to enable development for Xamarin.Android.
Xamarin.iOS   <IP_ADDRESS> (7656cc6)
Visual Studio extension to enable development for Xamarin.iOS.

Does anyone from the FFImage library has any idea why this is happening ? I assume that Xamarin Previwer does not load the Android Render .... but is there any way to disable this ?

Finally I was able to get a full stacktrace of the exception. The error is a ObjectNullReferenceException and the stack trace is:

I will keep digging trying to find a workaround

It's fixed in the latest prerelease nuget :) Thanks @tilutza for https://github.com/luberda-molinet/FFImageLoading/pull/612

Hi, have somebody did make it work using xaml previwer?
I'm trying to use an SvgCachedImage on my app but I'm getting this exception:
ERROR: : The renderer encountered an error while rendering this file: System.TypeInitializationException: The type initializer for 'FFImageLoading.Forms.CachedImage' threw an exception.
at FFImageLoading.Svg.Forms.SvgCachedImage.set_DownsampleUseDipUnits (System.Boolean value) in C:\projects\ffimageloading\source\FFImageLoading.Svg.Forms.Shared\SvgCachedImage.cs:15
at  at offset 12 in file:line:column :0:0
at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) in :0
And when I run the app, the image is not visible.
Any ideas about what I'm doing wrong?

We seem to be observing an issue with Xamarin Previewer with the latest FFImageLoading package. We first observed this on the previewer for both Android and iOS however after updating to latest FFImageLoading release, we only observe this on iOS. Getting an error:
An exception occurred while rendering the control.
System.TypeInitializationException was thrown.
The Type Initializer for "FFImageLoading.Forms.CachedImage" threw an exception.
##Version Information
=== Visual Studio Enterprise 2017 for Mac ===
Version 7.0.1 (build 24)
Installation UUID: 294bd613-94d8-444e-8fd2-8714ab8fb459
Runtime:
Mono <IP_ADDRESS> (2017-02/5077205) (64-bit)
GTK+ 2.24.23 (Raleigh theme)
            Package version: 500010001

=== NuGet ===
Version: <IP_ADDRESS>3
=== Apple Developer Tools ===
Xcode 8.3.3 (12175.1)
Build 8E3004b
=== Xamarin.Mac ===
Version: <IP_ADDRESS> (Visual Studio Enterprise)
=== Xamarin.Android ===
Version: <IP_ADDRESS> (Visual Studio Enterprise)
Android SDK: /Users/joseluizlacerdasoares/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
6.0 (API level 23)
7.0 (API level 24)
7.1 (API level 25)
SDK Tools Version: 25.2.5
SDK Platform Tools Version: 25.0.5
SDK Build Tools Version: 26.0.0
Java SDK: /usr
java version "1.8.0_101"
Java(TM) SE Runtime Environment (build 1.8.0_101-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.101-b13, mixed mode)
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL
=== Xamarin.iOS ===
Version: <IP_ADDRESS> (Visual Studio Enterprise)
Hash: d2270eec
Branch: d15-2
Build date: 2017-05-22 16:30:53-0400
=== Build Information ===
Release ID: 700010024
Git revision: 7ab1ca2ced6f584e56b7a0d4d321d00775cd95c9
Build date: 2017-05-19 05:44:51-04
Xamarin addins: 08d17158f3365beee5e60f67999e607cce4b3f93
Build lane: monodevelop-lion-d15-2
=== Operating System ===
Mac OS X 10.12.5
Darwin 16.6.0 Darwin Kernel Version 16.6.0
Fri Apr 14 16:21:16 PDT 2017
root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64

Indeed, this bug has not been fixed yet.

I think it's not necessary anymore
LiveXAML much better than XAML previewer
http://www.livexaml.com/

@vlkam so what you mean is that it works fine with LiveXAML? so no need to do anything in the end

Xamarin XAML previewer has a lot of limitation, bugs and requires the design time data.
LiveXAML works directly in simulators and devices, with real data and practically hasn't the limitation and doesn't require the design time data. LiveXAML works even with Fody and Realm
Therefore, by my opinion,  no reason for using Xamarin XAML previewer and no reason for solving this issue

@molinch @vlkam We should fix the official XAML Previewer anyway as @vlkam is a third party solution. I'll debug it later.

After trying to debug it, i got nowhere as Visual Studio for Mac IDE logs are not very useful. Could someone send me some full stack trace from Windows so I could move on?

https://github.com/luberda-molinet/FFImageLoading/issues/860


How to set vmax and vmin of cmap in ClusterMapPlotter()

Good morning,
I would like to set a same range, i.e. vmin and vmax, of cmap in the heatmap for several figures using the function ClusterMapPlotter(). My code is as follows:
heatmap = pch.ClusterMapPlotter(
data=heatmap_data.T,
top_annotation=col_ha,
col_cluster=False,
row_cluster=True,
label="values",
col_dendrogram=False,
row_dendrogram=True,
show_rownames=True,
show_colnames=False,
legend_gap=7,
vmin=0,
vmax=30,
cmap="parula",
rasterized=True,
tree_kws={"row_cmap": "Dark2"},
ylabel=f"Protein (10 in total)",
xlabel=f"Run (100 in total)",
legend_hpad=7,
yticklabels=True,
xticklabels_kws={"labelrotation": -45, "labelcolor": "blue"},
yticklabels_kws={"labelsize": 8},
)
However, the legend in the output figure did not show the range from 0 to 30, but the real minimum and maximum in the heatmap_data.
I would like to know how to solve this problem. Thank you!

Were you using the latest version of PyComplexHeatmap? If not, please upgrade to the latest version. @XuejianXiong

Thank you @DingWB. I upgraded to the version 1.7.3, but still have the same problem.

This is weird; this bug was fixed a long time ago.
Please print heatmap.vmin, heatmap.vmax, heatmap.legend_kws after running heatmap = pch.ClusterMapPlotter(**).

The output looks good "30 0 {'vmin': 0, 'vmax': 30}". However in the figures (attached), it still shows different values in the legend for different heatmaps.

Please install the latest version from github, and follow this example to see whether the legend has the same vmax and vmin as provided in the code (vmin=-2.2,vmax=2.3,).
If it is still not working, then you need to paste your data and code here.

Hi @XuejianXiong ,
How is it going? Did it work?

Hi @DingWB,
It worked! It seems my virtual environment caused issues with the package upgrading. I have to manually delete the old version and reinstall using pip.
Thank you for help!
Best,
Xuenjian


Create a better server launch flow

At the moment, we simply issue an API request to jupyterhub for a server spawn and wait until jupyterhub reports that this server is running. This is definitely not the way we want to handle server launches.
We should improve the sequence to provide some extra feedback to the user about what is happening. Most importantly, the notebooks service flask app should not block waiting for the server to spawn. We should discuss how to move this forward -- some options:

if the server for <namespace>/<project>/<sha> is not running, the endpoint should return an page that shows the status of the pod/server being launched and a 202 return code
if the server for <namespace>/<project>/<sha> is running, the server/notebook is returned as normal
the page with the status should redirect to the running server once it's up
we should use k8s/docker clients for checking on the server spawn so we can report errors as they arise. If the pod launch errors, call DELETE on the JH API for the server so that it is immediately removed from the proxy etc.
do we need a <namespace>/<project>/<sha>/status endpoint? Or just <namespace>/<project>/<sha>?status? No, use CRUD and make POST launch the notebook, GET gives the status of the launch

Once this is done, it will most likely fix #23

@rokroskar Closed via #25?

yep.


pandas not detected as module?

Pandas not detected as modul?
I have installed pandas module using pip install pandas in my env and the code has worked properly, but i still got notif error and problem in my vscode.
Any solution?

This is a badly formatted issue. Please follow the template so that the issue can be resolved. There could be a host of problems causing the issue, it is impossible to narrow it down without knowing more.
From just this information, I can recommend the following

Try Python: Restart Language Server in the command palette
See if the python version selected is really the you're running it with

the problem was gone when i installed the pandas module out of my environment.
but i still wondering why it's not detected on my env 🤔


please add scale by total pixels, mega pixel

Hello,
would you please add an option in the upscale image that needs to scale up or down based on the total megapixel target

Add to work list.

ImageScaleRestoreV2 node add the total_pixel method to scale images.

woow, it is looking great, thank you.

this is my solution to downscale images that match the latent size format.
Can you add the (multiple of) to your node?
By the way, I really appreciate all of your hard work and this great extension.

this is my solution to downscale images that match the latent size format. Can you add the (multiple of) to your node?

Look for this node, maybe it can meet your needs

this is great 👍 thank you.
also is it possible to add a scale ratio output (float) to show the scale ratio between input and output, it will be handy for ultimate SD upscale or other upscaler methods

also is it possible to add a scale ratio output (float) to show the scale ratio between input and output, it will be handy for ultimate SD upscale or other upscaler methods

It is easy to achieve using numerical calculation nodes.


clarify embed character limit

discordeno/discordeno#1913

The template repositories code and management is now located here: https://github.com/discordeno/discordeno/tree/main/template


Environment Configuration Options Missing from 'Publish To AWS' for Elastic Beanstalk

Describe the bug
There are numerous configuration options missing from 'Publish to AWS' for Elastic Beanstalk that require the environment be completely rebuilt or, when an initial deployment fails, leaves the user unable to reconfigure the environment via the Console to remedy if the environment is not in a 'Ready' state.
It would be helpful if these options were available in the AWS Toolkit UI when initially configuring an environment

Load Balancer Options: these changes require the recreation of a load balancer and therefore a new environment to change and aren't currently available in the AWS Toolkit UI

Internal vs Public
Shared vs Dedicated Application Load Balancer

Instance log streaming to CloudWatch logs

Log streaming: if there is an issue with an initial deployment and the user is unable to see the logs, he can't determine the root cause to resolve and can't turn this option on when the environment is not in a 'Ready' state

To Reproduce

Publish an Elastic Beanstalk environment through the 'Publish to AWS' context menu. During configuration, if you need to configure your Load Balancer to be internal or shared, you cannot. If you try to change these settings through the console you receive the error:
Updating Load Balancer listener named: arn:aws:elasticloadbalancing:xxxxx failed Reason: Resource handler returned message: "The following target groups cannot be associated with more than one load balancer: arn:aws:elasticloadbalancingxxxx (Service: ElasticLoadBalancingV2, Status Code: 400,..)" (HandlerErrorCode: ServiceLimitExceeded). Currently if you need these options you can't use the tool.
Publish an Elastic Beanstalk environment through the 'Publish to AWS' context menu. If there are any errors with the application deployment the environment configuration will complete, but the environment will be in a Severe state since the health check fails. At that point the only diagnostics to determine the root cause would be in the application logs. However, you cannot turn on Cloudwatch log streaming in the Console because the application is not in a 'Ready' state. In my case there was a dll that didn't work on the linux deployment and caused the application to fail during loading.

Expected behavior
These settings are available through the 'Publish to AWS' UI since they directly impact the user's ability to configure the initial environment.
Development System (please complete the following information):

Windows Version: Windows Server 2022
Visual Studio Version: Visual Studio 2022
AWS Toolkit for Visual Studio Version: 1.48

Hi @Pelias2525, this is great feedback - thanks for bringing this to our attention. We are investigating this further

Hi @Pelias2525, I'd like to understand a bit more about the nature of your deployments. In using Publish to AWS, are you curating the ideal development environment or are you using it to update your Production environment?

I'm looking at migrating my EC2 / IIS .Net sites to Elastic Beanstalk. AWS Toolkit is a large factor in determining the feasibility of such a migration. Currently I cannot use the Publish to AWS functionality, because I need to specify a shared load balancer, which is not offered by that UI. That forces me to create each Beanstalk environment through the Console (which is fine). However, after doing so, I still can't use the Publish to AWS functionality, since environments set up through the Console do not appear in the Publish To AWS UI. I can see the Console-created Environments using the Legacy UI, would work fine if it will continue to be supported. However, I know it's AWS' intention to remove the Legacy UI at some point. I am extremely concerned about the possibility of migrating my sites over to Elastic Beanstalk and at some point in the future being left in a situation where the Legacy tooling is removed and it's replacement (Publish to AWS) does not provide the ability to publish to the environments that I will have configured through the Console.

I can appreciate that different sized organizations have different needs for controlling their environments and that there can be value in Infrastructure as Code. From my understanding, if you want a high level of control over your environments you use IaC with Elastic Container Service and control every level of minutia.
On it's front page, Elastic Beanstalk is advertised as "AWS Elastic Beanstalk deploys web applications so that you can focus on your business." When I search for 'ASP.Net Core Elastic Beanstalk', I get this walkthrough from AWS showing how to deploy to Elastic Beanstalk using AWS Toolkit and the 'Publish to AWS' feature. In the AWS Tutorial for Deploying ASP.NET Applications to Elastic Beanstalk it states "If you use Visual Studio to develop your application, you can also use the AWS Toolkit for Visual Studio to deploy changes, manage your Elastic Beanstalk environments, and manage other AWS resources."
I want to use AWS Toolkit to deploy my code to Elastic Beanstalk as advertised. I'm fine if the toolkit wants to add IaC functionality, but as of right now:

It doesn't sync with configuration changes made to the environment through other means
It doesn't contain the core features needed to stand up an environment (for me, those items I outlined at the start of my ticket)
It doesn't allow me to deploy my code to an existing environment
It apparently isn't recommended for publishing to production systems

Before I go to the trouble of rearchitecting, I would like assurance from AWS that the toolkit will remain the tool that it is advertised to be. If that's the Legacy UI, then fine, but don't take it away in 2 years without a viable alternative after I've already committed to using it. Please advise. Thank you.

Our goal is to add additional capabilities to Publish to AWS experience to bring it to feature parity with the legacy Publish to Elastic Beanstalk experience. Once we have a better alternative to the legacy Publish to Elastic Beanstalk experience, we will announce a removal date for the Publish to Elastic Beanstalk experience.
As a reminder, the Publish to AWS and Publish to Elastic Beanstalk features are not intended for production deployments. Instead, we suggest using CDK or CloudFormation. See this reply from rbbarad for more details.

Appreciated. Please consider adding those items described in my original bug report, so that I can use the Publish to AWS experience as intended. Please also keep in mind that much of the value of the AWS Toolkit is to provide a UI mechanism within Visual Studio to automate deployments to Beanstalk Environments without having to setup command line interactions or reconfigure environments in every deployment. Not every user has their source code within an AWS hosted repository, nor needs the overhead of configuring the CDK / CloudFormation for simple use cases. Thank You.


Fixed #1713 Option to Suppress "SQLite does not support setting the s…

Only show the message that SQLite doesn't support schema changes once

This has been implemented as part of Flyway 5.0. Closing.


Assert valid learning rate in LAMB optimizer

All optimizers apart from LAMB use an assert statement to make sure that hyper_params.learning_rate is not None. This PR introduces such an assert also to LAMB.

Codecov Report

Merging #352 into master will decrease coverage by 0.32%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##           master     #352      +/-   ##
==========================================
- Coverage   79.33%   79.01%   -0.33%
==========================================
  Files          34       34
  Lines        2255     2249       -6
==========================================
- Hits         1789     1777      -12
- Misses        466      472       +6

Impacted Files
Coverage Δ

flax/optim/lamb.py
43.90% <0.00%> (-1.10%)
:arrow_down:

flax/nn/utils.py
93.33% <0.00%> (-4.67%)
:arrow_down:

flax/nn/stochastic.py
71.05% <0.00%> (-2.64%)
:arrow_down:

flax/jax_utils.py
62.37% <0.00%> (-2.05%)
:arrow_down:

flax/nn/base.py
91.70% <0.00%> (-0.23%)
:arrow_down:

flax/nn/normalization.py
82.71% <0.00%> (ø)

flax/nn/attention.py
78.71% <0.00%> (+0.10%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 81752d8...5f7c2e2. Read the comment docs.


Introduce CAN Bus support

CAN Bus support on EVE-OS
This PR introduces the support to the CAN Bus on EVE-OS.
In summary:

New I/O device types introduced in EVE-API are defined in pillar
A canbus pillar's library is introduced to control and setup CAN interfaces
Support to CAN interfaces is added to pillar: It reads configuration from device model and setup Physical and/or Virtual CAN interfaces defined in the model
CAN interface "passthrough" is implemented for KVM
The corresponding documentation is provided
The capability to list physical CAN interfaces is added to spec.sh script

The design document for this implementation can be accessed in the LF Edge's Wiki: https://wiki.lfedge.org/display/EVE/CAN+Bus+support
Summary of commits:
1d0b110b9  spec.sh: List physical CAN interfaces
d13e4af44 docs: Add CAN Bus documentation
4c9e53b15 kvm: Introduce CAN Bus passthrough
005287f65 pillar: Introduce CAN Bus support
a58276844 pillar: Introduce CAN Bus library
Regarding Yetus errors:
Error 1
The following error is also present in several other source files:
golangcilint: import 'github.com/vishvananda/netlink' is not allowed from list 'Main' (depguard)

Running Yetus locally I found more than 2k incidences of this error throughout the source code. I couldn't find the root cause and how to solve it.

Error 2
Regarding the following error:
golangcilint: var-naming: don't use ALL_CAPS in Go names; use CamelCase (revive)
revive: don't use ALL_CAPS in Go names; use CamelCase https://revive.run/r#var-naming

I won't fix because I just followed the constants defined for the netlink interface in the Linux kernel and in exactly the same way as other similar constants are defined (all in capital letters) in the unix package from golang: https://github.com/Golangltd/leafltd/blob/282493799a89/src/golang.org/x/sys/unix/ztypes_linux_amd64.go#L331

Edit: Yetus error were fixed and/or properly ignored.

There are no tests?

Regarding:
The following error is also present in several other source files:
golangcilint: import 'github.com/vishvananda/netlink' is not allowed from list 'Main' (depguard)
Running Yetus locally I found more than 2k incidences of this error throughout the source code. I couldn't find the root cause and how to solve it.
Seems like we need a config file per https://github.com/golangci/golangci-lint/issues/3877 to allow imports other than standard packages. Do we not have a config file at all?

Regarding: The following error is also present in several other source files:
golangcilint: import 'github.com/vishvananda/netlink' is not allowed from list 'Main' (depguard) Running Yetus locally I found more than 2k incidences of this error throughout the source code. I couldn't find the root cause and how to solve it.
Seems like we need a config file per golangci/golangci-lint#3877 to allow imports other than standard packages. Do we not have a config file at all?

AFAIK we don't. It should be the reason, as you pointed out. I will take a look.
I've addressed most of the comments and updated the PR.

There are no tests?

Tests were made on real hardware with CAN controllers from devices we support. Unfortunately unit-tests are a bit trick here because real CAN controllers supports in general only a set of the parameters that can be set by the library introduced here. For non-supported parameters I've checked the result comparing with returns from iproute2, which were the same. One option is to create a dummy CAN driver kernel module that would accept all the parameters, so we could check the whole data route from netlink IPC to the driver itself... For now I can just compromise with the tests on real hardware, this introduction can unblock those who are needing CAN device access, and I can continue to working on unit tests....

There are two ways:

mock the interface - which I think is not very useful as there is not much of a logic in the code here
use https://netmodule-linux.readthedocs.io/en/latest/howto/can.html

@christoph-zededa can you mark those of your comments which @rene and no longer open as resolved.
Kicking off tests.

I don't have a "Resolve" Button, but I wrote "resolved" for all resolved comments.

Updates in this PR:

Rebase on top of master
Addressed more comments from @christoph-zededa

Regarding: The following error is also present in several other source files:
golangcilint: import 'github.com/vishvananda/netlink' is not allowed from list 'Main' (depguard) Running Yetus locally I found more than 2k incidences of this error throughout the source code. I couldn't find the root cause and how to solve it.
Seems like we need a config file per golangci/golangci-lint#3877 to allow imports other than standard packages. Do we not have a config file at all?

@eriknordmark , I've opened the PR https://github.com/lf-edge/eve/pull/3813 to fix this error....

Updates in this PR:

Rebase on top of master
Address new comment from @christoph-zededa

LGTM

I don't have a "Resolve" Button, but I wrote "resolved" for all resolved comments.

Odd that the creator of a comment can't mark it as resolved; that's the only person who can tell that the answer was satisfactory IMHO. But the github creators have some other model in mind I guess.

Updates in this PR:

Fix more Yetus errors

Do we have other PRs related to this one? EVE-API?

First off, huge kudos for diving into adding CAN support – that's really cool and super valuable for the project! 😊 I've been checking out your PR, and it's clear you've put a ton of work into it. I've got a few thoughts and was hoping we could chat about them a bit.
From what I understand, the current approach introduces CAN-related code directly into the Pillar service. Given Pillar's role as primarily an orchestration service, I wonder if embedding low-level device handling directly into it might lead us away from its core responsibilities. Would it be possible to encapsulate this functionality into a dedicated package that Pillar could then utilize? This could help keep Pillar's codebase more focused and modular.
Alternatively, considering existing tools like can-utils could offer a way to leverage community-maintained solutions without reinventing the wheel. I recognize that relying on external tools introduces its challenges, especially around integrating and managing interactions. However, the benefits of such an approach, including reducing the complexity within Pillar and potentially increasing our project's maintainability, seem worth discussing.
My concern is primarily about the long-term sustainability and maintainability of our codebase. We've observed with the network subsystem that having a very narrow maintenance corridor can create bottlenecks. It might be prudent to explore strategies that avoid centralizing too much specialized knowledge within a small group, thus increasing our bus factor.
Of course, there are significant advantages to developing in-house solutions, particularly in terms of control and adaptability to our specific needs. Yet, I believe finding a balance that allows us to benefit from existing solutions while maintaining the flexibility we require is key.
This brings me to a broader question that I think is worth discussing as a team: When should we opt to reuse existing tools, and when is it preferable to implement from scratch? And if we decide to implement, where should this code live to best maintain the modular and maintainable nature of our project?
I hope these thoughts are taken in the collaborative spirit they're intended. Just wanted to say again how much I appreciate the effort you've put into this – it's really impressive work! I'm not suggesting we redo anything that's already been done; just thought it'd be great to chat about how we tackle similar challenges in the future.
Summoning @eriknordmark, @rouming, @milan-zededa, @christoph-zededa, and whoever is interested =)

@OhmSpectator , all your questions are 1000% valid for discussion. In respect to CAN library introduced in this PR, my very first attempt was to look at netlink's Go package in order to see if it fully supports our needs to manage CAN interfaces. Unfortunately, it does not. CAN management was introduced in beta version but still it doesn't support all the parameters that we want to support on EVE. The approach chosen for implementation it was to use the netlink kernel IPC to manage CAN interfaces. This is exactly how iproute, netlink and other tools do to control network interfaces. In the near future my plan is to port this library to netlink's package, since we also use this package to do network management.
Regarding what we should use on pillar, well, that's a broader discussion, outside the scope of this PR. So I suggest that we create an issue or drop these questions on our Slack channel so we can discuss with the whole EVE team. IMO run tools under the hood are more error prone and makes harder to parse arguments, input/output, receive asynchronous events.... However, I do agree that for some resources we can use dedicated daemons to control as long as we have a good IPC mechanism to talk to them. A very good example of a nice approach: ModemManager.... that's way better to delegate the complexity of modem management to the daemon service and just talk to this service using dbus, libraries and/or another good IPC. That's what @milan-zededa did. But in summary, pillar controls critical and many system resources, that inevitable to face and handle (at some point) low level code.
Again, thanks your comments and questions...

Do we have other PRs related to this one? EVE-API?

All changes for EVE-API and Kernel were already merged. The initial changes for EVE-API are here: https://github.com/lf-edge/eve-api/commit/5dda59a3d85efea7439b97406161eb097ed7e532 (they were updated by @christoph-zededa to add USB devices)

Updates in the PR:

Addressed latest comments.

Big kudos to @christoph-zededa which proposed the changes now implemented in SetupCAN() in order to remove the switch/case for checking properties, reducing the complexity of the code.

@OhmSpectator thanks for your very thoughful comments.
We definitely want to think about what we can leverage particularly when it makes our lives easier and less complex by leveraging a community which is handling the complexity. Modem manager is an example of that.
For CANbus the need is much simpler - be able to configure the CAN interfaces.
Ideally that should be done using something like the netlink package, but as Renê said that isn't there yet. We should consider switching to using it once it gets more support (or contribute the missing pieces once it is out of beta).
The can-utils do not seem to focus on the configuration but instead have lots of other useful tools to test/debug and use canbus from scripts etc.
The general question about both how we leverage existing pieces but also how we plug them into EVE is something we should talk more explicitly about (as well as figuring out which existing pieces we can replace and/or refactor.)

Build failed with
#19 5.072 ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.16/main: temporary error (try again later)
#19 5.072 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.16/main: No such file or directory
#19 10.08 ERROR: https://dl-cdn.alpinelinux.org/alpine/v3.16/community: temporary error (try again later)
#19 10.08 WARNING: Ignoring https://dl-cdn.alpinelinux.org/alpine/v3.16/community: No such file or directory
so I restarted it.

Re-run eden

ok @eriknordmark , in any case I will push an update for this PR... there is an small thing to fix....

LGTM, thanks @rene for this!
I have one general non-critical comment regarding canbus.go module: it looks like you have a group of functions which take link *CAN as first parameter, which makes it feel like a class method. For example
func SetTermination(link *CAN, termination uint16) error;

turns into
func (link *CAN) SetTermination(termination uint16) error;

So you can write this:
link.SetTermination(canProps.Termination)

@uncleDecart , the reason to not use a receiver in those functions it's mainly because I followed the same approach as in the netlink's Go package functions.... my plan for the future is to merge canbus library into netlink and push it upstream....

Updates in this PR:

Rebase on top of master
Addressed @uncleDecart 's comment
Rewrote SetupCAN: Although @christoph-zededa 's proposal had reduced the complexity, Yetus was still complaining about the complexity of SetupCAN, it was giving false positive for duplicated lines, and the function was indeed still complex... so at the end I rewrote this function with the following improvements:

Validation of properties passed in order to setup CAN
Removal of maps to different functions, now just a few auxiliary functions are needed
Incremental setup, i.e., as soon as we have all properties for a given parameter, setup the parameter in the CAN interface. For instance, as soon as we have all BitTiming properties, setup this parameter in the CAN interface

Just return at the first error


AttributeError: '_UserObject' object has no attribute 'add_slot'

Please make sure that this is a bug. As per our
GitHub Policy,
we only address code/doc bugs, performance issues, feature requests and
build/installation issues on GitHub. tag:bug_template
System information

Have I written custom code (as opposed to using a stock example script provided in TensorFlow): No
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Ubuntu 18.04
Mobile device (e.g. iPhone 8, Pixel 2, Samsung Galaxy) if the issue happens on mobile device: NA
TensorFlow installed from (source or binary): Source
TensorFlow version (use command below):v2.6.0-rc1-12-gb61c987109e 2.6.0-rc1
Python version:3.8.5
Bazel version (if compiling from source):3.7.2
GCC/Compiler version (if compiling from source):7.5.0
CUDA/cuDNN version: NA
GPU model and memory: NA

You can collect some of this information using our environment capture
script
You can also obtain the TensorFlow version with:

TF 1.0: python -c "import tensorflow as tf; print(tf.GIT_VERSION, tf.VERSION)"
TF 2.0: python -c "import tensorflow as tf; print(tf.version.GIT_VERSION, tf.version.VERSION)"

Describe the current behavior
git checkout r2.6
build the source (CPU only), install, generate mnist model, then try to convert it, has issue.

install
sudo apt install python3-dev python3-pip
pip install -U --user pip numpy wheel
pip install -U --user keras_preprocessing --no-deps

bazel build //tensorflow/tools/pip_package:build_pip_package
./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tmp/tensorflow_pkg
pip install /tmp/tensorflow_pkg/tensorflow-2.6.0rc1-cp38-cp38-linux_x86_64.whl
pip install tensorflow-model-optimization

generate model: refer to below code
convert:
bazel-bin/tensorflow/lite/python/tflite_convert --saved_model_dir mnist_qat_notdocker/float_model/ --output_file oo

error message:
...
File "/home/tensorflow2/tensorflow/bazel-bin/tensorflow/lite/python/tflite_convert.runfiles/org_tensorflow/tensorflow/python/saved_model/load.py", line 448, in _load_nodes
slot_variable = optimizer_object.add_slot(
AttributeError: '_UserObject' object has no attribute 'add_slot'
Describe the expected behavior
Contributing

Do you want to contribute a PR? (yes/no):
Briefly describe your candidate solution(if contributing):

Standalone code to reproduce the issue
Provide a reproducible test case that is the bare minimum necessary to generate
the problem. If possible, please share a link to Colab/Jupyter/any notebook.
import tempfile
import os
import tensorflow as tf
from tensorflow import keras
Load MNIST dataset
mnist = keras.datasets.mnist
(train_images, train_labels), (test_images, test_labels) = mnist.load_data()
Normalize the input image so that each pixel value is between 0 to 1.
train_images = train_images / 255.0
test_images = test_images / 255.0
Define the model architecture.
model = keras.Sequential([
keras.layers.InputLayer(input_shape=(28, 28)),
keras.layers.Reshape(target_shape=(28, 28, 1)),
keras.layers.Conv2D(filters=12, kernel_size=(3, 3), activation='relu'),
keras.layers.MaxPooling2D(pool_size=(2, 2)),
keras.layers.Flatten(),
keras.layers.Dense(10)
])
Train the digit classification model
model.compile(optimizer='adam',
loss=tf.keras.losses.SparseCategoricalCrossentropy(from_logits=True),
metrics=['accuracy'])
model.fit(
train_images,
train_labels,
epochs=1,
validation_split=0.1,
)
model.save('mnist_qat_notdocker/float_model')
Other info / logs Include any logs or source code that would be helpful to
diagnose the problem. If including tracebacks, please include the full
traceback. Large logs and files should be attached.

@tensorbuffer ,
I was able to execute the given code without any error.Please find the gist here.Also please take a look at this issue with similar error.It helps.Thanks!

@tilakrayal Thanks for the quick response. The problem happens when you convert, your link does not have the convert step.
Yes I have searched and read that issue you referred, in that case I think a custom OP is used, different than mine.
I also tried to do a new conda environment and do everything there (checkout source, build pip package, install, run python to save model and convert). I had the same issue with branch 2.6, however 2.5 branch works. So I do believe there's some problem.

Using the Python API seems to convert without problems. I think it might be a problem with tflite_convert built with Bazel.
https://colab.research.google.com/drive/1uuAl1BP8v1R1xhcOyBIDyh9nkO8AW6xD?usp=sharing
converter = tf.lite.TFLiteConverter.from_saved_model('mnist_qat_notdocker/float_model')
converter.target_spec.supported_ops = [tf.lite.OpsSet.TFLITE_BUILTINS, tf.lite.OpsSet.SELECT_TF_OPS]
tflite_model = converter.convert()
with open('mnist_qat_notdocker/float_model/model_float32.tflite', 'wb') as w:
    w.write(tflite_model)

Thanks. Ok I will try that. I remember that with python convert, I can set attributes like inference type, while in tflite_convert command line there's no such option. Wondering if they are the same.

I remember that command line tools were deprecated a long time ago. They are probably rarely maintained.

As mentioned by @PINTO0309 Python API is recommended for TF Lite converter,  command line tool support basic model conversion.
https://www.tensorflow.org/lite/convert
Feel free to reopen if running into issues using Python API for conversion. Thanks!

I have same problem, in TF2.6. But, I fix it just need append
from tensorflow.keras import Model
hope it's useful for you.