Alluxio master failed to start after stopping and restart with backup

Alluxio Version:
alluxio-2.4
Describe the bug
I have to stop alluxio master to change my ufs sync interval. After stooping when I try to start the master is not able to start.
I then tried to start it with backup in that case also it fails and not able to start.
It's giving this error
java.io.IOException: io.grpc.StatusRuntimeException: CANCELLED: HTTP/2 error code: CANCEL
Received Rst Stream
at alluxio.master.journal.raft.RaftJournalWriter.flush(RaftJournalWriter.java:100)
at alluxio.master.journal.AsyncJournalWriter.doFlush(AsyncJournalWriter.java:295)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: io.grpc.StatusRuntimeException: CANCELLED: HTTP/2 error code: CANCEL
Received Rst Stream
at io.grpc.Status.asRuntimeException(Status.java:533)
at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:449)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at alluxio.grpc.GrpcChannel$ChannelResponseTracker$1$1.onClose(GrpcChannel.java:172)
at io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:426)
at io.grpc.internal.ClientCallImpl.access$500(ClientCallImpl.java:66)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:689)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$900(ClientCallImpl.java:577)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:751)
at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:740)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
Starting master @  Logging to /home/ubuntu/alluxio-master/alluxio/logs
--- [ FAILED ] The master  is not serving requests.
--- Printing the log tail for /home/ubuntu/alluxio-master/alluxio/logs/master.log

BEGIN

2020-10-20 17:37:36,950 DEBUG AuthenticatedChannelClientDriver - Authentication established for GrpcChannelKey{ClientType=RaftClient, ClientHostname=ip-172-20-6-164.ap-south-1.compute.internal, ServerAddress=GrpcServerAddress{HostName=<IP_ADDRESS>, SocketAddress=<IP_ADDRESS>:19200}, ChannelId=541e6eb4-1165-4fb9-b657-092eacf08eca}
2020-10-20 17:37:36,951 DEBUG GrpcMessagingClient - Created a messaging client connection: GrpcMessagingClientConnection{ConnectionOwner=CLIENT, ConnectionId=189, TransportId=GrpcChannelKey{ClientType=RaftClient, ClientHostname=ip-172-20-6-164.ap-south-1.compute.internal, ServerAddress=GrpcServerAddress{HostName=<IP_ADDRESS>, SocketAddress=<IP_ADDRESS>:19200}, ChannelId=541e6eb4-1165-4fb9-b657-092eacf08eca}}
2020-10-20 17:37:36,951 DEBUG NettyClientHandler - [id: 0xd1a31255, L:/<IP_ADDRESS>:44770 - R:<IP_ADDRESS>/<IP_ADDRESS>:19200] OUTBOUND HEADERS: streamId=381 headers=GrpcHttp2OutboundHeaders[:authority: <IP_ADDRESS>:19200, :path: /alluxio.grpc.messaging.MessagingService/connect, :method: POST, :scheme: http, content-type: application/grpc, te: trailers, user-agent: grpc-java-netty/1.28.1, channel-id: 541e6eb4-1165-4fb9-b657-092eacf08eca, grpc-accept-encoding: gzip] streamDependency=0 weight=16 exclusive=false padding=0 endStream=false
2020-10-20 17:37:36,951 DEBUG ClientConnection - a75ad6e0-6ab3-4452-b81f-29753f5779dc - Setting up connection to <IP_ADDRESS>:19200
2020-10-20 17:37:36,951 DEBUG GrpcMessagingConnection - Submitted request(1) of type: io.atomix.copycat.protocol.ConnectRequest. Connection: GrpcMessagingClientConnection{ConnectionOwner=CLIENT, ConnectionId=189, TransportId=GrpcChannelKey{ClientType=RaftClient, ClientHostname=ip-172-20-6-164.ap-south-1.compute.internal, ServerAddress=GrpcServerAddress{HostName=<IP_ADDRESS>, SocketAddress=<IP_ADDRESS>:19200}, ChannelId=541e6eb4-1165-4fb9-b657-092eacf08eca}} FireAndForget: false
2020-10-20 17:37:36,951 DEBUG NettyClientHandler - [id: 0xd1a31255, L:/<IP_ADDRESS>:44770 - R:<IP_ADDRESS>/<IP_ADDRESS>:19200] OUTBOUND DATA: streamId=381 padding=0 endStream=false length=4108 bytes=00000010070a0208011a802001fc01002461373561643665302d366162332d343435322d623831662d3239373533663537373964630000000000000000000000...
2020-10-20 17:37:36,951 DEBUG GrpcMessagingServiceClientHandler - Creating a messaging server connection: GrpcMessagingServiceClientHandler{ServerAddress=<IP_ADDRESS>:19200, ClientAddress=/<IP_ADDRESS>:44770}
2020-10-20 17:37:36,951 DEBUG GrpcMessagingServiceClientHandler - Created a messaging server connection: GrpcMessagingServerConnection{ConnectionOwner=SERVER, ConnectionId=190, TransportId=GrpcMessagingServiceClientHandler{ServerAddress=<IP_ADDRESS>:19200, ClientAddress=/<IP_ADDRESS>:44770}}
2020-10-20 17:37:36,952 DEBUG GrpcMessagingConnection - Received a new message. Connection: GrpcMessagingServerConnection{ConnectionOwner=SERVER, ConnectionId=190, TransportId=GrpcMessagingServiceClientHandler{ServerAddress=<IP_ADDRESS>:19200, ClientAddress=/<IP_ADDRESS>:44770}}, RequestHeader: requestId: 1
, ResponseHeader:
2020-10-20 17:37:36,952 DEBUG GrpcMessagingConnection - Handling request(1) of type: io.atomix.copycat.protocol.ConnectRequest. Connection: GrpcMessagingServerConnection{ConnectionOwner=SERVER, ConnectionId=190, TransportId=GrpcMessagingServiceClientHandler{ServerAddress=<IP_ADDRESS>:19200, ClientAddress=/<IP_ADDRESS>:44770}}
2020-10-20 17:37:36,952 DEBUG GrpcMessagingConnection - Sending response of type: io.atomix.copycat.protocol.ConnectResponse for request(1). Connection: SERVER
2020-10-20 17:37:36,952 DEBUG NettyClientHandler - [id: 0xd1a31255, L:/<IP_ADDRESS>:44770 - R:<IP_ADDRESS>/<IP_ADDRESS>:19200] INBOUND HEADERS: streamId=381 headers=GrpcHttp2ResponseHeaders[:status: 200, content-type: application/grpc, grpc-encoding: identity, grpc-accept-encoding: gzip] padding=0 endStream=false
2020-10-20 17:37:36,952 DEBUG NettyClientHandler - [id: 0xd1a31255, L:/<IP_ADDRESS>:44770 - R:<IP_ADDRESS>/<IP_ADDRESS>:19200] INBOUND DATA: streamId=381 padding=0 endStream=false length=4110 bytes=00000010091204080110001a802001f50101d2010101000c3137322e32302e362e31363400004b0002b5000101d2010101000c3137322e32302e362e31363400...
2020-10-20 17:37:36,952 DEBUG GrpcMessagingConnection - Received a new message. Connection: GrpcMessagingClientConnection{ConnectionOwner=CLIENT, ConnectionId=189, TransportId=GrpcChannelKey{ClientType=RaftClient, ClientHostname=ip-172-20-6-164.ap-south-1.compute.internal, ServerAddress=GrpcServerAddress{HostName=<IP_ADDRESS>, SocketAddress=<IP_ADDRESS>:19200}, ChannelId=541e6eb4-1165-4fb9-b657-092eacf08eca}}, RequestHeader: , ResponseHeader: requestId: 1
isThrowable: false
2020-10-20 17:37:36,953 DEBUG GrpcMessagingConnection - Received response of type: io.atomix.copycat.protocol.CommandResponse for request(2). Connection: GrpcMessagingClientConnection{ConnectionOwner=CLIENT, ConnectionId=189, TransportId=GrpcChannelKey{ClientType=RaftClient, ClientHostname=ip-172-20-6-164.ap-south-1.compute.internal, ServerAddress=GrpcServerAddress{HostName=<IP_ADDRESS>, SocketAddress=<IP_ADDRESS>:19200}, ChannelId=541e6eb4-1165-4fb9-b657-092eacf08eca}}
<<< EOF
--- Printing the log tail for /home/ubuntu/alluxio-master/alluxio/logs/master.out

BEGIN
io.netty.handler.codec.http2.Http2Exception$StreamException: Received DATA frame for an unknown stream 377
at io.netty.handler.codec.http2.Http2Exception.streamError(Http2Exception.java:147)
at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.shouldIgnoreHeadersOrDataFrame(DefaultHttp2ConnectionDecoder.java:596)
at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder$FrameReadListener.onDataRead(DefaultHttp2ConnectionDecoder.java:239)
at io.netty.handler.codec.http2.Http2InboundFrameLogger$1.onDataRead(Http2InboundFrameLogger.java:48)
at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readDataFrame(DefaultHttp2FrameReader.java:422)
at io.netty.handler.codec.http2.DefaultHttp2FrameReader.processPayloadState(DefaultHttp2FrameReader.java:251)
at io.netty.handler.codec.http2.DefaultHttp2FrameReader.readFrame(DefaultHttp2FrameReader.java:160)
at io.netty.handler.codec.http2.Http2InboundFrameLogger.readFrame(Http2InboundFrameLogger.java:41)
at io.netty.handler.codec.http2.DefaultHttp2ConnectionDecoder.decodeFrame(DefaultHttp2ConnectionDecoder.java:174)
at io.netty.handler.codec.http2.Http2ConnectionHandler$FrameDecoder.decode(Http2ConnectionHandler.java:378)
at io.netty.handler.codec.http2.Http2ConnectionHandler.decode(Http2ConnectionHandler.java:438)
at io.netty.handler.codec.ByteToMessageDecoder.decodeRemovalReentryProtection(ByteToMessageDecoder.java:501)
at io.netty.handler.codec.ByteToMessageDecoder.callDecode(ByteToMessageDecoder.java:440)
at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:276)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357)
at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379)
at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365)
at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919)
at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:792)
at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:475)
at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:378)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base/java.lang.Thread.run(Thread.java:834)

<<< EOF
To Reproduce

Stop the master
Try to restart
Try to restart with a journal backup still it will not start.

Expected behavior

The Master should be able to start with the last checkpoint. If it failed due to some issues at least it should start from
backup.
It should give a proper error so that will be able to understand why it's failing to start.

Urgency
All alluxio data is getting loose whenever we restart the master.
logs_start.tar.gz
Additional context
Attaching the master log when I try to restart.
logs_start.tar.gz
start with backup master.log
start with backup master.out

@LuQQiu do you have any idea about this issue?

@bf8086 FYI

any thought around this as I am not able to use backup and every time my master down I have to format it.

@jainanuj07 The logs indicate there are some journal flushing issues. Do you mind sharing your Alluxio cluster site property file?

This is my alluxio-site.properties
alluxio.master.mount.table.root.ufs=s3://bounce-alluxio/
alluxio.user.file.metadata.sync.interval=30m
alluxio.worker.network.async.cache.manager.threads.max=10
alluxio.worker.tieredstore.levels=2
alluxio.worker.tieredstore.level0.alias=MEM
alluxio.worker.tieredstore.level0.dirs.path=/mnt/ramdisk
alluxio.worker.tieredstore.level0.dirs.mediumtype=MEM
alluxio.worker.tieredstore.level0.dirs.quota=32GB
alluxio.worker.tieredstore.level1.alias=SSD
alluxio.worker.tieredstore.level1.dirs.path=/alluxio_data
alluxio.worker.tieredstore.level0.dirs.mediumtype=SSD
alluxio.worker.tieredstore.level1.dirs.quota=400GB
alluxio.worker.network.max.inbound.message.size=320MB
alluxio.user.metrics.collection.enabled=true
alluxio.underfs.s3.directory.suffix=_$folder$
alluxio.master.journal.type=EMBEDDED
alluxio.master.daily.backup.enabled=true
alluxio.master.daily.backup.time=18:05
alluxio.user.streaming.data.timeout=90sec

@jainanuj07 Thanks for reporting the issue. There are two issues reported in your ticket,

Failed to start after stopping
Failed to start with backup.

I took a look at the first issue, there is no obvious error exist in the master.log, but master do takes longer than expected to start
2020-10-20 17:15:57,450 INFO  RaftJournalSystem - Initializing Raft Journal System
2020-10-20 17:15:57 to 17:16:13 Reading local copycat segment files
2020-10-20 17:16:13,680 INFO  AbstractPrimarySelector - Primary selector transitioning to SECONDARY
2020-10-20 17:16:14,567 INFO  JournalUtils - Reading journal entries
2020-10-20 17:16:15,314 INFO  Database - Adding new table ...
2020-10-20 17:16:14 to 16:59  Adding and Updating database tables
2020-10-20 17:17:03,100 INFO  AbstractPrimarySelector - Primary selector transitioning to PRIMARY
2020-10-20 17:17:03,108 INFO  AlluxioMasterProcess - All masters started
2020-10-20 17:17:08 to 2020-10-20 17:19:31  Database - Updating table ...
2020-10-20 17:19:31,082 INFO  RaftJournalSystem - Performing catchup. Last applied SN: 9920755. Catchup ID: -3286135040307079950
2020-10-20 17:19:32,090 Caught up  RaftJournalSystem - Caught up in 1008ms. Last sequence number from
2020-10-20 17:19:32,094 INFO  RaftJournalSystem - Shutting down raft journal
2020-10-20 17:19:32,127 INFO  RaftJournalSystem - Journal shutdown complete

The master takes 3.5mins to start which is longer than what alluxio-start.sh master tracks, that's why you saw the error log that master is not serving. After master starts, it immediately being killed maybe by alluxio-stop.sh master.
@jainanuj07 I noticed that the master takes a lot of time to updating database tables, are you using Alluxio catalog service and have many tables and/or partitions?
Will continue looking into the second issue.

For the failed to start from backup, I guess it takes way too long to restore from backup as well, from the logs, i saw
.....
2020-10-20 17:37:27,609 INFO  BackupManager - 1854417 entries from backup applied so far...
2020-10-20 17:37:44,696 INFO  AbstractMaster - TableMaster: Stopped secondary master.
2020-10-20 17:37:44,697 INFO  AbstractMaster - MetaMaster: Stopped secondary master.
2020-10-20 17:37:44,697 INFO  AbstractMaster - FileSystemMaster: Stopped secondary master.

start with backup master.log
2020-10-20 17:37:44,700 INFO  RaftJournalSystem - Shutting down raft journal
2020-10-20 17:37:44,703 INFO  RaftJournalWriter - Closing journal writer. Last sequence numbers written/submitted/committed: 1854416/1854416/1854130

2020-10-20 17:37:45,128 ERROR ProcessUtils - Uncaught exception while running Alluxio master @<IP_ADDRESS>:19998, stopping it and exiting. Exception "java.lang.RuntimeException: alluxio.exception.status.UnavailableException: Failed to complete request: Cannot flush. Journal writer has been closed", Root Cause "alluxio.exception.JournalClosedException: Cannot flush. Journal writer has been closed"

Could be restoring from backup takes unexpected long and then users think the master failed to start and killed it (so that rat journal gets shutting down).

The unexpected long time to start a master with journal/backup may come from the heavy usage of catalog service.
FYI @yuzhu @gpang

The current version is Alluxio-2.4.0-SNAPSHOT, is the version before merging Ratis back to master branch. so the cluster is running with original Copycat embedded journal.

Thanks, @LuQQiu  for looking at the issue.

are you using Alluxio catalog service and have many tables and/or partitions?
yes, I am using an alluxio catalog and I have around 300 tables with each table contains many partitions.
So is this expected behavior if the master is taking a long time to start it will throw an error and get killed ?
What is the solution to make this work. Is there any timeout that I can increase?

For the second question: If the master is taking a long time to start, the master process will not be killed (as I know), but the alluxio-start.sh command will error out since it's not able to check master serving in the given timeout.
I would say try to wait for the master to come up, let the master process running for like 5-10 minutes and see if the master is able to serve requests after that time.

@LuQQiu I do not see master process is running so it means it getting killed or not even started

For the logs_start.tar.gz, looks like master process should be able to come up after an unexpected long time (like 5-10mins), but user kill the process.
For the given start with backup master.log, the master does error out with
2020-10-20 17:37:44,700 INFO  RaftJournalSystem - Shutting down raft journal
2020-10-20 17:37:44,703 INFO  RaftJournalWriter - Closing journal writer. Last sequence numbers written/submitted/committed: 1854416/1854416/1854130

2020-10-20 17:37:45,128 ERROR ProcessUtils - Uncaught exception while running Alluxio master @<IP_ADDRESS>:19998, stopping it and exiting. Exception "java.lang.RuntimeException: alluxio.exception.status.UnavailableException: Failed to complete request: Cannot flush. Journal writer has been closed", Root Cause "alluxio.exception.JournalClosedException: Cannot flush. Journal writer has been closed"
java.lang.RuntimeException: alluxio.exception.status.UnavailableException: Failed to complete request: Cannot flush. Journal writer has been closed
	at alluxio.master.AlluxioMasterProcess.startMasters(AlluxioMasterProcess.java:219)
	at alluxio.master.FaultTolerantAlluxioMasterProcess.gainPrimacy(FaultTolerantAlluxioMasterProcess.java:127)
	at alluxio.master.FaultTolerantAlluxioMasterProcess.start(FaultTolerantAlluxioMasterProcess.java:90)
	at alluxio.ProcessUtils.run(ProcessUtils.java:36)
	at alluxio.master.AlluxioMaster.main(AlluxioMaster.java:55)
Caused by: alluxio.exception.status.UnavailableException: Failed to complete request: Cannot flush. Journal writer has been closed
	at alluxio.master.journal.MasterJournalContext.waitForJournalFlush(MasterJournalContext.java:80)
	at alluxio.master.journal.MasterJournalContext.close(MasterJournalContext.java:91)
	at alluxio.master.journal.StateChangeJournalContext.close(StateChangeJournalContext.java:53)
	at alluxio.master.BackupManager.lambda$initFromBackup$8(BackupManager.java:293)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: alluxio.exception.JournalClosedException: Cannot flush. Journal writer has been closed
	at alluxio.master.journal.raft.RaftJournalWriter.flush(RaftJournalWriter.java:84)
	at alluxio.master.journal.AsyncJournalWriter.doFlush(AsyncJournalWriter.java:295)
	... 1 more
2020-10-20 17:37:54,731 INFO  RaftJournalSystem - Journal shutdown complete
2020-10-20 17:37:54,731 ERROR ProcessUtils - Uncaught exception while stopping Alluxio master @<IP_ADDRESS>:19998, simply exiting. Exception "java.lang.IllegalStateException: Journal is not running", Root Cause "java.lang.IllegalStateException: Journal is not running"
java.lang.IllegalStateException: Journal is not running
	at com.google.common.base.Preconditions.checkState(Preconditions.java:508)
	at alluxio.master.journal.AbstractJournalSystem.stop(AbstractJournalSystem.java:56)
	at alluxio.master.AlluxioMasterProcess.stop(AlluxioMasterProcess.java:170)
	at alluxio.master.FaultTolerantAlluxioMasterProcess.stop(FaultTolerantAlluxioMasterProcess.java:175)
	at alluxio.ProcessUtils.run(ProcessUtils.java:43)
	at alluxio.master.AlluxioMaster.main(AlluxioMaster.java:55)
2020-10-20 17:37:54,731 DEBUG NettyClientHandler - [id: 0xd1a31255

@jainanuj07 Contacted you via Alluxio community slack, please take a look! Thanks

@LuQQiu  is issue here related to embedded journal? If yes, we will aim to fix it in the next release

@apc999 It's a joint effort from catalog service + restoring from backup (may related to journaling). I think we should get it fix or at least understand what's happening

PR #12649 helps Alluxio log the correct Ratis error that helps to debug the related journal issues.
PR #12650 expose Ratis journal size related configuration to Alluxio configuration and enlarge those values to support journaling entries larger than 3MB. Note that this is mainly helping the Ratis journal write path. Ratis journal read path has a hard limit of 30MB per entry.
PR #12658 change from logging a table with all its partitions to logging a table entry and multiple partitions entries based on alluxio.table.journal.partitions.chunk.size (=500 by default, a single entry only contains up to 500 partitions). In the future, Journal restore or restart will not be crashed by large table entries anymore.
PR #12695 helps resolve or prevents unexpected large journal entries coming from other places of Alluxio by flushing journal entries bigger than MASTER_EMBEDDED_JOURNAL_ENTRY_SIZE_MAX / 3 and log error when journal entries bigger than the MASTER_EMBEDDED_JOURNAL_ENTRY_SIZE_MAX.
@jainanuj07 Could you help validate if the above fixes solve the issue? We have done some testing on our side but may not be similar to your actual environment. Thanks!

PR #12649 helps Alluxio log the correct Ratis error that helps to debug the related journal issues.
PR #12650 expose Ratis journal size related configuration to Alluxio configuration and enlarge those values to support journaling entries larger than 3MB. Note that this is mainly helping the Ratis journal write path. Ratis journal read path has a hard limit of 30MB per entry.
PR #12658 change from logging a table with all its partitions to logging a table entry and multiple partitions entries based on alluxio.table.journal.partitions.chunk.size (=500 by default, a single entry only contains up to 500 partitions). In the future, Journal restore or restart will not be crashed by large table entries anymore.
PR #12695 helps resolve or prevents unexpected large journal entries coming from other places of Alluxio by flushing journal entries bigger than MASTER_EMBEDDED_JOURNAL_ENTRY_SIZE_MAX / 3 and log error when journal entries bigger than the MASTER_EMBEDDED_JOURNAL_ENTRY_SIZE_MAX.
@jainanuj07 Could you help validate if the above fixes solve the issue? We have done some testing on our side but may not be similar to your actual environment. Thanks!

resolved


Update Start

Update starting files:

Github action and issue template
Git configuration files
Updated README file
Add license file
Maven wrapper

Thanks. Looks like my code is all on a branch. Will try to push that in the next day or so.


speed up the helper

I am thinking of removing the .all() functions which will speed up the helper further (so release wont depend on test, and test wont depend on build).  This will mean that using npm scripts would be the recommended way of doing things for 'simple' users.  For those that want to know more,  they will look in the package.json and see that if they run component serve it will be quicker.
example package.json:
scripts:{
   build: 'component build',
   start: 'component build && component serve',
   test: 'component build && component test',
   release: 'npm test && component release'
}

upside

it's more explicit so people understand what is going on under the hood.
feels more configurable.
more intuitive.
less to document.

downside

it's more explicit, so feels likes less things 'for free'.
another breaking change

whoops. some how duplicated it :s


[New Feature] Adding query costs to debug logging to aid developer inner loop

Is your feature request related to a problem? Please describe.
Currently there is limited support to determine how many RUs are used for each query. This is currently only surfaced in the paging functionality.
Describe the solution you'd like
After discussions on twitter with @mumby0168 it was determined that we could add the costs to debug statements to allow for this to aid developers locally - https://twitter.com/billydev5/status/1482287510990426113

Thanks @WestDiscGolf for raising this I think you are right, it will really help people to get a feel of what their queries cost.


How i can use ocr command or code to extract text from image using tesseractmain.cpp

I need little help about usage of tesseract .I am not able to understand that how can i extract text from image using tesseractmain.cpp.I successfully Build the source code .But right stuck at how i can use it to perform basic function of ocr. Or how i can add some kind of wrapper for it .Thanks

Pleas ask direct in tesseract forum


Support downloading personal collections

Provide us a use case of the feature
Download collections.
Describe the solution you'd like
instaloader collection [collection ...]
Describe alternatives you've considered
If the feature request is accepted, would you be willing to submit a pull request?
No
Additional context
This would be very useful, there's no tool capable right now of downloading collections.

Duplicate of #544


FlutterFirebaseDynamicLinksPlugin.java uses unchecked or unsafe operations.

Bug report
while running:
$ flutter build appbundle
I get a warning:
Note: /Users/roberttessler/Documents/flutter/.pub-cache/hosted/pub.dartlang.org/firebase_dynamic_links-5.0.8/android/src/main/java/io/flutter/plugins/firebase/dynamiclinks/FlutterFirebaseDynamicLinksPlugin.java uses unchecked or unsafe operations.
Note: Recompile with -Xlint:unchecked for details.
Steps to reproduce
$ flutter build appbundle
Expected behavior
no warning
Sample project
Providing a minimal example project which demonstrates the bug in isolation from your main App greatly enhances the chance of a timely fix.
Please link to the public repository URL.

Additional context
Add any other context about the problem here.

Flutter doctor
Run flutter doctor and paste the output below:
Click To Expand
PASTE OUTPUT INSIDE HERE

Flutter dependencies
Run flutter pub deps -- --style=compact and paste the output below:
Click To Expand
PASTE OUTPUT INSIDE HERE

Same as https://github.com/firebase/flutterfire/issues/10122#issuecomment-1354488306


SQL lite file does not exist?

Hello, I installed the bot on a pterodatyl panel but the sql lite file does not exist, what should I do?

run the code on your computer and let it generate the sql file, after that upload it to the correct folder on your panel.


Update Article “the-fade”

Automatically generated by Netlify CMS

👷 Deploy Preview for dynamic-cheesecake-5fbd8b processing.

Name
Link

🔨 Latest commit
91311a5511af0c283948ba2d87cc23a27e5813d8

🔍 Latest deploy log
https://app.netlify.com/sites/dynamic-cheesecake-5fbd8b/deploys/633453075c8035000807f10a


Devops merge branch

Helpful commands:
python3 tools/init_db.py # inits db
python3 src/semaphoreRunner.py -d test_dspec.json -p<PHONE_NUMBER>00 # runs the demo test

https://github.com/orgs/conrad-blucher-institute/projects/17/views/2?pane=issue&itemId=47898966
We are aware of the test failing in outputs here is its ticket

I'm having the same issue as Savannah


Ingress may not work reliably on clusters without external LoadBalancer support (such as Kind clusters)

Cilium creates a service of type LoadBalancer for ingress support. In a managed Kubernetes cluster, this will typically result in an external load balancer being created that assigns an external IP address for that service.  On test clusters created with Kind or Minikube this doesn't automatically happen, so the Ingress will not be assigned an address.
It may be possible to use MetalLB or similar to provide the load balancer in Kind, but this has not been tested.
There is a further complication: the load balancer would need to be on a different node to the backend services being load balanced. This is because, in a multi-node Kind cluster, it's hard for socket-level eBPF programs to reliably detect the host network namespace for the node (because there are multiple nodes sharing the same kernel, unlike the situation in "production" Kubernetes where there is one node per VM). As a result, NodePort traffic originating in a Kind node's own network namespace will not reach its destination correctly. Traffic will reach its destination if that destination is on another node.

On test clusters created with Kind or Minikube this doesn't automatically happen, so the Ingress will not be assigned an address.

In this case it is possible to observe the assigned NodePort in kubectl get svc -A and use any of the Kind node addresses with it to reach the Ingress service. Not ideal, but works for testing when an external Load Balancer is not available.

As a result, NodePort traffic originating in a Kind node's own network namespace will not reach its destination correctly. Traffic will reach its destination if that destination is on another node.

There is a workaround for this documented in Cilium Kind Getting Started Guide:

To fully enable Cilium’s kube-proxy replacement (Kubernetes Without kube-proxy), cgroup v1 controllers net_cls and net_prio have to be disabled, or cgroup v1 has to be disabled (e.g. by setting the kernel cgroup_no_v1="all" parameter).

Turns out setting this kernel parameter allows Cilium Ingress to work properly regardless of the node where the traffic lands on.

On test clusters created with Minikube run minikube tunnel

May I request that the service type Cilium provisions is configurable? In clusters using outbound tunneling to ingress traffic (cloudflared, for instance), there is no LoadBalancer Kubernetes resource.


同步集群资源

configmsp 拼写错误， configmap

Thanksfor your feedback, it has been fixed: https://github.com/clusterpedia-io/website/pull/38


Added react-home page

Made Home Page with Reactjs,

The deployment have failed, what should I do ??

ADDED RESEARCH PAGE,


Master search

Have the categories root have a search that display 3-5 results per categories in a sectioned List.
We'll do that once we have CD plugged in.


[YouTube] Support super chats

Super chats are donations to a partnered YouTube streamer.

have messages if the donation amount is enough (for USD, this is $2 or greater)
pin themselves to the top of the chat window (up to 5 hours for the max donation. For USD, this is $500)
are colored based on the amount donated
can be made in currencies other than USD

Currently no super chat data is saved. Ideally the output would have several additional fields: color, pin duration, amount, and the original currency code. And the presence of amount would differentiate super chats from regular chat.
Example video with a large number of super chats: https://www.youtube.com/watch?v=UlemRwXYWHg

Thanks for the suggestion! I've been adding it and it's almost done!
What other information would you like to show? Currently I have the following:

Colours (For some reason it's in a format like "4280150454", and I'm not sure how that compares to RGB

The colour of the heading and
the colour of the body of the message

The amount in a given currency (e.g. $5.00)

and all the other information every other message has (text, timestamp, etc.)

Cool!
Yeah, those data points are all that I can think of. I googled<PHONE_NUMBER> and it led me to here:
https://docs-flutter-io.firebaseapp.com/flutter/material/Colors/tealAccent-constant.html
Suggesting that it was a teal super chat. If you convert it to hex, it's ARGB: FF1DE9B6. I don't mind which final format we pick. We could leave it as-is, or converted to a #-prefixed HTML color. Since HTML colors are RGB[A] we could probably just drop the alpha since I doubt there will ever be a transparent super chat.<PHONE_NUMBER> & 0xFFFFFF = #1DE9B6

Great! Thanks for that info.
I've actually been working on adding it for the past couple of hours and for some reason, there are cases where there are transparent super chats. I'll add it in two formats I suppose, rgba array and # prefixed HTML colour. They are indeed ARGB colours, and I can convert accordingly.
I'm quite close to getting it to work, but there are a ton of strange edge cases. In the example video URL, after getting the JSON (see attached txt file, since github doesn't allow JSON)
get_live_chat_replay.txt
of the chat info, there are a few important fields that are related to super chat messages, namely:
'liveChatPaidMessageRenderer','liveChatMembershipItemRenderer','liveChatPaidStickerRenderer' and 'liveChatTickerPaidMessageItemRenderer'. If you see any I missed, let me know 👍

That list has been updated to include all the following:
	__TYPES_OF_SUPER_CHAT_MESSAGES = [
		'liveChatTickerPaidStickerItemRenderer',
		'liveChatTickerPaidMessageItemRenderer',
		'liveChatMembershipItemRenderer',
		'liveChatTickerSponsorItemRenderer',
		'liveChatPaidMessageRenderer',
		'liveChatPaidStickerRenderer'
	]
	__TYPES_OF_MESSAGES_TO_IGNORE = [
		'liveChatViewerEngagementMessageRenderer', # message saying Live Chat replay is on
		'liveChatPlaceholderItemRenderer', # placeholder
		'liveChatModeChangeMessageRenderer' # e.g. slow mode enabled
	]

	__TYPES_OF_KNOWN_MESSAGES = [
		'liveChatTextMessageRenderer'
	] + __TYPES_OF_MESSAGES_TO_IGNORE + __TYPES_OF_SUPER_CHAT_MESSAGES

So, by selecting the option to display super chat messages, you will get TYPES_OF_SUPER_CHAT_MESSAGES, otherwise you will just get 'liveChatTextMessageRenderer' (the normal one, which is what we had previously)
And then, all items matching any of:
	'timestampUsec',
	'authorName',
	'purchaseAmountText',
	'message',
	'headerBackgroundColor',
	'bodyBackgroundColor',
	'timestampText',
	'amount',
	'durationSec',
	'detailText',
	'headerSubtext',
	'backgroundColor'

Let me know if this sounds like a good idea. 👍  (The above may be renames slightly)

Okay I will finish it off tomorrow - but let me know if you have any other questions in the meantime 👍 🔥

@libjared Okay - almost done! (So many small little things to worry about).
Just a question,

some super chat messages don't have messages, but have animated stickers. Do you need access to the sticker url?
Otherwise, when I am done with this update, you can test it out and let me know what things are missed and I will try my best to add it.

Okay, new update is live (d65753c) - let me know if it works 👍

The amount gets logged to console, but not to the outputted file (at least in the case of a text file).

Okay I can account for this, however, if you need more information stored, I would not recommend writing this data to text files. Instead, you should rather write it to a .json - as they have far better structure than text files.

@xnaas Let me know if this is better (efca2f8)
For example

That's really cool, thanks. I agree, json is probably a better format for this.

Yeah looks good, @xenova, I appreciate it. 😃


onReset hook

Hi,
i'm using bootstrap-tokenfield in autoform - all works great except Reset button. That's realted to https://github.com/sliptree/bootstrap-tokenfield/issues/111
and may be easily solved by adding onReset hook.
Please, consider.
Thank you.

It seems like that solution should work fine as written, without needing an onReset hook. You could replace the setTimeout call with Tracker.afterFlush for a little cleaner Meteor solution, but it should work.
In general, we should probably have onReset for custom input types, though, in which case you could define this as a custom input type.

Hi, I'll be very happy with a onReset method in custom type :) .
I'm currently writing a set of input types to edit geojson data (point, linestring, etc...) based on leaflet, but i haven't found a way to catch form reset inside the custom type. For exemple, even if I hide some inputs tag in the template to store values, there is no "change" event to catch the value reset when a form reset his fired.
Have you an idea to go arround with a onReset handler ?
Anthony.


Unmounting CardElement loses context

When a <CardElement /> unmounts and then re-appears on the page, props.stripe.createToken() yields:

submit.js:58 Error! Error: You did not specify the type of Source or Token to create.
We could not infer which Element you want to use for this operation.

In my use-case, I have a tab design that allows a user to select credit card or PayPal. Flicking between the two causes the error above.
When the component isn't unmounted/re-mounted, props.stripe.createToken() works fine.

Hi @leebenson
Thank you for reporting this!
I set up a quick jsfiddle with a makeshift "tab-switcher" that unmounts/remounts the card element, but was unable to recreate that issue: https://jsfiddle.net/y5wzu6dx/1/ (I also tried unmounting/remounting the whole StripeProvider or Elements subtree, but also did not see that error.)
Does my approach reflect your use case? Can you have a look and see where our implementations are different?

We recently identified one possible issue when un/re-mounting an element that would cause this kind of tokenization failure. We pushed a fix for it last week and have seen at least one case fixed as a result. Between that and not hearing back on a way to reproduce, I think I'm going to close this issue. If anyone runs into this again, very happy to reopen and investigate.


C100 stopped working, Error at 'getStatusAndUpdateCharacteristics'. Error: Invalid device confirm

Describe The Bug:
Plugin stopped working with C100 camera. Doesn't respond to commands from homebridge or home app.
Outputs errors in log.
Logs:
[5/8/2024, 7:00:24 PM] [tapo-camera] Error at 'getStatusAndUpdateCharacteristics'. Error: Invalid device confirm
    at TAPOCamera.refreshStok (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:205:15)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at TAPOCamera.getAuthenticatedAPIURL (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:335:19)
    at /var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:415:23
    at TAPOCamera.getStatus (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:555:18)
    at CameraAccessory.getStatusAndUpdateCharacteristics (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/cameraAccessory.ts:234:27)
    at CameraAccessory.setup (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/cameraAccessory.ts:265:7)
    at /var/lib/homebridge/node_modules/homebridge-tapo-camera/src/cameraPlatform.ts:30:9

Plugin Config:
{
    "bridge": {
        "name": "Homebridge",
        "username": "0E:67:F2:8B:4A:7E",
        "port": 51423,
        "pin": "150-86-555",
        "advertiser": "avahi",
        "bind": [
            "enp2s0"
        ]
    },
    "accessories": [],
    "platforms": [
        {
            "name": "Config",
            "port": 8581,
            "platform": "config"
        },
        {
            "cameras": [
                {
                    "name": "Tapo C100",
                    "ipAddress": "<IP_ADDRESS>",
                    "password": "....",
                    "streamUser": "....",
                    "streamPassword": "....",
                    "disableAlarmAccessory": true,
                    "privacyAccessoryName": "Camera"
                }
            ],
            "platform": "tapo-camera"
        }
    ],
}

Environment:

Plugin Version:
Homebridge Version: 1.8.1
Node.js Version: 20.12.2
NPM Version: Don't know
Operating System: Ubuntu Mantic (23.10)

if it helps any my plugin config:
{
"cameras": [
{
"name": "Living CM",
"ipAddress": "<IP_ADDRESS>",
"password": "",
"streamUser": "",
"streamPassword": "",
"pullInterval": 300000,
"debug": false,
"disablePrivacyAccessory": true,
"alarmAccessoryName": "Camera Alarm"
}
],
"_bridge": {
"username": "0E:A6:F9:7A:53:4C",
"port": 40873
},
"platform": "tapo-camera"
}
Homebridge Version: v1.8.2
Plugin Version: v2.3.4
OS: Ubuntu Jammy (22.04.3 LTS)
Node.js Version: v20.11.1

I got the same error with my C210 (fixed IP) :
[5/21/2024, 12:29:16 PM] [homebridge-tapo-camera] Error at 'getStatusAndUpdateCharacteristics'. Error: Invalid device confirm
    at TAPOCamera.refreshStok (/volume1/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:205:15)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at TAPOCamera.getAuthenticatedAPIURL (/volume1/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:335:19)
    at /volume1/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:415:23
    at TAPOCamera.getStatus (/volume1/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:555:18)
    at CameraAccessory.getStatusAndUpdateCharacteristics (/volume1/homebridge/node_modules/homebridge-tapo-camera/src/cameraAccessory.ts:234:27)
    at Timeout._onTimeout (/volume1/homebridge/node_modules/homebridge-tapo-camera/src/cameraAccessory.ts:212:7)

Here's my config :
{
    "cameras": [
        {
            "name": "Cage",
            "ipAddress": "<IP_ADDRESS>",
            "password": "",
            "streamUser": "",
            "streamPassword": "",
            "disableAlarmAccessory": true,
            "privacyAccessoryName": "Cage - Etat",
            "alarmAccessoryName": "Cage - Alarme"
        }
    ],
    "_bridge": {
        "username": "0E:2E:E9:31:7C:81",
        "port": 32217
    },
    "platform": "tapo-camera"
}

Environment:
Plugin Version: v2.3.4
Homebridge Version: v1.8.2
Node.js Version: v20.9.0
Operating System: Synology DSM (DS218)

Hello, having the same problem with C325WB. It was working fine, but suddenly stopped working. No changes were made to plugin or updated firmware of the camera.

Having same issue is there any solution or workaround identified?

Same here

Same for me with tapo C520WS... nothing changed on camera side...

I noticed the same issue with Tapo c125. It usually happens when camera got suddenly powered off.
Reboot the camera does not help. Everything works OK in Tapo App but homebridge seems to have « lost » the link to the camera.
Maybe an issue with IP or linking ?

TP-Link has changed the way the authentication works and apparently they're working on a new API. If your camera is still working, do NOT update the firmware
Here's a discussion on a similar project: https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/issues/551

Same here I updated the plugin to the lastest version and I get a very similar error in my terminal : [26/08/2024, 12:45:08] [Càmera PDA] Error getting status: Error: Invalid device confirm
at TAPOCamera.refreshStok (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:215:15)
at processTicksAndRejections (node:internal/process/task_queues:95:5)
at TAPOCamera.getStok (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:342:7)
at TAPOCamera.getAuthenticatedAPIURL (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:350:19)
at /var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:431:23
at TAPOCamera.getStatus (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:614:26)
at CameraAccessory.getStatusAndNotify (/var/lib/homebridge/node_modules/homebridge-tapo-camera/src/cameraAccessory.ts:214:28)
I just updated the whole HB software and node as well to the latest update:
Node.js Version	v20.17.0
Homebridge Tapo plugin v2.4.4
Camera: Tapo C200

Hey, please update to https://github.com/kopiro/homebridge-tapo-camera/releases/tag/v2.4.5 where I added more debug logs and post your logs again.

Hey, please update to https://github.com/kopiro/homebridge-tapo-camera/releases/tag/v2.4.5 where I added more debug logs and post your logs again.

Logs won't help much because it isn't a bug, TP-Link changed the deviceconfirm string in the camera firmware. The camera receives an admin password from the TP-Link cloud and uses that to generate the deviceconfirm, the plugin still uses the cloud password so they don't match anymore. Nonce and cnonce are correct and the hashing method is still SHA256.
The developer of the Home Assistant Tapo plugin has the same issue: https://github.com/JurajNyiri/HomeAssistant-Tapo-Control/issues/551

Update: you may be affected by this issue. Please follow this guide on how to fix it.

Error: Invalid device confirm
at TAPOCamera.refreshStok (/homebridge/node_modules/homebridge-tapo-camera/src/tapoCamera.ts:333:15)
at processTicksAndRejections (node:internal/process/task_queues:95:5)


test gha

Description
Changes proposed in this pull request:

...
...
...

Related issue(s)

No longer valid


Updated the readme to use the executable and fixed linesearch bug.

Note that the bug only comes apparent if the first iteration of the linesearch fails. Since this is a 'simple' problem this error did not make a difference in the figures. I only found it in the source inversion!


Smarter page titles & descriptions

Some page titles out there are either plain wrong or unhelpful. It's way worse for descriptions.
Most other search engines take some liberty and don't just use the <title> tag as source for the title or the <meta> tags for the description.
Other sources of data could include:

DMOZ (there is even a tag for avoiding it ) : #12
Wikipedia: #11
Other content in the page, like <h1> tags

Any other idea?
How to choose between these source will be a complex topic but we can build something reasonably simple in the short term. We already support a blacklist of titles and a few fallbacks in formatting.py

You should prefer .. Facebook / Twitter Cards when present .. these would usually give you better results then title/description.
http://ogp.me/
https://dev.twitter.com/cards/markup


No FilePath Bar

There is no filepath bar.

This was just added to the GenericFileBrowser view in v0.4.8. Give it a try!


Setup SSL for API

http://mcavage.github.io/node-restify/#Creating-a-Server

An alternative approach to adding this directly to ummon is to proxy ummon-server behind nginx or Apache using those for SSL termination. This is a common pattern for node apps, and simplifies things like certificate management by consolidating SSL termination in one place.
(This is the approach we plan to use for Way to Health.)


[Issue] How can I manually expire the cache in the following scenarios?

Scenario 1:
I have a paginated query for a product table. I need to cache the data whenever users switch pages. Each page switch results in caching of data, where the cache key is defined as key=$"{pageindex}{pagesize}{keywords}". This works as intended.
However, if I execute a delete or insert operation on the Product table, I need to refresh the related cached data for the Product. Here, the paginated query's data usually needs immediate refresh.
Furthermore, if a user has cached data based on a ProductId, and this specific ProductId data has just been modified (for instance, the corresponding product was deleted or updated), that cache should also expire.
How can I achieve manual cache refreshing or expiration in these cases? @jodydonetti

This is asking a lot of a cache library @neozhu. You will need some sort of cache invalidation strategy that keeps track of the types and their associated "primary keys" stored under each cache key. This is probably better handled at a level "above" the cache itself. To do it properly, the cache library would somehow have to know which of the types in the graph of objects you place into a cache bucket can be updated and what their "identity" is (i.e. primary key). Like, what if the cache bucket contained an object of type Order which contained a reference to the updated Product? How would it know, then, what keys to evict? This is not an easy problem to solve and I would think it to be out of scope for a project like FusionCache.
Aside: One major issue to note is that nodes other than the one that performed the SQL update may have in-memory cache keys that it doesn't. This would thus require a pub/sub mechanism to notify the other nodes that, e.g., Product(id=123) was modified in order for them to look up which of their cache keys contained that entity.

You have to store in a list all the keys you generate.
Every new key is added to the list.
Then after any edit of your product table get that list and expire all the keys.
Should work, hope it helps

Thanks a lot for your advice.

UPDATE: it is happening 🥳
Any help would be appreciated!

Hi all, v2.0.0-preview-1 is out 🥳
This includes Tagging and Clear() support!
🙏 Please, if you can try it out and let me know what you think, how it feels to use it or anything else really: your contribution is essential, thanks!


[vtk-dicom:x64-windows] build failure

Host Environment

OS: Windows 10 1903
Compiler:  MSVC 19.16.27031.1

To Reproduce
Steps to reproduce the behavior:
 ./vcpkg install --triplet x64-windows vtk-dicom[gdcm]
Failure logs

Console output.

vcpkg install --triplet x64-windows vtk-dicom[gdcm]
The following packages will be built and installed:
    vtk-dicom[core,gdcm]:x64-windows
Starting package 1/1: vtk-dicom:x64-windows
Building package vtk-dicom[core,gdcm]:x64-windows...
-- Using cached D:/LIBS/vcpkg/downloads/dgobbi-vtk-dicom-v0.8.9.tar.gz
-- Extracting source D:/LIBS/vcpkg/downloads/dgobbi-vtk-dicom-v0.8.9.tar.gz
-- Using source at D:/LIBS/vcpkg/buildtrees/vtk-dicom/src/v0.8.9-fa4d27bc42
-- Configuring x64-windows
CMake Error at scripts/cmake/vcpkg_execute_required_process.cmake:58 (message):
    Command failed: ninja -v
    Working Directory: D:/LIBS/vcpkg/buildtrees/vtk-dicom/x64-windows-rel/vcpkg-parallel-configure
    Error code: 1
    See logs for more information:
      D:\LIBS\vcpkg\buildtrees\vtk-dicom\config-x64-windows-out.log

Call Stack (most recent call first):
  scripts/cmake/vcpkg_configure_cmake.cmake:296 (vcpkg_execute_required_process)
  ports/vtk-dicom/portfile.cmake:25 (vcpkg_configure_cmake)
  scripts/ports.cmake:84 (include)

Error: Building package vtk-dicom:x64-windows failed with: BUILD_FAILED
Please ensure you're using the latest portfiles with `.\vcpkg update`, then
submit an issue at https://github.com/Microsoft/vcpkg/issues including:
  Package: vtk-dicom:x64-windows
  Vcpkg version: 2019.08.16-nohash

Additionally, attach any relevant sections from the log files above.

builtrees/vtk-dicom/config-x64-windows-out.log

[1/2] cmd /c "cd .. && "C:/Program Files/CMake/bin/cmake.exe" "D:/LIBS/vcpkg/buildtrees/vtk-dicom/src/v0.8.9-fa4d27bc42" "-DBUILD_PROGRAMS=OFF" "-DBUILD_EXAMPLES=OFF" "-DUSE_GDCM=ON" "-DUSE_DCMTK=OFF" "-DCMAKE_MAKE_PROGRAM=C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/LIBS/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v141" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/LIBS/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-G" "Ninja" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=D:/LIBS/vcpkg/packages/vtk-dicom_x64-windows""
FAILED: ../CMakeCache.txt
cmd /c "cd .. && "C:/Program Files/CMake/bin/cmake.exe" "D:/LIBS/vcpkg/buildtrees/vtk-dicom/src/v0.8.9-fa4d27bc42" "-DBUILD_PROGRAMS=OFF" "-DBUILD_EXAMPLES=OFF" "-DUSE_GDCM=ON" "-DUSE_DCMTK=OFF" "-DCMAKE_MAKE_PROGRAM=C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/Common7/IDE/CommonExtensions/Microsoft/CMake/Ninja/ninja.exe" "-DBUILD_SHARED_LIBS=ON" "-DVCPKG_CHAINLOAD_TOOLCHAIN_FILE=D:/LIBS/vcpkg/scripts/toolchains/windows.cmake" "-DVCPKG_TARGET_TRIPLET=x64-windows" "-DVCPKG_SET_CHARSET_FLAG=ON" "-DVCPKG_PLATFORM_TOOLSET=v141" "-DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_PACKAGE_REGISTRY=ON" "-DCMAKE_FIND_PACKAGE_NO_SYSTEM_PACKAGE_REGISTRY=ON" "-DCMAKE_INSTALL_SYSTEM_RUNTIME_LIBS_SKIP=TRUE" "-DCMAKE_VERBOSE_MAKEFILE=ON" "-DVCPKG_APPLOCAL_DEPS=OFF" "-DCMAKE_TOOLCHAIN_FILE=D:/LIBS/vcpkg/scripts/buildsystems/vcpkg.cmake" "-DCMAKE_ERROR_ON_ABSOLUTE_INSTALL_DESTINATION=ON" "-DVCPKG_CXX_FLAGS=" "-DVCPKG_CXX_FLAGS_RELEASE=" "-DVCPKG_CXX_FLAGS_DEBUG=" "-DVCPKG_C_FLAGS=" "-DVCPKG_C_FLAGS_RELEASE=" "-DVCPKG_C_FLAGS_DEBUG=" "-DVCPKG_CRT_LINKAGE=dynamic" "-DVCPKG_LINKER_FLAGS=" "-DVCPKG_TARGET_ARCHITECTURE=x64" "-DCMAKE_INSTALL_LIBDIR:STRING=lib" "-DCMAKE_INSTALL_BINDIR:STRING=bin" "-G" "Ninja" "-DCMAKE_BUILD_TYPE=Release" "-DCMAKE_INSTALL_PREFIX=D:/LIBS/vcpkg/packages/vtk-dicom_x64-windows""
-- The C compiler identification is MSVC 19.16.27031.1
-- The CXX compiler identification is MSVC 19.16.27031.1
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe
-- Check for working C compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe
-- Check for working CXX compiler: C:/Program Files (x86)/Microsoft Visual Studio/2017/Community/VC/Tools/MSVC/14.16.27023/bin/Hostx64/x64/cl.exe -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- vtkDICOM: Building vtkDICOM stand-alone
-- Found ZLIB: optimized;D:/LIBS/vcpkg/installed/x64-windows/lib/zlib.lib;debug;D:/LIBS/vcpkg/installed/x64-windows/debug/lib/zlibd.lib (found version "1.2.11")
CMake Error at CMakeLists.txt:148 (include):
  include could not find load file:

    D:/LIBS/vcpkg/installed/x64-windows/share/openjpeg/UseGDCM.cmake

-- Configuring incomplete, errors occurred!

GDCM was built previously with vcpkg with the same command.
ls installed/x64-windows/share/gdcm/
copyright  GDCMConfig.cmake  GDCMConfigVersion.cmake  GDCMTargets.cmake  GDCMTargets-debug.cmake  GDCMTargets-release.cmake  UseGDCM.cmake  XML/

ls installed/x64-windows/lib/gdcm*
installed/x64-windows/lib/gdcmcharls.lib  installed/x64-windows/lib/gdcmDSED.lib    installed/x64-windows/lib/gdcmjpeg12.lib  installed/x64-windows/lib/gdcmMEXD.lib
installed/x64-windows/lib/gdcmCommon.lib  installed/x64-windows/lib/gdcmgetopt.lib  installed/x64-windows/lib/gdcmjpeg16.lib  installed/x64-windows/lib/gdcmMSFF.lib
installed/x64-windows/lib/gdcmDICT.lib    installed/x64-windows/lib/gdcmIOD.lib     installed/x64-windows/lib/gdcmjpeg8.lib

 ls installed/x64-windows/include/gdcm*
installed/x64-windows/include/gdcm_j2k.h                                       installed/x64-windows/include/gdcmModuleEntry.h
installed/x64-windows/include/gdcm_jp2.h                                       installed/x64-windows/include/gdcmModules.h
installed/x64-windows/include/gdcmAAbortPDU.h                                  installed/x64-windows/include/gdcmMovePatientRootQuery.h
installed/x64-windows/include/gdcmAAssociateACPDU.h                            installed/x64-windows/include/gdcmMoveStudyRootQuery.h
installed/x64-windows/include/gdcmAAssociateRJPDU.h                            installed/x64-windows/include/gdcmMrProtocol.h
installed/x64-windows/include/gdcmAAssociateRQPDU.h                            installed/x64-windows/include/gdcmNActionMessages.h
installed/x64-windows/include/gdcmAbstractSyntax.h                             installed/x64-windows/include/gdcmNCreateMessages.h
installed/x64-windows/include/gdcmAnonymizeEvent.h                             installed/x64-windows/include/gdcmNDeleteMessages.h
installed/x64-windows/include/gdcmAnonymizer.h                                 installed/x64-windows/include/gdcmNestedModuleEntries.h
installed/x64-windows/include/gdcmApplicationContext.h                         installed/x64-windows/include/gdcmNetworkEvents.h
installed/x64-windows/include/gdcmApplicationEntity.h                          installed/x64-windows/include/gdcmNetworkStateID.h
installed/x64-windows/include/gdcmAReleaseRPPDU.h                              installed/x64-windows/include/gdcmNEventReportMessages.h
installed/x64-windows/include/gdcmAReleaseRQPDU.h                              installed/x64-windows/include/gdcmNGetMessages.h
installed/x64-windows/include/gdcmARTIMTimer.h                                 installed/x64-windows/include/gdcmNormalizedMessageFactory.h
installed/x64-windows/include/gdcmASN1.h                                       installed/x64-windows/include/gdcmNormalizedNetworkFunctions.h
installed/x64-windows/include/gdcmAsynchronousOperationsWindowSub.h            installed/x64-windows/include/gdcmNSetMessages.h
....

The weird thing is that CMake search the file UseGDCM.cmake inside openjpeg folder. Do I need to update CMAKE_PREFIX_PATH to give the path to GDCM CMake scripts?

Thanks for posting this issue. This issue has been fixed. So I am closing this issue for now.


Feature/customized theme

Uses the new colors across the AdminLTE theme with a custom build of AdminLTE version 2.4. Maybe we should create a fork of the AdminLTE repo with our colors, it's somewhat overkill though...
Also includes a revamped login page with particles-js in the background as it is fancy.
Bonus: Fixed user creation command

Can you please update the ReadMe to reflect this new package we use as well? :)


allow splittable user to use additional plugins in the deps finding pipeline

this is needed as there are some constructs that the jison library
generates that blows up with "use strict" directive. I need to be able
to pass "transform-remove-strict-mode" plugin

Its fine. We really just discover deps here. Should be less strict than
closure compiler.
On Mon, Sep 25, 2017 at 10:29 AM, erwin mombay<EMAIL_ADDRESS>wrote:

@erwinmombay commented on this pull request.
In splittable.js
https://github.com/cramforce/splittable/pull/51#discussion_r140843234:

@@ -238,9 +238,10 @@ exports.getGraph = function(entryModules, config) {
// directly and which we don't want to apply during deps finding.
transform(babel, {
babelrc: false,

plugins: [

plugins: Array.isArray(config.babel.plugins) ? config.babel.plugins.concat([

ideally it shouldn't since the "use strict" directive is desirable after
all (correct me if im wrong), its just unfortunate that jison uses a lot of
labels and jumps
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/cramforce/splittable/pull/51#discussion_r140843234,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAFeT2w7BtBu7aUZmvflag7PXof-CLYCks5sl-MQgaJpZM4PikFV
.

@cramforce ah right. that makes sense. ill make the changes and move the export babel plugin back into the splittable repo