libcamera-vid variability in frames captured with no reported frame drops

When running the following command: libcamera-vid --framerate 10 --width 4096 --height 3040 -t 24000 --awbgains 1,1 --shutter 100000 --denoise cdn_fast --codec yuv420 --save-pts timestamps.txt -o /dev/shm/camera.yuv The resulting output file sometimes contains 227 frames, and sometimes 231. Strangely I've never seen any other outcomes across ~10 attempts. I am calculating the number of frames by taking the byte size of the file and dividing by 18677760 (4096x3040x3/2):
4314562560 total bytes / 18677760 bytes per frame = 231 frames
4239851520 total bytes / 18677760  bytes per frame = 227 frames
Based on the 10 second timeout, I would expect to see 240 frames (10fps * 24 seconds), but that's potentially a different issue, or just not guaranteed with a time-based timeout. On a slight tangent, it would be nice to be able to provide a num-buffers parameter to capture an exact number of frames (important in my use-case). See here for a similar feature request for libcamerasrc.
Back to the variable frame output. You can find both timestamp files here: 227.txt, 231.txt
In the 227 timestamp file, there is a 4 frame gap between 1100 and 1600. However, in both cases libcamera-vid never logs a frame drop (ie. "Dropping unmatched input"). See the debug log for a 231 frame capture here - 231_Frame_Log.txt and for the 227 capture here - 227_Frame_Log.txt
Are you able to replicate this behavior? Let me know if there's anything else I can get.
I am testing with an Arducam Mini IMX477 module + Compute Module 4 + Official IO Board + Bullseye Lite 64bit + 5.10.63-v8+ Kernel running headless over ssh. Full system info here - https://pastebin.com/654vBepL
$ libcamera-hello --version
libcamera-apps build: 2a38ae93f143 21-10-2021 (14:53:29)
libcamera build: v0.0.0+3156-f4070274
$ vcgencmd version
Oct 29 2021 10:47:33
Copyright (c) 2012 Broadcom
version b8a114e5a9877e91ca8f26d1a5ce904b2ad3cf13 (clean) (release) (start)

Based on the 10 second timeout, I would expect to see 240 frames (10fps * 24 seconds), but that's potentially a different issue, or just not guaranteed with a time-based timeout.

I think this is just down to the inaccuracies of using an simple elapsed time based test to signal a timeout.  I doubt this can be improved upon much.

On a slight tangent, it would be nice to be able to provide a num-buffers parameter to capture an exact number of frames (important in my use-case). See here for a similar feature request for libcamerasrc.

This is a good idea and probably something that should not be too difficult to implement.

I think this is just down to the inaccuracies of using an simple elapsed time based test to signal a timeout. I doubt this can be improved upon much.

But what do you think about that gap in the timestamp file? To me that would seem to indicate that it's not just a time counting difference.

This is a good idea and probably something that should not be too difficult to implement.

Ok, great. Having both libcamera-vid and libcamerasrc support num-buffers would be awesome. Thanks!

But what do you think about that gap in the timestamp file? To me that would seem to indicate that it's not just a time counting difference.

There has been a bug we fixed very recently to do with dropped frames.  If you haven't updated your firmware in a while, you may be hitting this.  To confirm, can you temporarily put isp_freq=500 in your /boot/config.txt file and see if this stops the issue?

This is a good idea and probably something that should not be too difficult to implement.

Ok, great. Having both libcamera-vid and libcamerasrc support num-buffers would be awesome. Thanks!

We will add support in libcamera-vid, but for libcamerasrc, you would have to send the request through either the libcamera mailing list or through bugzilla (which I see you have already raised).  The latter is not something that is maintained by us.

There has been a bug we fixed very recently to do with dropped frames. If you haven't updated your firmware in a while, you may be hitting this. To confirm, can you temporarily put isp_freq=500 in your /boot/config.txt file and see if this stops the issue?

I will try updating my firmware later today, but wanted to quickly let you know before then, that my previous testing was with gpu_freq=800 (which also sets isp_freq=800 - verified by watching the clocks during camera capture). I'll let you know what it looks like after an update.

And a quick update from testing with no gpu_overclock (no overclocks at all), and I did a few tests of the above command. Of the 5 tests, I actually saw 2 report a dropped frame, so I discarded those results (potentially related to the bug you were describing). Of the other 3 tests, 2 captured 231 frames, and 1 actually captured only 221<PHONE_NUMBER> bytes) frames. The latter had no reported frame drops in the console. From the timetamps.txt file, you can see a 10 frame gap between 1099.984 and 2199.971 - 221.txt

And a quick update from testing with no gpu_overclock (no overclocks at all), and I did a few tests of the above command. Of the 5 tests, I actually saw 2 report a dropped frame, so I discarded those results (potentially related to the bug you were describing). Of the other 3 tests, 2 captured 231 frames, and 1 actually captured only 221<PHONE_NUMBER> bytes) frames. The latter had no reported frame drops in the console. From the timetamps.txt file, you can see a 10 frame gap between 1099.984 and 2199.971 - 221.txt

Is there anything else running in the system when you run this application?  If you set your output file to /dev/null, does it eliminate these drops?  I want to rule out the possibility of file IO causing this drop.

Outputting to /dev/null does seem to result in consistent 231 frames every time across 10 tests (with an overclock). So this raises a few thoughts for me:

Is it really possible we're hitting memory bandwidth issues when writing to /dev/shm? If so, is there any way to tweak this tempfs filesystem setup to be better optimized for high bandwidth YUV capture? My worry is that if we're hitting issues at 10fps, there's no way it's going to handle full resolution at 30fps :(
Beyond catching the gap in the timestamps file, I don't get any frame drop debug messages in the log. Do you think it's possible to find out exactly where those drops are occurring, and why they're not caught by the log (maybe it's something that could be tweaked and not a pure hardware/memory bandwidth issue...)
If it really does seem like a /dev/shm memory bandwidth limitation (and no tweaks to the tempfs setup helps), is it at all possible to get libcamera/libcamera-apps to buffer using high memory rather than CMA? Then, for my rather special usecase (limited to 300 frame bursts), I could maybe just patch the sources to have a much larger number of buffers. Do you think that would that be feasible?

Thanks again.

Outputting to /dev/null does seem to result in consistent 231 frames every time across 10 tests (with an overclock). So this raises a few thoughts for me:

Ok, that at least puts us at a baseline.

Is it really possible we're hitting memory bandwidth issues when writing to /dev/shm? If so, is there any way to tweak this tempfs filesystem setup to be better optimized for high bandwidth YUV capture? My worry is that if we're hitting issues at 10fps, there's no way it's going to handle full resolution at 30fps :(

It's possible, but I've not done any tests to say conclusively.  However, I am curious about the /dev/shm device.  On my pi, df -h reports the size of the disk as 4.0Gb.  This is a bit suspicious, as that is the total memory available on my board.  Could it be that /dev/shm actually swaps out to disk (even though it is meant to be a ramdisk) if it fills to a certain limit?  What does the results show if you create your own ramdisk to a particular size using fstab?

Beyond catching the gap in the timestamps file, I don't get any frame drop debug messages in the log. Do you think it's possible to find out exactly where those drops are occurring, and why they're not caught by the log (maybe it's something that could be tweaked and not a pure hardware/memory bandwidth issue...)

There is a possible framedrop in the kernel device driver that libcamera has no possibility of knowing.  If you were to enable verbose kernel logging, this should show up.

If it really does seem like a /dev/shm memory bandwidth limitation (and no tweaks to the tempfs setup helps), is it at all possible to get libcamera/libcamera-apps to buffer using high memory rather than CMA? Then, for my rather special usecase (limited to 300 frame bursts), I could maybe just patch the sources to have a much larger number of buffers. Do you think that would that be feasible?

Unfortunately not, the hardware requires buffers to be in the CMA region or they will not be addressable.
I may have asked this before, but do you need YUV420 output?  Would MJPEG with a suitable high quality encode be acceptable for your usecase?  I seem to be able to save 10fps MJPEG stream without framedrops on a standard sdcard:
libcamera-vid --frames 300 -o test.mjpeg -v --codec mjpeg --save-pts pts.txt --width 4096 --height 3040

It's possible, but I've not done any tests to say conclusively. However, I am curious about the /dev/shm device. On my pi, df -h reports the size of the disk as 4.0Gb. This is a bit suspicious, as that is the total memory available on my board. Could it be that /dev/shm actually swaps out to disk (even though it is meant to be a ramdisk) if it fills to a certain limit? What does the results show if you create your own ramdisk to a particular size using fstab?

Yeah, I'm a little unsure how it handles situations where /dev/shm is full but the OS needs some of that ram... I will try testing that out later. Throughout all of my tests I have actually been using a 6GB ramdisk specified via fstab: none /dev/shm tmpfs defaults,size=6G 0 0

There is a possible framedrop in the kernel device driver that libcamera has no possibility of knowing. If you were to enable verbose kernel logging, this should show up.

Interesting. I will try to enable verbose logging and will see if I can catch those drops.

Unfortunately not, the hardware requires buffers to be in the CMA region or they will not be addressable.

Bummer. And I've tried several ways to get more than ~576MB CMA and failed. I'm not sure it's possible on the Pi, but would love to be told otherwise. Do you think there's maybe a way to copy the CMA buffer -> high memory buffer inside either libcamera/libcamera-apps in a way that would exceed the performance of /dev/shm (which probably has some overhead)?

I may have asked this before, but do you need YUV420 output? Would MJPEG with a suitable high quality encode be acceptable for your usecase? I seem to be able to save 10fps MJPEG stream without framedrops on a standard sdcard:

Unfortunately I need to hit at least 20-24fps (ideally 30fps). Once I get the IMX477 4 lane CSI driver working, I will run some tests to see just how high the MJPEG encoder will go.
Thanks for all your help.

Quick update. Looks like /dev/shm isn't nearly as fast as I would expect:
pi@raspberrypi:~ $ sudo dd if=/dev/zero of=/dev/shm/zero bs=4k count=100000
100000+0 records in
100000+0 records out
409600000 bytes (410 MB, 391 MiB) copied, 0.825578 s, 496 MB/s
pi@raspberrypi:~ $ sudo dd if=/dev/zero of=/dev/shm/zero bs=4k count=100000
100000+0 records in
100000+0 records out
409600000 bytes (410 MB, 391 MiB) copied, 0.777537 s, 527 MB/s
pi@raspberrypi:~ $ sudo dd if=/dev/zero of=/dev/shm/zero bs=4k count=100000
100000+0 records in
100000+0 records out
409600000 bytes (410 MB, 391 MiB) copied, 0.768167 s, 533 MB/s

Interesting. Raising the block size helps:
pi@raspberrypi:~ $ sudo dd if=/dev/zero of=/dev/shm/zero bs=64k count=10000
10000+0 records in
10000+0 records out
655360000 bytes (655 MB, 625 MiB) copied, 0.807093 s, 812 MB/s

Do you happen to know how libcamera writes the data to disk?

Do you think there's maybe a way to copy the CMA buffer -> high memory buffer inside either libcamera/libcamera-apps in a way that would exceed the performance of /dev/shm (which probably has some overhead)?

This is not easy libcamera-apps, it was not structured to do this.  However, it would be entirely possible to create a new app that just memcpys the CMA buffer into another buffer or memory space and return the CMA buffer back for the hardware to reuse.
You might also want to try creating a ramdisk not called /dev/shm (maybe /ramdisk) with a fixed size to see if that improves things.  /dev/shm seems to have a specific usage in Linux.

Unfortunately I need to hit at least 20-24fps (ideally 30fps). Once I get the IMX477 4 lane CSI driver working, I will run some tests to see just how high the MJPEG encoder will go.

Unfortunately I need to hit at least 20-24fps (ideally 30fps). Once I get the IMX477 4 lane CSI driver working, I will run some tests to see just how high the MJPEG encoder will go.

Does the size of the YUV image matter much?  If you get the ISP to scale that down, your IO and memory bandwidth will be reduced linearly, and that might help.

Do you happen to know how libcamera writes the data to disk?

This happens using standard file IO operations, see https://github.com/raspberrypi/libcamera-apps/blob/main/output/file_output.cpp.  Perhaps there is scope to improve performance there as well.

This is not easy in libcamera-apps, it was just not structured to do this. However, it would be entirely possible to create a new libcamera-app that just memcpys the CMA buffer into another buffer or memory space and return the CMA buffer back for the hardware to reuse.

Interesting. That's probably a bit beyond my skills, but I'll keep it in mind.

You might also want to try creating a ramdisk not called /dev/shm (maybe /ramdisk) with a fixed size to see if that improves things. /dev/shm seems to have a specific usage in Linux.

I did a little research on the internet and it seems, performance-wise, tempfs is about the same compared to ramfs. I may give it a try just to see later this week.

Does the size of the YUV image matter much? If you get the ISP to scale that down, your IO and memory bandwidth will be reduced linearly, and that might help.

Yeah, for our use-case, we need as much resolution as possible (180x140ish degree fisheye capture and reprojecting to equirectangular).

This happens using standard file IO operations, see https://github.com/raspberrypi/libcamera-apps/blob/main/output/file_output.cpp. Perhaps there is scope to improve performance there as well.

Interesting. Every last bit (pun half intended) of bandwidth will help when trying to capture 560MB/s (YUV420 4056x3040 @ 30fps).
For now though, I first want to figure out where those missing frames are. I'll try to do some kernel logging and let you know what I find. Thanks for the help.

Hmm, I just tried to update the kernel log level via echo "1" | sudo tee /proc/sys/kernel/printk but that doesn't seem to show any logs when capturing and monitoring via journalctl -f - Do you think I need to build a kernel with the v4l2 dev log option enabled?

To enable the Unicam kernel debug logging, you can append bcm2835_unicam.debug=5 to /boot/cmdline.txt.
However, I've just had a quick peek, and unfortunately, there is no clear single line of logging that indicates a dropped frame.  Rather, it has to be inferred from multiple lines of trace logging which is not very convenient.  I'll look to add a single line of logging and tag you in the PR.

Sounds good. I'll be ready to test. Thanks!

Do we have sequential sequence numbers out of the unicam which would indicate frame loss/drops that we can catch/report in libcamera?

Do we have sequential sequence numbers out of the unicam which would indicate frame loss/drops that we can catch/report in libcamera?

Yes we do, and this can be used to indicate frame drops as well!

Maybe this is worthy of something like:
int dropped = currentSequence - previousSequence - 1;
if (dropped)
LOG(V4L2, Debug) << "Dropped " << dropped << " frame(s)"
in the V4L2VideoDevice then?

LOG(V4L2, Debug) is only output when v4l2 debug messages are set in the kernel flags, right? If so, I think it would be nice to show all frame drop related messages to the user/console regardless of v4l2 debug settings.

With the latest kernel (and updated libcamera and libcamera-apps) I am able to catch those frame drops. This timed capture (24 seconds @ 10fps) recorded 224 frames. libcamera-vid did catch one frame drop:
[0:08:02.959242278] [3065]  WARN RPI raspberrypi.cpp:1862 Dropping unmatched input frame in stream Unicam Embedded
[0:08:02.959277519] [3065] DEBUG RPI raspberrypi.cpp:1876 Could not find matching embedded buffer

And the new kernel logging caught a few more (see full log below for all drops):
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 16, lines done 760
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 16, lines done 1520
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 16, lines done 2280
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC002, sequence 16, lines done 3039
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4001, sequence 17, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [0] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [1] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x5003, sequence 17, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 17, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 17, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC000, sequence 17, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4003, sequence 18, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [0] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [1] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x5001, sequence 18, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 18, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 18, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC002, sequence 18, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4001, sequence 19, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [0] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [1] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x5003, sequence 19, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 19, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 19, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC000, sequence 19, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4003, sequence 20, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [0] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [1] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x5001, sequence 20, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 20, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 20, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC002, sequence 20, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4001, sequence 21, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [0] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [1] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x5003, sequence 21, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 21, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 21, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC000, sequence 21, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4003, sequence 22, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [0] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [1] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x5001, sequence 22, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 22, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 22, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC002, sequence 22, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4001, sequence 23, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: [0] Dropping frame, buffer not available at FS
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 23, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 23, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 23, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC000, sequence 23, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4003, sequence 24, lines done 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 0
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: Scheduling dummy buffer for node 1
Nov 30 23:41:38 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 24, lines done 760
Nov 30 23:41:39 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4003, sequence 24, lines done 1520
Nov 30 23:41:39 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x4, STA: 0x4001, sequence 24, lines done 2280
Nov 30 23:41:39 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x6, STA: 0xC002, sequence 24, lines done 3039
Nov 30 23:41:39 raspberrypi kernel: unicam fe801000.csi: ISR: ISTA: 0x1, STA: 0x4001, sequence 25, lines done 0

One other interesting note is that the kernel log shows that 240 frames were requested. And even when it captures 231 (the max I've ever seen with 24 seconds @ 10fps), and you add the 7 intentionally dropped frames: [0:07:59.853805174] [3069] DEBUG IPARPI raspberrypi.cpp:273 Drop 7 frames on startup, the sum doesn't add up to 240.  And during that 231 frame case, the kernel log doesn't report any frame drops.
All of the logs can be found below:
231timestamps.txt
231framesKernel.txt
224timestamps.txt
224framesKernel.txt
224framesApp.txt
Let me know if there's anything specific you'd like me to test. I'm going to also try the new --frames parameter in libcamera-vid to see if the discrepancies are any different.

Maybe this is worthy of something like:
int dropped = currentSequence - previousSequence - 1; if (dropped) LOG(V4L2, Debug) << "Dropped " << dropped << " frame(s)"
in the V4L2VideoDevice then?

Interestingly, we've just hit exactly this on the IPU3 as well - so I've started adding frame drop detection logs in to V4L2VideoDevice.
Will probably want some rate-lmiited logging infrastructure in libcamera LOG_RATELIMITED(n/sec,...)
On the IPU3 our sequence numbers for the completed buffers come from the ISP which are always sequential even if the CSI receiver dropped a frame. I thought the ISP should preserve the sequence number of any buffer input into it, as the output - but it doesn't seem to.
What happens on RPi - does the ISP take any incoming sequence number and keep it as the outgoing sequence number? or does it generate a monotonic sequence number on the ISP outputs ?

What happens on RPi - does the ISP take any incoming sequence number and keep it as the outgoing sequence number? or does it generate a monotonic sequence number on the ISP outputs ?

Yes, it does independent monotonic sequence counting in the ISP driver and Unicam drivers...

Let me know if there's anything specific you'd like me to test. I'm going to also try the new --frames parameter in libcamera-vid to see if the discrepancies are any different.

Could you run libcamera-vid again, but with:

-o /dev/null instead of saving the output.
Do not supply a -o parameter at all.

I've had a bit more of a play with this and here are some further observations:

If I write a file to /dev/shm/, I do get multiple frame drops with the above command.  It seems like a bulk of the drops always happen at the 1s mark.  This is similar to your behaviour as well, which is suspicious.
We have a --flush command that flushes the IO on every write.  This seems to help, but not by much.
If we increase the internal buffer count to 10 frames, the drops reduce significantly.  However, we still see 1 or 2 drops per run (again, at around the 1s mark).
If we delete the output file in /dev/shm/ manually before running libcamera-vid, I see no frame drops at all!  This is with both 10 buffers, and the default 6 buffers.  It seems the file truncation operation is surprisingly slow when the IO is flushed to the disk.  @dustinkerstein could you try this and report back.

Another thing I noticed, your commands seem to use --width 4096 which is almost certainly not what you want.  The sensor output is 4056 pixels wide, so you will be getting a small resample on the output to resize to 4096 pixels wide, and this will impact image quality.  If you are doing this because of 32-byte wide alignment constraints, you probably want to use --width 4032 and add --roi=0,0,0.99409,1 to ensure the ISP crops an output window from the frame without resizing.

Good stuff. Seems like we've isolated things a bit. Confirming your results, when outputting to /dev/null I see no frame drops,
LIBCAMERA_LOG_LEVELS=0 ./libcamera-vid -t 24000 --framerate 10 --width 4056 --height 3040 --awbgains 1,1 --shutter 10000 --denoise cdn_fast --codec yuv420 --save-pts ~/timestamps.txt -o /dev/null

And also no drops when using no -o parameter:
LIBCAMERA_LOG_LEVELS=0 ./libcamera-vid -t 24000 --framerate 10 --width 4056 --height 3040 --awbgains 1,1 --shutter 10000 --denoise cdn_fast --codec yuv420 --save-pts ~/timestamps.txt

If I write a file to /dev/shm/, I do get multiple frame drops with the above command.  It seems like a bulk of the drops always happen at the 1s mark.  This is similar to your behaviour as well, which is suspicious.
Yeah, I consistently see the drops when writing to /dev/shm/ around that 1 second mark
If we delete the output file in /dev/shm/ manually before running libcamera-vid, I see no frame drops at all!  This is with both 10 buffers, and the default 6 buffers.  It seems the file truncation operation is surprisingly slow when the IO is flushed to the disk.  @dustinkerstein could you try this and report back.
Yep! I see the same thing. If that file exists already, I will see drops. If /dev/shm is empty I do not see drops.
Another thing I noticed, your commands seem to use --width 4096 which is almost certainly not what you want. The sensor output is 4056 pixels wide, so you will be getting a small resample on the output to resize to 4096 pixels wide, and this will impact image quality. If you are doing this because of 64-byte wide alignment constraints, you probably want to use --width 4032 and add --roi=0,0,0.99409,1 to ensure the ISP crops an output window from the frame without resizing.
Yeah, I was trying to accommodate the stride requirement (incorrectly). All of my new testing has been with --width 4056.

So it seems like the net results from testing so far are:

Writing to tempfs /dev/shm can cause frame drops if the output file already exists (or maybe just having other files in /dev/shm - need to test this as tempfs does some fancy memory management when it detects system ram is running low - though I do have swap disabled and I believe that is what it does when it needs to free space)
The new kernel logging does catch these frame drops more accurately than libcamera-vid. Could libcamera-apps (and libcamera generally) be tweaked to be able to catch those drops directly?
There is a suspicious time range around the 1 second mark that appears to be more susceptible to drops.

I'll post a few more testing results from my ISP bandwidth testing / hacking (capture 30fps 1080p but trick the ISP to process 4056x3040 frames.

The new kernel logging does catch these frame drops more accurately than libcamera-vid. Could libcamera-apps (and libcamera generally) be tweaked to be able to catch those drops directly?

This would have to go into libcamera - either in the pipeline handler or the V4L2 core.  @kbingham did you say you may be already doing this in the V4L2 class?

There is a suspicious time range around the 1 second mark that appears to be more susceptible to drops.

I believe this is down to the the same file truncation overhead.  The system is buffering up IO for approximately 1s then doing a large block write (involving truncating the file first causing the drops).

Note - This post is specific to ISP bandwidth testing / hacking (to see if the ISP can handle 4056x3040 12bit @ 30fps) and should not be looked at in context of this #157 dropped frames issue.
With the following changes to raspberrypi.cpp:
 // ISP HACK
format.size = Size(4056, 3040);

LOG(RPI, Debug) << "Setting " << stream->name() << " to "
		<< format.toString();

ret = stream->dev()->setFormat(&format);
if (ret)
	return -EINVAL;

// ISP HACK
// if (format.size != cfg.size || format.fourcc != fourcc) {
// 	LOG(RPI, Error)
// 		<< "Failed to set requested format on " << stream->name()
// 		<< ", returned " << format.toString();
// 	return -EINVAL;
// }

I'm not entirely sure this is a valid test, as the ISP is receiving junk buffers so I'm not sure if that makes it easier (and thus invalid) to process frames compared to a true 4056x3040 frame. Log output seems to confirm CSI interface is receiving 2028x1080 frames, and the ISP is set to 4056x3040:
[11:53:58.898419666] [162540]  INFO RPI raspberrypi.cpp:677 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1080-SBGGR12_1X12 - Selected unicam format: 2028x1080-pBCC
[11:53:58.898618367] [162540] DEBUG RPI raspberrypi.cpp:710 Setting ISP Output0 to 4056x3040-YU12
[11:53:58.898746013] [162540] DEBUG RPI raspberrypi.cpp:778 Setting ISP Output1 (internal) to 1200x898-YU12

When running both:
LIBCAMERA_LOG_LEVELS=0 ./libcamera-vid --framerate 30 --width 1920 --height 1080 -t 10000 --awbgains 1,1 --shutter 10000 --denoise cdn_fast --codec yuv420 --save-pts ~/timestamps.txt -o /dev/null

LIBCAMERA_LOG_LEVELS=0 ./libcamera-vid --framerate 30 --width 1920 --height 1080 -t 10000 --awbgains 1,1 --shutter 10000 --denoise cdn_fast --codec yuv420 --save-pts ~/timestamps.txt

I get about 30 drops.
But when I drop the framerate to 24 (still outputting to /dev/null) I get no frame drops. That's good to see, but feels somewhat contradictory to what https://bugs.libcamera.org/show_bug.cgi?id=77 describes. But I'll ignore that for now.
When I try to write either 24 or 30 fps to /dev/shm/ I get many more drops, and the effective non-drop framerate seems to be around 17fps for me.
To me, it seems like this ISP hacked bandwidth test appears to be at least somewhat valid given the change in behavior. If we consider it valid, then it seems the ISP can handle 4056x3040 12bit @ 24fps. The issue then becomes the fact that writing to /dev/shm (even when it's empty) is still causing drops. Since this particular behavior is somewhat related to #157, I'm curious if anyone here thinks there's a way to improve writing to /dev/shm.

I believe this is down to the the same file truncation overhead. The system is buffering up IO for approximately 1s then doing a large block write (involving truncating the file first causing the drops).

Kk, so one workaround is just to delete that file before writing to it. But do you think there's any way to handle it better in the case that it does? And related to my ISP testing / hacking above, it seems there are potentially general bandwidth IO issues when trying to write 4056x3040 beyond ~17fps to /dev/shm. Do you think there's anything that could be improved in that area? Thanks again!

But when I drop the framerate to 24 (still outputting to /dev/null) I get no frame drops. That's good to see, but feels somewhat contradictory to what https://bugs.libcamera.org/show_bug.cgi?id=77 describes. But I'll ignore that for now.

If you increase core_freq= and isp_freq= to something like 550, this might be enough of an overclock to get your required 30fps.  You may also have to use --denoise cdn_off, not too sure.

Kk, so one workaround is just to delete that file before writing to it. But do you think there's any way to handle it better in the case that it does? And related to my ISP testing / hacking above, it seems there are potentially general bandwidth IO issues when trying to write 4056x3040 beyond ~17fps to /dev/shm. Do you think there's anything that could be improved in that area? Thanks again!

Writing to /dev/shm seems to have more of an overhead than a standard memcpy (not surprising). We have a mode of operation where a large circular buffer is used to store the images until exit, where it is then flushed to disk.  You invoke this by the --circular command line option.  Currently, this circular buffer is 4Mb in size (see here).  You could perhaps investigate increasing that size to something like 6Gb to store all 300 12Mpix YUV frames, and avoid any IO operations until the end of the program.  Of course, to allocate and address 6Gb of memory, you have to be running in a 64-bit environment.

If you increase core_freq= and isp_freq= to something like 550, this might be enough of an overclock to get your required 30fps. You may also have to use --denoise cdn_off, not too sure.

I'm actually already testing with gpu_freq=800 (which sets ISP = 800) so I'm probably close to maxing things out.

Writing to /dev/shm seems to have more of an overhead than a standard memcpy (not surprising). We have a mode of operation where a large circular buffer is used to store the images until exit, where it is then flushed to disk. You invoke this by the --circular command line option. Currently, this circular buffer is 4Mb in size (see here). You could perhaps investigate increasing that size to something like 6Gb to store all 300 12Mpix YUV frames, and avoid any IO operations until the end of the program. Of course, to allocate and address 6Gb of memory, you have to be running in a 64-bit environment.

Now that sounds intriguing. I will test that tonight, and maybe also trying to integrate FFmpeg so I can start encoding in parallel (using FFmpeg to do the necessary tiling / cropping in software to not further tax the ISP). Will let you know how that goes. Thanks!

I'm actually already testing with gpu_freq=800 (which sets ISP = 800) so I'm probably close to maxing things out.

In that case try --denoise cdn_off and see if that helps.

I notice that the size_, rptr_ and wptr_ members of the circular buffer output class are unsigned ints (here). I think they can just be replaced by uint64_t and it will all work just the same. Of course you could start with < 4GB initially, just to see if it seems to work.

The new kernel logging does catch these frame drops more accurately than libcamera-vid. Could libcamera-apps (and libcamera generally) be tweaked to be able to catch those drops directly?

This would have to go into libcamera - either in the pipeline handler or the V4L2 core.  @kbingham did you say you may be already doing this in the V4L2 class?

Yes, I've started adding this already.

I notice that the size_, rptr_ and wptr_ members of the circular buffer output class are unsigned ints (here). I think they can just be replaced by uint64_t and it will all work just the same. Of course you could start with < 4GB initially, just to see if it seems to work.

Just gave 6GB a shot with the following modifications:

And it does avoid frame drops, but seems to have trouble after caching ~2GB. When capturing 4056x3040 @ 10fps, I can only get it to write 114 frames (even though it captured > 114 frames). The following command:
LIBCAMERA_LOG_LEVELS=0 ./libcamera-vid --circular --framerate 10 --width 4056 --height 3040 --frames 114 --awbgains 1,1 --shutter 10000 --denoise cdn_fast --codec yuv420 --save-pts ~/timestamps.txt -o ~/camera.yuv

Captures 114 frames. But when setting frames>114, it still only writes 114 frames. Here's the tail end of a log when I tried capturing for 24 seconds @ 10fps:
[21:49:37.932779112] [323746] DEBUG DelayedControls delayed_controls.cpp:179 Queuing Analogue Gain to 853 at index 241
[21:49:37.932869555] [323746] DEBUG DelayedControls delayed_controls.cpp:179 Queuing Exposure to 350 at index 241
[21:49:37.932950128] [323746] DEBUG DelayedControls delayed_controls.cpp:179 Queuing Vertical Blanking to 460 at index 241
[21:49:37.933052997] [323746] DEBUG RPISTREAM rpi_stream.cpp:237 Queuing buffer 0 for ISP Stats
[21:49:37.933108959] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1593 /dev/video16[18:cap]: Queueing buffer 0
[21:49:37.933322752] [323746] DEBUG Request request.cpp:287 Request(232:C:0/1:0)
[21:49:37.933412491] [323748] DEBUG Camera camera.cpp:1126 Stopping capture
[21:49:37.944448107] [323746] DEBUG Request request.cpp:287 Request(233:X:0/1:0)
[21:49:37.944503477] [323746] DEBUG Request request.cpp:287 Request(234:X:0/1:0)
[21:49:37.944536884] [323746] DEBUG Request request.cpp:287 Request(235:X:0/1:0)
[21:49:37.944600679] [323746] DEBUG Request request.cpp:287 Request(236:X:0/1:0)
[21:49:37.944690270] [323746] DEBUG Request request.cpp:287 Request(237:X:0/1:0)
[21:49:37.944959803] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1459 /dev/video0[13:cap]: Releasing buffers
[21:49:37.945026802] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1177 /dev/video0[13:cap]: 0 buffers requested.
[21:49:37.955564704] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1459 /dev/video1[14:cap]: Releasing buffers
[21:49:37.955687739] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1177 /dev/video1[14:cap]: 0 buffers requested.
[21:49:37.955936365] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1459 /dev/video13[15:out]: Releasing buffers
[21:49:37.955974697] [323746] DEBUG V4L2 v4l2_videodevice.cpp:201 Cache misses: 240
[21:49:37.959748173] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1177 /dev/video13[15:out]: 0 buffers requested.
[21:49:37.959811116] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1459 /dev/video14[16:cap]: Releasing buffers
[21:49:37.959986428] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1177 /dev/video14[16:cap]: 0 buffers requested.
[21:49:37.963854310] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1459 /dev/video15[17:cap]: Releasing buffers
[21:49:37.963924401] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1177 /dev/video15[17:cap]: 0 buffers requested.
[21:49:37.964562465] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1459 /dev/video16[18:cap]: Releasing buffers
[21:49:37.964619760] [323746] DEBUG V4L2 v4l2_videodevice.cpp:1177 /dev/video16[18:cap]: 0 buffers requested.
NullEncoder closed
Wrote<PHONE_NUMBER> bytes (114 frames)

Any idea on why that might be happening?

You've spotted a couple more uint64_ts which is great, but I think we might need to change the return type of the Available() function to uint64_t too. I have a feeling the while loop in the outputBuffer() function will keep dragging the front of the buffer forward once that function returns a small number (modulo 2^32).

That all sounds good. I'm starting to appreciate how expensive time-wise memory allocation is (6GB seems to take ~10 seconds). Do you think there's any feasibility of allocating/copying single frame buffers on-the-fly, or does it just take too much time?

That all sounds good. I'm starting to appreciate how expensive time-wise memory allocation is (6GB seems to take ~10 seconds). Do you think there's any feasibility of allocating/copying single frame buffers on-the-fly, or does it just take too much time?

I doubt that 300x single frame allocations are going to be much quicker.  But it should be quick enough for you to test this in a simple program.  If you do go down this route, you also cannot use the --circular functionality already provided and will have to implement something new that provides similar functionality.

I'm not entirely sure this is a valid test, as the ISP is receiving junk buffers so I'm not sure if that makes it easier (and thus invalid) to process frames compared to a true 4056x3040 frame. Log output seems to confirm CSI interface is receiving 2028x1080 frames, and the ISP is set to 4056x3040:

One issue I see with this is that you are not modelling the Unicam -> memory write correctly.  Instead of writing 4056x3040@30fps to SDRAM, we are doing 2028x1080@30fps, so there will be addition memory bandwidth that needs to be accounted for.
Did you manage to run with --denoise cdn_off to see if the ISP would hit 30fps with your overclock?

I doubt that 300x single frame allocations are going to be much quicker. But it should be easy enough for you to test this in a simple program. If you do go down this route, you also cannot use the --circular functionality already provided and will have to implement something new that provides similar functionality.

Yeah, I might give it a shot later this week. First test would be to just see how much time it takes to copy the buffer, and if it's reasonable I'll try to do a simple poc.

One issue I see with this is that you are not modelling the Unicam -> memory write correctly. Instead of writing 4056x3040@30fps to SDRAM, we are doing 2028x1080@30fps, so there will be addition memory bandwidth that needs to be accounted for.

True, but do you think that affects the ISP's resources directly, or more overall system memory bandwidth?

Did you manage to run with --denoise cdn_off to see if the ISP would hit 30fps with your overclock?

I just did and it seems to perform the same as with cdn_fast with an effective rate of ~26fps when writing to /dev/null

This seems to imply that memory bandwidth has become a limiting factor. Theoretically, the ISP should be able to keep up with an 800Mhz clock.

Ok, interesting. It's very close... I'll keep hacking around to see if I can figure out a suitable buffering solution.
Feel free to close this issue unless anyone feels there's anything left actionable. Thanks everyone for the help.
PS - Arducam wasn't too keen on providing a quote for a 4 lane CSI IMX477 patch given their limited resources, and the limited functionality it would have (ie. only really being suitable for small burst captures of YUV/Bayer to ramdisk). I'll try to convince them to at least give me a quote, but if anyone knows of another way to get a 4 lane CSI patch (or is interested in helping via a bounty), feel free to reach out to me directly at dustin @ panomoments.com

This seems to imply that memory bandwidth has become a limiting factor. Theoretically, the ISP should be able to keep up with an 800Mhz clock.

Ahha! It looks like the ISP clock slows down from an overclocked 850mhz -> 300mhz after around 5 seconds, and that's where the frame drops start happening. Note that I'm testing using the hack to force ISP Output0 to 4056x3040 with the sensor running at 2028x1080 (though I see this same clock slowdown without the hack - just no frame drops). If I use force_turbo=1, the ISP clock stays at 850mhz, and I don't see any drops. Ie. 4056x3040 30fps ISP output seems to work when writing to /dev/null. Has anyone seen the ISP clock slowing down like that?
ISPClockSlowdown.txt
ISPClockForceTurbo.txt

Ahha! It looks like the ISP clock slows down from an overclocked 850mhz -> 300mhz after around 5 seconds, and that's where the frame drops start happening. Note that I'm testing using the hack to force ISP Output0 to 4056x3040 with the sensor running at 2028x1080 (though I see this same clock slowdown without the hack - just no frame drops). If I use force_turbo=1, the ISP clock stays at 850mhz, and I don't see any drops. Ie. 4056x3040 30fps ISP output seems to work when writing to /dev/null. Has anyone seen the ISP clock slowing down like that?

Try using gpu_freq_min instead of gpu_freq.  The latter still allows clock scaling, but the former sets a lower bound.  Good to hear this keeps up without frame drops.  Are you using --denoise cdn_off as well?  I suspect you can wind down the 850Mhz to something like 550-600Mhz and still keep up.

Try using gpu_freq_min instead of gpu_freq and remove force_turbo. The gpu_freq still allows clock scaling, but gpu_freq_min sets a lower bound. Good to hear this keeps up without frame drops. Are you using --denoise cdn_off as well? I suspect you can wind down the 850Mhz to something like 550-600Mhz and still keep up.

Ok cool. I'll give that a try with cdn_off and cdn_fast (usually I'm testing with the latter as I will want to use this NR setting).
The bigger news is that I think I found the bottleneck which has been causing lots of head scratching (at least on my part). Writing to tempfs /dev/shm isn't the issue. It appears as though libcamera-vid's output itself is the bottleneck. Check out these tests and let me know what you think:
Pipe Bandwidth
dd if=/dev/zero bs=6M count=1000 | dd of=/dev/null bs=6M
6291456000 bytes (6.3 GB, 5.9 GiB) copied, 8.75606 s, 719 MB/s

Tempfs /dev/shm Bandwidth
dd bs=6M count=1000 if=/dev/zero of=/dev/shm/test
6291456000 bytes (6.3 GB, 5.9 GiB) copied, 8.9798 s, 701 MB/s

libcamera-vid Bandwidth (using hacked code from above to output 4096x3040 YUV frames @ 30fps)
libcamera-vid --width 1920 --height 1080 --framerate 30 -t 10000 --gain 1 --awbgains 1,1 --shutter 1000 --denoise off --codec yuv420 -o - | dd of=/dev/null bs=6M
3698196480 bytes (3.7 GB, 3.4 GiB) copied, 11.2344 s, 329 MB/s
3698196480 bytes / 18677760 bytes per frame = 198 frames (101 drops - 1 frame is always missing with -t parameter)
libcamera-vid --width 1920 --height 1080 --framerate 18 -t 10000 --gain 1 --awbgains 1,1 --shutter 1000 --denoise off --codec yuv420 -o - | dd of=/dev/null bs=6M
3343319040 bytes (3.3 GB, 3.1 GiB) copied, 10.884 s, 307 MB/s
3343319040 bytes / 18677760 bytes per frame = 179 frames (0 drops - 1 frame is always missing with -t parameter)

At around 19fps, libcamera-vid starts to drop frames. It appears the max throughput of libcamera-vid seems to be around 330MB/s-350MB/s, and that's why I am dropping frames when writing to /dev/shm (or piping to FFmpeg/Gstreamer).  Do you think that's something that can be investigated? Let me know if you'd like me to test anything else. Thanks!

Reading/copying those buffers seems to be the culprit. For 4096x3040 frames it appears to take ~55ms for fwrite(mem, size, 1, fp_), and I get the same time if I manually copy the buffer with memcopy. 1 / 0.055 = 18.2, which is the FPS threshold where I start seeing drops. 55ms for copying 18.7MB is only 340MB/s, compared to the nearly double throughput of the dd / pipe tests above. I'll keep poking around. Have a great weekend.

Another quick update. I tested --circular again to see how it compared, even though for my use-case it isn't suitable due to the initial ~10 second memory allocation time. The Write method takes ~38ms for 4056x3040 frames, and is able to successfully capture at 26fps. Beyond that, it starts dropping frames. Here is the command I was testing with (along with the hacked code from above to output 4096x3040 YUV frames from the ISP):
libcamera-vid --circular 6000 --framerate 26 --width 1920 --height 1080 -t 8000 --codec yuv420 -o /dev/null

So, pre-allocating the buffers does save a little time (55ms -> 38ms), but it still is bottlenecked by the memcopy / fwrite. From some quick research, it does seem like memory copies from CMA -> User Memory is known to be slow. Do you think that's what we're hitting?

Just did a few more tests. Hopefully they provide some insight. I tried the following command: libcamera-vid --circular 6000 --framerate 30 --width 1920 --height 1080 -t 8000 --codec yuv420 -o /dev/null with varying buffers defined in libcamera_app.cpp, up to the max my Pi allowed.
8 seconds @ 30fps, 15 buffers - Wrote<PHONE_NUMBER> bytes (210 frames), Wrote<PHONE_NUMBER> bytes (212 frames)
8 seconds @ 30fps, 14 buffers - Wrote<PHONE_NUMBER> bytes (211 frames), Wrote<PHONE_NUMBER> bytes (210 frames)
8 seconds @ 30fps, 13 buffers - Wrote<PHONE_NUMBER> bytes (210 frames), Wrote<PHONE_NUMBER> bytes (209 frames)
8 seconds @ 30fps, 12 buffers - Wrote<PHONE_NUMBER> bytes (209 frames), Wrote<PHONE_NUMBER> bytes (209 frames)
8 seconds @ 30fps, 10 buffers - Wrote<PHONE_NUMBER> bytes (206 frames), Wrote<PHONE_NUMBER> bytes (206 frames)
8 seconds @ 30fps, 8 buffers - Wrote<PHONE_NUMBER> bytes (206 frames), Wrote<PHONE_NUMBER> bytes (206 frames)
8 seconds @ 30fps, 6 buffers - Wrote<PHONE_NUMBER> bytes (204 frames), Wrote<PHONE_NUMBER> bytes (202 frames)
8 seconds @ 30fps, 2 buffers - Wrote<PHONE_NUMBER> bytes (198 frames), Wrote<PHONE_NUMBER> bytes (201 frames)
8 seconds @ 30fps, 1 buffers - Wrote<PHONE_NUMBER> bytes (157 frames), Wrote<PHONE_NUMBER> bytes (157 frames)

Note that as with the previous tests, this is with the ISP forced to output 4056x3040 frames. It seems as though increasing the amount of buffers matters, but has diminishing returns.

The CMA buffers have to be uncached, so that is probably there is a performance penalty compared to user memory buffers.
Regarding the 10s allocation time with --circular, it turns out that the std::vector constructor will allocate as well as zero the memory allocated if a size parameter is provided.  This is why it takes ~10s.  You can avoid the zeroing of the memory by not passing the size into the constructor, and calling vector.reserve(SIZE) instead.  A quick test shows that this takes the same amount of time as a standard malloc(SIZE) call (60us for a 2GB buffer).  However, when you come to access the memory, pages do get allocated on-the-fly.  This will mean that as you fill the circular buffer, there may be slowdowns along the way.  If you are lucky, the additional frame buffers may be able to take the slack and keep up without dropping the frame.
So this may come down to a choice of taking 10s up-front and ensure all the pages are allocated, or allocating the pages on-the-fly, and having enough buffering in the camera pipeline to avoid any frame drops.

Ok interesting. I'll play around with that later today. On a related note, do you happen to know of any way to get the Pi to allocate CMA beyond the 1GB barrier? I've heard it might break some peripherals, but that might be ok for me depending on what it breaks. While a bit ugly, it's temping to just have 300 buffers in CMA... Thanks for your help.

You can use the vc4-kms-v3d (Bullseye) overlay to change cma size.  Try dtoverlay -h vc4-kms-v3d for more info.
Note, that neither the ISP nor the Unicam hardware can address that high memory, so it would be pointless.

I'll close this down as the original issue should be resolved.


[lore-generate-new] provide flag to create project with yarn support over npm

Yarn is out. People seem to like it as an alternative to npm. The faster install times are nice and I'm a fan of the default lockfile, especially since lerna doesn't support minor versioning as a first-class concept.
Lerna really wants packages to be using major versions to represent breaking changes in order to benefit the most from it's bootstrapping and publishing behaviors. This is a challenge between it means code has to be published to npm manually if I want to keep the ~0.10.x as the package version in package.json files instead of ^0.10.x, which Lerna automatically changes packages to.
Lockfile's would reduce the risk surrounding publishing minor version changes that a user might unintentionally install, as the lockfile would hold them to whatever version of packages their code was using yeseterday when it worked.

Closing this as npm v5 has been released which is considerably faster than v4, and now defaults to generating a lockfile (package-lock.json), which addresses the two largest issues that made yarn an attractive alternative.
I haven't looked into the nuts and bolts of what's now different between the two package managers, but with those changes, the value of switching to yarn is unclear, and there are no longer any plans use yarn for managing lore or to provide a --yarn flag for new project generator.


[Feature] Connect starrocks custom metrics to datadog

Starrocks has custom metrics available for pulling.
However, it requires manual steps to configure Prometheus server to pull the metrics.
We need to make the operator  automatically setup the metrics pulling and connect it to datadog.
Ref:
https://docs.starrocks.io/en-us/latest/administration/Monitor_and_Alert
https://docs.datadoghq.com/containers/kubernetes/prometheus/?tab=kubernetesadv2

@kevincai @shileifu
It seems that as long as you allow adding annotation to the fe/be pod in values.yaml , we could configure the promethus scrape with DD.
Can you support this?
    ad.datadoghq.com/<CONTAINER_IDENTIFIER>.checks: |
      {
        "openmetrics": {
          "init_config": {},
          "instances": [
            {
              "openmetrics_endpoint": "http://%%host%%:%%port%%/<PROMETHEUS_ENDPOINT> ",
              "namespace": "<METRICS_NAMESPACE_PREFIX_FOR_DATADOG>",
              "metrics": [{"<METRIC_TO_FETCH>":"<NEW_METRIC_NAME>"}]
            }
          ]
        }
      }


Update 094.1-2048.md

Demo

Test:
https://mayank2424.github.io/2048-Puzzle-Game/

Closed via #607

Sorry ,Wait It will be open shortly
On Thu, May 24, 2018, 2:47 PM Nils Weber<EMAIL_ADDRESS>wrote:

Closed #606 https://github.com/CodingTrain/website/pull/606.
—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/CodingTrain/website/pull/606#event-1643742803, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AbXAXEgKEpYdDaZLfX4bXaaPoDx_IVAyks5t1nq_gaJpZM4ULlie
.


fsync head chunks file before closing temporary file

Even though we first write the segment file to a temporary file and then rename, the Close() method does not assure writes to be reflected to disk (TIL). We faced the mmap: invalid argument error again with the temp file protection.
We should be syncing the temporary file before closing, similar to what we do for compaction. While this is on the hotpath, the sync call won't be that often and should not be a problem in most cases.

While this is on the hotpath, the sync call won't be that often and should not be a problem in most cases.

Syncs can take seconds, so this could cause quite a stutter. What's the failure mode that this is meant to be covering? This should only matter if there's a machine crash within a minute of the write.

What's the failure mode that this is meant to be covering?

Prometheus/TSDB not able to start on crash if it was unlucky to hit this case. Another option is to ignore an empty last file while reading the files.

A Prometheus crash or a machine crash? Sync should never make a difference for a Prometheus crash.

Machine crash, hence a force shutdown on Prometheus

Could we do it async as we do elsewhere?
Being resilient in the read path is also a good idea.

Could we do it async as we do elsewhere?

We want data to go to disk before rename, so async might not be helpful.

Being resilient in the read path is also a good idea.

I will do this rather.


修复linux依赖检测

PR Type
What kind of change does this PR introduce?

[ ] Bugfix
[ ] Feature
[ ] Code style update (formatting, local variables)
[ ] Refactoring (no functional changes, no api changes)
[ ] Other... Please describe:

What is the current behavior?

Issue Number: N/A
What is the new behavior?
Does this PR introduce a breaking change?

[ ] Yes
[ ] No

Other information

apt info package不管是否安装都会存在关键字"installed",导致qinglong安装linux依赖永远都是已安装。“apt-manual-installed”的值为yes或no，用于区别是否为手动安装，正好可以判定当前是否已安装。

👍


[Schema] Enhance import design schema

Description
This PR fixes #9145
This PR implements conditional logic in the import design schema to display only 'URL Import' option exclusively when the user selects 'Helm Chart' as the design type, as file upload is not supported in this case. For other design types, the PR provides the option to choose between 'URL Import' and 'File Upload.'
Additionally, I updated the title format from Title Case to Sentence case for improved consistency and enhanced field descriptions with additional helpful information and relevant documentation links.
Here is a video of the updated import modal
https://github.com/meshery/meshkit/assets/71271069/60c8bd3b-4c62-4657-83f0-076440447b1c
Notes for Reviewers
Signed commits

[ ] Yes, I signed my commits.

Thanks @theBeginner86 for reviewing, I'm moving forward here

FYI @leecalcote

Yay. Looks good. “Url import” ✅

In general, if we can hyperlink text and not show the actual hyperlink, that’s better.

Sounds reasonable I'll look


Inconsistent .NET Core 3.1 dependencies

The "Dependencies" Section for .NET Core 3.1 does not match with the linked .NET Core 3.1 Supported OS Versions document.
The other document lists Windows 7 SP1 and Windows 10 1607, this page does not list Windows 7 and it lists Windows 10 1609 (is this the same as 1607?)
Which of these tables is correct?

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 1eeab451-820f-de42-983e-4d0d5764f6ab
Version Independent ID: d11d0d14-8a03-50e4-5c00-9a4417ad4d0a
Content: Install .NET Core on Windows
Content Source: docs/core/install/windows.md
Product: dotnet-core
GitHub Login: @adegeo
Microsoft Alias: adegeo

I just saw that I can view all issues for this page. I guess this issue is a duplicate of #19384 but there is still the discrepancy between this document and the linked document. This issue can be closed if everything can be tracked in #19384

Thanks, yeah looks like duplicate 😄 But thank you for filing an issue and interacting with the docs!

Duplicate of #19384


Windows dll build for 1.6.16

is there a windows dll build available for 1.6.16?
If not could you provide clear instructions on how to create one?
Thanks

Sorry, there's no official support for windows. You can likely build it using WSL

ok ty.


Would be possible to watch only one function?

Currently I have npm run watch set to run webpack-cli -w and my webpack.config.js is the same as the one shown at the beginning of the README here on Github.
Currently in our SAM project my dev machine takes more than a minute to build. So I was wondering if there was a way to create a command where I can run npm run watch myFunction and it will only watch that single function? Usually when doing dev work I'm often only working on one single function at a time so this would be super helpful.
I suspect that there is a way but my knowledge of Webpack & of the aws-sam-webpack-plugin is not great. Any ideas?

Replying to myself...
I am now able to do the following:
Assuming that the handler is in src/handlers/users/create.ts and the function name in template.yml is createUser, then I build like this:
npm run build -- --env handler=users/create.ts --env functionName=createUser
and I can watch with:
npm run watch -- --env handler=cards/blocks.ts --env functionName=customerBlocks
This assumes that all handlers are in src/handlers. If you don't have a handlers folder then modify entry below.
const path = require("path");
const AwsSamPlugin = require("aws-sam-webpack-plugin");

module.exports = (env) => {
  let awsSamPlugin = {}
  const singleFunction = (env.handler && env.functionName);
  if (singleFunction) {
    awsSamPlugin = {
      entry: () => '',
      filename: () => ''
    }
  } else {
    awsSamPlugin = new AwsSamPlugin();
  }
  return {
    // Loads the entry object from the AWS::Serverless::Function resources in your
    // SAM config. Setting this to a function will
    entry: singleFunction ? './src/handlers/' + env.handler : () => awsSamPlugin.entry(),

    // Write the output to the .aws-sam/build folder
    output: {
      filename: singleFunction ? './.aws-sam/build/' + env.functionName + '/app.js' : (chunkData) => awsSamPlugin.filename(chunkData),
      libraryTarget: "commonjs2",
      path: path.resolve(".")
    },

    // Create source maps
    devtool: "source-map",

    // Resolve .ts and .js extensions
    resolve: {
      extensions: [".ts", ".js"]
    },

    // Target node
    target: "node",

    // AWS recommends always including the aws-sdk in your Lambda package but excluding can significantly reduce
    // the size of your deployment package. If you want to always include it then comment out this line. It has
    // been included conditionally because the node10.x docker image used by SAM local doesn't include it.
    externals: process.env.NODE_ENV === "development" ? [] : ["aws-sdk"],

    // Set the webpack mode
    mode: process.env.NODE_ENV || "production",

    // Add the TypeScript loader
    module: {
      rules: [{ test: /\.tsx?$/, loader: "ts-loader" }]
    },

    // Add the AWS SAM Webpack plugin
    plugins: singleFunction ? [] : [awsSamPlugin]
  }
};

@graphboss I saw the source of aws-sam-webpack-plugin and I think this could be achieved by adding an optional parameter to entry like so:
awsSamPlugin.entry(functionName)
then it would be possible to do
npm run watch -- --env functionName=myFunction
thus eliminating the need to create two env variables (since aws-sam-webpack-plugin is already able to find the path of each handler).
If you think this is a viable change I can make a PR.


Feature Request: Automatic turn OFF or ON after configurable time

Example:
A user switches ON a lamp, and the plugin makes sure it will turn of after a predefined interval.
As far as I know, this kind of automation is not possible through home app.
I thing it would be a good idea to implement such function inside the plugin. If I'm wrong, I do apologize in advance.
Thanks,

It's already a feature in the broadlink rm homepridge plugin, so hope this helps: https://github.com/lprhodes/homebridge-broadlink-rm


SharePoint Connector

Awaiting input on the testing approach.

Can you please add a proper file structure. Currently, it looks like

The SharePoint folder should contain everything related to the module and should have an app.json file.
Inside the code should be in src folder, permissions folder could be either in the root folder or in src.
Just follow the structure of some of the other modules.

@mazhelez , @IhorHandziuk any feedback on test app?


How to include custom, local, plugin

Is there a way to add custom plugins to SVGO? I have a small plugin I'd like to include as a build step in a project.
For example, I'd like to run svgo with a command line option pointing to a directory that contains custom plugins. Or point to said directory in my .svgo.yml configuration file. Is this possible?

No, it's not possible for now.

@GreLI would you be happy for a contribution adding this ability?

Yes

So looking at the source I think something like this is already supported. When I pass in a configuration looking like this:
{
  plugins: [
    {
     custom: { // this could be any name except the already registered ones
        type: 'full', // full, perItem or perItemReverse
        description: 'My custom plugin',
        params: {}, // some arbitrary data
        fn: function(data, params) {
          // custom plugin code goes here
          return data;
        }
     }
  ]
}```

I get to execute my code in a custom manner.

It seems that the only limitation is:

- Only supports inline code, so you have to invoke SVGO via the API rather than using command line or YAML.

Oh sweet! Thanks for looking into this @gampleman!
In my opinion it would be ideal to have a config (or command line) option that points to a local directory of custom plugins that following the conventions of the plugins bundled inside SVGO. This directory would essentially be merged with bundled plugins.
I peeked at the source when I created this issue but didn't have time to investigate further.


Tilting-pad thermo-hidro-dynamic bearing models

Creation of the hydrodynamic tilting-pad bearing model for their properties determination inside ROSS.
This is a WIP, currently being ported to Python from a functional and validated Matlab code. The code is in Running through all relevant formulation. The current development target is the post-processing phase, translating the bearing fields into practical K and C coefficients.

Hello and good afternoon!
This is the pull request created for the tilting pad bearings model we are implementing. Since we have a working code on Matlab, it is a matter of translating and converting it to object-oriented programming style. This work has been started recently, and should take some time until it nears its completion, but if possible, we would like to request a sample of the results you obtain from your commercial code, so that we can standardize our code outputs.
Changes in the output generation will undoubtedly incur in many structural modifications inside the code, so the sooner we have a defined standard to follow the better. We have a sample of this that has been provided for the cylindrical bearing model, and a similar one for the tilting-pad bearing should be enough to set the standard for this code too.
Thanks in advance, and have a great evening!
(@bela-bs , @marcusfilipesr , @hiagopinacio , @raphaeltimbo , @rodrigomoliveira1 )

Hello everyone. I am joining you in this work. Any questions about hydrodynamic bearing will be available to help.

Hello everyone. I am joining you in this work. Any questions about hydrodynamic bearing will be available to help.

@elyqg is our expert on the bearing models, and both me (@verg1lio) and @marcusfilipesr will be assisting on the coding and debug.

Hello, @verg1lio, good morning.
It's a very nice initiative. Besides, I think @JuliaMota and @davidjgm can help you out, since they work mainly with the FluidFlow code.

Hello there!
Just a quick update here... I have been translating the iterative loops to build the variable fields inside the bearings, and since they are all one inside the other and very much recursive in nature, they will be done all at once. So, on Trello (https://trello.com/c/zm7tJ29x/11-bearing-tilting-pad-thermohydrodynamic) my Tilting pad bearing card will be updated in a burst when the loops are done. After that the process should be more streamlined, but until then i'll focus here to get it right.
Hope you guys are having a productive week!

Hello, and good afternoon!
I have just finished the rough translation of the code into Python language. Will now proceed to finalize any complex matlab function of architecture left behind, and then debug the code until it matches the results produced by the Matlab original one.
After thet point, the next steps will be the architectural change into Object Oriented programming and the modularization of the internal loops of the code, in order to turn it into a more useful and generalized program, as per our agreement.
Wish you all a great week!
Vergilio

Hi, and good morning! (@raphaeltimbo and @rodrigomoliveira1)
I have finished conversion of the code and it runs through the whole bearing simulation, but as of today the result is still not quantitatively correct. Therefore, @marcusfilipesr will be starting the conversion to modular programming and object oriented paradigm, keeping this code and the one for cylindrical bearings standardized, while i finish ironing out the last minor convergence errors in this one, matching its results to our Matlab original one.
Hope you guys have a great day, and i'll keep you updated of our progress.
Cheers!

Hi there!
I have marked this issue as stale because it has not had activity for 45 days.
Consider the following options:

If the issue refers to a large task, break it in smaller issues that can be solved in
less than 45 days;
Label the issue as wontfix or wontfix for now and close it.

Dear all (@raphaeltimbo , @marcusfilipesr , @hiagopinacio , @elyqg , @bela-bs ), good morning!
This THD tilting pad bearing model is accurately matching our results from other methodologies and previous benchmarks found in the specific literature. I believe it is ready for merging in the main ROSS code.
Is there something else to make on it or we can proceed with the merging?
Thanks, and have a great day!
P.S. I am working locally on the THD thrust tilting pad bearing model, and should finish its rough translation soon. Once that is done it will be committed to its branch for debugging and corrections to its code.

Dear all (@raphaeltimbo , @marcusfilipesr , @hiagopinacio , @elyqg , @bela-bs ), good morning!
This THD tilting pad bearing model is accurately matching our results from other methodologies and previous benchmarks found in the specific literature. I believe it is ready for merging in the main ROSS code.
Is there something else to make on it or we can proceed with the merging?
Thanks, and have a great day!
P.S. I am working locally on the THD thrust tilting pad bearing model, and should finish its rough translation soon. Once that is done it will be committed to its branch for debugging and corrections to its code.

I have also provided the relevant information for a simulation on MaxBRG for comparison, but never got the results back. I therefore utilized literature benchmarks and experimental tests performed in our lab (LMEst) for this model validation. The code IS VALIDATED both experimentally and numerically, as of today, matching our benchmark results to its numerical precision.

Dear @raphaeltimbo, I hope to find you in better health!
We (@verg1lio and @elyqg) will take this week to add the K and C coefficients determination to this methodology, adapting the modular code made for the THD cylindrical bearings model. I will get back to you once its done, to remove the WIP from here and do the final conference and merge.
Cheers!

Hi @verg1lio ! Thank you for the message.
Ok. Let me know if you need any help in the implementation.

Dear @raphaeltimbo , good morning!
I believe that this function is now finished and ready for merging. @elyqg and I did the inclusion of the K and C coefficients determination procedure to it, and it is working as intended. If there is any modification or correction still to be done, please let us know.
I am currently doing the hard translation of the Thrust THD bearing model from Matlab to Python, and have about a third of it done as of now. In the next few weeks I will upload the translated code to our Thrust bearing branch and start the debugging.
Have a great day!

Hi @verg1lio !
You have some doc tests failing for this PR. You can check that by clicking 'Details' in the Tests / build actions above.
As an example:
=================================== FAILURES ===================================
______ [doctest] ross.fluid_flow.tilting_pad.tilting_pad_bearing_example _______
1577     This function returns pressure and temperature fields as well as the dynamic
1578     coefficients. The purpose is to make available a simple model so that a doctest
1579     can be written using it.
1580     Returns
1581     -------
1582     TiltingPadBearing : ross.TiltingPadBearing Object
1583         An instance of a termo-hydrodynamic tilting pad bearing model object.
1584     Examples
1585     --------
1586     >>> bearing = tilting_pad_bearing_example()
UNEXPECTED EXCEPTION: AttributeError("'TiltingPadBearing' object has no attribute 'equilibrium_pos'")
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest ross.fluid_flow.tilting_pad.tilting_pad_bearing_example[0]>", line 1, in <module>
  File "/home/runner/work/ross/ross/ross/fluid_flow/tilting_pad.py", line 1674, in tilting_pad_bearing_example
    THD_tilting_pad.coefficients()
  File "/home/runner/work/ross/ross/ross/fluid_flow/tilting_pad.py", line 1459, in coefficients
    if self.equilibrium_pos is None:
AttributeError: 'TiltingPadBearing' object has no attribute 'equilibrium_pos'

Hi @verg1lio !
You have some doc tests failing for this PR. You can check that by clicking 'Details' in the Tests / build actions above.
As an example:
=================================== FAILURES ===================================
______ [doctest] ross.fluid_flow.tilting_pad.tilting_pad_bearing_example _______
1577     This function returns pressure and temperature fields as well as the dynamic
1578     coefficients. The purpose is to make available a simple model so that a doctest
1579     can be written using it.
1580     Returns
1581     -------
1582     TiltingPadBearing : ross.TiltingPadBearing Object
1583         An instance of a termo-hydrodynamic tilting pad bearing model object.
1584     Examples
1585     --------
1586     >>> bearing = tilting_pad_bearing_example()
UNEXPECTED EXCEPTION: AttributeError("'TiltingPadBearing' object has no attribute 'equilibrium_pos'")
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.8.10/x64/lib/python3.8/doctest.py", line 1336, in __run
    exec(compile(example.source, filename, "single",
  File "<doctest ross.fluid_flow.tilting_pad.tilting_pad_bearing_example[0]>", line 1, in <module>
  File "/home/runner/work/ross/ross/ross/fluid_flow/tilting_pad.py", line 1674, in tilting_pad_bearing_example
    THD_tilting_pad.coefficients()
  File "/home/runner/work/ross/ross/ross/fluid_flow/tilting_pad.py", line 1459, in coefficients
    if self.equilibrium_pos is None:
AttributeError: 'TiltingPadBearing' object has no attribute 'equilibrium_pos'

Good morning @raphaeltimbo , and thanks for the input!
I will get to work on this right away, and work out the details for it to be successfully merged to the main.
Have a great week,
Vergílio

Hi there!
I have marked this issue as stale because it has not had activity for 45 days.
Consider the following options:

If the issue refers to a large task, break it in smaller issues that can be solved in
less than 45 days;
Label the issue as wontfix or wontfix for now and close it.

I'll be closing this PR since it has not any updates for a while.

I'll be closing this PR since it has not any updates for a while.


Support for fragment imports

graphql-tag's webpack loader uses the following syntax to include fragments from within .graphql files:
#import "./UserInfoFragment.graphql"
Right now, imports are being completely ignored in the Jest transform.

Fixed in: https://github.com/remind101/jest-transform-graphql/pull/4


resources required for training with 72b language models

Hi, thanks for your great work. I was wondering the how many gpus are needed to training internvl with 72b llm.

Hello, if you start with separate ViT and LLM models, connect them with a randomly initialized MLP projector, and perform complete pretraining and SFT, you will need at least 256 GPUs with 80GB memory each. However, if you are fine-tuning based on our released InternVL2-Llama3-76B model, we recommend using 32 GPUs.


Locale && Plugin update

1 rollup bundle
2 export utils
3 set en (English) as default locale
4 local add ordinal for plugin advancedFormat

Codecov Report

Merging #141 into master will not change coverage.
The diff coverage is 100%.

@@          Coverage Diff          @@
##           master   #141   +/-   ##
=====================================
  Coverage     100%   100%
=====================================
  Files           4      4
  Lines         253    252    -1
  Branches       46     46
=====================================
- Hits          253    252    -1

Impacted Files
Coverage Δ

src/utils.js
100% <100%> (ø)
:arrow_up:

src/constant.js
100% <100%> (ø)
:arrow_up:

src/index.js
100% <100%> (ø)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d9a01e...5e5780b. Read the comment docs.


Accessibility: Add keyboard handling for XArray HTML view

The default html view generated via XArray uses hidden checkboxes to achieve collapse expand behaviour. This PR -
Makes the hidden checkboxes visible with 0 opacity , to make them accessible via keyboard Make style changes to associated labels for focus, to highlight where the keyboard focus is Change the grid layout to account for the invisible checkboxes
https://github.com/user-attachments/assets/3f7cb0c9-2b10-4abd-9098-96e653d2392b
issue - https://github.com/pydata/xarray/issues/9397

Very nice! Thank you.

Thanks @srijan55 ! Any other keyboard shortcuts very welcome!


installation script proposal

Can I propose that an installation script be introduced that will download the latest release binary executable and install it in your GOPATH/bin directory?
I am attaching such a script. Not sure where to put it (which is why I didn't submit a PR), but this works for me. Was hoping you guys would take it (or make something like it) so I wouldn't have to maintain it myself :)
It works very simply.  Make sure your GOPATH is set properly, and run the script (with no arguments). The script will put the latest release of dep in GOPATH/bin. Under the covers, the script uses 'uname'  to figure out what artifact it needs and then uses the github API to determine where the latest release is for that artifact. It then downloads it directly to GOPATH/bin. You can run such a script in a Go project's Makefile as a way for a new developer to download dep quickly and easily or have Travis run the make target to download dep so Travis can then use dep to pull down vendor deps (assuming the vendor deps aren't already committed to git).
The single script is found in this zip file: get-go-dep.zip

#1533 should resolve this.


Add branch

I am trying this test link:
https://fuseio.app.link/SdgodDm4s2
It should switch to this community:
0x324C621ECC464B5Ee82BdA93EEAaf7E787d1eCD4


Unitize relationships data passed to api

Belongs to many

Now there was two ways of updating BelongsTo relation. Example with book
POST /books
    {
        "name" : "Batman",
        "author" : 1, // id of author
    }

or
POST /books
    {
        "name" : "Batman",
        "author" :{
              id : 1
         }
    }

And to unionize approach first way should be deprecated and no longer be maintained. It is confusing to assign an integer to key which may indicate to object.

Feature was marked as deprecated for now.

As i gave this issue a thought i figured out that this limmitation we want to achieve can be completed by new laravel validation in validation form.
For example - if we want to set BelongsToMany relation our problem for now is fact that client may send request with properties which does not exists in pivot table.
{
     "name" : "Batman",
      "tags" :[{
            "id" : 1,
            "name" : "superhero"
       }]
}

Because name field does not in user_tag table an error has been thrown,
What if we could use validation from FormRequest of new version of laravel to our advantage. Ex $request->safe()
Rules should look like this
public function rules() : array
{
   return [
     'name'=> 'required',
     'tags.*.id' => 'required'
   ];
}

So  $request->safe() will not pass field tag.name to create or update method.
Therefore we can avoid pivots.


chore: disable @yarnpkg/plugin-compat

Codecov Report

Merging #274 (c38221e) into trunk (01d8bfe) will not change coverage.
 Consider uploading reports for the commit 283a89d to get more accurate results

@@           Coverage Diff           @@
##            trunk     #274   +/-   ##
=======================================
  Coverage   97.56%   97.56%
=======================================
  Files           5        5
  Lines         205      205
  Branches       64       64
=======================================
  Hits          200      200
  Misses          5        5

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.


Should LocalStorage Source have a remove?

Right now https://github.com/martyjs/marty-lib/blob/master/modules/local-storage-state-source/localStorage.js has get and set, but the localStorage API also has removeItem() method.
Should the LocalStorage Source also accomodate for this?

Yeah, it definitely should. I will try find some time to add this to the API