Missing required module 'minizip'

We wanted to incorporate Zip framework within our open-source QuantiLogger framework and struggled with Missing required module ‘minizip’ problem. We have spent couple days trying to solve the problem without any success. We have also observed existing issues related to this problem and tried all proposed solutions. Finally we ended up using ZipFoundation instead.
In order to demonstrate the problem, we created a simple example app. It is just an empty Simple View App that does not actually do anything apart from including the Zip framework. We used Carthage to import the framework. After I prepared the demonstrating app, it all built successfully even with included Zip framework. I also tried to create a new folder where I cloned the demonstrating app directly from Github and again, built successfully. The problem occured when another team member cloned the repository. After he did so, he build with Missing required module ‘minizip’ failure. Then he performed carthage update and suddenly all built successfully.
You can hopefully recreate the problem as well by cloning the demonstrating app and trying to build it.
As mentioned before, we solved the problem by using another framework (unfortunately). However we are very curious why the problem occurs since we spent much time trying to solve it ourselves.

I got the same problem trying to import the IOS-Pods-DFU-Library library via Carthage

This should be fixed with #119.

We updated the Zip library within the demonstrating app in order to test the fix and the problem did not disappear! The only difference is that instead of Missing required module ‘minizip’ we are now having Missing required module ‘Minizip’ failure.

@troupmar  Unbelievable ,but today I met this weird issue ........OMG

@AvdLee  I find all the links about this problem, but cannot find the suitable method to solve it .so I think there is a smart way to solve this ...

Reopening this as it seems to not be fixed yet. @simonseyer you've mentioned in your PR before that this should have been fixed.
Are you able to test this with @troupmar 's test app and submit a fix?

@Avdlee I had tried once more and then find the issue is not existing (If I use Cocoapods or Carthe ),work well,The issue is from I did that manually without dependency management tool .As the library I used iOS-DFU-Library depended on your library ,so pls ensure the issue .

We have encountered the same issue. Could it be related to https://github.com/Carthage/Carthage/issues/1903?

Just found a way to reproduce this issue (got it on our Jenkins, then replaced my local Carthage/Build folder to the one built on jenkins side, and got the same issue), then I make it simple (and local):
kenji@macboo:~/source-cache/project$ carthage bootstrap
kenji@macboo:~/source-cache/project$ xcodebuild -derivedDataPath ./DerivedData .....
kenji@macboo:~/source-cache/project$ echo $?
0
kenji@macboo:~/source-cache/project$ cd ..
kenji@macboo:~/source-cache$ mv project other
kenji@macboo:~/source-cache$ cd other
kenji@macboo:~/source-cache$ rm -rf DerivedData
kenji@macboo:~/source-cache/other$ xcodebuild -derivedDataPath ./DerivedData .....
CompileSwift normal x86_64 /Users/kenji/source-cache/other/Some/Some.swift (in target: Target)
[big command]
<unknown>:0: error: missing required module 'minizip'
	CompileSwift normal x86_64 /Users/kenji/source-cache/project/Some/Some.swift
	CompileSwiftSources normal x86_64 com.apple.xcode.tools.swift.compiler
(150 failures)
kenji@macboo:~/source-cache/other$ echo $?
65

Moving back other to project make xcodebuild builds fine (after removing DerivedData)
I think it is related to bcsymbolmap file containing absolute paths...

I'm also looking for solution for this issue. Unfortunately, it is related not only to this repository, but all frameworks which uses modulemap inside. Easiest way to reproduce it - to build a framework(manually or via carthage) and then remove/move sources to another folder.
Example:
carthage update
mv Carthage/Checkout Carthage/Checkout_

Why does it happen?
Seems like when a framework project contains a module with C code and as result a module map to this code, then result framework will contain links to the same C code with absolute paths. You can see that inside Zip.framework/Modules/Zip.swiftmodule/. Just open any swiftmodule file with any text editor and look for old modulemap path.
I mean, this C code/headers(maybe just headers, not sure about that) should be placed at the same place for using the framework, otherwise framework won't find this C-module (in this case - minizip)
Workaround
Only workaround what I've found is to copy minizip folder(folder with C code) inside your project (where you have to use Zip.framework) and configure Import Paths to this directory. Then, when Zip.framework have to find mini zip module, it will use the project settings (module is configured and load to the project already).
So, either do not remove Checkout directory for carthage (because framework will look for minizip module there) and as result do not use any caches for carthage (like rome), or add minizip directory to your project and configure Import Paths.
If someone knows how it could be avoided, I would gladly to use it, because I don't like either solutions. Much better when a framework contains all code/information and we do not need to do anything for working with it

@AlexIzh
Thank you, I've been experiencing same issue with iOS-DFU-Library (we were excluding Carthage/Checkouts in repo)
and I couldn't find a solution.
Adding 'minizip' folder to project and adding Import Paths worked! Thanks!

I have the same issue. But I can't add folder as I am doing this inside framework project and I need to be able to copy all require frameworks to target which will use framework I am working on. Furthermore I can not change Zip to anything else, as I am using MetaWear framework which does depends on your framework :(

This issue is still present. It's been more than an year. Since I do not like the idea of adding a subdependency for a carthage dependenc manually, I'm going to switch back to CocoaPods for now. Sad.

This issue is present also in latest version 2.0.0 using Swift Package Manager on XCode 11.4.1.
Please fix it as soon as possible, or, in alternative, make version 2.0.0 available on CocoaPods too.
Thanks

Hi all, sorry for those struggling with this. Is it still an issue with the latest release?

@marmelroy Haven't tried it in a while now. Checking it now to see if it persists.

@marmelroy Just tried with Carthage on Xcode 11.7. Now getting No such module Zip on import Zip statement. Works fine with SPM though 👍🏽

@marmelroy It has not been fixed.
Tried on XCode 12.0.1 using Zip v. 2.1.1. The error persists:

Has anyone solved this for Swift Package Manager?

Bump.  Still seeing this.  Xcode 12.4, targeting iOS 14.4.

In order to solve this it is possible to stop depending on the "custom" version of Minizip in this repository, and instead do one of the following options:

Create a proper dependency on the official Minizip link 1, link 2
Create a an extra podspec for the "custom" version of Minizip used in this repo
In any case, it means start treating Minizip as a fully fledge standalone dependency instead of using the current integration

I got the issue when trying to build my app on CI env without the Carthage/Checkouts directory.


Feature Suggestion: Auto-accept mode

New option: --auto-accept (or give it a better name)
It makes fzf immediately accept what is selected when all candidates are loaded and the query has narrowed them down to only one.
This option will work well with options like --query=^ and --exact and allows for using fzf like menu selection UI.
I can't think of a good use case with --multi, so it may be just for single-selection mode.

[x] I have read through the manual page (man fzf)
[x] I have the latest version of fzf
[x] I have searched through the existing issues

Info

OS

[x] Linux
[x] Mac OS X
[ ] Windows
[ ] Etc.

Shell

[ ] bash
[x] zsh
[ ] fish

Problem / Steps to reproduce
You are always required to press Enter when you see there's only one matching candidate.
I tried to achieve enter-less acceptance by using key bindings like --bind 'x:change-query(^x)+accept' but it did not seem to work.

Have you tried --bind one:accept?

I haven't.  And aw, I guess this is what I wanted. 😳


Added 'lanczos' to the image interpolation function list.

I fixed the scipy.misc.imresize function to add 'lanczos' interpolation. This was just a case of adding the option that PIL already supports. Unit test also added to confirm the fix.

@@            master   #6022   diff @@
======================================
  Files          238     236     -2
  Stmts        43822   43908    +86
  Branches      8215    8238    +23
  Methods          0       0
======================================
+ Hit          34253   34302    +49
- Partial       2602    2610     +8
- Missed        6967    6996    +29

Review entire Coverage Diff as of 28570e9

Powered by Codecov. Updated on successful CI builds.

Thanks @beyondmetis


Allow missing whitelist property

Currently, the following .licensee.json file is treated as an invalid configuration.
{
  "license": "(MIT OR BSD-2-Clause OR BSD-3-Clause OR Apache-2.0)"
}

I think it would be fine to allow a missing whitelist property.

Feel free to send a PR!


Fails to start with Config error - Cannot read property 'split' of undefined

I added the config exactly as per the first example, and checked the entire config.json with a JSON parser, but I get the following error (truncated to just show config ui related messages for readability):
Starting Node.js HomeKit Server...
Started Node.js HomeKit Server.
[2018-6-10 09:45:54] Loaded config.json with 0 accessories and 4 platforms.
[2018-6-10 09:45:57] Loaded plugin: homebridge-config-ui-x
[2018-6-10 09:45:57] Registering platform 'homebridge-config-ui-x.config'
[2018-6-10 09:45:57] ---
[2018-6-10 09:46:01] [Config] Initializing config platform...
[2018-6-10 09:46:01] [Config] Spawning homebridge-config-ui-x with PID 28806
Load homebridge-bravia-tv.BraviaTV
[2018-6-10 09:46:04] [Config] Failed to load /var/homebridge/config.json - Unexpected token :
[2018-6-10 09:46:05] [Config] TypeError: Cannot read property 'split' of undefined

I'm running this on a Raspberry Pi 3 with nodejs v8.11.2 and Homebridge 0.4.43

Can you post your full config.json?

{
  "bridge": {
    "name": "RaspBridge",
    "username": "CC:22:3D:E3:CE:30",
    "port": 51826,
    "pin": "191-20-203"
  },
  "description": "Casa MC Homekit Bridge",
  "platforms": [
    {
      "platform": "HarmonyHub",
      "name": "Harmony Hub"
    },
    {
      "platform": "cmdSwitch2",
      "switches": [
        {
          "name": "PS4 Pro",
          "manufacturer": "Sony Corporation",
          "on_cmd": "ps4-waker",
          "off_cmd": "ps4-waker standby",
          "state_cmd": "ps4-waker search | grep -i '200 Ok'"
        }
      ]
    },
    {
      "platform": "config",
      "name": "Config",
      "port": 8080,
      "sudo": false
    }
  ]
}

Config looks fine here. If you're still having issues with this feel free to re-open.


NO-ISSUE: Unable to copy and paste nodes in the new DMN Editor

Attempting to copy and paste a node in the editor resulted in a node being added to the DRG nodes menu, but not to the editor. The problem was that a new ID was never being generated.
This fix resolves that by using the randomize function similarly to how we use it in the BEE.
And ensures that the pasted nodes have a new ID and that the copied nodes retain their original IDs.
https://github.com/apache/incubator-kie-tools/assets/92726146/6893eae9-3cdc-42a5-a581-cae0ff884fe4

Hi Jozef, I had the same thought! :D I was still able to replicate the issue even when on Tiago's branch containing the fix, so this PR is needed 👍


TOC with fuzzy search

Is there a way to generate toc with fuzzy search .

There is no fuzzy search feature available for Toc. However it can be a good feature.


Update language-sass

autocomplete-css specs are testing scopes that changed with the bump to 0.50.0.  Will merge once I fix those specs and the build goes :green_heart:.

The inconsistency introduced with the scope boundary changes ended up not being worth it, so I've temporarily reverted them.  Once the rest of the language-sass PRs get merged I'll work on improving the autocomplete-css scope matching.


Add additional unit tests

Added additional Unit Tests to increase coverage.

Thanks @TheRealHaui
I'd like to cherry-pick 7e11b54 rather than merging the PR as is, as the PR includes the "parallel" changes to the ITs that haven't got any effect (as there is only one class per execution anyway).

You're welcome.
Don't know why the change included the pom.xml changes.
Explicitly created an own branch for that.
Anyway ...


Add namespaced example

@christophfroehlich is this PR review for review?

@ahcorde I added a similar example in #181


Minio example doesn't start, exited status

The minio stack example unfortunately does not work, and the container is in "exited" status. No ports appear, something is wrong with the port config in the stack template: https://portainer-templates.netlify.app/minio
Perhaps I am doing something wrong, examples would be appreciated.

See the recently updated app list with a new Report Issue link next to each app.

Thanks, I'll give it another try one day.


Error when running ./rails_setup_osx.sh

error when we try to run:  ./rails_setup_osx.sh
Error message:
./rails_setup_osx.sh: line 79: syntax error near unexpected token else' ./rails_setup_osx.sh: line 79:     else'

Hi @kevineo. Thanks for reporting the issue. It's confirmed as bug due to only else in the script.


Run tests with short flag by default

IMO, we want to default to -short because editor-based tools should (generally) return as rapidly as possible. Long-running tests should be reserved for CI. I'm open to changing the default to false if I hold a minority opinion, however.

This seems reasonable. Thanks for the contribution!


Update backend.properties to None for simulators

As per conversations, update the backend.properties() methods for local and remote
simulators, so they return None, adjusting the BaseBackend method
accordingly.
Addresses partly #1332.

Summary
Details and comments

A simulator does not have properties. I agree we should fix the spec

This is true, the spec does say that a backend will return a backend.props data structure for the properties .  So it is required.  However the info inside is optional.  So a quick 'will' -> 'can' should fix it.


Make it easier to see linter issues

What changed?
I changed the format of the api-linter to:

Only show the rule URL and location
Make the location a clickable link
Group violations by rule

Why?
The output previously was not very easy to parse:
Before:

After:

Breaking changes

Wish we didn't have to have platform dependent build scripting and require extra dependencies like "jq". We have a perfectly fine language for executing processes, parsing JSON, etc.

Thanks for the suggestion, @stephanos! gojq seems to virtually be a drop-in replacement for jq. Also, the install already has a good home in api-linter-install. Hopefully this addresses your concerns about portability, @cretz and your concern about installing jq @bergundy.


random you tube video audio

Script version
v4.0
Describe the bug
I click to watch a video, but in addition to the video I'm watching i also get audio from a random you tube video.
Expected behavior
To skip the ad that plays before a video without any other issues

There's some video playing in the background. I noticed the YouTube shorts aren't working too. I click and start playing, but the shorts get a black screen.


Drop unnecessary context arg from PinSubmitter class hierarchy

What
Drop unused context arg in PinSubmitter constructors.

Why
Neither AbstractVaultSubmitter nor BasisTheoryPinSubmitter made use of the context field; only VGSPinSubmitter needed access to context. But, VGSPinSubmitter already has access to context via foragePinEditText.context. Less code means less surface area for bugs!

Test Plan

❌ Did not add unit tests but updated existing tests to no longer depend on `context
❌ No need to manually test. The CI tests passing is sufficient!

Demo
N/A since this only changes implementation details

How
Can be rolled out immediately, once approved

#253
#252  👈
main

This stack of pull requests is managed by Graphite. Learn more about stacking.

Join @devinmorgan and the rest of your teammates on  Graphite

Merge activity

Jun 13, 4:49 PM EDT: @devinmorgan merged this pull request with Graphite.


ojbect with circular reference causes isPojo to error out with 'maximum call stack exceeded'

    var obj = { a: 1, b: 2, c: [3, 4] };
    obj.c.push(obj);

    console.log(isPojo(obj))

This code (which has a circular reference at property c) errors out with
RangeError: Maximum call stack size exceeded     at isAllowedJSONProperty (file://.../node_modules/@zerodep/is-pojo/esm.mjs:1:546)

Hi @chrisjameslennon,
I appreciate your time finding and reporting these issues!
v2.0.7 of @zerodep/is-pojo resolves this issue.


API Restore JS Minification

BUG Fix incorrect cache on CacheGeneratedAssetHandler
This change was lost from the original pull request at https://github.com/silverstripe/silverstripe-framework/pull/4680

cc @chillu

Merged :) Sorry again for mucking up your other PR there!


Router has Problems in Cordova-App

In Cordova-Apps the router has problems with links as absoluterelative pathes like "/homepage/...".
Please change the RE_ORIGIN-RegEx in index.js:12 from
/^.+?\/\/+[^\/]+/
to
/^.+?\/\/([^\/]+|)/
The following patterns are then valid an the router works in webapps and Cordova-Apps as well:

http://host:80/home
http://host:80/home/someMore
file:///home
file:///home/someMore
etc.

Best regards,
Marcel

Closing this issue because it's related to an old router version. Please update to the latest @riotjs/route version if you can.


Allow set metadata parameters on JettonInit phase

At the moment only following parameters could be set on jetton deploy/init:

jetton_name
jetton_description
jetton_symbol
max_supply

And if jetton master should be initialized with any other paramter like image_data or decimals it will only be possible via another request to jetton (JettonSetParameter)
I suggest to update JettonInit message with dynamic list of input fields to allow initialize jetton with any metadata fields
But how to do this? At the moment i don't have good solution for it

All allowed metadata fields for jetton

My first thought is to adopt a similar approach as the optional custom_payload used in the JettonTransfer message. In this case, we could introduce a parameter named something like metadata , which would serve as a flexible container for additional fields such as image_data, decimals and others.
The metadata parameter could map to a struct that accommodates all optional parameters.
Do you prefer a dynamic list (dictionary) of input fields for this?

Do you prefer a dynamic list (dictionary) of input fields instead of this?

I think it should be dynamic list because we can't describe all possible fields of metadata in advance. Some jettons theoretically could have custom logic for metadata fields.

The metadata parameter could map to a struct that accommodates all optional parameters.

You mean something like this?
message JettonInit{
  // non metadata fields first
  owner: Address;
  max_supply: Int;
  // metadata fields will be stored like in `contracts/teps/tep64.tact`
  // key - sha256 of string key
  metadata: map<Int, Cell>
}

I was thinking something like the below, a structural predefined optional values. Your suggestion to have metadata as a map will be more future proof to changes
message JettonInit{
  // non metadata fields first
  max_supply: Int;
  //....
  //.....
  //.....
  metadata: Cell?;
}

struct Metadata {
    decimals: Int? as uint256;
    totalSupply: Int? as uint256;
    name: Slice?;
    string: Slice?;
    attributes: Cell?;
    //....
    //....
    ///...
}


docs: add webhooksPath option for createNodeMiddleware

This PR adds the missing webhooksPath option to the docs for createNodeMiddleware
fixes: #1587

View rendered docs/development.md

should I also update the docs that use the old options?
example: https://github.com/probot/probot/blob/master/docs/deployment.md#vercel

should I also update the docs that use the old options?
example: https://github.com/probot/probot/blob/master/docs/deployment.md#vercel

yes, please! Great catch!

ok done, for real this time 😛


New package: Nexa v0.1.0

Registering package: Nexa
Repository: https://github.com/SabeDoesThings/Nexa.jl
Created by: @SabeDoesThings
Version: v0.1.0
Commit: 4d72040d6b8d7e0bc034246c70158cd6341b8272
Git reference: main
Description: A simple upcoming game library for the Julia programing language
Release notes:

the first release with some of the basic usage

I only started working on it 2 days ago.

I'm not necessarily opposed to the name Nexa, but it may be a bit premature to register it quite that early. We can make exceptions for "nice" names shorter than 5 characters, but generally, I would endorse that only for packages that are on a clear path to be relatively mature and well-supported. Meanwhile, it might be better to use a LocalRegistry for development.

oh ok. Yeah I wanted to try to make a simple game with it but didn't want the entire package source code in the project. I will definitely look into making a LocalRegistry. Does that make it to where I can just add a package with the github url instead of registering a whole package?

Does that make it to where I can just add a package with the github url

That you can do without any registration at all, see "Adding unregistered packages" in the Pkg documentation.
What a LocalRegistry would do is to allow you to Pkg.add it as a registered package. The General registry is just the default one, but you can have others. A lot of development workflows get easier with registered packages, but it doesn't matter if they are registered in General or any other registry.
The point of General is to make the package more visible and easier to install for the entire Julia community, and, most importantly, packages that are dependencies for other packages in General must also be registered in General. But until there is some other package that should be in General that has Nexa as a dependency, a LocalRegistry is sufficient in principle.


Feat: support Quarto docs

Is your feature request related to a problem? Please describe.
I could be oblivious and this is already supported by I feel like this plug-in takes care of all my markdown rendering needs besides quarto markdown files (.qmd).
Describe the solution you'd like
I would love if this plug-in could render these file types!
Describe alternatives you've considered
Using a secondary markdown rendering app like headlines has worked for .qmd files in the past but I'd rather not have double dependencies.
Additional information
No response

I believe this is the same as: https://github.com/MeanderingProgrammer/render-markdown.nvim/issues/76.
The following seemed to work in that case:
require('render-markdown').setup({
    file_types = { 'markdown', 'quatro' },
})

ahhh its looking like this is an issue with lazy loading! I can get it to trigger if i explicitly add cmd = {"RenderMarkdown"} in my config for the plugin. I am investigating further to figure out why this won't auto enable when opening a quarto document now...will report back here so others may figure this out if they struggle like myself....PS i should have searched the issues better...I don't know why I got lazy this time.

Adding event = "LazyFile" to my lazy config for this plugin and keeping the rest of the regular config items related to cmd and file_types solved this issue for me!

Adding event = "LazyFile" to my lazy config for this plugin and keeping the rest of the regular config items related to cmd and file_types solved this issue for me!

 Glad it's working :)


Issue with /docs/user-guide/pods/single-container/

Issue with /docs/user-guide/pods/single-container/
Hi Team,
I was following following link http://kubernetes.io/docs/user-guide/pods/single-container/
to create pod .

looks like an unfinished submission


File writer tests

Adds a test for the file/writer node. The test tries to read and replicate a file on disk. as soon as the DONE port from the file/writer sends '29' ( the number of bytes on the file) it will be closed as passed.

Updated pull request, closing this one.


Markdown: GFM-style code-block (fenced) support?

This one works:
    var code = true;

But this one, does not:
    ```javascript
    var code = true;
    ```

Reason: https://github.com/isagalaev/highlight.js/blob/master/src/languages/markdown.js#L52-L61
This only captures the four spaces at the beginning. If I wanted to extend upon that, how would I capture the beginning and end properly? It would span over multiple lines. So, what would be the right regex for that?

I think something along these lines should work:
{
  className: 'code',
  begin: /^```\w*\s*$/, end: /^```\s*$/
}

Would you be up to making a pull request? Here's the doc on how to build and run tests for the library: http://highlightjs.readthedocs.org/en/latest/building-testing.html


Build proxy support

The build script wasn't working for me because the http proxy information wasn't being propagated to the docker build environment (and the Dockerfile was running apk). This pull request will pass the proxy if set as a build-arg

Closing this.
It's possible to expose ENV instead of passing flags: https://docs.docker.com/engine/admin/systemd/#httphttps-proxy


[Feature request]: Leaving Work Mantra

Version

[X] I'm using version 1.7.0

Existing requests

[X] I've checked Existing requests

What operating system are you using?
Windows
Operating System Version
Windows 10 19043.1081
Problem description
Cal Newport's Deep Work suggested creating a mantra for giving people the time to leave work and relax.
Possible solution
Cues of Closing Work

Check To-Do List
Schedule Urgent Work
Skim Tomorrows Task
Plan for Tomorrow

Things to do as leisure: ...
Alternative solutions
https://github.com/skidding/jobs-done
Additional information
No response
Code of Conduct

[X] I agree to follow this project's Code of Conduct

Hi, I don't plan to add this to Stretchly, as it is out of scope, but I also have another app where you can create custom reminders: https://lateron.app

@hovancik a reminder app that takes over the whole screen would be more useful than just pinging in the corner.

There's issue about this: https://github.com/hovancik/later-on/issues/2, but not sure when and such


Shippable multi-repo CI testing

Shippable has added support for multi-repo testing, i.e. for downstream repos (repos that depend on your repo) to be tested when upstream repo changes are made. This allows cross-repository integration tests to be run. We should try it for uproxy-* and freedom-* repos.

We no longer use Shippable.


Use shared lib for some samples

context: https://code.google.com/p/android/issues/detail?id=222124

audio-echo is using the shared_lib for now for the current workaround implementation. closing this for now


Remove text about ss3sim for the path #66

Remove the text about ss3sim in the path because the ss3sim will reference this information here soon.

This is a very simple change, and I did a "formal" pull request just to work on "workflow" of (1) issue, (2) commit, (3) pull request, (4) review, and (5) closure.


Add bower package configuration and .editorconfig, apply formatting

Reveal.js itself is a Bower package. It makes it easier to install and update the plugins in an presentation.
external.js contained mixed indentation (tabs and spaces). The .editorconfig helps to avoid that - it is supported by a lot of editors/IDEs.

Hi Thomas,
thanks for the PR. I published the package to bower right a few minutes ago. Now it is installable via "bower install reveal_external".
Thanks and have a nice day,
Jan


There is a double CloseHandle in runner-win.c

Version: 1.13.1
Platform: Windows x64 build 1607; Visual Studio 2017

process_copy_output in runner-win.c creates a POSIX-style file descriptor from the process's stdout Win32 HANDLE using open_osfhandle, then closes the file descriptor, which closes the HANDLE. There's then a double close in process_cleanup. I imagine this normally isn't noticeable, but if you're running in the VS2017 debugger then it causes the process to stop in the debugger with a warning popup.
I don't know what the best solution for this is, but what I've done in my copy is make process_copy_output first duplicate the handle using DuplicateHandle. (See the code below. I also added a couple of fclose calls that looked to be missing.)
The tests then run to completion under the debugger without throwing up any obvious errors.
Thanks,
--Tom
int process_copy_output(process_info_t* p, FILE* stream) {
  char buf[1024];
  int fd, r;
  FILE* f;
  HANDLE dup;

  if (!DuplicateHandle(GetCurrentProcess(), p->stdio_out, GetCurrentProcess(), &dup, 0, FALSE, DUPLICATE_SAME_ACCESS))
    return -1;

  fd = _open_osfhandle((intptr_t)dup, _O_RDONLY | _O_TEXT);
  if (fd == -1) {
    CloseHandle(dup);
    return -1;
  }
  f = _fdopen(fd, "rt");
  if (f == NULL) {
    _close(fd);
    return -1;
  }

  r = fseek(f, 0, SEEK_SET);
  if (r < 0) {
    fclose(f);
    return -1;
  }

  while (fgets(buf, sizeof(buf), f) != NULL)
    print_lines(buf, strlen(buf), stream);

  if (ferror(f)) {
    fclose(f);
    return -1;
  }

  fclose(f);
  return 0;
}

Hi Tom! Mind sending it as a PR?

Yes, though I can't promise a timely one. But it's on my to do list, so it will get done eventually (apologies - I'm not working with a setup that makes PRs quick to create).
Thanks,
--Tom

@tom-seddon did you mean to close this?


restic fatal error: runtime: out of memory

This was from trying to run a first time backup from my laptop via sftp to a backup repository which is 5 TiB in size (from backing up a server & workstation).
Output of restic version
restic 0.1.0 (v0.1.0-548-g795e3d5)
compiled at 2016-05-24 22:26:48 with go1.5.4
Actual behavior
fatal error: runtime: out of memory
runtime stack:
runtime.throw(0xa37550, 0x16)
/usr/lib/golang/src/runtime/panic.go:527 +0x90
runtime.sysMap(0xc98a740000, 0x5a000000, 0x118f00, 0xc94d58)
/usr/lib/golang/src/runtime/mem_linux.go:203 +0x9b
runtime.mHeap_SysAlloc(0xc75b00, 0x5a000000, 0xc820001c80)
/usr/lib/golang/src/runtime/malloc.go:426 +0x160
runtime.mHeap_Grow(0xc75b00, 0x2d000, 0x0)
/usr/lib/golang/src/runtime/mheap.go:628 +0x63
runtime.mHeap_AllocSpanLocked(0xc75b00, 0x2d000, 0x100)
/usr/lib/golang/src/runtime/mheap.go:532 +0x5f1
runtime.mHeap_Alloc_m(0xc75b00, 0x2d000, 0x100000000, 0xc800000001)
/usr/lib/golang/src/runtime/mheap.go:425 +0x1ac
runtime.mHeap_Alloc.func1()
/usr/lib/golang/src/runtime/mheap.go:484 +0x41
runtime.systemstack(0x7fc0f294fe28)
/usr/lib/golang/src/runtime/asm_amd64.s:278 +0xab
runtime.mHeap_Alloc(0xc75b00, 0x2d000, 0x10100000000, 0x4357af)
/usr/lib/golang/src/runtime/mheap.go:485 +0x63
runtime.largeAlloc(0x5a000000, 0x7fc000000001, 0xc80000001d)
/usr/lib/golang/src/runtime/malloc.go:748 +0xb3
runtime.mallocgc.func3()
/usr/lib/golang/src/runtime/malloc.go:637 +0x33
runtime.systemstack(0xc820020a00)
/usr/lib/golang/src/runtime/asm_amd64.s:262 +0x79
runtime.mstart()
/usr/lib/golang/src/runtime/proc1.go:668
goroutine 82 [running]:
runtime.systemstack_switch()
/usr/lib/golang/src/runtime/asm_amd64.s:216 fp=0xc824197608 sp=0xc824197600
runtime.mallocgc(0x5a000000, 0x94a5c0, 0x7fc000000001, 0x429622)
/usr/lib/golang/src/runtime/malloc.go:638 +0x9c4 fp=0xc8241976d8 sp=0xc824197608
runtime.newarray(0x94a5c0, 0x200000, 0x4bd4693f)
/usr/lib/golang/src/runtime/malloc.go:780 +0xc9 fp=0xc824197718 sp=0xc8241976d8
runtime.hashGrow(0x84f740, 0xc822d9a000)
/usr/lib/golang/src/runtime/hashmap.go:761 +0x82 fp=0xc824197748 sp=0xc824197718
runtime.mapassign1(0x84f740, 0xc822d9a000, 0xc824197900, 0xc8241978b8)
/usr/lib/golang/src/runtime/hashmap.go:481 +0x55b fp=0xc8241977f0 sp=0xc824197748
restic/repository.DecodeIndex(0x7fc0f2951530, 0xc822d9a030, 0xc8200fe070, 0x0, 0x0)
/tmp/restic-build-781614583/src/restic/repository/index.go:519 +0x5ba fp=0xc824197a68 sp=0xc8241977f0
restic/repository.LoadIndexWithDecoder(0xc8200942a0, 0xe579cf34c883a906, 0x375522f06072b6de, 0x7a12e8d1a4c630ce, 0x14e7bc67aaec6f38, 0xac7120, 0x40, 0x0, 0x0)
/tmp/restic-build-781614583/src/restic/repository/index.go:568 +0x2d2 fp=0xc824197bb0 sp=0xc824197a68
restic/repository.LoadIndex(0xc8200942a0, 0xe579cf34c883a906, 0x375522f06072b6de, 0x7a12e8d1a4c630ce, 0x14e7bc67aaec6f38, 0x375522f06072b6de, 0x0, 0x0)
/tmp/restic-build-781614583/src/restic/repository/repository.go:380 +0x60 fp=0xc824197c78 sp=0xc824197bb0
restic/repository.(*Repository).LoadIndex.func1(0xe579cf34c883a906, 0x375522f06072b6de, 0x7a12e8d1a4c630ce, 0x14e7bc67aaec6f38, 0xc820094c00, 0x0, 0x0)
/tmp/restic-build-781614583/src/restic/repository/repository.go:348 +0x59 fp=0xc824197d78 sp=0xc824197c78
restic/repository.ParallelWorkFuncParseID.func1(0xc820099080, 0x40, 0xc820094c00, 0x0, 0x0)
/tmp/restic-build-781614583/src/restic/repository/parallel.go:89 +0x269 fp=0xc824197e60 sp=0xc824197d78
restic/repository.FilesInParallel.func1(0xc8200f4770, 0xc820094c60, 0xc8200f4760, 0xc820094c00, 0xc820094cc0)
/tmp/restic-build-781614583/src/restic/repository/parallel.go:53 +0x114 fp=0xc824197f68 sp=0xc824197e60
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1721 +0x1 fp=0xc824197f70 sp=0xc824197f68
created by restic/repository.FilesInParallel
/tmp/restic-build-781614583/src/restic/repository/parallel.go:63 +0x185
goroutine 1 [chan receive, 4 minutes]:
restic/repository.(_Repository).LoadIndex(0xc8200942a0, 0x0, 0x0)
/tmp/restic-build-781614583/src/restic/repository/repository.go:367 +0x17a
main.CmdBackup.Execute(0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffe97eed4e9, 0x1d, 0x0, 0xa943e2, ...)
/tmp/restic-build-781614583/src/cmds/restic/cmd_backup.go:302 +0x59c
main.(_CmdBackup).Execute(0xc82001c240, 0xc820013dd0, 0x1, 0x3, 0x0, 0x0)
:14 +0xe0
github.com/jessevdk/go-flags.(_Parser).ParseArgs(0xc820078870, 0xc82000a250, 0x3, 0x3, 0x0, 0x0, 0x0, 0x0, 0x0)
/tmp/restic-build-781614583/src/github.com/jessevdk/go-flags/parser.go:278 +0x8e3
github.com/jessevdk/go-flags.(_Parser).Parse(0xc820078870, 0x0, 0x0, 0x0, 0x0, 0x0)
/tmp/restic-build-781614583/src/github.com/jessevdk/go-flags/parser.go:154 +0x9b
main.main()
/tmp/restic-build-781614583/src/cmds/restic/main.go:26 +0x1d1
goroutine 17 [syscall, 4 minutes, locked to thread]:
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1721 +0x1
goroutine 5 [syscall, 4 minutes]:
os/signal.loop()
/usr/lib/golang/src/os/signal/signal_unix.go:22 +0x18
created by os/signal.init.1
/usr/lib/golang/src/os/signal/signal_unix.go:28 +0x37
goroutine 6 [chan receive, 4 minutes]:
restic.init.1.func1.1()
/tmp/restic-build-781614583/src/restic/lock.go:262 +0x16f
created by restic.init.1.func1
/tmp/restic-build-781614583/src/restic/lock.go:265 +0x2b
goroutine 18 [select, 4 minutes, locked to thread]:
runtime.gopark(0xac7450, 0xc820026728, 0x9d4da8, 0x6, 0x44ed18, 0x2)
/usr/lib/golang/src/runtime/proc.go:185 +0x163
runtime.selectgoImpl(0xc820026728, 0x0, 0x18)
/usr/lib/golang/src/runtime/select.go:392 +0xa64
runtime.selectgo(0xc820026728)
/usr/lib/golang/src/runtime/select.go:212 +0x12
runtime.ensureSigM.func1()
/usr/lib/golang/src/runtime/signal1_unix.go:227 +0x353
runtime.goexit()
/usr/lib/golang/src/runtime/asm_amd64.s:1721 +0x1
goroutine 7 [chan receive, 4 minutes]:
main.CleanupHandler(0xc82001c1e0)
/tmp/restic-build-781614583/src/cmds/restic/cleanup.go:58 +0x7b
created by main.init.1
/tmp/restic-build-781614583/src/cmds/restic/cleanup.go:25 +0x12a
goroutine 8 [syscall, 1 minutes]:
syscall.Syscall(0x0, 0x5, 0xc823bbbca0, 0x4, 0x18, 0x18, 0x0)
/usr/lib/golang/src/syscall/asm_linux_amd64.s:18 +0x5
syscall.read(0x5, 0xc823bbbca0, 0x4, 0x4, 0x18, 0x0, 0x0)
/usr/lib/golang/src/syscall/zsyscall_linux_amd64.go:783 +0x5f
syscall.Read(0x5, 0xc823bbbca0, 0x4, 0x4, 0x18, 0x0, 0x0)
/usr/lib/golang/src/syscall/syscall_unix.go:160 +0x4d
os.(_File).read(0xc82002e240, 0xc823bbbca0, 0x4, 0x4, 0x18, 0x0, 0x0)
/usr/lib/golang/src/os/file_unix.go:211 +0x53
os.(_File).Read(0xc82002e240, 0xc823bbbca0, 0x4, 0x4, 0x18, 0x0, 0x0)
/usr/lib/golang/src/os/file.go:95 +0x8a
io.ReadAtLeast(0x7fc0f4999568, 0xc82002e240, 0xc823bbbca0, 0x4, 0x4, 0x4, 0x0, 0x0, 0x0)
/usr/lib/golang/src/io/io.go:298 +0xe6
io.ReadFull(0x7fc0f4999568, 0xc82002e240, 0xc823bbbca0, 0x4, 0x4, 0x18, 0x0, 0x0)
/usr/lib/golang/src/io/io.go:316 +0x62
github.com/pkg/sftp.recvPacket(0x7fc0f4999568, 0xc82002e240, 0xc82003af70, 0x0, 0x0, 0x0, 0x0, 0x0)
/tmp/restic-build-781614583/src/github.com/pkg/sftp/packet.go:143 +0xde
github.com/pkg/sftp.(*Client).recv(0xc820078960)
/tmp/restic-build-781614583/src/github.com/pkg/sftp/client.go:160 +0x7f
created by github.com/pkg/sftp.NewClientPipe
/tmp/restic-build-781614583/src/github.com/pkg/sftp/client.go:77 +0x2d6
goroutine 11 [select, 4 minutes]:
main.refreshLocks(0xc6d760, 0xc820094ae0)
/tmp/restic-build-781614583/src/cmds/restic/lock.go:69 +0x434
created by main.lockRepository
/tmp/restic-build-781614583/src/cmds/restic/lock.go:46 +0x196
goroutine 12 [semacquire, 4 minutes]:
sync.runtime_Semacquire(0xc8200f477c)
/usr/lib/golang/src/runtime/sema.go:43 +0x26
sync.(_WaitGroup).Wait(0xc8200f4770)
/usr/lib/golang/src/sync/waitgroup.go:126 +0xb4
restic/repository.FilesInParallel(0x7fc0f4999f20, 0xc82012c000, 0x9d37b0, 0x5, 0x14, 0xc8200f4760, 0x0, 0x0)
/tmp/restic-build-781614583/src/restic/repository/parallel.go:66 +0x1ad
restic/repository.(_Repository).LoadIndex.func2(0xc820094ba0, 0xc820094b40, 0xc8200942a0, 0xc820142760)
/tmp/restic-build-781614583/src/restic/repository/repository.go:364 +0xc4
created by restic/repository.(*Repository).LoadIndex
/tmp/restic-build-781614583/src/restic/repository/repository.go:365 +0x143
Steps to reproduce the behavior
Simply try to run the first time backup.
I believe the problem is that the memory requirements for reading the index files is too large. I recently did a rebuild index on another machine. It was super memory intensive and required the machine to have 8GiB + swap to complete the job.
On the backup drive:
du -sh index
939M    index

Thanks for giving restic a try. We'll need to investigate and get the memory usage down.
I think this is a bug.

I think this is the same issue I've been experiencing for a while. Is it?
 Backing up 1TB in linux-64bit, the process takes over 2GB of RAM.
I don't know why it takes so much memory... it is mainly one index (blob id) entry per every file / 1MB-block, for what I understand. It shouldn't be that much.

Is there any update on this? I really like restic, but this problem makes it more or less unusable for me.
I try to backup the www-data from within a number of small Vultr vm's. In one case restic tries to backup 950 MB in 28000 files and panics at ~10%. The peak memory consumption is 450 MB.
I can understand that its problematic to backup TB of data with a small amount of memory, but 950 MB should be possible.

No update yet, sorry. I'm currently working on to restructure the code (see PR #600), afterwards I'll rework buffer and memory management for restic. Then we'll (hopefully) see an improvement.
One (minor) change was introduced recently: #577 introduced auto-configuring parameters for scrypt(). Before  that change, at least 200 MiB of RAM were (temporarily) required to compute the low-level AES and MAC keys from the user's password. That has been lowered to about 60 MiB.

I didn't know scrypt required so much ram for anything. I will read again how it works.

Yes, recently I've implemented several measures to bring down memory usage a bit, this is contained in 0.4.0. Since then, there are several more improvements.
You can do two things:

Use a recent version of Go, at least 1.7 (1.8 is even better)
Use the latest master

Please report back if one of these (or even better: both combined) work for you.

Testing again with:
restic 0.4.0 (v0.4.0-58-g0674f32)
compiled with go1.8 on linux/amd64
Will let know the outcome when it it's done.

Same result:
...
Feb 20 07:05:58 web kernel: [145119.949097] [16184]     0 16184  1043575   921409    2037    91804             0 restic
...
Feb 20 07:05:58 web kernel: [145119.949116] Out of memory: Kill process 16184 (restic) score 861 or sacrifice child
Feb 20 07:05:58 web kernel: [145119.955614] Killed process 16184 (restic) total-vm:4174300kB, anon-rss:3685636kB, file-rss:0kB

I scaled up the VM with more memory to speedup the initial backup (switching from duplicity). The original VM memory has 512Mb (simple static web server but with a lot of files and 512Mb is more then enough). This will be a no go i guess for now.
Duplicity did not gave issues with the memory but it has a local (on disk) state monitor so it does not keep everything in memory.
Is that what makes it different and restic needs more memory with more files?

Which operation did get restic killed? backup or prune?
Would you mind running it again with debug built in (run go run build.go -tags debug), then add a parameter --mem-profile, e.g.: restic --repo s3:[...] --mem-profile restic.pprof backup [...], then upload the binary you used and the memory profile (in restic.pprof) somewhere and send me the link. Maybe this reveals what's causing the excess memory usage.
At the moment, restic does not have any local state (which is different from duplicity). It may indeed one factor.

@johansmitsnl I think it's important to distinguish between Restic running out of memory + crashing, and the scenario where your Linux kernel is lying about the amount of available memory, and then failing to deliver it when the application demands it.
Greedy applications, like deduplication caches as implemented in Restic, or ZFS ARC, do not fare well when you lie to them.
As a first step, perhaps you could turn off the OOM killer so we can get the debug output?
Here are the commands to temporarily disable it, try running these, and then the restic job:
sudo sysctl vm.overcommit_memory=2   # always check, never overcommit
sudo sysctl vm.oom-kill=0            # dont use the oomkiller

@fd0 running in debug mode now. It will take some time to run and fail.
I'll gather the full output when it fails + the restic.pprof file.
@cfcs the task is already running, will check it after this run.

./restic -p pass backup / --exclude-file=restic-exclude --mem-profile restic.pprof
debug enabled
scan [/]
scanned 54469 directories, 1582081 files in 7:30
Killed17] 99.24%  12.329 MiB/s  407.627 GiB / 410.747 GiB  1286512 / 1636550 items  0 errors  ETA 4:19

Feb 21 16:34:47 web kernel: [265648.004726] restic invoked oom-killer: gfp_mask=0x280da, order=0, oom_score_adj=0
Feb 21 16:34:47 web kernel: [265648.004730] restic cpuset=/ mems_allowed=0
Feb 21 16:34:47 web kernel: [265648.004733] CPU: 1 PID: 7246 Comm: restic Not tainted 3.13.0-108-generic #155-Ubuntu
Feb 21 16:34:47 web kernel: [265648.004734] Hardware name: Xen HVM domU, BIOS 4.2.amazon 11/11/2016
Feb 21 16:34:47 web kernel: [265648.004736]<PHONE_NUMBER>000000 ffff8800b58dda70 ffffffff8172d229 ffff88010132e000
Feb 21 16:34:47 web kernel: [265648.004739]<PHONE_NUMBER>000000 ffff8800b58ddaf8 ffffffff817277c8<PHONE_NUMBER>000000
Feb 21 16:34:47 web kernel: [265648.004741]<PHONE_NUMBER>000000<PHONE_NUMBER>000000<PHONE_NUMBER>000000<PHONE_NUMBER>000000
Feb 21 16:34:47 web kernel: [265648.004743] Call Trace:
Feb 21 16:34:47 web kernel: [265648.004751]  [<ffffffff8172d229>] dump_stack+0x64/0x82
Feb 21 16:34:47 web kernel: [265648.004754]  [<ffffffff817277c8>] dump_header+0x7f/0x1f1
Feb 21 16:34:47 web kernel: [265648.004758]  [<ffffffff81156d81>] oom_kill_process+0x201/0x360
Feb 21 16:34:47 web kernel: [265648.004763]  [<ffffffff812dde55>] ? security_capable_noaudit+0x15/0x20
Feb 21 16:34:47 web kernel: [265648.004765]  [<ffffffff81157511>] out_of_memory+0x471/0x4b0
Feb 21 16:34:47 web kernel: [265648.004768]  [<ffffffff8115d82c>] __alloc_pages_nodemask+0xa6c/0xb90
Feb 21 16:34:47 web kernel: [265648.004773]  [<ffffffff8119e25a>] alloc_pages_vma+0x9a/0x140
Feb 21 16:34:47 web kernel: [265648.004777]  [<ffffffff8117ea41>] handle_mm_fault+0xbb1/0xf10
Feb 21 16:34:47 web kernel: [265648.004779]  [<ffffffff81183b0a>] ? vma_merge+0x10a/0x340
Feb 21 16:34:47 web kernel: [265648.004782]  [<ffffffff817393c4>] __do_page_fault+0x184/0x560
Feb 21 16:34:47 web kernel: [265648.004784]  [<ffffffff81185a3e>] ? do_mmap_pgoff+0x34e/0x3d0
Feb 21 16:34:47 web kernel: [265648.004787]  [<ffffffff81170319>] ? vm_mmap_pgoff+0x99/0xc0
Feb 21 16:34:47 web kernel: [265648.004789]  [<ffffffff817397ba>] do_page_fault+0x1a/0x70
Feb 21 16:34:47 web kernel: [265648.004793]  [<ffffffff81735ae8>] page_fault+0x28/0x30
Feb 21 16:34:47 web kernel: [265648.004794] Mem-Info:
Feb 21 16:34:47 web kernel: [265648.004796] Node 0 DMA per-cpu:
Feb 21 16:34:47 web kernel: [265648.004797] CPU    0: hi:    0, btch:   1 usd:   0
Feb 21 16:34:47 web kernel: [265648.004799] CPU    1: hi:    0, btch:   1 usd:   0
Feb 21 16:34:47 web kernel: [265648.004799] Node 0 DMA32 per-cpu:
Feb 21 16:34:47 web kernel: [265648.004801] CPU    0: hi:  186, btch:  31 usd:   0
Feb 21 16:34:47 web kernel: [265648.004802] CPU    1: hi:  186, btch:  31 usd:  30
Feb 21 16:34:47 web kernel: [265648.004803] Node 0 Normal per-cpu:
Feb 21 16:34:47 web kernel: [265648.004804] CPU    0: hi:   90, btch:  15 usd:   0
Feb 21 16:34:47 web kernel: [265648.004805] CPU    1: hi:   90, btch:  15 usd:   5
Feb 21 16:34:47 web kernel: [265648.004808] active_anon:794385 inactive_anon:173877 isolated_anon:32
Feb 21 16:34:47 web kernel: [265648.004808]  active_file:112 inactive_file:100 isolated_file:0
Feb 21 16:34:47 web kernel: [265648.004808]  unevictable:0 dirty:0 writeback:20 unstable:0
Feb 21 16:34:47 web kernel: [265648.004808]  free:20981 slab_reclaimable:4936 slab_unreclaimable:4038
Feb 21 16:34:47 web kernel: [265648.004808]  mapped:7357 shmem:8079 pagetables:4676 bounce:0
Feb 21 16:34:47 web kernel: [265648.004808]  free_cma:0
Feb 21 16:34:47 web kernel: [265648.004810] Node 0 DMA free:15904kB min:264kB low:328kB high:396kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15988kB managed:15904kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
Feb 21 16:34:47 web kernel: [265648.004814] lowmem_reserve[]: 0 3744 3933 3933
Feb 21 16:34:47 web kernel: [265648.004817] Node 0 DMA32 free:64776kB min:64072kB low:80088kB high:96108kB active_anon:3102648kB inactive_anon:620464kB active_file:428kB inactive_file:400kB unevictable:0kB isolated(anon):128kB isolated(file):0kB present:3915776kB managed:3836704kB mlocked:0kB dirty:0kB writeback:80kB mapped:28976kB shmem:31000kB slab_reclaimable:13948kB slab_unreclaimable:10940kB kernel_stack:800kB pagetables:16512kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:1296 all_unreclaimable? yes
Feb 21 16:34:47 web kernel: [265648.004820] lowmem_reserve[]: 0 0 189 189
Feb 21 16:34:47 web kernel: [265648.004822] Node 0 Normal free:3244kB min:3244kB low:4052kB high:4864kB active_anon:74892kB inactive_anon:75044kB active_file:20kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:262144kB managed:194224kB mlocked:0kB dirty:0kB writeback:0kB mapped:452kB shmem:1316kB slab_reclaimable:5796kB slab_unreclaimable:5212kB kernel_stack:672kB pagetables:2192kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:30 all_unreclaimable? yes
Feb 21 16:34:47 web kernel: [265648.004826] lowmem_reserve[]: 0 0 0 0
Feb 21 16:34:47 web kernel: [265648.004828] Node 0 DMA: 0*4kB 0*8kB 0*16kB 1*32kB (U) 2*64kB (U) 1*128kB (U) 1*256kB (U) 0*512kB 1*1024kB (U) 1*2048kB (R) 3*4096kB (M) = 15904kB
Feb 21 16:34:47 web kernel: [265648.004836] Node 0 DMA32: 816*4kB (UE) 764*8kB (UE) 483*16kB (UEM) 296*32kB (UE) 162*64kB (UEM) 78*128kB (UEM) 44*256kB (EM) 5*512kB (E) 2*1024kB (E) 1*2048kB (R) 0*4096kB = 64848kB
Feb 21 16:34:47 web kernel: [265648.004845] Node 0 Normal: 292*4kB (UEM) 1*8kB (M) 0*16kB 1*32kB (M) 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB (R) 0*4096kB = 3256kB
Feb 21 16:34:47 web kernel: [265648.004853] Node 0 hugepages_total=0 hugepages_free=0 hugepages_surp=0 hugepages_size=2048kB
Feb 21 16:34:47 web kernel: [265648.004854] 22997 total pagecache pages
Feb 21 16:34:47 web kernel: [265648.004856] 14711 pages in swap cache
Feb 21 16:34:47 web kernel: [265648.004857] Swap cache stats: add 2001075, delete 1986364, find 2329853/2480173
Feb 21 16:34:47 web kernel: [265648.004858] Free swap  = 0kB
Feb 21 16:34:47 web kernel: [265648.004859] Total swap = 524284kB
Feb 21 16:34:47 web kernel: [265648.004860] 1048477 pages RAM
Feb 21 16:34:47 web kernel: [265648.004860] 0 pages HighMem/MovableOnly
Feb 21 16:34:47 web kernel: [265648.004861] 16980 pages reserved
Feb 21 16:34:47 web kernel: [265648.004862] [ pid ]   uid  tgid total_vm      rss nr_ptes swapents oom_score_adj name
Feb 21 16:34:47 web kernel: [265648.004866] [  500]     0   500     4868        0      13       81             0 upstart-udev-br
Feb 21 16:34:47 web kernel: [265648.004868] [  506]     0   506    12468        0      26      147         -1000 systemd-udevd
Feb 21 16:34:47 web kernel: [265648.004870] [  660]     0   660     3814        0      12       58             0 upstart-socket-
Feb 21 16:34:47 web kernel: [265648.004871] [  661]     0   661     5854       15      17       51             0 rpcbind
Feb 21 16:34:47 web kernel: [265648.004873] [  700]     0   700     2645       67       7      604             0 dhclient
Feb 21 16:34:47 web kernel: [265648.004875] [  875]     0   875     2554       53       8      522             0 dhclient
Feb 21 16:34:48 web kernel: [265648.004876] [ 1109]   102  1109     9807        0      23       96             0 dbus-daemon
Feb 21 16:34:48 web kernel: [265648.004878] [ 1170]     0  1170    10861        1      26       89             0 systemd-logind
Feb 21 16:34:48 web kernel: [265648.004880] [ 1189]   101  1189    65049      169      33     1568             0 rsyslogd
Feb 21 16:34:48 web kernel: [265648.004881] [ 1216]     0  1216     3818        0      11       74             0 upstart-file-br
Feb 21 16:34:48 web kernel: [265648.004883] [ 1327]     0  1327     3634        2      12       39             0 getty
Feb 21 16:34:48 web kernel: [265648.004884] [ 1330]     0  1330     3634        2      12       38             0 getty
Feb 21 16:34:48 web kernel: [265648.004885] [ 1334]     0  1334     3634        1      12       38             0 getty
Feb 21 16:34:48 web kernel: [265648.004887] [ 1335]     0  1335     3634        1      12       41             0 getty
Feb 21 16:34:48 web kernel: [265648.004888] [ 1336]     0  1336    57460     1511      28      891             0 amazon-ssm-agen
Feb 21 16:34:48 web kernel: [265648.004890] [ 1338]     0  1338     3634        2      11       38             0 getty
Feb 21 16:34:48 web kernel: [265648.004891] [ 1370]     0  1370    15344       27      35      142         -1000 sshd
Feb 21 16:34:48 web kernel: [265648.004893] [ 1377]     0  1377     4783        0      13       49             0 atd
Feb 21 16:34:48 web kernel: [265648.004894] [ 1378]     0  1378     5912       22      17       43             0 cron
Feb 21 16:34:48 web kernel: [265648.004896] [ 1386]     0  1386     4797       31      13       31             0 irqbalance
Feb 21 16:34:48 web kernel: [265648.004898] [ 1414]     0  1414     1091        1       7       35             0 acpid
Feb 21 16:34:48 web kernel: [265648.004899] [ 1542]   113  1542    14271       27      29      320             0 freshclam
Feb 21 16:34:48 web kernel: [265648.004901] [ 1562]     0  1562     3188        1      10       49             0 incrond
Feb 21 16:34:48 web kernel: [265648.004902] [ 1579]   114  1579    81347        0      25      122             0 memcached
Feb 21 16:34:48 web kernel: [265648.004904] [ 1595]   112  1595     5833       12      15      113             0 nrpe
Feb 21 16:34:48 web kernel: [265648.004905] [ 1618]     0  1618    23862        0      18       80             0 pcscd
Feb 21 16:34:48 web kernel: [265648.004907] [ 1716]     0  1716     6334       26      16       53             0 master
Feb 21 16:34:48 web kernel: [265648.004909] [ 1725]   106  1725     6891       40      17       49             0 qmgr
Feb 21 16:34:48 web kernel: [265648.004910] [ 1754]     0  1754    31075     6393      63     2259             0 puppet
Feb 21 16:34:48 web kernel: [265648.004912] [ 2020]   110  2020     7863       38      19      113             0 ntpd
Feb 21 16:34:48 web kernel: [265648.004914] [ 2136]     0  2136   134596       35     161     2815             0 apache2
Feb 21 16:34:48 web kernel: [265648.004915] [ 2170]     0  2170     3634        2      12       39             0 getty
Feb 21 16:34:48 web kernel: [265648.004916] [ 2171]     0  2171     3195        1      11       37             0 getty
Feb 21 16:34:48 web kernel: [265648.004918] [ 7198]     0  7198     6487       22      16      157             0 screen
Feb 21 16:34:48 web kernel: [265648.004919] [ 7199]     0  7199     5009        2      14      186             0 bash
Feb 21 16:34:48 web kernel: [265648.004921] [ 7213]     0  7213  1020617   911496    1997   105850             0 restic
Feb 21 16:34:48 web kernel: [265648.004923] [27099]    33 27099   139643     6104     195     6269             0 apache2
Feb 21 16:34:48 web kernel: [265648.004924] [27131]    33 27131   139702     8404     196     4063             0 apache2
Feb 21 16:34:48 web kernel: [265648.004926] [27586]    33 27586   137364     7094     184     1764             0 apache2
Feb 21 16:34:48 web kernel: [265648.004927] [27587]    33 27587   137056     5022     180     2359             0 apache2
Feb 21 16:34:48 web kernel: [265648.004929] [27592]    33 27592   139684    11058     194     1806             0 apache2
Feb 21 16:34:48 web kernel: [265648.004930] [27595]    33 27595   137445     8227     183     1737             0 apache2
Feb 21 16:34:48 web kernel: [265648.004932] [27596]    33 27596   137256     6150     181     1950             0 apache2
Feb 21 16:34:48 web kernel: [265648.004933] [27597]    33 27597   137902     7513     186     1987             0 apache2
Feb 21 16:34:48 web kernel: [265648.004935] [27598]    33 27598   137342     7713     184     1777             0 apache2
Feb 21 16:34:48 web kernel: [265648.004936] [27599]    33 27599   137384     7125     184     1789             0 apache2
Feb 21 16:34:48 web kernel: [265648.004938] [28710]   106 28710     6850       75      17        0             0 pickup
Feb 21 16:34:48 web kernel: [265648.004939] Out of memory: Kill process 7213 (restic) score 865 or sacrifice child
Feb 21 16:34:48 web kernel: [265648.011392] Killed process 7213 (restic) total-vm:4082468kB, anon-rss:3645648kB, file-rss:336kB

The restic file + profile dir https://file.leftclick.eu/upload/20170222065253_restic-debug.tar
I noticed the profile dir has one empty file. This is not correct right? Did I do something wrong?

Hm, did you create the dir restic.pprof? The parameter --mem-profile expects a file name as the argument, not a directory name. And it shouldn't be empty...

No I did not create the directory. Removed the directory and touched the file so that it will have a file.
Had this output:
./restic -p pass --exclude-file=exclude --mem-profile restic.pprof backup /
debug enabled
2017/02/23 07:15:27 profile: could not create initial output directory: mkdir restic.pprof: not a directory

In the ENV there are these variable set:

AWS_ACCESS_KEY_ID='key id'
AWS_SECRET_ACCESS_KEY='secrey'
RESTIC_REPOSITORY='s3:s3.amazonaws.com/bucket/restic'

If you use my restic compiled app can you reproduce this?
The backup runs for quite some time and I can see a lot of traffic to S3. It seems to trip just before the end of the backup run.
Please let me know how to assist in debugging this.

I'll need to reproduce this locally, that'll probably take some time, then I can try to find the cause for this. I'll let you know if you can help in any way, thanks!

Repost because you failed to do it (Restic is not crashing; your badly configured Linux system is killing it. If you would reconfigure it to not kill restic, perhaps we could get on with the investigation of why Restic is using so much memory):
@johansmitsnl I think it's important to distinguish between Restic running out of memory + crashing, and the scenario where your Linux kernel is lying about the amount of available memory, and then failing to deliver it when the application demands it.
Greedy applications, like deduplication caches as implemented in Restic, or ZFS ARC, do not fare well when you lie to them.
As a first step, perhaps you could turn off the OOM killer so we can get the debug output?
Here are the commands to temporarily disable it, try running these, and then the restic job:
sudo sysctl vm.overcommit_memory=2   # always check, never overcommit
sudo sysctl vm.oom-kill=0            # dont use the oomkiller

Not able to use Restic simply because of the out of memory issue. Backing up a 950mb wp-content folder, and just about 10% into backup, out of memory kills it.
 Have you tried giving it the GOGC environment variable? E.g. GOGC=20 restic ... backup .... If not, please do and see if it makes any difference.

@rawtaz My complete command is restic backup foldername/ Is that right?
I tried the GOGC=20 restic backup foldername/ and still error similar to @egalanos .
I'm using DigitalOcean Spaces and following this tutorial: https://www.digitalocean.com/community/tutorials/how-to-back-up-data-to-an-object-storage-service-with-the-restic-backup-client#backing-up-a-directory

@seanmavley Too bad, the GCGO=20 helped a lot in one of my memory-constrained systems. I could try GOGC=10 just for kicks, might be lucky and it works. How much memory do you have available for restic to use anyway? Yeah, the backup command is fine.

@rawtaz Not much memory I have available. It is a 1Gig RAM box with some 300Mb to spare. Using GOGC=1 still even throws the errors.
What stops restic from using the available possible memory, even if that'll mean longer time to backup?
I just wanna backup files which even if it takes an hour to do, I don't care. They're not critical need to back them instant. I don't fully understand how restic works, but it getting in conflict with the kernel over memory struggles isn't cool.
Heck, I wouldn't even allow a mere backup process to demand more than 100Mb of my Server RAM resources, when my Nginx serving live traffic as a proxy to some 7 applications is even using less.

@seanmavley 300 MB isn't much, I wouldn't count on that working out well for anything but very small repos. But I'm not the one to give you exact information on this, so take it with a grain of salt. Yeah, there's probably room for improvement on the memory handling.

I'll chime in and add a bit of background information. Restic is still a young project, so there are several rough edges that we're aware of. Memory usage is one of them.
Restic mainly uses too much memory in two areas:

Index handling: At the start, restic reads all the index files from a repository to learn what data is already stored there (and where). This is then kept in memory. Since these index files can be quite large, and restic processes them in parallel, it'll use much memory in the beginning for loading and decrypting the index files. Later, only the containing information is stored for further use, but even that's sometimes too big (depending on the amount of data stored in the repo).
Encrypting/decrypting blobs during backup: Restic at the moment uses way too much concurrency during backup for small boxes. I'm currently reworking the archiver code allowing this to be tweaked, and I also plan to add a check for how much concurrency makes sense for the current machine.

So, maybe this helps to understand a bit better what our challenges (as a project) are at the moment. Restic isn't suited for machines without much memory (yet).

As a workaround I've configured machine with more RAM to rsync whole filesystem of Raspberry Pi and backup it using restic backup --hostname my-raspberry. Not perfect but works.

Are there any news on this? I deployed restic 0.7.4 to about 130 systems all backing up to one repo. What looked very promising at the beginning now looks like being unusable to me: After several days of doing backups the repo grew to about 1TB of repo data. Now restic is eating so much memory on backup that systems start failing because the linux OOM-killer starts killing random processes on various systems (VMs) with less then 8GB of RAM. This seems like a real showstopper to me.
So are there any plans or work being done to address this?
Maybe it would be good to NOT let restic load the whole index at once? I'd love to see some config option to set a hard memory limit. I am aware that this might slow down backup a lot but anything is better than an exploding system because the backup job at night uses all of its memory ...
I noticed that rebuilding the index decreased memory usage a lot. But it was still too much. Also playing with GOGC did not really help much.
Thanks for this great piece of software. I am really looking forward to see some progress here.

No news directly, but I'm making good progress with reimplementing the archiver. That's the next step.


show specific err msg on password form

Details

Proposal: https://github.com/Expensify/App/issues/4856#issuecomment-906656039
Approval : https://github.com/Expensify/App/issues/4856#issuecomment-908557025
As discussed, I've added conditions for 3 specific error messages on password form.
Fixed Issues

$  https://github.com/Expensify/App/issues/4856
QA Steps

Launch the app
Enter email or phone number & proceed
Click on signin without entering password

Tested On

[x] Web
[x] Mobile Web
[ ] Desktop
[ ] iOS
[x] Android

Screenshots

Web

Mobile Web

Desktop

iOS

Android

Making the required changes!

:hand: This PR was not deployed to staging yet because QA is ongoing. It will be automatically deployed to staging after the next production release.

🚀 Deployed to staging by @puneetlath in version: 1.0.92-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
failure ❌

🕸 web 🕸
success ✅

This has been deployed to production and is now subject to a 7-day regression period.
If no regressions arise, payment will be issued on 2021-09-10.  :confetti_ball:

🚀 Deployed to production by @roryabraham in version: 1.0.93-1 🚀

platform
result

🤖 android 🤖
success ✅

🖥 desktop 🖥
success ✅

🍎 iOS 🍎
success ✅

🕸 web 🕸
success ✅


Add beaconChainExplorerUriTemplate property

Summary
Due to restrictions of staking, we link to the Beacon Chain explorer to see the value of specific validators. At the moment, this is done via a client-side hardcoded value.
This adds a new beaconChainExplorerUriTemplate property, similar to blockExplorerUriTemplate. The reason for not adding the value directly is for extensibility as it is also possible to search for other values other than public keys, e.g. block number/hash, graffiti or state hash.
Changes

Add nullable beacon_chain_explorer_uri_public_key_template property.
Add relevant serializer and map property
Add mock value for tests.
Generate migration and add relevant test.

Pull Request Test Coverage Report for Build<PHONE_NUMBER>9
Details

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+0.005%) to 99.501%

Totals

Change from base Build<PHONE_NUMBER>9:
0.005%

Covered Lines:
1197

Relevant Lines:
1203

💛  - Coveralls


fixed blunder i made in plant.py

had changed default values of the car which should not have happened.

Thanks! This is why we started with the tests first, so we can make mistakes before jumping on the car :)


Can't compile Netatalk

Info

Which version of Pi are you using:
Pi3b
Which github revision of software:
v22.02.01
Which board version:
FULL
Which computer is the RaSCSI connected to:
N/A

Describe the issue
I'm having problems in compiling Netatalk, It finished with the error
Making all in initscripts
make[3]: Entering directory '/home/pi/Netatalk-2.x/distrib/initscripts'
make[3]: *** No rule to make target 'a2boot.service', needed by 'all-am'.  Stop.
make[3]: Leaving directory '/home/pi/Netatalk-2.x/distrib/initscripts'
make[2]: *** [Makefile:426: all-recursive] Error 1
make[2]: Leaving directory '/home/pi/Netatalk-2.x/distrib'
make[1]: *** [Makefile:479: all-recursive] Error 1
make[1]: Leaving directory '/home/pi/Netatalk-2.x'
make: *** [Makefile:409: all] Error 2

I also tried not to enable a2boot, and just configure it with
./configure --enable-systemd --enable-zeroconf
But that didn't fix the problem. Please, suggest what did I miss.

It has been addressed here:
https://github.com/rdmark/Netatalk-2.x/issues/6
Documentation needs to be updated though. New service names don't include netatalk prefix. So atalkd, papd and etc...

@Deftaudio Good catch re: the wiki documentation. I've updated it.


[BOSS IDEA] aura damage boss

No idea what to call this but I'll give it a shot. Basically, the boss has an auto attack that does 0 damage but has 100% chance to do a small aoe bash (so that melee characters behind the boss are not hit). It has the capability to permabash anyone within its area of effect.
The boss's damage comes in the form of an aura that deals heavy damage to a portion of the arena (big enough that no hero can our range the aura).
The general idea of the boss is that it requires some amount of teamwork. The boss must be stunned periodically so you can switch out the person who is essentially permastunned in from of him before he dies to the aura. At this point, they can leave the aura to heal up (if you wanted to keep it quick you could give te boss a second percentage based regen aura that affected the entire boss zone but was canceled by the damage aura.
The team basically has to cycle to prevent the boss from rampaging and stunning the whole team while also killing it quick enough that they lose to attrition.  As the boss gets to lower levels of health you could add other effects like silence, blind, break etc. to the aura, though this would make it far more technical of a fight since you would eventually lose your ability to stun the boss at all.
Its counter-play is built around organization and movement without having too many difficult to understand mechanics. If I failed to explain something and you're curious feel free to ask.

Interesting idea but baumi is against bosses that are gambling and chance to permbash whole team is kind of it.

The aoe is small (~250) and only prevents more than one unit being directly in front of the boss to split aggro. The bash would never affect the rest of the team unless they got too close.

Permastun is uninteresting, closed.


Cannot find serialize/deserializer ... Multiple possible serializers found for type [Map ...]: []

Expected Behavior
I believe this is again related to #761. Before v2.8.2, it was working fine. I have two custom serializers marked as @Secondary for specific use cases of Map<String, Any?> that I use with @Serdeable.Serializable(using = MyCustomSerde::class).
Actual Behaviour
With v2.8.2+, for any field of type Map not flagged with @Serdeable.Serializable(using = MyCustomSerde::class), I get the message
Cannot find serialize/deserializer for type: class ....Organization. Error resolving serializer for property [BeanProperty{beanType=class ....Organization, type=interface java.util.Map, name='name'}] of type [java.util.Map]: Multiple possible serializers found for type [Map name]: []

It seems as though the default beans serdes cannot be found anymore.
Steps To Reproduce
No response
Environment Information

JDK 17

Example Application
No response
Version
4.3.8

@dstepanov Thanks, that seems to do the trick.


Video Viewed from only the Device on which Relay Server Runs.

Hi,
It is very good application for peer to peer video call. I am facing some issues as explained below.
I am running Relay Server on <IP_ADDRESS>, Also running one client application on the same device with username Test1.
I am running client application on 2 more devices with usernames Test2 and Test3. In these devices application setting host IP is provided as <IP_ADDRESS>. So I am able to see Online peers.
Case1: When I call from Test1 to Test2, video is shown on Test2, but there is no video and audio incoming on Test1.
Case2: Call between Test2 and Test3, no video seen on both devices.
So my observation is Video is transferred from only device with username Test1, where the Relay server is running.
Please let me know what is the issue?
Is it possible to have communication between end devices without having Relay server?

Thanks for the comment and raising this issue.
Strange, I have tested this app on my work environment with my colleagues with Lan \ WIFI in similar setup.
I have few questions:

Do you have device permissions to access camera and video?
Do you use holepunch feature?
-What's the resolutions of your cameras? (is it very high resolution?)

Can you also try this and get back to me, it helps me identify the issue

Change the transport layer to TCP and see if the call works
Send a small file between peers to see if it transfers.

I will investigate it, btw there will be huge patch on the network side soon.

Thanks for the response.

Do you have device permissions to access camera and video?
Yes
Do you use holepunch feature?
I have enabled checkbox Auto HolePunch from Settings.
What's the resolutions of your cameras? (is it very high resolution?)
I am using Getac T800 Rugged Tablet it has 8MP Camera resolution. I have also tried to connect external camera Logitech HD 720p. could not succeed, as there is no option to switch the camera.

As you suggested :

I have tried to establish call by changing the transport layer to TCP from the settings. But still video is not shown from Getac T800.
Tested File transfer, it is working fine.

Also can you clarify which codec is used for video call? Because in settings it shows webp and jpg.
For low bandwidth can we use VP8 or H.264 in this app?

So problem seems to be not on network but on acquisition of devices.
I am assuming you are using a pre-built version from one of the releases (i.e. v0.2.0-alpha) so we dont have version incompatibility.
I'm using Open CV wrapper for getting the device camera, and NAudio for the audio. Both takes the default device they can find. You pointed that there is no audio and video, so both of them is failing.
To verify this, you can check it by calling from a device which the cam feed doesn't work,  you will see yourself on right top corner Inside the application. If it doesn't show anything or shows a black rectangle, means camera is not acquired. For audio similarly there is an option in setting "listen yourself" works when you are on call.
My suspicion is that what you are using is a windows tablet, I never used or tested on windows tablets, only on home Pcs and some thinkpad laptops. Maybe they have more security to determine which app can access to device.  I assume the one that works is not a tablet? Have you tired to run as admin?
About H.264, I intend to implement it, on C# there is no direct implementation or readily available library. I probably need to do a wrapper around Cisco's H.264 encoder(C++).  However there is no documentation so it will take a bit of time.
So current codec is  is stream of Jpg images or WebP(smaller size) images. You can call it MJpeg I guess. They take about 200-400 Kb/s on my device. You can also reduce or bump the quality or fps manually with a slider.
Bare in mind this app is a toy app i did on few weekends, mainly to test my network library. So unfortunately, it not feature rich yet.

Forgot to ask you, have you tried share screen? Does it work?

Hi,
Share screen is working fine, also I have checked self view of video is shown in the Getac Tablet. Randomly sometimes video also transmitted successfully, when I had run app as admin. But it won't work on everytime call initiated from either side. Success rate is very less. Not sure why it behaves so?

Thanks for your tests and responses.
I think I have an idea on your last issue where images are transmitted "sometimes".
After image is acquired from camera, its compressed to either Jpg or WebP. If the compressed image is more than 64Kb, it will be dropped because it cant fit on single UDP datagram.
Normally when you enable "Congestion Control" on video settings it should automatically drop the quality. Or you can disable it and manually drop the quality.
Another reason could be that, the WebP is computationally intensive image compression. Maybe the tablet CPU is not powerful enough to handle it on real-time. So you can try to switch on JPG. Share screen also uses WebP compression. So maybe that's not it on your case.
In any case I will patch the points you raised on future release soon.
Cheers

OK thank you so much for the responses. Lets hope you will be able to the patch all the points.

If you have time, just to confirm the theory, can you try it by switching to JPG and reducing the quality manually (disable the congestion control, lower the slider), to see if it transmits on lower qualities? you can also try the transport layer at TCP to see if it makes any difference.
Thanks

Ok,
I implemented H264 encoding and Jumbo Udp frame support for large udp messages.  There is now also camera selection and test options on settings section for multi-camera setups. You can check the new version on latest release.
This should resolve the issue.
Note that if your tablet is based on ARM processor, application is not supported yet.
Closing this issue, if new one arises, don't hesitate to create one.


Adds service catalog details view to the selfservice ui service catalog list

Updates favicon.ico with the appropriate manageiq file.

@AllenBW Can you add some screenshots to the initial PR comment?  Thx, Dan

Go home @miq-bot...you're drunk

@AllenBW Please set your git config for manageiq to pull.rebase true.  The reason the bot is giving you false positives is because you did a merge instead of a rebase when you updated your branch.


Remove deprecated Layer.findAll

Description
This removes the deprecated findAll method from the Layer class that has reached the EOL .
Instead, users should call Layer.findAllAnnotations(...).annotations.
Part of deprecations that are slated for removal for next release in https://flutter.dev/go/deprecation-lifetime
More context: https://medium.com/flutter/deprecation-lifetime-in-flutter-e4d76ee738ad
Related Issues
Deprecated in https://github.com/flutter/flutter/pull/42953
Checklist
Before you create this PR, confirm that it meets all requirements listed below by checking the relevant checkboxes ([x]). This will ensure a smooth and quick review process.

[x] I read the Contributor Guide and followed the process outlined there for submitting PRs.
[x] I signed the CLA.
[x] I read and followed the Flutter Style Guide, including Features we expect every widget to implement.
[x] I read the Tree Hygiene wiki page, which explains my responsibilities.
[x] I updated/added relevant documentation (doc comments with ///).
[x] All existing and new tests are passing.
[x] The analyzer (flutter analyze --flutter-repo) does not report any problems on my PR.
[x] I am willing to follow-up on review comments in a timely manner.

Breaking Change
Did any tests fail when you ran them? Please read Handling breaking changes.

[x] No, no existing tests failed, so this is not a breaking change.

This is waiting on a fuchsia roll

This is waiting on a fuchsia roll


vimPlugins: updates

Motivation for this change
packer.nvim is getting increasingly popular in neovim, especially because it can deal with plugin dependency via luarocks.
Things done

[ ] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[ ] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

Was it autogenerated?

Was it autogenerated?

yes it was, I've just added sqlite as a buildInput of sql.nvim

yes it was, I've just added sqlite as a buildInput of sql.nvim

@SuperSandro2000 I dont get your comment, this is all generated as recommended by:
pkgs/misc/vim-plugins/update.py  --add tami5/sql.nvim --add nvim-telescope/telescope-frecency.nvim --add wbthomason/packer.nvim
Checking out changes commit by commit is quite straightforward.

Next time just run ./pkgs/misc/vim-plugins/update.py before adding the plugins.

Next time just run ./pkgs/misc/vim-plugins/update.py before adding the plugins.

but there is an "update commit" before all of them, that's what the script does...

but there is an "update commit" before all of them, that's what the script does...


WmAgentScripts/assignProdTaskChain.py

Assignment via script for taskchain failed with the following
Only change to checked out script is:
jen_a@vocms049:~/WmAgentScripts_testbed $ diff assignProdTaskChain.py assignProdTaskChain.py~
166,167c166
< #    url = 'cmsweb.cern.ch'
<     url = 'cmsweb-testbed.cern.ch'

url = 'cmsweb.cern.ch'

jen_a@vocms049:~/WmAgentScripts_testbed $ python assignProdTaskChain.py -w sryu_TaskChain_Task_Multicore_reqmgr2_test_160223_223034_4279
Traceback (most recent call last):
File "assignProdTaskChain.py", line 317, in
main()
File "assignProdTaskChain.py", line 270, in main
datasets = schema["OutputDatasets"]
KeyError: 'OutputDatasets'

@jenimal, Jen, could you run this test again with updated patch.
https://github.com/CMSCompOps/WmAgentScripts/pull/129
I am getting some DBS permission error. I just want to check whether you are getting a same error.
/SMS-T2-4bd_mStop-250_mLSP-170to240_TuneCUETP8M1_13TeV-madgraphMLM-pythia8/FAKE-MCRUN2_71_V1-v1/LHE
Traceback (most recent call last):
  File "rejectAndClone.py", line 82, in <module>
    main()
  File "rejectAndClone.py", line 75, in main
    dbs3.setDatasetStatus(ds, 'INVALID', files=True)
  File "/afs/cern.ch/user/s/sryu/WmAgentScripts/dbs3Client.py", line 162, in setDatasetStatus
    dbsapi.updateDatasetType(dataset=dataset, dataset_access_type=newStatus)
  File "/data/srv/wmagent/v1.0.8.patch1/sw.pre/slc6_amd64_gcc481/cms/dbs3-client/3.2.11d-comp3/lib/python2.6/site-packages/dbs/apis/dbsClient.py", line 1358, in updateDatasetType
    return self.__callServer("datasets", params=kwargs, callmethod='PUT')
  File "/data/srv/wmagent/v1.0.8.patch1/sw.pre/slc6_amd64_gcc481/cms/dbs3-client/3.2.11d-comp3/lib/python2.6/site-packages/dbs/apis/dbsClient.py", line 193, in __callServer
    self.__parseForException(http_error)
  File "/data/srv/wmagent/v1.0.8.patch1/sw.pre/slc6_amd64_gcc481/cms/dbs3-client/3.2.11d-comp3/lib/python2.6/site-packages/dbs/apis/dbsClient.py", line 221, in __parseForException
    raise HTTPError(http_error.url, data['exception'], data['message'], http_error.header, http_error.body)
RestClient.ErrorHandling.RestClientExceptions.HTTPError: HTTP Error 403: You are not allowed to access this resource

@jenimal @vlimant should we close this issue now?


feat: confirm overwrite when importing

SUMMARY

This PR changes the import API to require an explicit flag in order to overwrite existing assets.
BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

N/A
TEST PLAN

Added unit tests, both for the commands and the API.
ADDITIONAL INFORMATION

[ ] Has associated issue:
[ ] Changes UI
[ ] Requires DB Migration.
[ ] Confirm DB Migration upgrade and downgrade tested.
[ ] Introduces new feature or API
[ ] Removes existing feature or API

Codecov Report

Merging #11982 (c7b9a02) into consolidate_import_logic (2297f9b) will decrease coverage by 4.44%.
The diff coverage is 97.56%.

@@                     Coverage Diff                      @@
##           consolidate_import_logic   #11982      +/-   ##
============================================================
- Coverage                     67.67%   63.22%   -4.45%
============================================================
  Files                           939      466     -473
  Lines                         45408    28481   -16927
  Branches                       4361        0    -4361
============================================================
- Hits                          30728    18008   -12720
+ Misses                        14577    10473    -4104
+ Partials                        103        0     -103

Flag
Coverage Δ

cypress
?

javascript
?

python
63.22% <97.56%> (-0.94%)
:arrow_down:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

superset/commands/importers/v1/__init__.py
95.94% <95.23%> (+0.63%)
:arrow_up:

superset/charts/api.py
80.60% <100.00%> (+0.06%)
:arrow_up:

superset/charts/commands/importers/v1/__init__.py
100.00% <100.00%> (ø)

superset/dashboards/api.py
86.52% <100.00%> (+0.05%)
:arrow_up:

...erset/dashboards/commands/importers/v1/__init__.py
100.00% <100.00%> (ø)

superset/databases/api.py
83.87% <100.00%> (-5.20%)
:arrow_down:

...perset/databases/commands/importers/v1/__init__.py
100.00% <100.00%> (ø)

superset/datasets/api.py
92.16% <100.00%> (+0.03%)
:arrow_up:

...uperset/datasets/commands/importers/v1/__init__.py
100.00% <100.00%> (ø)

superset/db_engines/hive.py
0.00% <0.00%> (-85.72%)
:arrow_down:

... and 503 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2297f9b...4ac8cdd. Read the comment docs.


High-level API suggestion

I've used something similar to this code in isolation and it works well and isolates end users from any of the nuts and bolts that they otherwise need to know.
I know this is a research project but had you thought about adding testing?

What I thought is that the users may call demucs.separate.main(["args"]) if they want to make it simple, but use the api module if they want to use more complicated methods. I created this module because I made an effort to figure out how to separate an audio without saving it to the disk, and I wanted to make it easier for other users to achieve things like this.

Besides, please run the workflows:
Go to https://github.com/boxabirds/demucs/actions/workflows/linter.yml and click Run Workflow

As a user of the module I don’t care about any of those details I just want
the stems so that’s the wrapper I’ve used and thought I’d share it because
likely there are others who take a similar high level view — demucs is one
of 5 or 6 AI tools in my application’s workflow.
I like taking the approach of user-centred design: listen to users of my
product and observe behaviour.
(I’m not sure about workflows I don’t use them.)
On Sun, 7 May 2023 at 18:31, Weiqi Gao @.***> wrote:

Besides, please run the workflows:
Go to https://github.com/boxabirds/demucs/actions/workflows/linter.yml
and click Run Workflow
—
Reply to this email directly, view it on GitHub
https://github.com/CarlGao4/demucs/pull/1#issuecomment-1537500026, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AABD62JBWNUGQOK4CLGEAI3XE7L6PANCNFSM6AAAAAAXYGVDHA
.
You are receiving this because you authored the thread.Message ID:
@.***>

I see, but doesn't demucs.separate.main(["track.mp3"]) fulfill your requirement? Or what other functions do you want to use?


set the policy to allow manifest v2 extensions by default

Description
set the policy ExtensionManifestV2Availability to a value of 2 to allow v2 extensions
Who's implementing?

[ ] I'm willing to implement this feature myself

The problem
google is getting rid of v2 extensions and this will block ublock origin and other important privacy software.  we need that back
Possible solutions
set the value in your builds
Alternatives
No response
Additional context
No response

already doing this https://github.com/ungoogled-software/ungoogled-chromium/blob/master/patches/core/ungoogled-chromium/extensions-manifestv2.patch


uboot: missing perl dependency

Description of changes
Depending on the configuration, perl might be required. Perl is used to build the OID database, which is required for EFI secureboot (at least).
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)

[ ] sandbox = relaxed
[ ] sandbox = true

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

Add a :+1: reaction to pull requests you find important.

cc @bartsch @lopsided98 @dezgeg

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/prs-ready-for-review/3032/4750


UserManager.getUser: user not found in storage

UserManager.getUser: user not found in storage

Did you get any resolution for this?

any resolution for this?


HDDS-4720. RenameKey : add unit test to verify bucket#renameKey

What changes were proposed in this pull request?
This task is to verify the functionality of ozonebucket#renameKey client API. Basically the existing o3fs#rename(path, path) logic is internally invoking bucket#renameKey, so functionality should work fine.
What is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-4720
How was this patch tested?
Added new UTs

LGTM, +1. @rakeshadr , please double check if failed ut is related with this, thanks.

Thanks @linyiqun for the reviews. TestOzoneFileSystem is failing, which is unrelated to this PR. Failure will be analysed separately along with HDDS-4486 or other jira tasks.

LGTM, +1. @rakeshadr , please double check if failed ut is related with this, thanks.

Thanks @linyiqun for the reviews. TestOzoneFileSystem is failing, which is unrelated to this PR. Failure will be analysed separately along with HDDS-4486 or other jira tasks.


No-names tag does not work with default_icons

When no-names is set to true, the names of applications that use the default_icon are still shown

Hi, thank you for reporting. There was a decision made in a previous pull request, that it doesn't make that much sense to hide the name when default icon is enabled, how would you tell apart applications?
If this is a problem for you I'll path it, but out of curiosity whats the use-case here?

Hi. The main issue is that some applications have really long names that take almost half of the bar. When multiple applications without custom icons are open, the polybar i3 module pushes other modules out of view.
I also think that it does not make much sense to hide the name of some applications but not of others. I use the icons so that a quick glance at the corner of my screen is enough to know what I have open in each workspace. That is not possible if some application display their titles.
I set up custom icons to common applications that are always open. The default icon would be enough to find out in which workspace the other applications are.

Thanks for the explanation, makes sense. I pushed a fix under the hotfix branch, would you mind trying it out before I draft a new version?

Thank you. It seems to be working fine with no issues so far.


[Suggestion] Add the number of joined servers at the end of the process.

So I had this small suggestion, as the title says it would be very useful to add the number of servers that have been joined, it can be displayed at the end of the process at the same moment when the time taken is displayed.
Not always all the invites result valid so even if I input 100 invite codes in the .txt files, I'm not always 100% sure that all of the 100 invites have been joined or worked. Having a final count would help a lot to check effectively how the joining went.
That being said, that's a very good joined, keep the work up!

Nice idea, but how do you plan on this being logged?

Possibly have a counter running and have it increment +1 for each Success (since the success on join gets logged as far as I can see in the console), and display at the end the final count?
Honestly I'm not into GO development so I can't help much code-wise, I just came out with the idea.


Feature to update the refresh token.

Is your feature request related to a problem? Please describe.
I'm frustrated when the refresh token expires, I have to recall what to do which essentially is to delete the integration and re-set it up.
Describe the solution you'd like
An option in configuration which allows you to enter a new token.

We already have code to reauth if the refresh token expires.


Add :wide modifier

I bring an incredibly important change -- the addition of a :wide modifier for emotes, with similar behavior to widePeepoHappy and widePeepoSad on Twitch.
Here's what it looks like in a text chat:

Yeah, it looks awful, but that's the point.

funny, but if I'm not mistaken there is css settings like for example image-rendering that could make it less blurry and thus look a little better

funny, but if I'm not mistaken there is css settings like for example image-rendering that could make it less blurry and thus look a little better

I believe the problem is that the renderer grabs the 1x resolution file (28x28px) from BTTV's servers, so even regular emotes look bad when they're the only content of the message (therefore larger). The stretching makes that issue more obvious, but isn't the root of the problem.
Also, the quality of the result of changing image-rendering depends on the content of the image -- for some images it might look better, for others it might be worse. It's also a bit subjective.

Emotes will be broken out into their own plugin, so not planning to merge this


Perfect Scrollbar not working properly inside twitter bootstrap tabs

I observed that when i place the code under twitter bootstrap tabs the scrollbars where not showing on hover. However when i hit the down page arrow key they appear.

Closed for the inactivity.


Check combined node flag for deprecated

Fixes a part of #39374
Related: https://github.com/microsoft/TypeScript/pull/39323

@typescript-bot perf test.

What's up

I'm not sure? Maybe because I mentioned it in #39323?

oh. nope. it's because it's based on the branch node-based-deprecated-checks. It needs to be rebased from master if that's possible or else cherry-picked over there.

Never mind. I have another PR.


Sf container support

Support for Symfony Dependency Injection Container

Duplicate of #164

I'm closing this PR. We experienced some bugs in the code for the Aura.DI container and we can't push fixes as that code isn't in a package. Therefor we are moving the code for Aura.DI and Pimple into their own packages in #177. We would like to see the same approach for this container.
Once the package is created, tested and stable, you can create a new PR or we can help you out integrating it into the installer.


Command: toggle_comment

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/1197070-command-toggle_comment?utm_campaign=plugin&utm_content=tracker%2F282001&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F282001&utm_medium=issues&utm_source=github).

Migrated to limetext/lime-backend#16.


Click events not working on iPad

Hello @rkusa  again, I am trying your plugin with bootstrap modal window, I want to drag a modal window, Everything is fine but when modal window is opened none of anchor works properly.
Here is demo : http://jsfiddle.net/indianwebdesigner/csuscr9v/3/
~Sahib

Thanks for reporting! I'll look into it at the weekend

Yes Sure! take your time :)

What do you mean with "anchor"? There are no HTML anchors, so you probably mean something else?

mean to say like close buttton not working if i added new anchor then it also not working on iPad

The close button should work now. I still don't geht the anchor part. Could you update your example to show your anchor issue?


Improve visual feedback on element :focus

Navigation using the tab key is tough because visual feedback is either too subtle or nonexistent when an element gains :focus, e.g. the "Select image" button, or warning dialog button(s).

Pinging @taahirisaacs and @konmouz for input on visual feedback.

This is happening in https://github.com/resin-io/etcher/pull/1745

Closing as duplicate of #1734


cdn.jsdelivr.net bundled version outdated(?)

Hi,
first of all, thanks for the great library, really makes my life easier 👏🏻
I'm using the library within a web browser, and I import it as follows:
<script type="text/javascript" src="https://cdn.jsdelivr.net/npm/swissqrbill/lib/browser/bundle/index.js"></script>

const svg = new SwissQRBill.SVG(data, {language: "FR"});
document.getElementById('qr-bill').append(svg.element)

it works perfectly fine, except that I have some é and other accented characters in my strings, and that the generated QR code outputs strangely encoded chars.
I searched a bit in the issue, and found that you recently updated the QR code library: https://github.com/schoero/SwissQRBill/pull/367 to solve exactly this issue.
it seems that the bundled version I'm using, https://cdn.jsdelivr.net/npm/swissqrbill/lib/browser/bundle/index.js, is outdated, can that be?
If yes, could you update it? or advise another means of using an up-to-date version from within the browser?
Best regards,
Clément

looking into this with more attention, it seems the issue could come from something else, as I've been able to generate a properly encoded QR with the link I referenced.
if you have an idea what could cause this (html accented chars to javascript ?) it might help, but I think I'll soon be able to close my issue anyways

confirming that there is no issue with your library, it works perfectly!
I had an issue where my underlying application was sending me text represented with html entities (i.e. stuff like é).
using the following Stackoverflow answer permitted me to solve this! https://stackoverflow.com/a/9609450/5556775


port to AWS Lambda and package a deployment workflow

SNS -> Lambda looks ultra-straightforward: https://docs.aws.amazon.com/lambda/latest/dg/with-sns-create-package.html
Ideally both setup and teardown can be automated.

event is a Python dict which looks like:
{
  "Records": [
    {
      "EventVersion": "1.0",
      "EventSubscriptionArn": "arn:aws:sns:EXAMPLE",
      "EventSource": "aws:sns",
      "Sns": {
        "SignatureVersion": "1",
        "Timestamp": "1970-01-01T00:00:00.000Z",
        "Signature": "EXAMPLE",
        "SigningCertUrl": "EXAMPLE",
        "MessageId": "95df01b4-ee98-5cb9-9903-4c221d41eb5e",
        "Message": "Hello from SNS!",
        "MessageAttributes": {
          "Test": {
            "Type": "String",
            "Value": "TestString"
          },
          "TestBinary": {
            "Type": "Binary",
            "Value": "TestBinary"
          }
        },
        "Type": "Notification",
        "UnsubscribeUrl": "EXAMPLE",
        "TopicArn": "arn:aws:sns:EXAMPLE",
        "Subject": "TestInvoke"
      }
    }
  ]
}

The AWS setup things that need to happen are:

create a lambda execution role in IAM

create the lambda function

give SNS permission to call the function; looks something like:

aws lambda add-permission
--region us-west-2
--function-name helloworld
--statement-id 1
--principal sns.amazonaws.com
--action lambda:InvokeFunction
--source-arn arn:sns_topic_arn
* subscribe the lambda function to the SNS topic using [the SNS API](https://docs.aws.amazon.com/sns/latest/dg/sns-lambda.html)

Completed.


edwards: avoid inversions in Add in extended points

Description
This PR proposes solving a performance problem while doing Add in Edwards curves.
The TL;DR of the problem is that this operation does an equality checking, which involves a very slow inversion.
Type of change

[x]  Performance improvement

How has this been tested?
This PR doesn’t change the logic or add new border cases, so existing tests cover correct behavior.
How has this been benchmarked?
name                    old time/op    new time/op    delta
MixedAdd/Extended-16       221ns ± 2%     220ns ± 2%      ~     (p=0.393 n=10+10)
Add/Extended-16           3.70µs ± 1%    0.20µs ± 2%   -94.52%  (p=0.000 n=10+10)

name                    old alloc/op   new alloc/op   delta
MixedAdd/Extended-16       0.00B          0.00B           ~     (all equal)
Add/Extended-16            0.00B          0.00B           ~     (all equal)

name                    old allocs/op  new allocs/op  delta
MixedAdd/Extended-16        0.00           0.00           ~     (all equal)
Add/Extended-16             0.00           0.00           ~     (all equal)

@yelhousni, @gbotrel, rebased.


Contributing docs doesn't include method to build docs

I followed the guidelines at https://ml.dask.org/contributing.html:
$ conda env create -f ci/environment-3.7.yaml --name=dask-ml-dev
$ python -m pip install -e ".[dev]"

This worked. I made some documentation changes, so I had to build the docs. Typing make html in the docs/ directory yielded some errors, and I had to install extra packages:
$ cd docs
$ make html
$  pip install IPython
$ make html
$ pip install jupyter_client

Sorry about that. You can use the env ci/environment-docs.yaml, which should have everything necessary.