GirCore: Update to 0.5.0

Pull request type
Please check the type of change your PR introduces:

[ ] Bugfix
[ ] New feature or enhancement
[ ] UI change (please include screenshot!)
[ ] Code style update (formatting, renaming)
[ ] Refactoring (no functional changes, no api changes)
[ ] Build related changes
[ ] Documentation content changes
[ ] Internationalization and localization
[x] Other (please describe): Update of GirCore dependency to 0.5.0 without a functional change.

What is the current behavior?

Issue Number: N/A
What is the new behavior?

No functional change

Other information

Quality check
Before creating this PR:

[x] Did you follow the code style guideline as described in CONTRIBUTING.md
[x] Did you build the app and test your changes?
[ ] Did you check for accessibility? On Windows, you can use Accessibility Insights for this.
[ ] Did you verify that the change work in Release build configuration
[x] Did you verify that all unit tests pass
[x] If necessary and if possible, did you verify your changes on:

[ ] Windows
[ ] macOS
[x] Linux

The CI error is a bit irritating for me. I executed the unit tests with linux and there were no errors. The tests run fine on the linux runner, too. For me it looks like the macos build is not using GirCore at all. So I wonder why there is an exception in the macos build if GirCore was only added to the DevToys.Linux project.
The stack trace looks like:
 01:53:30 [DBG]   Failed DevToys.UnitTests.Core.AppHelperTests.CheckForUpdate_Failed_Async [18 ms]
  01:53:30 [DBG]   Error Message:
  01:53:30 [DBG]    System.ArgumentNullException : Value cannot be null. (Parameter 'factory')
  01:53:30 [DBG]   Stack Trace:
  01:53:30 [DBG]      at System.ThrowHelper.Throw(String paramName)
  01:53:30 [DBG]    at System.ThrowHelper.ThrowIfNull(Object argument, String paramName)
  01:53:30 [DBG]    at Microsoft.Extensions.Logging.LoggerFactoryExtensions.CreateLogger(ILoggerFactory factory, Type type)
  01:53:30 [DBG]    at DevToys.Api.LoggingExtensions.Log(Type forType) in /Users/runner/work/DevToys/DevToys/src/app/dev/DevToys.Api/Core/LoggingExtensions.cs:line 31
  01:53:30 [DBG]    at DevToys.Core.AppHelper.CheckForUpdateAsync(IWebClientService webClientService, IVersionService versionService, CancellationToken cancellationToken) in /Users/runner/work/DevToys/DevToys/src/app/dev/DevToys.Core/AppHelper.cs:line 138
  01:53:30 [DBG]    at DevToys.UnitTests.Core.AppHelperTests.CheckForUpdate_Failed_Async() in /Users/runner/work/DevToys/DevToys/src/app/tests/DevToys.UnitTests/Core/AppHelperTests.cs:line 182
  01:53:30 [DBG] --- End of stack trace from previous location ---

This messages seems to indicate that the LoggingExtensions.LoggerFactory property is not initialized. Checking it's references it gets initialized in the linux project here.
A corresponding call is missing for the macos version. Could this be a bug in general?
Funny thing is if I execute the failed test AppHelperTests.CheckForUpdate_Failed_Async via Rider it fails for linux, too as the factory is not set in the unit tests either. Another potential bug?
Executing all tests works with rider. This looks like the tests are not really reliable, especially as async code is used. I had problems with XUnit and async code before. But this probably requires a deeper look into the root cause of the problem. I don't think my change is related to it.
Do you have any suggestions on how to proceed?

Thank you @badcel ! I didn't have time to test it yet. I see your concern with the CI. I'm aware there's a race condition impacting only MacOS build and haven't found out why yet (I don't reproduce it locally). I re-run the CI and it looks all good now :-)

No problem take your time I'm in no hurry.

Thank for offering this change, it works like a charm! 😍

You are welcome 👍. I will slowly create more PRs and keep them  in small logical units to make it easy for you to follow along.


No theme within the terminal

Description
I've just installed (and reinstalled) hyper-pokemon and I've set up my .hyper.js as specified in the Usage section but I am not seeing the theme in the terminal. The tabs are styled properly, I've even got the poketab( and love it! ), but nothing inside the terminal.
I'm using:
Hyper 2.1.1,
hyper-pokemon 0.3.4,
Mac ElCapitan 10.11.6
I have checked the package in ~/.hyper_plugins/node_modules/hyper-pokemon everything seems to be in order there. There's nothing strange in the app console as far as I can tell.
I'm just getting started using Hyper, so I'm not sure where else I should look for the issue.

.hyper.js
module.exports = {
  config: {
    // default font size in pixels for all tabs
    fontSize: 16,

    // font family with optional fallbacks
    fontFamily: 'Menlo, "DejaVu Sans Mono", "Lucida Console", monospace',

    // terminal cursor background color and opacity (hex, rgb, hsl, hsv, hwb or cmyk)
    cursorColor: 'rgba(248,28,229,0.8)',

    // `BEAM` for |, `UNDERLINE` for _, `BLOCK` for █
    cursorShape: 'UNDERLINE',

    // color of the text
    foregroundColor: '#fff',

    // terminal background color
    backgroundColor: '#000',

    // border color (window, tabs)
    borderColor: '#333',

    // custom css to embed in the main window
    css: 'text-shadow: 0px 0px 15px rgb(255, 255, 255);',

    // custom css to embed in the terminal window
    termCSS: '',

    // custom padding (css format, i.e.: `top right bottom left`)
    padding: '12px 14px',

    // the full list. if you're going to provide the full color palette,
    // including the 6 x 6 color cubes and the grayscale map, just provide
    // an array here instead of a color map object
    colors: {
      black: '#000000',
      red: '#ff0000',
      green: '#33ff00',
      yellow: '#ffff00',
      blue: '#0066ff',
      magenta: '#cc00ff',
      cyan: '#00ffff',
      white: '#d0d0d0',
      lightBlack: '#808080',
      lightRed: '#ff0000',
      lightGreen: '#33ff00',
      lightYellow: '#ffff00',
      lightBlue: '#0066ff',
      lightMagenta: '#cc00ff',
      lightCyan: '#00ffff',
      lightWhite: '#ffffff'
    },
      pokemon: 'pikachu', // Define your favorite pokemon theme
      pokecursor: 'true', // Activate your theme's pokecursor
      pokemonSyntax: 'dark', // Define the color of the terminal tabs
      unibody: 'true', // Define the color of the Hyper window header
      poketab: 'true',

    // the shell to run when spawning a new session (i.e. /usr/local/bin/fish)
    // if left empty, your system's login shell will be used by default
    shell: '/bin/zsh',

    // for setting shell arguments (i.e. for using interactive shellArgs: ['-i'])
    // by default ['--login'] will be used
    shellArgs: ['--login'],

    // for environment variables
    env: {},

    // set to false for no bell
    bell: 'SOUND',

    // if true, selected text will automatically be copied to the clipboard
    copyOnSelect: false,

    // URL to custom bell
    // bellSoundURL: 'http://example.com/bell.mp3',

    // for advanced config flags please refer to https://hyper.is/#cfg

  },

  // a list of plugins to fetch and install from npm
  // format: [@org/]project[#version]
  // examples:
  //   `hyperpower`
  //   `@company/project`
  //   `project#1.0.1`
    plugins: ['hyper-pokemon'],

  // in development, you can create a directory under
  // `~/.hyper_plugins/local/` and include it here
  // to load it and avoid it being `npm install`ed
  localPlugins: []
};

@easilyBaffled At this moment, the theme supports only the stable channel of Hyper, which includes all releases where their first/major semantic natural number is less than 2.X.X, so downgrading to the latest stable release 1.4.8 should fix the issue : )

@klauscfhq thanks for the quick solution.
I have a few days left before I start my new job, meaning I have a few days of free time left. Do you know why it doesn't work with the latest versions of Hyper? I'd like to help.

After playing around a bit I found that, at least part of it, was my own fault. In my .hyper.js config I had css: 'text-shadow: 0px 0px 15px rgb(255, 255, 255);', which is not a valid css rule so when it got combined in
css: `
				${config.css || ''}
				.terms_terms {
				  background: url("file://${pathToTheme}") center;
				  background-size: cover;
				}

that whole block of styling was made in valid.
As for the cursor, I believe the new class name is .terminal-cursor, but I'm still playing with that.

Fixed on c2ba434.


Auto-cancel

Hello. You know how i can cancels orders when i can ask item only from BUFF APP(not the seller send me)? I tried this
cancel_data = {     "bill_order_id": i['id'],     "game": "csgo" } cancel_order = s.post("https://buff.163.com/api/market/bill_order/deliver/cancel", json=cancel_data, headers={"Referer":"https://buff.163.com/market/buy_order/history?game=csgo&state=trading","X-CSRFToken": s.cookies.get_dict()['csrf_token']})
But response is:
{'code': 'BillOrder Invalid', 'error': 'The order is abnormal, please contact support', 'extra': None}
But i know that is possible
this code also work in ussually cancel order

I am not sure what you mean. (ask item only from BUFF APP?)
Could you please take some screenshots of this action and make a description?

I wrote my bot that auto buy skins and chooses Send offer by seller, and also automatically accepts exchanges on Steam.

But the problem is that sometimes, when buying, it is impossible for the seller to send me an offer.

And if you do not create an exchange 3 times through the BUFF application, then a purchase ban arrives. There is no button on the site to cancel such an order. But my friend said that his bot can cancel such orders through a request, but my request is not core

Well, i got your point.
But currently i dont have time to reproduce this and make further investigation.
So maybe asking your "friend" is the quickly way to achieve this.
Sorry for not helping you.

How do you automate trading in the presence of such an error?

Actually, "auto trading" things are not been developed since last year, you can take a look at the history:
https://github.com/hldh214/buff2steam/commits/master/buff2steam/buff.py
And in the early 2019, buff only has one trade method, which is using bot account to be the middle between buyer and seller, seller send the skin to the bot, bot send it to the buyer.
So at that time we dont meet that situation.


All Company Tags are not coming up in list. For eg. Goldman Sachs which are present on leetcode

For eg. Goldman Sachs which are present on leetcode is not coming on VS code  and Google has over 1198 questions while in VS code only around 300 questions are present


Update Character “bass-note”

Automatically generated by Netlify CMS

👷 Deploy Preview for dynamic-cheesecake-5fbd8b processing.

Name
Link

🔨 Latest commit
4967d4fdb4ac890d4c78f6d7a0a34df915e64afb

🔍 Latest deploy log
https://app.netlify.com/sites/dynamic-cheesecake-5fbd8b/deploys/63339579510b170009d428ce


django-meta

Improve SEO by using the django-meta package
https://django-meta.readthedocs.io/en/latest/index.html

Not implementing for now.  Closing.


Firmware Dump Analysis

I have a firmware file I have read from a device running RTL8710, but it can't be analyzed with ltchiptool.
I receive the following error:
`E: UPK: The chosen device doesn't contain pin configuration.
Possible causes:

it has vendor-specific firmware
it uses TuyaMCU (report error if that's the case!)

Auto-generating ESPHome YAML is not possible.
`
Which is fine, makes sense because this device is unable to be flashed using tuya-cloudcutter.
I have flashed it with a blank config and am able to access the web server, so OTA updates via ESPHome should be possible.
How can the firmware be analyzed to find the correct configuration for this device?

Dump Analysis currently only works with Beken chips.  You can post the dump to the #dump channel in the LibreTiny discord channel, and we can see if it is something that is available or not, but there is currently not enough information to attempt to adapt the tools to do this automatically, as the sample of Realtek dumps is still quite small.

Perfect, thanks. File has been posted


Update ember-cli-build-config-editor to v5.1

I released it as a patch version upgrade, but it appears to have a problem in e-b with esprima version conflicts and I changed the exact text of how the properties are written to use identifiers over literal strings as keys where possible.

@srvance what's the status with this? Should this be updated, or rather closed?

Your choice. It will expand the configs the blueprint can handle, but I won’t be able to work on it for awhile.

This has since been replaced.

@srvance What do you mean by "has been replaced"? ember-cli-build-config-editor is still locked at 0.5.0.

I thought it had been replaced with the interactive configuration tool. We can re-open if we want to keep enhancing it.


membership of the hackspace

The membership of the hackspace rule contains amongst other provisions
"Nottingham Hackspace Ltd is required under Companies Act [1] to keep a record of all members legal name and current address."
I think this section may be confusing the term "Member" as used by the hackspace and "Member" as used by the companies act.
In the companies act a "Member of the company" is a shareholder of the company.
As far as I am aware "Members of the hackspace" are not shareholders of the limited company and hence not in the legal sense members of the company ?.
As a matter of interest who are the shareholders of the limited company ?

There are no shareholders as the company is limited by guarantee, not by
shares.
All members of the hackspace are members of the company. The trustees are
directors.
Technically, you are liable for a pound if the hackspace goes under, but in
practical terms we count this in your membership payment. As part of our
banking reform, we will start keeping that money seperate
On 24 Dec 2015 12:27 p.m., "drazziweht"<EMAIL_ADDRESS>wrote:

The membership of the hackspace rule contains amongst other provisions
"Nottingham Hackspace Ltd is required under Companies Act [1] to keep a
record of all members legal name and current address"
I think this section may be confusing the term "Member" as used by the
hackspace and "Member" as used by the companies act
In the companies act a "Member of the company" is a shareholder of the
company
As far as I am aware "Members of the hackspace" are not shareholders of
the limited company and hence not in the legal sense members of the company
?
As a matter of interest who are the shareholders of the limited company ?
—
Reply to this email directly or view it on GitHub
https://github.com/NottingHack/rules/issues/18.

James, your conclusion is therefore that the Space is under a legal obligation to keep full names and addresses?

Yes
Happy to be corrected if I'm wrong, but that is how we currently operate
On 24 Dec 2015 12:44 p.m., "rlp10"<EMAIL_ADDRESS>wrote:

James, your conclusion is therefore that the Space is under a legal
obligation to keep full names and addresses?
—
Reply to this email directly or view it on GitHub
https://github.com/NottingHack/rules/issues/18#issuecomment-167105566.

Ahh, should have realised it would be ltd by guarantee rather than by shares,
Yes, in this case I think you are correct, there does need to be a members register and it looks like the requirements are the same as shareholders of a company ltd by shares, i.e. do need name + address, although there is less requirement to disclose the list.
Given this I will close the issue.
Chris


Adding signin functionality

sign in to the endpoint that takes email and password from the request body.
authenticating the user and generating a new jwt token valid for 30 days.

Sign-in endpoint that gets email and password from request body.
Validate user credentials
Generating access token that is valid for 30 days.

Response Example
Successful authentication
Request data
{

    "email"<EMAIL_ADDRESS>    "password":"3213215646574"

}

Valid credentials response
{
    "code": 200,
    "message": "user authenticated successfully",
    "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VyX2lkIjoiNjQ0NzE3NzhiNjhjNTYyYzY5ZjVjYWUwIiwiZXhwIjoxNjg1MDIxNjc2LjQwMTU5N30.8chNSFjPOWrqxFzjP9Vl8bUuCF849ZnP_bJxwgFmQnk",
    "user_id": "64471778b68c562c69f5cae0"
}

Bad credentials
Failed authentication response
{
    "code": 400,
    "message": "Invalid credentials"
}


fix missing ineffassign warnings

Signed-off-by: André Martins<EMAIL_ADDRESS>development workflow is currently broken due these warnings

This change is

test-me-please


Many i686 packages are not build with large file support

See the comments of https://github.com/NixOS/nixpkgs/pull/10181 for previous discussion.
In short, 32-bit programs that are not built with largefile support break in spectacular ways when executed on a 64-bit system running on a filesystem containing inode numbers above 32 bits. In particular, ext4 seems to be generally safe from this issue but e.g. XFS is not.
This issue isn't easy to solve. I suggest that we ignore the problem and wait until nobody cares about i686 support, then drop i686.

I primarily consider this as issue(s) upstream, though I don't expect significant improvement from that side anymore, after all those years.  We will probably want to "support" e.g. 32-bit wine for quite a long time, but judging from the lack of activity on the previous discussion, I imagine the problem doesn't happen that often in practice.

Are there any updates to this issue, please?


Workaround for repos with recursive submodules

I am using the latest version and trying to add https://github.com/go-goracle/goracle as package however the sub-directory/module in this project (shown below) is not downloaded. I tried Glide and it works fine for this. I remember encountering this kind of issues for different packages as well.
Are there any workarounds for such cases?

Duplicate of #1633.

I just manually copied the folder as a workaround for now.
Closing this in favor of the former issue.

I haven't tried it myself, but it could be possible that not pruning unused packages makes it work?
[prone]
  unused-packages = false

Maybe this whole "distributed package management" thing is very difficult to achieve after all and there should be a package registry for Go 🤔

Maybe this whole "distributed package management" thing is very difficult to achieve after all and there should be a package registry for Go 🤔

I believe that the sweet spot is somewhere in between. Having tools to facilitate working with distributed packages while having somewhere to find high quality packages.
The namespace problem of other languages have been solved in Go by relying on URLs as a unique identifier for import paths, thus leveraging the distributed nature of DNS for infrastructure. This is great, and it is worth remembering times coding in C, when you have be careful with naming your exported declarations so they don't collide with other source files with which you are linking. In this regard, Go shines at "distributed package management".
However, finding high quality packages with stability promises and semantic versioning is another area where Go could learn from other programming language communities, and for this a "meta" packet registry would be helpful.
I wholeheartedly believe that packages should be distributed in a similar fashion as they are now, and that versioning and package management using dep will solidify over time and become pleasant. What we would need as a community is some site similar to godoc.org that lists packages, but giving more choices for finding packages such as searching by category, avg. number of users (cloned statistics from GitHub), rough stability guarantees (e.g. uses semantic versioning), etc. With such a site, the Go community could start to reach consensus for how to solve X with a given package and thus focus efforts and attention to solving the more interesting problems left to solve.
tl;dr; dep is a wonderful step in the right direction! We also need to work on other fronts to make the story of distributed package management a wonderful experience for developers and users.


New permission handling and ReportNG report builder

This is the PR corresponding to #1149. Please see the issue for a detailled explanation.

[X] Your code is flake8 compliant
[X] If this is a new feature and not a bug fix, you've included the proper documentation in the ReadTheDocs documentation folder. https://github.com/DefectDojo/Documentation/tree/master/docs or provide feature documentation in the PR. (https://github.com/DefectDojo/Documentation/pull/60)
[X] Model changes must include the necessary migrations in the dojo/dd_migrations folder.
[ ] Add applicable tests to the unit tests.

I think the idea is very nice but we should verify if this approach also works if we integrate more roles, for example @valentijnscholten suggestions #1048. For me the few rolls are one of the main drawbacks in DefectDojo.

I think the idea is very nice but we should verify if this approach also works if we integrate more roles, for example @valentijnscholten suggestions #1048. For me the few rolls are one of the main drawbacks in DefectDojo.

I don't see a reason it won't work for more roles. You can define custom predicates for permission checking (as done in reportng.models.is_report_requester and their evaluation is very cheap. Of course, we have to define what roles exist first. That's outside the scope of this implementation.

The extended Q object implementation and QProvider have now been generalized and I plan to release it as a library, unless it's accepted by upstream Django of course. Anyway, this makes the additions to Dojo even smaller and the API simpler.

Is this version able to run? When I start it with docker-compose I get some errors in the log and a server error 500 when I visit defectdojo in the browser.
For example:
initializer_1   |   Applying dojo.0010_auto_20190711_0940...Traceback (most recent call last):
...
initializer_1   |   File "/usr/local/lib/python3.7/site-packages/django/db/models/fields/related.py", line 604, in resolve_related_fields
initializer_1   |     raise ValueError('Related model %r cannot be resolved' % self.remote_field.model)
initializer_1   | ValueError: Related model 'dojo.User' cannot be resolved

and
uwsgi_1         | Internal Server Error: /
uwsgi_1         | Traceback (most recent call last):
uwsgi_1         |   File "/usr/local/lib/python3.7/site-packages/django/contrib/sessions/backends/base.py", line 189, in _get_session
uwsgi_1         |     return self._session_cache
uwsgi_1         | AttributeError: 'SessionStore' object has no attribute '_session_cache'
...
uwsgi_1         | MySQLdb._exceptions.ProgrammingError: (1146, "Table 'defectdojo.django_session' doesn't exist")

Ah, there were some changes for which migrations weren't right yet, but now it should run just fine.

This has now been rebased onto latest dev and thus also uses the newer base image.

I'm currently trying to get generic parts of the model permission registration and view mixins upstream into the rules library, which will move another ~100 lines out of Dojo.

hey, @efficiosoft great work. Here is one more point regarding permission. it would be great if you can add this or suggest to us how we can apply this:-
let say I'm a user with no staff permission (only active member). I am authorized for only one product.
--> currently matrix section implemented for same for all. it shows the same data to admin, staff and user. what I thinking is it should only show matrix for products that are allowed to me.
--> same as in metrics/product/type it should show the only information of findings which are allowed to me. Currently, it is showing all the findings.

Hi @dr3dd589
Sure, that's going to be super convenient.

I already converted Product, Engagement, Test and Finding to the new DojoModel and DojoModelManager, which gives each of these models a Model.objects.for_user(user) method that builds a base queryset with only those objects the user has read access to. The goal is to move all models, or at least all models for which views exist to this pattern.
All class-based views having DojoPermissionViewMixin mixed in automatically use that per-user restricted base queryset and hence don't even see the objects to which the user has no read access.
When it's needed to get a user-restricted base queryset somewhere else, e.g. in a model-specific method which computes metrics, that method can simply use Model.objects.for_user(user) or Model.objects.for_user.as_q(user) to get such a restricted queryset.

thank @efficiosoft  Any idea when you are ok to merge this so that I can implement this as you suggested.

I want to see what happens to https://github.com/dfunckt/django-rules/pull/99 and https://github.com/dfunckt/django-rules/pull/100 first, because that will reduce the amount of code to be added to Dojo.
Besides that, the ReportNG component still requires some testing and I can't give a timeline for it yet. Expect some weeks please.

@dr3dd589 However, you can look at the views in dojo.reportng.views to get an idea of how to implement these kind of class-based views beforehand.

Sure, @efficiosoft  I'll see this. thanks!

Just want to give an update on the process of this PR.

django-flexquery, which will do the user-based queryset filtering for us, now has 100% test coverage.
I added tests for the generic permission checking parts that'll go upstream into django-rules soon.

The good news about this is that we don't have to test those generic parts ourselves. Once 2. happened I'll be able to rebase the PR and get it ready for merging.

Hi @devGregA, @valentijnscholten, @dr3dd589, @alles-klar and others who showed interest in this PR,
All additions to third-party libraries are now merged and released and I rebased the PR for merging finally.
Best regards
Robert

And, of course, just ask if you face any issue while migrating a view.

@Maffooch Thanks for the feedback.
As you have read in the docs, ReportNG is just the framework the TeX builder was built ontop of, so I didn't enable a builder by default. However, enabling it with the default configuration is quite simple with an environment variable like this:
DD_REPORTNG_BUILDERS="['dojo.reportng.builders.tex.TeXReportBuilder']"

This is also denoted in the docs as an example.
Given that the TeX builder isn't of any use without at least one report template, and an admin has to read the docs anyway in order to write templates properly, I don't want to enable something which is going to require manual configuration by default.
Do you have concrete suggestions how to improve the current behavior?

@efficiosoft Thats the largest issue I have with this. The user has two choices: Configure something basic based on what they find in the docs, or configure something that fits their needs exactly. One way or another, configuring is going to take place. Like you said, reporting is useless without at least one template. Maybe a possible solution would be to add an example page in the docs where a user could copy/paste without anything else? Another one would be to make a bare bones config by default. If it doesn't fit the users needs, they can spend the time reading the docs, understanding the TeX framework, and playing around with thier configs to get something they like. Not all users will want to take that time to do that. I don't want to see all your hard work wasted.

@efficiosoft Thats the largest issue I have with this. The user has two choices: Configure something basic based on what they find in the docs, or configure something that fits their needs exactly.

The docs explain all available options, not just basic things.

One way or another, configuring is going to take place. Like you said, reporting is useless without at least one template.

Doesn't exactly this speak against enabling the TeX builder by default?

Maybe a possible solution would be to add an example page in the docs where a user could copy/paste without anything else?

Copy/paste what? The value to set for DD_REPORTNG_BUILDERS? This already exists.

Another one would be to make a bare bones config by default. If it doesn't fit the users needs, they can spend the time reading the docs, understanding the TeX framework, and playing around with thier configs to get something they like. Not all users will want to take that time to do that. I don't want to see all your hard work wasted.

They get a barebone config with just a single line. But that doesn't free them from learning how to write a template, which is what the whole Tex builder docs are about.

It seems I still don't understand how this is should be working then. I suppose I'm off to read the docs again and try to figure it out.

If all you want is provide a way to quickly create a report, no matter how that looks... then maybe just create a static, unconfigurable asciidoc builder for ReportNg and enable that by default? I don't see a use case for using the TeX builder without an own template.

Just something for the uninitiated to start generating reports out of the gate. I mean, after thirty minutes in the docs yesterday, I am evidently still missing the point here. It has got to be dummy proof.

It's all about how you're going to use the created reports. If they should simply serve as a kind of printable dump of the findings, then maybe some kind of default template is enough for you and can be included with the docs. However, I'm afraid I can't create one that looks good.
If however, the reports are something you want to hand out to customers you tested some product for, then you surely want (or even are forced to use) a highly-customized template, which is where the power of TeX really pays out.

Ah, and even with a sample template, you still need to either install texlive together with dojo (quite large with a useful set of packages) or compile the report locally.

I understand that. I doesn't have to be something you'd hand to your boss. Anything would work really. Just something that enables the report builder to be at least usable. I thought something installed incorrectly when there was no option to create a new report. Little things like that gave me the impression this was a WIP instead of a completed project.

Well, we can of course enable the TeX builder by default, since it's the only implementation at this time. But you'll still have to fetch the sample template from some source, let's say the docs, and copy it into place. But then there's the question: Which of the many, many fields of the models (product, engagement, test, finding) should be included in that template and how should they be displayed?

Such a thing can easily reach ~1000 lines.

So something nice like that would take ~1000 lines? A user has to take the time to learn how to do this and then write a small novel to get it looking presentable? This is exactly why It think a dinky default is necessary. Maybe the default can be a simplified version of the docs? It would show a POC of the UI working and the other integrated features. It could be the selling point for whether the user is willing to create their custom template.

If I am the only one with these concerns, then they can be put to bed. No point in implementing something that big to appease one person.

So something nice like that would take ~1000 lines?

If you want to include all the different fields of tests and findings, generate conditional output for booleans, display endpoints properly etc... remember, this is TeX ;)

Maybe the default can be a simplified version of the docs? It would show a POC of the UI working and the other integrated features. It could be the selling point for whether the user is willing to create their custom template.

I don't like duplicating content because of maintainability reasons, but a POC should be doable, although nobody will use it for something real.

If I am the only one with these concerns, then they can be put to bed. No point in implementing something that big to appease one person.

I'm sure you're not, but then there is again the question: Who will use such a sample template?
A pretty short POC template is already included in the docs, however it doesn't generate meaningful content, just demonstrates how to output the different fields.

I'll check out the POC template in the docs and get back to you. My gripes may have been in vain as I was/am too dense to realize they are already there!

No problem ;)
IIRC, that one just outputs finding titles for demonstration purposes... but it works as a starting point for something usable.
I could maybe add two more fields, such as description and mitigation to demonstrate how to convert from the markdown fields to TeX.

I'm mainly here for the permission refactoring (could it be a separate PR?), but it sounds like the reporting stuff needs some (basic) default report otherwise nobody will use it. And reporting is, as you said, an important feature for a large set of usecases.

@valentijnscholten Separating the PR would be possible, but be quite involved because ReportNG uses the permission system and there are the template snippets and filterset which sit somewhere in between.
@Maffooch and @valentijnscholten I extended the sample template in the docs to demonstrate more of the concepts, but it really is a POC and nothing for real world usage for the reasons I already described above. It really depends on what you want to use it for and I expect people to write their custom ones for different occasions and report types.
Maybe we can link to one (or more) community-provided template in the future?

@efficiosoft I have my dunce cap waiting on standby. I cannot for the life of me figure out how to get a template loaded. I made one based off the sample one you made in the docs. Its located in media/reporting/tex_templates. I also used the sample Reporting Builders config you provided as well that points to that location. What am I doing wrong here?

reporting -> reportng ;)

I'll change the template loader to try to create the directory if it doesn't exist, which will make it harder to run into these kind of slips.

And please use the latest revision, because I implemented a fix in there today.

@Maffooch TeX builder is now enabled by default and the template loader will create the configured template directories if they don't exist.

Finally got a report to build after you set some defaults! Got a .tex report generated and then converted afterwards, Could not get the pdf to generate due to a missing lualatex module. This going to be super sweet in the right hands (I don't know latex enough to make something cool). Fabulous work.

Yeah, as described in the docs, the texlive suite isn't included in the docker image, because a full texlive has ~5Gb, which I think isn't justifiable to ship by default. That's why there is the option to only get the source code as well.
Maybe we can create a repo to which community members can contribute their report templates for public use. Seems more appropriate to me than assuming a fixed default for everybody.

A report repo is an excellent idea! The default should stay as well as a tangible example of it working. It would also make it easier for users to find the correct place to put new templates if they don't read the docs, or do a poor job of doing so (as I did). Would be simple enough to just replace the default with something from the repo or a handmade template.

You still have to place the sample template into the tex_templates directory yourself. Just the empty directory now gets created for you.

I'm not going to rebase and redo the migrations for this PR again and again until it has been agreed to merge it, so please ignore the CI failures for now.

Im cool with reporting now with a default. @valentijnscholten I think the new permission system would be your forte.

Just to demonstrate how easy it is, I added another builder implementation that can generate HTML reports in ~100 lines. If one wants to, he could port the existing HTML template of the old report builder over.
Finding image handling in reports has also been reworked to support the new image captions and more fields that might be added in the future.
Give it a try if you like.

... and docs for the HTML builder are now added as well, including the necessary markdown rendering filter.

Any new objections on this? If you would like to merge it anytime soon, I'd go and resolve the merge conflicts, but I don't want to take the effort of rebasing again and again.
Best regards
Robert

The reason why I haven't looked at this in detail is because it's a massive PR with 76 files changed and seems to implement 2 things in one PR: permission changes and reporting features. Is there any chance it can be split up? I'm very interested in the permission changes, but don't have enough knowledge about the reporting part and Django UI stuff in general.

It is huge but would likely take a ton of work to split into two. I had tested the reporting and made a fuss about it a few months ago. It's all good and ready to roll. The new permissions system is something I think you'd be more familiar with and could suggest some valuable feedback.

@valentijnscholten The ReportNG component depends on the new permission handling, and since I developed all this during an internship as part of my computer science program at university I couldn't wait to get the permission stuff merged before building ReportNG.
However, both parts of the PR can be distinguished solely by file/directory. The ReportNG work completely lies inside the dojo/reportng and dojo/templates/dojo/reportng directories. Everything else is the permission handling and acompanying template snippets to make use of it.

I have looked at this, and:

A large part is indeed the reporting part. I haven't looked at that
The other part is around permissions/queries/simplification. It all looks OK to me, but to be honest I don't know enough about python/django/django-orm to easily approve this. There are also some things I don't fully understand, such as the Engagement class using the Product manager to return objects? Same for Test class using the Engagement manager, etc.
I think it might be better to cherry-pick the things we think are useful and implement them in their own PR(s).
I need at least one more person/mod to work with me on this.

Hi @valentijnscholten

I have looked at this, and:
* A large part is indeed the reporting part. I haven't looked at that

Right, @Maffooch looked at the reporting part some months ago already and deemed it ok, I made no changes since then.

* The other part is around permissions/queries/simplification. It all looks OK to me, but to be honest I don't know enough about python/django/django-orm to easily approve this.

Let me try to explain it to you in more detail. The main problems we have so far with permission handling in dojo are massive redundancy and widely distributed enforcing of role permissions all over the different views. My PR addresses these two things:

You declare once who should be able to view, change, delete or add a specific object right in its model definition. This is handled by the rules framework (which I contributed the model declaration part to).

You tell which action a particular view wants to perform on what object. Now, the rules framework checks whether the current user has permission to do so, just by looking at the predicates set inside the model definition. In case you're using django's class-based CRUD view types, all of this checking is even done transparently (also contributed this to rules).

So after all models have been annotated with permission predicates and all views were converted to class-based ones (which can be a bit-by-bit process), the views no longer have to be aware of permissions and roles at all, which finally solves the two aforementioned problems.

There are also some things I don't fully understand, such as the Engagement class using the Product manager to return objects? Same for Test class using the Engagement manager, etc.

As I said, permission declarations are baked into the model definitions directly. Now, permission checking using these declarations can reason whether a user may perform an action on a specific object. In django, however, a queryset containing all objects of a model is generated by a model manager, not the Model class itself (usually using the .all() or .filter() etc. methods). To fully isolate objects a user should not even know of, the new permission system needs a way to restrict the base queryset all operations of a user operate on. For that purpose, I extended model managers so that they can provide all views/filtersets/API viewsets etc. with such a user-specific base queryset. In case of Engagement and Test, for example, we see that the base queryset of engagements depends on the products that are in the same user's base queryset (aka the products a user should be able to see). The tests a user should have access to depend on the engagements he has access to, the findings depend on the tests and so on. This is what the custom managers do, delegate the checks. And to make all of this efficient with a single db query and no joins and sub-queries, I created and used the django-flexquery library for this purpose.
That way, the question "Has a user access to finding X?" can be answered by"Yes, if he has access to the test finding X belongs to." instead of hard-coding the check if the user is in a product's authorized_users list everywhere all over the place.

  I think it might be better to cherry-pick the things we think are useful and implement them in their own PR(s).

The permission part is quite dense (it hasn't lots of lines anyway), and IMHO you won't be able to break some parts out of it without breaking the whole idea, too.
Just ask if you've got more questions about the implementation, I'll be happy to help.
Best regards
Robert

Thanks for the info. One reason also for suggesting to cherrypick the permission/role part is the conflicts we have against current dev.
To get this PR merged I think we would need:

me + 1 (or more) persons looking at the permission/role code
2 or more other persons looking at the report code
you doing one more rebase on / merge with dev so we can take it from there.
Tonight we have a moderator meeting. It's not on the agenda, but let me see if I can recruit some volunteers.

Thanks for the info. One reason also for suggesting to cherrypick the permission/role part is the conflicts we have against current dev.

Actually, the conflicts all regard the permission handling part, ReportNG is mostly self-contained and doesn't cause conflicts. Even the existing conflicts can easily be resolved and shouldn't cause any interference with the recent changes I think.

To get this PR merged I think we would need:
* me + 1 (or more) persons looking at the permission/role code

* 2 or more other persons looking at the report code

* you doing one more rebase on / merge with dev so we can take it from there.

Tonight we have a moderator meeting. It's not on the agenda, but let me see if I can recruit some volunteers.

I'm quite busy at the moment and won't have time for any rebasing before March 18th. But then, it should be doable.

Hi,
As Promised, here is the rebase onto latest dev.
Best regards
Robert

Thanks, I tried to push a commit to fix the flake8 errors, but permission denied.

Did a quicktest, but everything regarding product seems to raise an error (/product and /product/1)
File "/c/Data/django-DefectDojo/dojo/product/views.py" in product
  56.     prod_filter = ProductFilter(request.GET, queryset=prods, user=request.user)

File "/c/Data/django-DefectDojo/dojo/filters.py" in __init__
  341.         super(DojoFilter, self).__init__(*args, **kwargs)

Exception Type: TypeError at /product
Exception Value: __init__() got an unexpected keyword argument 'user'

Also took a look at the reporting part. I like the idea, but it is extremely slow for some reason. Page load times sometimes over 60 seconds. Lots of database queries, I think prefetching is needed there.

Fixed the errors regarding /product, something got messed up there during rebasing.

Also took a look at the reporting part. I like the idea, but it is extremely slow for some reason. Page load times sometimes over 60 seconds. Lots of database queries, I think prefetching is needed there.

For me a page load in the builder takes ~200ms, though admittedly I don't have a huge dataset. It get's significantly better in production mode, because then, templates only need to be compiled once and are then cached.
It'd be interesting to know whether the evaluation of queries needed for working out all the available filtering choices takes most time or those for rendering the result lists. I'll add a select_related() tomorrow and see how that goes.

The 'select all' tickboxes do not seem to work, or maybe they are very slow because they trigger some request to the server.

They do work, but yes, they trigger a request. And that could be the bottleneck actually, because they count database objects filtered by a list of primary keys using the SQL IN operator and then send all the PKs back as hidden HTML form fields. I think it could become problematic when selecting around 10000+ findings at once to be included in a single report... Who does so btw? What magnitude of number of products, engagements, tests and findings do you have in your db and how many did you select?

It also says I need to select a template, but the dropdown is empty.
OK just read above about the templates. As I said I'm not really looking for reporting, but I think it could come in handy someday. But having a reporting feature without some template out of the box doesn't make sense to me.

The documentation tells you how to set up a template and provides a minimalist example for demonstration purposes. But yeah, I guess somebody will write and share a template that could be included by default some day. It depends on what you want to use the reports for after all.

Thanks, I tried to push a commit to fix the flake8 errors, but permission denied.

Hmm, I checked "Allow edits from maintainers", don't know if there's anything else to activate.

Found violations: dojo/filters.py
	F811 @ 575:1 - redefinition of unused 'EngagementFilter' from line 547
Found violations: dojo/models.py
	F811 @ 408:5 - redefinition of unused 'Meta' from line 394
Found violations: dojo/reportng/builders/html/__init__.py
	F401 @ 6:1 - '..base.ReportGenerationError' imported but unused

still couple of flake8 error so can't merge it locally

I have looked at the permission part and so far my understanding:

API v2 automatically user the restricted queryset because of the mixins
Our legacy (View based) codebase need to be updated to user the for_user() explicitly (intead of filter("test__engagement__product__authorizedusers_in", ...) kinda stuff

This all makes sense and we can use that. What I do not understand fully is how the permission system will restrict access in the same way. Or maybe it doesn't? For example how to I prevent users from deleting findings for products they are not authorized for? Or is the idea that if we use for_user everywhere, it will never return findings the user is not authorized for?

First, you need to switch Finding to inherit from dojo.models_base.DojoModel and declare appropriate predicates for object-level permissions in its Meta using the rules_permissions setting. These will describe whether a user will be allowed to add/change/delete a particular finding. dojo.models_base.DojoModel.preprocess_rules_permissions() sets some defaults. In the case of findings, you probably will set:
rules_permissions = {"add": rules.is_authenticated, "change": rules.always_true}

This is safe since the base queryset will only contain findings of products the user is authorized for anyway, and the user should be able to change/delete all of these findings.
dojo.reportng.models.ReportNG shows how object-level permissions are declared. It's pretty simple and self-contained actually.
In legacy function-based views, you'll then need to use:

Finding.objects.for_user(request) to get the queryset of all findings to be listed to the user. That's defined by the custom model manager attached to the Finding model and works straight away.
Before deleting a finding, check the result of request.user.has_perm(Finding.get_perm("delete"), concrete_finding_obj) to check whether the user may delete the particular finding.

Note, the view functions don't need to know that there's a relation between a user's permission to delete a finding and whether he's in the product's autorized users list.
All these checks would be handled automatically when using django's native class-based views instead of plain functions, at least for CRUD actions. See dojo/reportng/views.py to get a sense of how clean that would look, especially ReportNGDeleteView, ReportNGDownloadView, ReportNGDetailView and ReportNGFilterView, all of which have automated permission checking via the mixin and don't even need to do db lookups themselves.
The API viewsets will need the dojo.mixins.DojoPermissionAPIViewSetMixin mixed in for the automatic permission checking and queryset restriction to work.
Honestly, would it help you any if we had a talk together? Writing all of this down turns out to be quite time-consuming.

This all makes sense and we can use that. What I do not understand fully is how the permission system will restrict access in the same way. Or maybe it doesn't? For example how to I prevent users from deleting findings for products they are not authorized for? Or is the idea that if we use for_user everywhere, it will never return findings the user is not authorized for?

The idea is to use for_user() everywhere indeed, and for object-level perms on top of that see my previous comment.

I also think that if a user tries to retrieve an object it has no access to it will generate a 404 instead of 401/403?

That's the consequence if the user should not even be able to see the object, aka not know of its existence at all. When just an object-level permission check fails, e.g. user tries to delete an object he only ahs read-only access to, he'll get a 403.

BTW, we could write a simple decorator for legacy function-based views that executes model.objects.for_user(request).get(pk=pk), performs an appropriate object-level permission check for e.g. change or delete on the fetched object and, if all went well, passes the object on to the regular view function. That way, view functions that act on a single object don't need to do the boilerplate get_object_or_404() + permission checking stuff.

Well you're welcome on the slack channel: https://owasp-slack.herokuapp.com/ I think I did sent an invite to one of your e-mail a while ago.
The problem with the PR is that it is doing multiple things in one PR modifying 76 files. And some of the queryset/permission parts are implemented some of them are not. I was under the impression this PR contained a fully implemented new way of handling permissions, but it turns out it is not and there is more implementation work to be done in various places.
I really would have liked a separate PR for the queryset and permission part only. But when you stated that you couldn't make one I decided we will have to make our own containing the stuff we want/like to have. This will take more time from everyone, so yes time could have been saved along the way.
One of the biggest problems with the UI today is that the authorized users thing doesn't work. You need to be superuser to be able to modify findings and I haven't seen yet if this PR can help with that.
For now I like the way we can more easily restrict the querysets for listing objects. I don't like the 404s for objects that do exist but the user has no access to.
But may I don't know enough about django and permissions etc and somebody else will need to handle this PR.

Well you're welcome on the slack channel: https://owasp-slack.herokuapp.com/ I think I did sent an invite to one of your e-mails a while ago.

Joined, thanks.

The problem with the PR is that it is doing multiple things in one PR modifying 76 files.

85% of the changes are ReportNG + template snippets. Permission stuff is only these files IIRC:

dojo/mixins.py
dojo/models_base.py
dojo/models.py
rules and flexquery in requirements.txt

And some of the queryset/permission parts are implemented some of them are not.

Not quite. The permission checking and queryset building is fully implemented.
The only thing you maybe need to tweak is the default object-level permissions in dojo.models_base.DojoModel.preprocess_rules_permissions(). A user could be allowed to write to all objects he has in his base queryset, for instance.

I was under the impression this PR contained a fully implemented new way of handling permissions, but it turns out it is not and there is more implementation work to be done in various places.

The permission checking is ready to be used, however, your code base is massive and I told you from the beginning that I didn't touch the existing components. Due to the way the existing views were written, it has to be applied component by component. So, to put it all in a nutshell, these places are:

The models have to be switched to dojo.models_base.DojoModel, which involves no db changes, just switching the base class.
The models optionally could get a rules_permissions setting.
The views need to use the user-specific base queryset, maybe even via a decorator which would make things easier for legacy view functions.

I really would have liked a separate PR for the queryset and permission part only. But when you stated that you couldn't make one I decided we will have to make our own containing the stuff we want/like to have. This will take more time from everyone, so yes time could have been saved along the way.

I have been employed for developing ReportNG in the first place, and time was rare. On the other hand, hard-coding the permissions was no option. That's why I developed both hand in hand and ReportNG now demonstrates how to use the permission system, which, by itself, is very slim.

One of the biggest problems with the UI today is that the authorized users thing doesn't work. You need to be superuser to be able to modify findings and I haven't seen yet if this PR can help with that.

I think I explained that it can.

For now I like the way we can more easily restrict the querysets for listing objects. I don't like the 404s for objects that do exist but the user has no access to.

That lies in the nature of filtering the base queryset. Of course one could do another db query if an object wasn't found in the restricted queryset to see if it would have existed without queryset restrictions, but that seems like a bad idea to me, considering it'd need another query and the objects that cause a 404 aren't even listed to the user, so how should he get redirected to them at all?

But may I don't know enough about django and permissions etc and somebody else will need to handle this PR.

I'll contact you on Slack and we'll see if we can sort things out more easily.

Right, you now have something concrete to work with. I built a decorator (dojo.utils.ensure_permission()) that makes using the new permission checking paradigm a breeze, even for legacy view functions. It's documented and I applied it to delete_finding() and edit_finding() for demonstration purposes. With that in place, an authorized user should be able to access the finding edit page without being staff, although all the other views need to be adjusted so that he can actually see the products/engs/tests/findings and navigate appropriately, but that's technically it.
And don't forget to switch the models to DojoModel, like I did with Finding already.

@efficiosoft Today we had a moderator meeting and decided that we will create new branch for the next release soon. Afterwards we want to merge this PR to dev, so we have some time to get feedback and improve this new features.

We discussed some PRs during our monthly call. Next week dev will be promoted as RC for the next release. Once that RC branch is there, we can merge this PR here into dev to further test & improve it.

Hi, I'm hoping to find some help with this. I've installed this and have been following the Documents to find where to put the templates. I've found it says:
List of directories to scan for templates.
["<MEDIA_ROOT>/reportng/html_templates"]
My installation directory is /opt/django-DefectDojo so i've tried putting the template in the root of that file so it would be:
/opt/django-DefectDojo/reportng/html_templates
That didn't work so I tried it in
/opt/django-DefectDojo/dojo/reportng/html_templates
I've tried looking around to see where in the files it points to ["<MEDIA_ROOT>/reportng/html_templates"] but can't for the life of me find it. I'm on the Slack under the name "Michael Aguilera".

@Michael-Aguilera The media root is the directory you configured for storing uploaded files, such as finding images. At the docker-based installs, it's by default a directory named media, don't know where you have it though.

There is a PR that make findings/media work in at least the dev docker setup: #2056

Okay I found the directory you're talking about! Me being me was looking at the files stored on my file directory rather than the docker vms.... The default user is "I have no name!", any idea what the root password is?

@efficiosoft We are ready to merge this into dev and finalize it there. Would you be willing to help by:

resolving conflicts one more time
I am thinking about marking the new reporting feature as "beta" or "preview" in the menu as during my testing I saw some issues that I think need to be resolved before it is production ready.

We should care about the migrations. At the moment the current migration file " dojo/db_migrations/0015_findingimage_caption.py" is changed. Can we create a new migration file instead? Otherwise we will get in trouble when updating old DD installations.

Looks like there is some pickling / serialization issue, possibly the rules/permission lru cache?
uwsgi_1_819b0a12f074 | Internal Server Error: /product/5/import_scan_results
uwsgi_1_819b0a12f074 | Traceback (most recent call last):
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/serialization.py", line 50, in _reraise_errors
uwsgi_1_819b0a12f074 |     yield
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/serialization.py", line 221, in dumps
uwsgi_1_819b0a12f074 |     payload = encoder(data)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/serialization.py", line 350, in pickle_dumps
uwsgi_1_819b0a12f074 |     return dumper(obj, protocol=pickle_protocol)
uwsgi_1_819b0a12f074 | _pickle.PicklingError: Can't pickle <functools._lru_cache_wrapper object at 0x7f966bd35198>: it's not the same object as django.contrib.auth.models.PermissionsMixin.has_perm
uwsgi_1_819b0a12f074 |
uwsgi_1_819b0a12f074 | During handling of the above exception, another exception occurred:
uwsgi_1_819b0a12f074 |
uwsgi_1_819b0a12f074 | Traceback (most recent call last):
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in inner
uwsgi_1_819b0a12f074 |     response = get_response(request)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 115, in _get_response
uwsgi_1_819b0a12f074 |     response = self.process_exception_by_middleware(e, request)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/django/core/handlers/base.py", line 113, in _get_response
uwsgi_1_819b0a12f074 |     response = wrapped_callback(request, *callback_args, **callback_kwargs)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
uwsgi_1_819b0a12f074 |     return view_func(request, *args, **kwargs)
uwsgi_1_819b0a12f074 |   File "./dojo/product/views.py", line 492, in import_scan_results_prod
uwsgi_1_819b0a12f074 |     return import_scan_results(request, pid=pid)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/django/contrib/auth/decorators.py", line 21, in _wrapped_view
uwsgi_1_819b0a12f074 |     return view_func(request, *args, **kwargs)
uwsgi_1_819b0a12f074 |   File "./dojo/engagement/views.py", line 695, in import_scan_results
uwsgi_1_819b0a12f074 |     url=reverse('view_test', args=(t.id, )))
uwsgi_1_819b0a12f074 |   File "./dojo/notifications/helper.py", line 20, in create_notification
uwsgi_1_819b0a12f074 |     process_notifications(event, system_notifications, *args, **kwargs)
uwsgi_1_819b0a12f074 |   File "./dojo/notifications/helper.py", line 121, in process_notifications
uwsgi_1_819b0a12f074 |     send_alert_notification_task.delay(event, notifications.user, *args, **kwargs)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/celery/app/task.py", line 425, in delay
uwsgi_1_819b0a12f074 |     return self.apply_async(args, kwargs)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/celery/app/task.py", line 568, in apply_async
uwsgi_1_819b0a12f074 |     **options
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/celery/app/base.py", line 775, in send_task
uwsgi_1_819b0a12f074 |     amqp.send_task_message(P, name, message, **options)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/celery/app/amqp.py", line 559, in send_task_message
uwsgi_1_819b0a12f074 |     **properties
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/messaging.py", line 169, in publish
uwsgi_1_819b0a12f074 |     compression, headers)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/messaging.py", line 252, in _prepare
uwsgi_1_819b0a12f074 |     body) = dumps(body, serializer=serializer)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/serialization.py", line 221, in dumps
uwsgi_1_819b0a12f074 |     payload = encoder(data)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/contextlib.py", line 77, in __exit__
uwsgi_1_819b0a12f074 |     self.gen.throw(type, value, traceback)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/serialization.py", line 54, in _reraise_errors
uwsgi_1_819b0a12f074 |     reraise(wrapper, wrapper(exc), sys.exc_info()[2])
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/vine/five.py", line 194, in reraise
uwsgi_1_819b0a12f074 |     raise value.with_traceback(tb)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/serialization.py", line 50, in _reraise_errors
uwsgi_1_819b0a12f074 |     yield
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/serialization.py", line 221, in dumps
uwsgi_1_819b0a12f074 |     payload = encoder(data)
uwsgi_1_819b0a12f074 |   File "/usr/local/lib/python3.5/site-packages/kombu/serialization.py", line 350, in pickle_dumps
uwsgi_1_819b0a12f074 |     return dumper(obj, protocol=pickle_protocol)
uwsgi_1_819b0a12f074 | kombu.exceptions.EncodeError: Can't pickle <functools._lru_cache_wrapper object at 0x7f966bd35198>: it's not the same object as django.contrib.auth.models.PermissionsMixin.has_perm

Too sad if this work would be lost.
After you got the permission stuff implemented, any thoughts on merging ReportNG?


Update vue and vue-router

There were breaking changes between Vue 2 and 3, which caused Vue Router to also break on upgrade. This PR fixes both.
Tested here: https://afe2e0a36-global-fd.azurefd.net/

nobody promised that this process would be fun ;)


docs: clarify commit message requirements in contribution.md

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 69.38%. Comparing base (7fc8567) to head (b9e1c83).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #1416      +/-   ##
==========================================
- Coverage   69.52%   69.38%   -0.15%
==========================================
  Files         186      186
  Lines       18326    18326
==========================================
- Hits        12742    12715      -27
- Misses       4909     4933      +24
- Partials      675      678       +3

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

🚨 Try these New Features:

Flaky Tests Detection - Detect and resolve failed and flaky tests


What is the difference between exception with 'name' and 'type'

Looking though the code for returning exception I see sometimes 'name' is used to e.g. return 'ParseError'.
But other times the 'type' is used to e.g. return 'Syntax'.
Is there any special reason to make this difference, and if so what is the difference?

And further more is one of the 2 always filled in case of an exception?
Or can an exception be thrown without this filled.

I see also 'extract' and 'callExtract', and 'line' and 'callLine'.
And wonder what the difference is between both?

I think it's an inconsistency, it shouldn't matter.

extract: the 3 lines of source where the error happened
line: the line number where the error happened
callLine: if the error happened in a mixin, the line number which called that mixin
callExtract: the source where the call to the mixin happened

So the call params are only filled if the error happened inside a mixin.

For me it matters :-) to provide the right error message when compiling the LESS through Rhino.
It would be improvement to make this more consistent in a future release.
To explicitly declare what kind of error information can be expected when compilation fails.

Hi Alexis,
I just released lesscss4j 1.0.0 and I would appreciate it if you would RT my
last message on Twitter about it (http://twitter.com/#!/marceloverdijk).
Thanks in advance.
Cheers,
Marcel


network_timeout is not respected

I'm using network_timeout = 1.0 on a FastHttpLocust subclass, but the slow pages still succeed. I would expect that after 1.0s, the request would stop and another one would start. Additionally, I would expect the page load to count as an error. What I'm seeing though is that Locust is waiting for the page to load and counting it as a success.
I know there's a test in the codebase for this, but it only checks if there was an error in the slow page, but this could be due to other things.

Code for my Flask test web server `app.py`
import random
import time
from flask import Flask
app = Flask(__name__)

@app.route('/')
def hello_world():
    time.sleep(random.random() * 0.9)
    return 'Hello, World!'

@app.route('/slow')
def slow():
    sleep = 3 + random.random()
    time.sleep(sleep)
    return f'Took me {sleep} s.'

@app.route('/400')
def bad_request():
    return 'Bad Request for your data', 400

My `locust.py` file:
from locust import TaskSet
from locust.contrib.fasthttp import FastHttpLocust
from locust.wait_time import constant_pacing

def index(l):
    l.client.get("/slow")

class UserBehavior(TaskSet):
    tasks = {index: 1}

class WebsiteUser(FastHttpLocust):
    task_set = UserBehavior
    wait_time = constant_pacing(1.0)
    network_timeout = 1.0
    connection_timeout = 1.0

I run the web server with flask run --reload (need pip install flask) and run the locust without the GUI using locust -f locust.py --no-web -c 10 -r 10 -t 10s --host http://localhost:5000.

Here are the results
locust -f locust.py --no-web -c 10 -r 10 -t 10s --host http://localhost:5000
[2020-04-03 12:07:19,301] Roks-MacBook-Pro-2.local/INFO/locust.runners: Hatching and swarming 10 users at the rate 10 users/s (0 users already running)...
[2020-04-03 12:07:20,229] Roks-MacBook-Pro-2.local/INFO/locust.runners: All locusts hatched: WebsiteUser: 10 (0 already running)
[2020-04-03 12:07:20,306] Roks-MacBook-Pro-2.local/INFO/locust.main: Run time limit set to 10 seconds
[2020-04-03 12:07:20,306] Roks-MacBook-Pro-2.local/INFO/locust.main: Starting Locust 0.14.5
 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)       0       0       0  |       0    0.00    0.00

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                         0     0(0.00%)       0       0       0  |       0    0.00    0.00

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
 GET /slow                                                         10     0(0.00%)    3574    3100    4001  |    3400    0.00    0.00
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                        10     0(0.00%)    3574    3100    4001  |    3400    0.00    0.00

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
 GET /slow                                                         10     0(0.00%)    3574    3100    4001  |    3400    0.00    0.00
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                        10     0(0.00%)    3574    3100    4001  |    3400    0.00    0.00

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
 GET /slow                                                         20     0(0.00%)    3516    3095    4001  |    3400    1.43    0.00
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                        20     0(0.00%)    3516    3095    4001  |    3400    1.43    0.00

[2020-04-03 12:07:30,307] Roks-MacBook-Pro-2.local/INFO/locust.main: Time limit reached. Stopping Locust.
[2020-04-03 12:07:30,309] Roks-MacBook-Pro-2.local/INFO/locust.main: Shutting down (exit code 0), bye.
[2020-04-03 12:07:30,309] Roks-MacBook-Pro-2.local/INFO/locust.main: Cleaning up runner...
[2020-04-03 12:07:30,309] Roks-MacBook-Pro-2.local/INFO/locust.main: Running teardowns...
 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s failures/s
--------------------------------------------------------------------------------------------------------------------------------------------
 GET /slow                                                         21     0(0.00%)    3502    3095    4001  |    3400    1.91    0.00
--------------------------------------------------------------------------------------------------------------------------------------------
 Aggregated                                                        21     0(0.00%)    3502    3095    4001  |    3400    1.91    0.00

Percentage of the requests completed within given times
 Type                 Name                                                           # reqs    50%    66%    75%    80%    90%    95%    98%    99%  99.9% 99.99%   100%
------------------------------------------------------------------------------------------------------------------------------------------------------
 GET                  /slow                                                              21   3400   3600   3700   3800   4000   4000   4000   4000   4000   4000   4000
------------------------------------------------------------------------------------------------------------------------------------------------------
 None                 Aggregated                                                         21   3400   3600   3700   3800   4000   4000   4000   4000   4000   4000   4000

You can see that all the requests were successful, but all of them should have failed and there should have been more requests, as each one should fail after 1.0 s.
Environment:

OS: Mac OS
Python version: Python 3.7.1
Locust version: 0.14.5

I've added some additional checks to the test but that did not make it fail: 12c4d33b5a37cf71d22c7fc3d78b700f4778e974

@rokcarl Can you try pulling latest master and see if it works? Since https://github.com/locustio/locust/pull/1304 , Locust classes must explicitly be declared abstract, so you may have ended up running the base FastHttpLocust class instead of WebsiteUser

Can you try pulling latest master and see if it works? Since #1304 , Locust classes must explicitly be declared abstract, so you may have ended up running the base FastHttpLocust class instead of WebsiteUser.

Good catch!

I tried it from master, but now I get the following error:
[2020-04-06 17:46:46,736] docker-desktop/ERROR/locust.core: Cannot choose from an empty sequence
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/locust/core.py", line 260, in run
    self.schedule_task(self.get_next_task())
  File "/usr/local/lib/python3.7/site-packages/locust/core.py", line 419, in get_next_task
    return random.choice(self.locust.tasks)
  File "/usr/local/lib/python3.7/random.py", line 261, in choice
    raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence

Which is weird because there's a test in locust to make sure that I get a different error if the tasks are empty. Have the same code as I had above. I also tried tasks = [], the same. I tried extending HttpLocust instead of FastHttpLocust, but it was the same again.

pip freeze
certifi==2<IP_ADDRESS>
chardet==3.0.4
click==7.1.1
ConfigArgParse==1.2
Flask==1.1.2
gevent==1.5a3
geventhttpclient-wheels==1.3.1.dev3
greenlet==0.4.15
idna==2.9
itsdangerous==1.1.0
Jinja2==2.11.1
locustio==0.14.5
MarkupSafe==1.1.1
msgpack==1.0.0
psutil==5.7.0
pyzmq==19.0.0
requests==2.23.0
six==1.14.0
urllib3==1.25.8
Werkzeug==1.0.1

I got this ☝ by basically running pip install https://github.com/locustio/locust/archive/master.zip.

Hmm. Not sure what is the problem, but if you use this syntax it should work:
class WebsiteUser(FastHttpLocust):
    @task
    def index(self):
        self.client.get("/slow")
    wait_time = constant_pacing(1.0)
    network_timeout = 1.0
    connection_timeout = 1.0

Yes, this works and the timeout works as well, I ran it against my mock website with a timeout of 0.5 + random.random() (so between 0.5 and 1.5) and half of the requests were successful and half were a failure.
Thank you!


phone signup password reset function needed

Feature request
we need a function to reset users passwords from phone signups just like the function<EMAIL_ADDRESS>

hi @HassanIyan, if you want to reset a user's password for phone-password signups, you can just call the signInWithOtp() method - it will send an OTP to your phone number which you can use to sign in


Link to a non-existent route gives blank page and going back gives an error

What version of Remix are you using?
2.1.0
Are all your remix dependencies & dev-dependencies using the same version?

[X] Yes

Steps to Reproduce
Add a link to a route that doesn't exist
<NavLink to="/examples/dashboard">
        Overview
</NavLink>

In this case, the /examples/dashboard route does not exist.
Click on this route and go back from the browser back button.
Expected Behavior
There should be a 404 page returned/rendered when you click on that link. When you go back, you should see the same page you were on before and not an error.
Actual Behavior
When the link is clicked, I get a blank page instead of 404. This gets solved by adding an Error Boundary to catch all though, but ideally, it should have been a 404 browser page by default. I suspect this has something to do with client side routing.
The bigger bug however is when I click on the browser back from this route, I get an error that says Cannot read properties of null (reading 'appendChild')

Here is a video showing the whole thing as well
https://github.com/remix-run/remix/assets/40840079/27398cde-86c9-4875-8db0-97ac7be6a1b3

Can you provide a minimal reproduction?  Adding a 404 link to https://remix.new works as expected, showing the built-in default 404 error boundary and back-button routing also works as expected?
https://stackblitz.com/edit/remix-run-remix-xwspfu?file=app%2Froutes%2F_index.tsx

Correct, it works with the boiler plate. I think it has something to do with Clerk and its error boundary, not able to replicate that on stack blitz. Might have to look into my project itself. Thanks for the help and sorry for the misinformed issue!

Sorry, ignore the above message. I think I have found the root.

Commenting these lines make the error go away, ill try to replicate this on stack blitz


Code coverage should respect assembly level ExcludeFromCodeCoverage attribute

Description
We would like to exclude our test binaries from code coverage, and the only options we have today are either decorating each and every class with [ExcludeFromCodeCoverage] or have crazy regexes in our runsettings.json. testfx should respect assembly level ExcludeFromCodeCoverage instead. This is related to https://github.com/microsoft/vstest/issues/1050.
Steps to reproduce

Create a simple Hello World project.
Create a unit test project that calls Hello World and specify this in csproj:

  <ItemGroup>
        <AssemblyAttribute Include="System.Diagnostics.CodeAnalysis.ExcludeFromCodeCoverageAttribute" />
    </ItemGroup>

Run unit test with code coverage enabled.

Expected behavior
The test binary is not included in coverage results.
Actual behavior
The test binary is included in coverage results.
Environment

OS: Windows 11
Build version of vstest.console: VS2022.
Package version of MSTest framework and adapter: 2.2.3

This is already available in our latest package: https://dev.azure.com/dnceng/public/_artifacts/feed/test-tools/NuGet/Microsoft.CodeCoverage/overview/17.3.0-preview-20220601-01


Unable/Cannot find electron app

Just wanted to give this project a try but I can't seem to find the application after running the following commands
yarn install

npm run build:64

This the screenshot of after running the following commands:

Built app should be inside the build/ directory. Sorry for any confusion leaded by the docs.

nah the documentations are clear, I am just not sure if the folder build is created when running the following command npm run build:64. But I will try it again and see what happens.

Great. Readme sated that the app would be inside dist when this issue was raised, hence my comment. I hope you were able to build your app!


Abstract: DXVK_HUD="all"

i'm usually using DXVK_HUD='devinfo,fps,frametimes,drawcalls,pipelines,memory,version'  for debugging and so i think that shorting this command into DXVK_HUD="all" might prove helpful.

There is DXVK_HUD="1" which i wasn't aware of. Closing


Not able to paste blobs between one regular storage account and one ADLS Gen2 storage account

Storage Explorer Version: 1.22.0-dev
Build Number: 20211123.1
Branch: main
Platform/OS: Windows 10/Linux Ubuntu 20.04/MacOS Monterey 12.0.1
Architecture: ia32/x64
How Found: From running test cases
Regression From: Previous release (1.21.3)
Steps to Reproduce

Expand one non-ADLS Gen2 storage account -> Blob Containers.
Open one blob container -> Select one blob -> Click 'Copy'.
Expand one ADLS Gen2 storage account -> Blob Containers.
Open one blob container -> Check whether the 'Paste' button is enabled.

Expected Experience
The 'Paste' button is enabled.
Actual Experience
The 'Paste' button is disabled.
Additional Context
When copying one ADLS Gen2 blob, the 'Paste' button in the regular blob container is disabled.

Verified on the build main/20211125.2. Fixed.


fix(ar/egydead): fix streamwish extractor and dood

fix extractor
Checklist:

[x] Updated extVersionCode value in build.gradle for individual extensions
[x] Referenced all related issues in the PR body (e.g. "Closes #xyz")
[x] Added the containsNsfw = true flag in build.gradle when appropriate
[x] Have not changed source names
[x] Have explicitly kept the id if a source's name or language were changed
[x] Have tested the modifications by compiling and running the extension through Android Studio

Why not just use the Streamwish extractor lib ?

Why not just use the Streamwish extractor lib ?

there's so much to keep up with i didn't touch aniyomi since long time :(
but i'll add it


Observable.call(this)

Hi, following your blog posts and very excited about the simplicity of FRZR (great name by the way) I have a question? in view.js in the constructor function is the first line:
Observable.call(this);
which will create:
this.listeners = {};
on the View instance, so how come the prototype methods of on, off, trigger are added to the new View without instantiating Observable with new, I recreated the code and got .on is not a function etc…
regards,
Jamie

Hi!
This line inherits View from Observable:
https://github.com/pakastin/frzr/blob/master/src/view.js#L39
The inherits function can be found from here:
https://github.com/pakastin/frzr/blob/master/src/utils.js#L28
I checked with codepen and methods seem to work just fine:
http://codepen.io/pakastin/pen/NxpmWK
The point is to instantiate View with new View(options, data), otherwise use the view(options, data) notation (it will add new keyword for you). Also using var CustomView = View.extend(options) will require you to use new CustomView(data) and var customView = view.extend(options) doesn't.. Makes sense? :)
They're defined here:
https://github.com/pakastin/frzr/blob/master/src/view.js#L238

There's also ViewList (needs new keyword) and viewList (doesn't need).
Think View and ViewList as classes and view and viewList as factories.
Maybe I should make a check to View that the user is using key..

Hi Juha,
Thank you for the in-depth response, I missed the Inherits utility function, makes perfect sense now, I really like where you are going with the simplicity of FRZR, your API is a very transparent thin layer on the Dom API, great work.
Do you have any ideas for reusable components? say for example a counter component that embeds say two swatch components that display count and clicks, where you can easily reuse and add these component group instances to the Dom
Also when do you think part 2 in your blog series will be published?
regards,
Jamie

Here is an example of making components:
http://codepen.io/pakastin/pen/MKpMyM
Btw. I just added server-side rendering as a beta (v. 0.8.1). When you're using FRZR from Node.js, you'll get this new View.render() method, which will return HTML. You also have frzr.server -variable, which will return true when running on the server.
Have fun! ;)

Btw. If you need more complex components (or just like more traditional approach), you can create your own custom Views manually:
function CustomView (data) {
  ...
  frzr.View.call(options, data);
  ...
}
frzr.inherits(CustomView, frzr.View);
frzr.define(CustomView.prototype, {
  customMethod: function () {
    ...
  }
});

Or with ES6:
class CustomView extends frzr.View {
  constructor (data) {
    super(options, data);
    ...
  }
}

Here's another example of "nested" components:
http://pakastin.fi/frzr-test/
Source:
http://pakastin.fi/frzr-test/main.js (view logic)
http://pakastin.fi/frzr-test/api.js (data logic inside web worker)
I will probably publish that example in github in some point. I was just testing a redux-like logic with web workers. Turned out it works quite fast and smooth, even on mobile ;)

Here's similar example in Github:
https://github.com/pakastin/worker-example


use ubuntu 20.04 for PR workflow to make it consistent with release workflow

Refer to #168 for historical context of why we are using 20.04 here

I'm going to bypass the failed rust lint check on this one as it is certainly unrelated to the PR.

Ugh I saw https://github.com/fermyon/spin/actions/runs/8336227041/job/22813048050 and assumed these lint failures were for the same reason but they were not.
This certainly shouldn't have caused lint errors based on the CI workflow but apparently it did. Mea culpa. :facepalm:

Should we revert for now until we figure out what the issue is?

It looks like this was caused by caching. I deleted the GH rust cache and reran the job which appears to be succeeding now: https://github.com/fermyon/spin/actions/runs/8343891983/job/22837378173


default config contains a duplicate key

I have noticed the following under user-config
[user-config]
USER_AGENT : /usr/share/seclists/Passwords/Common-Credentials/best15.txt
[...]
USER_AGENT : celerystalk

thanks for this.  don't know how i missed it! fixed in build 211


Test discovery slows loading of LS

As a result, auto completion is slow to load.

I notice 1-2 seconds delay, if any. Things seem almost the same for me.

I removed LS so i had to download it.

Please test with LS steady downloaded. Yes downloading will be slow.
What was experienced was slow LS due to test discovery.

I tested with LS already downloaded as well. I do not notice any significant difference.
MY EXPERIMENTAL SETUP

Sets clock to t = 0

Activate the extension
Discovers the tests (for 1st situation), LS gets initialized
Hovers over, sometimes loading... appears for a moment (in both cases) before displaying the intellisense.

Time t at the end of these steps observed to be almost the same.

Will reopen if someone else has similar issues.


Show 'my vote' on proposal page

Note - the response data in types is incorrect - need to regenerate based on regen ledger


Adding rustdoc-args = ["--generate-link-to-definition"] in [package.metadata.docs.rs] makes documentation coverage dissapear

Its seems that all crates that are documented with the option rustdoc-args = ["--generate-link-to-definition"] do not have the "documentation coverage" section in the crates' drop down menu in docs.rs.
For example: clap, vs tokio
Why is that? Why can't there always be the coverage info displayed?

I'm not sure where the problem is here,
my hunch would be that the rustdoc command we generate for coverage can't handle this specific arg, but I need to sift through some logs to find the actual error.
( currently we skip over coverage and just build the docs in case of an error generating coverage)

Maybe its with the rustdoc-args parameter as a whole, because there is the --show-coverage argument which you can pass to rustdoc to have an overview of your docs coverage printed to stdout. I assume this argument is used to get the coverage for docs.rs, and it overlaps with crates providing their own sets of rustdoc-args.

I needed to jump though some hoops to find the actual error message.
It is :

[stderr] error: --generate-link-to-definition option can only be used with HTML output format

@GuillaumeGomez do you have an idea how to solve this best?
in my mind, we probably need some of the rustdoc-args for coverage, but filtering would be hard.
Could rustdoc just ignore ( and warn) ?

We could indeed emit a warning and mention that the option will be ignored.

We could indeed emit a warning and mention that the option will be ignored.

@GuillaumeGomez that would be awesome, thanks!

Already opened  rust-lang/rust#129050. :p

I wonder if we should collect & store the coverage logs too...

I mean, that would have helped here.

https://github.com/rust-lang/rust/pull/129050 is now merged, so tomorrow it should be fixed on docs.rs as well. Closing then.

Just validated that the change in nightly

is on docs.rs
is working

( I ran a rebuild for clap, and now we have coverage)


Need to pass Security Labels and potentially USERNAMESPACE mappings to volumes.

We have been playing around with a lvm volume plugin for docker.  One problem we are having is how to setup the security constraints of the volume.
https://github.com/shishir-a412ed/docker-lvm-plugin
The way this volume is working currently is during docker create the volume gets created without any SELinux labels associated with it and without any User Namespace.  Then when you run docker run it will get mounted into the container with a label that is blocked by SELinux and if you were running with User Namespace, owned by UID=0, which would not be accessible to the container.
We have looked at making changes to docker and doing a relabel during setupMounts, but this means every time you started the container, you would be running recursive changes of SELinux Labels and potentially UID/GID parings.  We want to only do this during container creation.  Or in the case of lvm we could mount the volume with context='SELINUX_LABEL".  But none of this information is available to the volume plugin.
Just wanted to open a discussion on this issue.

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.
If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.
For more information about reporting issues, see CONTRIBUTING.md.
You don't have to include this information if this is a feature request
(This is an automated, informational response)

BUG REPORT INFORMATION
Use the commands below to provide key information from your environment:
docker version:
docker info:
Provide additional environment details (AWS, VirtualBox, physical, etc.):
List the steps to reproduce the issue:
1.
2.
3.
Describe the results you received:
Describe the results you expected:
Provide additional info you think is important:
----------END REPORT ---------
#ENEEDMOREINFO

@cpuguy83 I guess you would be the lead on this discussion.

@rhatdan Does the userns example not work on 1.10?

In 1.10 the content of the container fs should be copied into the volume, taking the uid/gid with it.

cpuguy83, if I used one of your new "mount flags, I believe I could set the UID to match, but I don't see how the top level pid would get created with the correct UID.  We could also use the SELinux labels on the -o option to set up the labels to be shared but I can't set them to be private to the container, since I don't know the MCS Label until the container has been created.
In the future if I we added the ability to specify the UID/GID Usernamespace fields on the docker run command, how would the UID/GID of the ownership pid of the volume be set?
Currently we are setting local volumes during the creation stage, but if the actual volumes are not created/mounted until the mount stage, I don't see them being set correctly.

@cpuguy83 @rhatdan what's the status on this one? Is this still needed?

I am not sure

this bug effects me, would be great to see it fixed..thanks.  running 1.12 in enforcing mode, I have to run the container as privledged to get around this using the local volume driver.

Is this related to https://github.com/docker/docker/pull/23024 ?

Yes this fixes this issue at least as far as SELinux is concerned.


Skip pywin32 226

Release 0.1.4

This is a temporary workaround which would otherwise be needed in our downstream package Azure SDK for Python.

CC: @chlowell

@rayluo that should be a short term fix only, because you are forcing customers to freeze this dep in time. If they use it in their own application for something else, they might be broken with a "dependency hell" at some point.
Could we agree that long term fix should be to remove that? Or at least consider this as an option if thye are not fixing this fast enough?

@lmazuel Thanks for calling this out! We did mention "temporary" in the proposed release note in this PR's description. And yes, once the upstream pywin32 fixes this from their end, we would then have to issue yet another patch to revert this change.
To be more clear, we wish we would NOT have to cut such a temporary release in the first place, to work around something that is not even in the scope of this repo/project.
We do it here, for Azure SDK for Python team, because this repo is more close to the upstream issue than Azure SDK for Python is. :-)

FYI: Our upstream package portalocker has kindly adopted a similar change (with a reasonable assumption that the next release of pywin32 would fix it). Credit goes to @wolph
Stay tuned.

@chlowell @lmazuel:
Thanks to its author @wolph, our upstream package portalocker 1.5.2 has just been released to PyPI, which excludes the problematic pywin32 226.
We no longer need this PR here.


Add CurrencyConverter

CurrencyConverter
This plugin lets you convert real and crypto currencies quickly.
By submitting this pull request I confirm I've read and complied with the below requirements 🖖
Requirements for your pull request

You have read and understood the Contribution Guidelines.
This pull request has a title in the format Add [Name of Plugin].

✅ Add GEmojiSharp
❌ Update readme.md
❌ Adding GEmojiSharp
❌ Added GEmojiSharp

Your entry should include a short description of the plugin. The first character should be uppercase and the description should end in a dot. It should be an objective description and not a tagline or marketing blurb.

✅ - [GEmojiSharp](…) - Find and copy GitHub Emoji to the clipboard.
❌ - [GEmojiSharp](…) - A GitHub Emoji plugin for PowerToys Run.
❌ - [GEmojiSharp](…)

The link text should be the Name of the plugin found in the plugin.json file. The URL should be the link to the GitHub repo, or a link to an anchor in the readme.

✅ - [GEmojiSharp](https://github.com/hlaueriksson/GEmojiSharp) - …
❌ - [GEmojiSharp.PowerToysRun](…) - …
❌ - [GEmojiSharp](https://github.com/hlaueriksson/GEmojiSharp/releases/download/v1.0.0/GEmojiSharp.PowerToysRun.1.0.0.zip) - …

Your entry should be added at the bottom of the appropriate category.

@hlaueriksson Did you get a chance to look into the plugin?

Hi @Advaith3600
Thanks for your contribution!
I have tried the plugin and I could not get it to work on my machine.
I created two issues in your repo.
I hope you have time to take a look.

Hey, I have released a new version, v1.0.2 with all the changes and fixes. Please have a look.

The v1.0.2 release works great.
Nice plugin! Thanks for your contribution 🚀

Hi @Advaith3600
I just wanted to let you know that this list is now included in the main awesome list.
And that means that you can add a badge to your repo 😎


Put the current version of the kit in the footer

This would help with debugging as you would be able to tell which version of the kit is running at a glance.

good idea

It's currently inserted as an html comment on each page. Are you suggesting it be visibly printed too?

@edwardhorsford yes because that wasn't apparent to me. I am suggesting it's visible in the page footer.