Add support for requestTemplates parameter in apigateway integrations mock

This PR adds support for the requestTemplates=... parameter for the put_integration method in apigateway mock. See http://boto3.readthedocs.io/en/latest/reference/services/apigateway.html#APIGateway.Client.put_integration
This is a non-breaking change. New test cases have been added to ensure we keep backward compatibility.
@spulec Thanks for maintaining this awesome project! :)

Coverage increased (+0.0007%) to 95.035% when pulling 5075f6684fa1c2837659bfff31e0e13f699bf896 on whummer:feature/apigateway-req-templates into ae938223d4fb077510df81bee8db782fb6c3318f on spulec:master.

Thanks!


adding windsor connector

@mkzsg - Thank you for your contribution. Would you take a moment to sign the Contributor License Agreement (CLA)? After the CLA is signed, someone can review your pull request. Thanks!

#label:"aq-pr-triaged"
@MicrosoftDocs/public-repo-pr-review-team

@microsoft-github-policy-service agree [company="Windsor.ai"]

@microsoft-github-policy-service agree company="Windsor.ai"

Close open to trigger a build.

quick ping @mkzsg to please address the comments shared

quick ping @mkzsg to please address the comments shared

@ptyx507x pushed fixes. hope this is ready to go now.

@mkzsg - You also need to add the Windsor connector to the list of all connectors in the Power Query documentation. The list of all connectors can be found in the connectors/index.md file. The list is in alphabetical order. Also, since your Windsor icon (windsor-64.png) is black with a transparent background, be aware that if someone is using dark mode in their web browser, your icon will disappear against the black background. If you want it to remain visible in dark mode, you should probably use a white background.

@mkzsg - Also, since this is a beta release of your connector, you should add "(Beta)" to the Windsor name in the table of contents (toc.yml), the list of all connectors (connectors/index.md), and to heading 1 of the connector article.

@DougKlopfenstein fixed.

@DougKlopfenstein can this be merged now?

@mkzsg
I've done an edit pass to check spelling, grammar, formatting, and Microsoft style. Please take a look at the article to make sure that I didn't change the meaning of some of the more esoteric sentences. If they are OK, just leave a comment here and I'll merge the PR.

@mkzsg I've done an edit pass to check spelling, grammar, formatting, and Microsoft style. Please take a look at the article to make sure that I didn't change the meaning of some of the more esoteric sentences. If they are OK, just leave a comment here and I'll merge the PR.

Yes, all OK. LGTM. Let's merge.

@Court72 can you review sometime today? Thanks, Jason

@DougKlopfenstein @JasonWHowell - usually PR review would prefer to move the commits for new articles to the private repo. Getting a staged page ensures that (among other things) images display correctly and Acrolinx helps us check for editorial issues like unapproved acronyms, typos, and spelling errors. That said, we'll merge this with the following recommendation:
@mkzsg - can you check the article after it goes live to ensure all the images display correctly and the page renders as you intended? thanks!


Number of results on the Results page

To display a number of results.

To change the font size and make it bold.

Committed code in branch fb_581_rbl (sorry @Vino-S - it was the closest I could manage!) with suggested more friendly wording:

Code (re-)added to fb_575_rbl. Should be on test3 soon.

Done, closing


Using the software to simulate fully actuated hexarotor dynamics

Hello CTU team,
I have been using your software for a while and I would like to use it to study the dynamics of fully actuated hexarotor.
I have noticed that in the models there is already an hexarotor f550, but to make it fully actuated, I need to be able to tilt the rotors. Is it possible to do it directly in the xacro file ? Or do I need some extra files and build another xacro file ?
Best regards,
Frank

Hello, you can just modify the .xacros to tilt the rotors:

But we use PX4 as a low level controller, and to my knowledge it does not support fully actuated platforms. But you can maybe try something like this - https://github.com/castacks/PX4-fully-actuated
Cheers, Dan H.

Hi @kefran88,
Since Dan has answered your question (looks crazy if you ask me), I am closing the issue, if you have more questions - feel free.


what would happen if future iOS introduced radical changes in design paradigm

This is a hypothetic question.
Let's say Apple introduced a completely redesign in iOS 15 next year, much like what happened on iOS 7 by Jony Ive https://en.wikipedia.org/wiki/IOS_7#Design. What does it mean for flutter ?
Since flutter does its own UI drawing instead of delegating to platform UI controls, I assume flutter team will need to maintain support for two versions of UI drawing

For flutter App installed on iOS < 15, keep using the old design, even if the flutter app gets upgraded flutter version.
For flutter App installed on iOS >= 15, use the new design.

Now flutter engine would switch on platform version (iOS 15) to draw differently internally while dart code doesn't need change much.
Am I getting this right ? Thanks !

Hey @shuoch-g ,
This issue tracker is used for tracking bugs and feature requests for the flutter framework, engine and the flutter tool.
Please see https://flutter.dev/community for resources and asking queries doubts and assistance on personal code. You may get some help if you post it on StackOverflow and if you need help with your code, please check https://www.reddit.com/r/flutterhelp/.
Thanks


[BUG] Unable to set timezone_id or locale using pytest.mark fixture

I have tried the example outlined in the Pytest Plugin Reference doc:
@pytest.mark.browser_context_args(timezone_id="Europe/Berlin", locale="en-GB")
def test_browser_context_args(page):
    assert page.evaluate("window.navigator.userAgent") == "Europe/Berlin"
    assert page.evaluate("window.navigator.languages") == ["de-DE"]

However, it doesn't work for either:
First assertion:
E       AssertionError: assert 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36' == 'Europe/Berlin'
E         - Europe/Berlin
E         + Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/<IP_ADDRESS> Safari/537.36

Second assertion
E       AssertionError: assert ['en-US'] == ['de-DE']
E         At index 0 diff: 'en-US' != 'de-DE'
E         Full diff:
E         - ['de-DE']
E         + ['en-US']

Also, just noticed that your example has you set en-GB, but the test is asserting de-DE
    @pytest.mark.browser_context_args(timezone_id="Europe/Berlin", locale="de-DE")
    def test_browser_context_args(page):
        # assert page.evaluate("window.navigator.userAgent") == "Europe/Berlin"
>       assert page.evaluate("window.navigator.languages") == ["de-DE"]
E       AssertionError: assert ['en-US'] == ['de-DE']
E         At index 0 diff: 'en-US' != 'de-DE'
E         Full diff:
E         - ['de-DE']
E         + ['en-US']

I still doesn't change it, regardless.


[BUG] BOM no longer removed from Service Bus incoming messages on newer extension versions

Library name and version
Microsoft.Azure.WebJobs.Extensions.ServiceBus 5.2.0
Describe the bug
Some messages sent to a Service Bus queue can include a BOM (Byte Order Mark) (0xEFBBBF for UTF-8 encoding) at the beginning of the message, for example if the message is JSON-serialized from a stream. Other messages, for example if using a simple JSON encoding directly on an object, will not have the leading BOM.
In SB extension version 4.3.0, the BOM is removed on the receiving side by the ServiceBusTrigger. The incoming string parameter or Message.Body property does not include it. This meant that regardless of how the sender serialized the message, it could still be treated the same way.
In SB extension version 5.2.0, the BOM is still present on the receiving side when the ServiceBusTrigger function executes. This is the case if the Function is binding to a string parameter or ServiceBusMessage. As a result, messages that were sent without a BOM will deserialize cleanly without any further processing. Any messages sent with a BOM will throw an exception upon deserializing.
Expected behavior
The leading BOM is removed from SB queue messages before being sent to the SB queue trigger function, same as the behavior using extension version 4.3.0.
Actual behavior
The leading BOM is still present in the arguments received by the SB queue trigger function, which is a different behavior than the older version of the extension.
Reproduction Steps
The following code sends two messages to a SB queue named test-sb-queue, assuming the connection strings are replaced appropriately. One message is created using a stream, and therefore has the leading BOM, while the other is stringified simply and doesn't contain the BOM.
using System.Globalization;
using System.Text;
using Azure.Messaging.ServiceBus;
using Microsoft.Extensions.Configuration;
using Newtonsoft.Json;
using Newtonsoft.Json.Converters;

try {
    string payload = "Hello";

    {
        // send payload with stream
        var payloadStream = CreatePayloadWithStream(payload);
        Console.WriteLine("Now sending a message serialized from a stream.");
        Console.WriteLine($"Payload bytes: {Convert.ToHexString(payloadStream)}");
        var message = new ServiceBusMessage(payloadStream);
        var client = new ServiceBusClient(/*REPLACE WITH OWN CONNECTION STRING*/);
        var sender = client.CreateSender("test-sb-queue");
        await sender.SendMessageAsync(message);
    }

    {
       // send simple payload
        var payloadSimple = CreatePayloadSimple(payload);
        Console.WriteLine("Now sending a message serialized simply.");
        Console.WriteLine($"Payload bytes: {Convert.ToHexString(payloadSimple)}");
        var message = new ServiceBusMessage(payloadSimple);
        var client = new ServiceBusClient(/* REPLACE WITH OWN CONNECTION STIRNG */);
        var sender = client.CreateSender("test-sb-queue");
        await sender.SendMessageAsync(message);
    }
} catch (Exception e) {
    Console.WriteLine(e);
}

/// Serialize payload by creating a unicode string with a stream. This results in BOM added to beginning
/// of string.
static byte[] CreatePayloadWithStream(string message)
{
    JsonSerializerSettings serializerSettings = new JsonSerializerSettings
    {
        TypeNameAssemblyFormatHandling = TypeNameAssemblyFormatHandling.Simple,
        TypeNameHandling = TypeNameHandling.None,
        Converters = { new IsoDateTimeConverter { DateTimeStyles = DateTimeStyles.RoundtripKind } }
    };
    JsonSerializer serializer = JsonSerializer.Create(serializerSettings);

    var stream = new MemoryStream();

    var streamWriter = new StreamWriter(stream, Encoding.UTF8);
    var jsonWriter = new JsonTextWriter(streamWriter) { Formatting = Formatting.None };
    serializer.Serialize(jsonWriter, message);
    jsonWriter.Flush();
    stream.Position = 0;

    return stream.ToArray();
}

/// Serialize payload by just calling JsonConvert.SerializeObject(). This doesn't
/// result in BOM added to beginning of string.
static byte[] CreatePayloadSimple(string messagePayload)
{
    var item = JsonConvert.SerializeObject(messagePayload);

    return Encoding.UTF8.GetBytes(item);
}

Running the above code yields the following output:
Now sending a message serialized from a stream.
Payload bytes: EFBBBF2248656C6C6F22
Now sending a message serialized simply.
Payload bytes: 2248656C6C6F22

The leading BOM (0xEFBBBF) is present in the first message, but not the second.
Now, we can write a simple SB queue trigger function to see how the messages are received.
using System;
using System.Text;
using Microsoft.Azure.WebJobs;
using Microsoft.Azure.WebJobs.Host;
using Microsoft.Extensions.Logging;
using Newtonsoft.Json;

namespace Company.Function
{
    public class ServiceBusQueueTrigger1
    {
        [FunctionName("ServiceBusQueueTrigger1")]
        public void Run([ServiceBusTrigger("test-sb-queue", Connection = "testsb")]string myQueueItem, ILogger log)
        {
            log.LogInformation($"String: {myQueueItem}");
            var bytes = Encoding.UTF8.GetBytes(myQueueItem);
            log.LogInformation($"Bytes: {Convert.ToHexString(bytes)}");
            // attempting to deserialize here will throw an error in newer versions of the extension
            var deserialized = JsonConvert.DeserializeObject<string>(myQueueItem);
        }
    }
}

Running the above function using version 4.3.0 of the extension yields the following output:
String: "Hello"
Bytes: 2248656C6C6F22
String: "Hello"
Bytes: 2248656C6C6F22

The old extension version removes the leading BOM and the message can be handled the same way, regardless of how it was encoded. The function does not throw an error on attempting to deserialize the incoming message.
However, running the same function using extension version 5.2.0 yields the following output:
String: ?"Hello"
Bytes: EFBBBF2248656C6C6F22
String: "Hello"
Bytes: 2248656C6C6F22

The BOM is still present at the start of the message. An error is thrown on attempting to deserialize since the string starts with an invalid character:
System.Private.CoreLib: Exception while executing function: ServiceBusQueueTrigger1. Newtonsoft.Json: Unexpected character encountered while parsing value: ?. Path '', line 1, position 1.
Environment
Window 11 (OS Version 10.0.22000) with .NET version 6.0.1.
Verified on VS Code and Visual Studio 2022

You are correct that the new version of the extension does not strip out the BOM. The data in ServiceBusMessage.Body and when binding to a string is representing the data that was sent as-is.
The default behavior when using StreamWriter without specifying an encoding would be to emit UTF8 without a BOM. Would it be possible to just not add a BOM when sending?
If this is not possible, you could strip off the BOM yourself by constructing a UTF8Encoding instance with the constructor that allows you to specify whether a BOM should be included.
Would either of these options work?

Thanks for the quick response! I opened this ticket on behalf of a customer who filed an IcM incident saying they have a lot of functions in production that rely on the old behavior and changing the code of each function (or each application that sends messages) would have to touch a lot of their codebase. Is there a solution that doesn't require code changes?
Also, is there a reason the behavior was changed? We thought this was a bug rather than an intentional change. Was it mentioned in the release notes somewhere? Thanks!

Thanks for the quick response! I opened this ticket on behalf of a customer who filed an IcM incident saying they have a lot of functions in production that rely on the old behavior and changing the code of each function (or each application that sends messages) would have to touch a lot of their codebase. Is there a solution that doesn't require code changes?

No, I don't think there is a way to avoid code changes. Code changes are likely necessary anyway when upgrading to the new version.

Also, is there a reason the behavior was changed? We thought this was a bug rather than an intentional change. Was it mentioned in the release notes somewhere? Thanks!

The message body is stored in a BinaryData instance. We considered excluding the BOM from the BinaryData.ToString() method when adding this type, but we were advised not to by the .NET folks. This behavior was carried forward to the string binding, as it would be strange if the ToString of the message body and the string binding values differed. I don't believe this was mentioned in the release notes anywhere.

That makes sense, thank you! Will get back to the cx with this feedback. Closing this issue.


No menu- When I press HOME, I get a black screen with no menu.

When i compile, any method of compiling I press HOME, i get no menu, only a more ''black'' screen.
When I download release I get ''memory... error''
So...  Also mb current (cheat) procces already exists.

"Memory init failed" displays then csgo process not found.

I don't know why you close it... Cs2 was open, it's just doesn't attach to it and says memory init or wtv.
Also, when I do compile it myself I get the menu etc on my main pc.  Even if the ''menu loads'' the esp etc, when in a game don't appear...
Do I need to update offsets or something or I'm just not able to make it work ?
Also how do you compile ''AOT native'' i have no idea how to do that even after looking ytb videos ;-; . S.O.S


readme updates

docs(readme): adding summary of package.json scripts
chore(ignores): add playwright  folders to gitignore
chore(images): moved all sourced images into data
docs(readme): first draft of file layout
build(app): clean up electron build
chore(cleanup): simplifying tests until I'm ready to go bigger

Pull Request Test Coverage Report for Build<PHONE_NUMBER>5
Details

0 of 16 (0.0%)  changed or added relevant lines in 3 files are covered.
11304 unchanged lines in 165 files lost coverage.
Overall coverage decreased (-71.4%) to 3.063%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

src/cards/Train.jsx
0
2
0.0%

src/map/MapPlayers.jsx
0
5
0.0%

src/data/index.js
0
9
0.0%

Files with Coverage Reduction
New Missed Lines
%

src/hooks/alert.js
2
0.0%

src/state/storage.js
3
70.97%

src/atoms/Industry.jsx
3
0.0%

src/atoms/TunnelEntrance.jsx
3
0.0%

src/atoms/Good.jsx
4
0.0%

src/util/Page.jsx
4
0.0%

src/pages/Elements.jsx
5
0.0%

src/atoms/Border.jsx
5
0.0%

src/hooks/index.js
5
0.0%

src/SyntaxHighlighter.js
5
0.0%

Totals

Change from base Build<PHONE_NUMBER>8:
-71.4%

Covered Lines:
428

Relevant Lines:
15783

💛  - Coveralls


[FEATURE REQUEST] - shell completion

Is your feature request related to a problem? Please describe.

Those are the most used shells and known to me that have completion (more to come if someone helps), I want that to be improved.

https://github.com/nyxnor/onionservice/blob/main/etc/bash_completion/onionservice-cli
https://github.com/nyxnor/onionservice/tree/main/etc/zsh_completion

Describe the solution you'd like

Bash completion official package and debian guide

Additional context

Low priority, it will be very useful but not intrinsic required.

only bash, https://unix.stackexchange.com/a/158523
https://maskedbyte.com/how-to-create-a-custom-bash-completion-script/

Planning to move this to a new repository onionjuggler-shell-completion
This will avoid having bash and zsh code with posix, because of shellcheck and codefactor lintering.

https://zsh.sourceforge.io/FAQ/zshfaq04.html#l54

https://github.com/scop/bash-completion/blob/master/bash_completion

9a00d26

8d36854

ca13e72

in the end it was only bash completion, because sh does not have one and I won't learn zsh completion.
Also, only possible to complete services by configuration, because it is readable.
Not possible to complete clients server and client side because it is owned by the tor user.
Also, if the completion was made, it would only be valid to the root user, and I don't want anyone running root user by default just for completion.


Error with 2.2.0 Error: Input does not meet YAML 1.2 "Core Schema" specification: setup

@DavidMc2021 did you update the workflow file it has changed from before.
name: Notion Board
on:
  issues:
  issue_comment:
  workflow_dispatch:
    inputs:
      setup:
        description: 'Populate your notion database with all the requried properties'
        type: boolean
      syncIssues:
        description: 'sync all other existing issues in this repo'
        type: boolean
      issueType:
        type: choice
        description: 'The issue type you want to sync'
        options:
          - all
          - open
          - closed
        default: open
jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - name: Notion Board
        uses<EMAIL_ADDRESS>        with:
          setup: ${{github.event.inputs.setup}}
          syncIssues: ${{github.event.inputs.syncIssues}}
          issueType: ${{github.event.inputs.issueType}}
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
          NOTION_API_KEY: ${{ secrets.NOTION_API_KEY }}
          NOTION_DATABASE: ${{ secrets.NOTION_DATABASE }}

@Souvikns - That is correct, I updated per your comment and it ran without error. I also deleted out all previous issues in the notion DB and it populated from scratch in around 30sec. Impressive!
Thanks for the quick support on this!

Awesome 👍🏻
Let me know if you face any other errors.


Darin vocab

Checklists
General Checks

[ ] the branch is up to date with main/master
[ ] the code works when pulled and run locally
[ ] All CI checks pass
[ ] all conflicts are resolved (if any)
[ ] PR has a descriptive title
[ ] PR has appropriate labels and milestones for easy identification
[ ] PR it is assigned to the owner
[ ] reviewers are assigned
[ ] the PR contributes only one focused change
[ ] It is in the appropriate column in the project board (if necessary)
[ ] has short and clear description
[ ] is linked to an issue (if it is related)
[ ] feedback is addressed (if any and if it is appropriate feedback.)

Markdown

[ ] the markdown source is formatted
[ ] spelling and grammar is correct in all text
[ ] The markdown looks correct when you preview the file
[ ] all links and images work
[ ] any embedded code snippets are formatted

@Darin-Hamouda can you please fix this conflict?

@gelilaa  conflict resolved
and I did not commit any snippet in this branch.


psqworker fails and can't be killed if GCS_MOUNT_DIR points to a non existing directory in a non writable parent

[INFO] Starting PSQ listener on queue turbinia-psq
[INFO] Running Turbinia PSQ Worker.
[INFO] Starting Task StatTask c45b770ca70b4c518a6970a67df6d2b8
[INFO] Creating new directory /mnt/turbinia/output/1534207046-c45b770ca70b4c518a6970a67df6d2b8-StatTask
[ERROR] StatTask Task failed with exception: [Permission error ([Errno 13] Permission denied: '/mnt/turbinia')]
[ERROR] Traceback (most recent call last):
  File "/home/romaing/src/turbinia/turbinia/workers/__init__.py", line 399, in run_wrapper
    self.result = self.setup(evidence)
  File "/home/romaing/src/turbinia/turbinia/workers/__init__.py", line 318, in setup
    self.output_manager.setup(self)
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 156, in setup
    self._output_writers = self.get_output_writers(task)
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 64, in get_output_writers        unique_dir=unique_dir)]
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 221, in __init__
    super(LocalOutputWriter, self).__init__(*args, **kwargs)
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 177, in __init__
    self.create_output_dir()
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 235, in create_output_dir
    raise TurbiniaException(msg)
TurbiniaException: Permission error ([Errno 13] **Permission denied: '/mnt/turbinia')**
[ERROR] No TurbiniaTaskResult object found after task execution.
[DEBUG] Checking TurbiniaTaskResult for serializability
[INFO] Result check: Successful
[ERROR] Task 1693f13511f24a9c93ce72788fe1f2c7 failed.
Traceback (most recent call last):
  File "/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/psq/task.py", line 90,
in execute
    result = self.call_func()
  File "/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/psq/task.py", line 104, in call_func
    return self.f(*self.args, **self.kwargs)
  File "/home/romaing/src/turbinia/turbinia/task_manager.py", line 73, in task_runner
    return obj.run_wrapper(*args, **kwargs)
  File "/home/romaing/src/turbinia/turbinia/workers/__init__.py", line 450, in run_wrapper
    if original_result_id != self.result.id:
AttributeError: 'NoneType' object has no attribute 'id'
[ERROR] Task 1693f13511f24a9c93ce72788fe1f2c7 failed.
[ERROR] Task 1693f13511f24a9c93ce72788fe1f2c7 failed.                                                      [38/188]
Traceback (most recent call last):
  File "/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/psq/task.py", line 90,
in execute
    result = self.call_func()
  File "/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/psq/task.py", line 104, in call_func                                                                                return self.f(*self.args, **self.kwargs)                                                                         File "/home/romaing/src/turbinia/turbinia/task_manager.py", line 73, in task_runner
    return obj.run_wrapper(*args, **kwargs)
  File "/home/romaing/src/turbinia/turbinia/workers/__init__.py", line 450, in run_wrapper
    if original_result_id != self.result.id:
AttributeError: 'NoneType' object has no attribute 'id'
[INFO] Starting Task PlasoTask e160edc505bf47d893c4d9e9cb93f0d5
[INFO] Creating new directory /mnt/turbinia/output/1534207047-e160edc505bf47d893c4d9e9cb93f0d5-PlasoTask
[ERROR] PlasoTask Task failed with exception: [Permission error ([Errno 13] Permission denied: '/mnt/turbinia')]
[ERROR] Traceback (most recent call last):
  File "/home/romaing/src/turbinia/turbinia/workers/__init__.py", line 399, in run_wrapper
    self.result = self.setup(evidence)
  File "/home/romaing/src/turbinia/turbinia/workers/__init__.py", line 318, in setup
    self.output_manager.setup(self)
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 156, in setup
    self._output_writers = self.get_output_writers(task)
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 64, in get_output_writers
    unique_dir=unique_dir)]
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 221, in __init__
    super(LocalOutputWriter, self).__init__(*args, **kwargs)
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 177, in __init__
    self.create_output_dir()
  File "/home/romaing/src/turbinia/turbinia/output_manager.py", line 235, in create_output_dir
    raise TurbiniaException(msg)
TurbiniaException: Permission error ([Errno 13] Permission denied: '/mnt/turbinia')[ERROR] No TurbiniaTaskResult object found after task execution.
[DEBUG] Checking TurbiniaTaskResult for serializability
[INFO] Result check: Successful
[ERROR] Task 06d539c4cfac49c5a0ccf58addb3bc1d failed.
Traceback (most recent call last):
  File "/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/psq/task.py", line 90,
in execute
    result = self.call_func()
  File "/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/psq/task.py", line 104$
 in call_func
    return self.f(*self.args, **self.kwargs)
  File "/home/romaing/src/turbinia/turbinia/task_manager.py", line 73, in task_runner
    return obj.run_wrapper(*args, **kwargs)
  File "/home/romaing/src/turbinia/turbinia/workers/__init__.py", line 450, in run_wrapper
    if original_result_id != self.result.id:
AttributeError: 'NoneType' object has no attribute 'id'
[ERROR] Task 06d539c4cfac49c5a0ccf58addb3bc1d failed.
Traceback (most recent call last):
  File "/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/psq/task.py", line 90,
in execute
    result = self.call_func()
  File "/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/psq/task.py", line 104$
 in call_func
    return self.f(*self.args, **self.kwargs)
  File "/home/romaing/src/turbinia/turbinia/task_manager.py", line 73, in task_runner
    return obj.run_wrapper(*args, **kwargs)
  File "/home/romaing/src/turbinia/turbinia/workers/__init__.py", line 450, in run_wrapper
    if original_result_id != self.result.id:
AttributeError: 'NoneType' object has no attribute 'id'
^[[5~^C^C^C^C^C^C^C^C^C^C

$ turbiniactl server
[DEBUG] Recieved pubsub message: {"recipe": {}, "request_id": "877b2ac92f664b918921520726b5d396", "type": "Turbinia
Request", "context": {}, "evidence": [{"processed_by": [], "description": null, "tags": {}, "local_path": "/var/log
/dmesg", "copyable": false, "source": null, "saved_path": null, "cloud_only": false, "request_id": null, "config":
{}, "type": "Directory", "saved_path_type": null, "name": "/var/log/dmesg"}]}
[INFO] Processing PubSub message<PHONE_NUMBER>34310
[INFO] Received evidence [Directory:/var/log/dmesg:/var/log/dmesg] from PubSub message.
[INFO] Adding new evidence: Directory:/var/log/dmesg:/var/log/dmesg
[INFO] Adding StatJob job to process /var/log/dmesg
[INFO] Writing new task StatTask into Datastore
[INFO] Adding PSQ task StatTask with evidence /var/log/dmesg to queue
[INFO] Adding PlasoJob job to process /var/log/dmesg
[INFO] Writing new task PlasoTask into Datastore
[INFO] Adding PSQ task PlasoTask with evidence /var/log/dmesg to queue
[DEBUG] Task 1693f13511f24a9c93ce72788fe1f2c7 not yet created
[DEBUG] Task 06d539c4cfac49c5a0ccf58addb3bc1d not yet created
[INFO] 2 Tasks still outstanding.
[DEBUG] Updating Task StatTask in Datastore
[DEBUG] Updating Task PlasoTask in Datastore
[WARNING] Task 1693f13511f24a9c93ce72788fe1f2c7 failed.
[WARNING] Task 06d539c4cfac49c5a0ccf58addb3bc1d failed.
[INFO] 0 Tasks still outstanding.
[DEBUG] Updating Task StatTask in Datastore
[DEBUG] Updating Task PlasoTask in Datastore

$ turbiniactl -w -a -p 10 directory -l /var/log/dmesg
/home/romaing/venvs/turbinia-dev/local/lib/python2.7/site-packages/google/auth/_default.py:66: Use
rWarning: Your application has authenticated using end user credentials from Google Cloud SDK. We recommend that mo
st server applications use service accounts instead. If your application continues to use end user credentials from
 Cloud SDK, you might receive a "quota exceeded" or "API not enabled" error. For more information about service ac$
ounts, see https://cloud.google.com/docs/authentication/.                                                            warnings.warn(_CLOUD_SDK_CREDENTIALS_WARNING)
[DEBUG] Setting up PSQ Task Manager requirements on project romaing-turbinia-dev
[DEBUG] Trying to create pubsub topic projects/romaing-turbinia-dev/topics/turbinia-dev                            [DEBUG] PubSub topic projects/romaing-turbinia-dev/topics/turbinia-dev already exists.
[DEBUG] Setup PubSub publisher at projects/romaing-turbinia-dev/topics/turbinia-dev
[INFO] Creating request 877b2ac92f664b918921520726b5d396 with evidence /var/log/dmesg                              [INFO] Published message<PHONE_NUMBER>34310 to topic turbinia-dev
[INFO] Waiting for request 877b2ac92f664b918921520726b5d396 to complete
[DEBUG] Calling Cloud Function [gettasks] with args [{u'instance': u'turbinia-dev', u'kind': u'TurbiniaTask', u're$uest_id': '877b2ac92f664b918921520726b5d396'}]
[INFO] 0 Tasks found, 0 completed. Waiting 10 seconds.
[DEBUG] Calling Cloud Function [gettasks] with args [{u'instance': u'turbinia-dev', u'kind': u'TurbiniaTask', u'request_id': '877b2ac92f664b918921520726b5d
<repeats forever>

Same "Unkillable" issue if Plaso is broken:
Traceback (most recent call last):
  File "/usr/bin/log2timeline.py", line 12, in <module>
    from plaso.cli import log2timeline_tool
  File "/usr/lib/python2.7/dist-packages/plaso/cli/log2timeline_tool.py", line 19, in <module>
    from plaso.cli import extraction_tool
  File "/usr/lib/python2.7/dist-packages/plaso/cli/extraction_tool.py", line 12, in <module>
    from plaso import parsers  # pylint: disable=unused-import
  File "/usr/lib/python2.7/dist-packages/plaso/parsers/__init__.py", line 9, in <module>
    from plaso.parsers import bsm
  File "/usr/lib/python2.7/dist-packages/plaso/parsers/bsm.py", line 74, in <module>
    class BSMParser(interface.FileObjectParser):
  File "/usr/lib/python2.7/dist-packages/plaso/parsers/bsm.py", line 90, in BSMParser
    IPV4_STRUCT = construct.UBInt32('ipv4')
AttributeError: 'module' object has no attribute 'UBInt32'
[INFO] Execution failed with status 1
[INFO] Execution failed with status 1
[WARNING] New file path [/mnt/turbinia/output/1534208316-acba9089469a431dbde6a57b3598906e-PlasoTask/worker-log.txt] already exists.
[INFO] Writing /mnt/turbinia/output/1534208316-acba9089469a431dbde6a57b3598906e-PlasoTask/worker-log.txt to GCS path output/1534208316-acba9089469a431dbde6a57b3598906e-PlasoTask/worker-log.txt
[DEBUG] Result close successful. Status is [Execution failed with status 1]
[DEBUG] Checking TurbiniaTaskResult for serializability
[INFO] Result check: Successful
[DEBUG] Returning original result object 6b364a20787e43c4946437f9a3732539 after task execution
^C^C^C^C^C^C

There are a few different things going on here:

An error that could potentially be caught early on, but isn't checked for.  This is what I'll keep this bug open to fix.
The error then causes Turbinia worker to behave badly (the error should be caught and returned instead of causing a stack trace).  I have another PR in progress to fix this.
PSQ being uninterruptible:  This is a known issue with PSQ, and unfortunately is the case whether or not there is a problem in the worker.  I can't seem to find the issue now, but I'll see if I can dig it up.  I added a note about this in the developer doc.

So for the third bullet point above, I ended up filing this bug: https://github.com/GoogleCloudPlatform/psq/issues/41
I think the previous issue I was thinking about was this one, but it's been closed since then:
https://github.com/GoogleCloudPlatform/psq/issues/17

FYI: For point 2 above, I have PR #235

I'm going to close this issue out now.  Issue 1 was fixed in #230, Issue 2 was fixed in #235, and issue 3 is tracked in  GoogleCloudPlatform/psq#41.


Menu doesn't work

On a (new, out of the box) SmartEVSE with firmware 3.0.1, the menu doesn't work.
Chrome devtools shows:
"caught ReferenceError: openNav is not defined  at HTMLSpanElement.onclick ((index):28:92)"
The function "openNav()" is defined in script.js, but that file isn't loaded from index.html

If you mean the web page menu, yes the "hamburger" does not work yet.
Currently it's only a status page that shows the Phase currents , Charging state and Errors (if any)

+1

function "openNav()" is defined in script.js, but that file isn't loaded from index.html

and script.js is present in spiffs, so simply adding 1 line to index.html would solve that:
<script src="script.js" type="text/javascript"></script>
However, I guess the subsequent js/html code isn't production ready yet...
Are there any prognoses on issue solving / further development ?
or should we all refer to the serkri fork ?

Yes, dingo35 is creating a stable 3.5 version, that will have a working webpage.
https://github.com/dingo35/SmartEVSE-3.5
Once that's stable, i'll program the controllers with this software by default.

From now on i'll program controllers with v3.6.0. You can of course upgrade the fw of your controller to this version.


update to version 1 compatible

also, tests for everything

Wow this is great! I'll look it over tonight

There's some formatting issues, but I'm not worried about them. It looks good to me after that comment above gets addressed

Awesome, thanks @calvinmetcalf for all of this


Normalize rustfmt.toml and CODING_STYLE.md

They differ. Having worked on multiple Rust codebases at this point, I would probably prefer deviating less.

Of:
max_width = 79
comment_width = 79
force_explicit_abi = false
fn_args_layout = "Compressed"
use_small_heuristics = "Max"
match_arm_blocks = false
reorder_imports = true
tab_spaces = 2
use_field_init_shorthand = true
trailing_comma = "Never"

Only those two are quite noticeable to somebody coming from most of the other rust projects:
tab_spaces = 2
trailing_comma = "Never"

fn_args_layout is also problematic as it's nightly rustfmt only, which makes it way more obnoxious to use.

I personally prefer to avoid trailing commas, though it is not for very rational reasons (I mostly think it looks ugly, and weird syntactically).

I prefer trailing_comma = "Vertical" only for the reason that the unified diffs for inserting an item into a long list don't contain any deleted lines.

I would argue that also max_width = 79 is problematic once we have editorconfig trying to be "helpful".


which infrastructure should i choose?

Section Number 3.4
Issue * i found a discrepancy between the figure and the text below. The figure seems to say add MHD to XDS or XCA infrastracture but the text it says "Where none of these conditions exist, then the choice could be XDS or MHDS based" but in the figure above if no infrastructure points only to MHDS*
Proposed Change If we want them to use MJHD or MHDS henge the text otherwise the figure should be changed according to the fact that if i have no infrastucture at all i can choose an XDS infrastructure
Priority: low

the diagram is bias... it was initially to help those focused on FHIR... where as those that don't care about FHIR should still feel that XDS and XCA are recommended.
new diagram needs to make this clear.

updated on call today


Service Connection not being used that is earlier created

For which extension would you like to add a feature request?
Question
I have created a new task in power BI releases pipelines-> power BI Action -Update Parameters
Here it is asking for service connection , so i already used one service connection in my earlier tasks, so i want to use that only. but is giving invalid error.
What to do now.

@SatyamGupta621428, with a new version of the task we also had to adjust the service connection. So It could be the case that it is different.
Normally any approved service connection should also pop up in the drop down and you can use the service connections over the different tasks. So please check the configurations of the tasks.

Now i am cloning tasks so that service connection issue is resolved. In PowerBi Actions Update parameters there is one field
Update parameter Json in which what value will go . I need help.

In the hover there should be a example.

I have used the JSON but it is giving invalid request when i am creating release. SO what can be the error??

Please include a variable called system.debug and set the value to true.

Bro error is coming again invalid request although i added system.debug variable and set its value to true

Could you then share more info. With the info share now I can not help you.


Understanding normalization and log transformation

[X] I have checked that this issue has not already been reported.
[X] I have confirmed this bug exists on the latest version of scanpy.
[ ] (optional) I have confirmed this bug exists on the master branch of scanpy.

This is probably a bug in my thinking, but naively I thought that sc.pp.normalize_total() normalizes counts per cell, thus allowing comparison of different cells by correcting for variable sequencing depth. However, the log transformation applied after normalisation seems to upset this relationship, example below. Why is this not problematic?
Incidentally, I first noticed this on my real biological dataset, not the toy example below.
Edit: relevant paper

We can show, mathematically, that if we normalize expression profiles to have the same mean across cells, the mean after the equation [log] transformation used for RNA-Seq data will not be the same, and it will depend on the detection rate...

And this one:

One issue of particular interest is that the mean of the log-counts is not generally the same as the log-mean count [1]. This is problematic in scRNA-seq contexts where the log-transformation is applied to normalized expression data.

Minimal code sample
>>> from anndata import AnnData
>>> import scanpy as sc
>>> import numpy as np
>>> adata = AnnData(np.array([[3, 3, 3, 6, 6],[1, 1, 1, 2, 2],[1, 22, 1, 2, 2], ]))
>>> X_norm = sc.pp.normalize_total(adata, target_sum=1, inplace=False)['X']
>>> X_norm_log = np.log1p(X_norm)
>>> X_norm_again = np.expm1(X_norm_log)
>>> adata.X.sum(axis=1)
array([21.,  7., 28.], dtype=float32)  # Different counts for each cell
>>> X_norm.sum(axis=1)
array([1., 1., 1.], dtype=float32)  # Normalisation means same counts for each cell
>>> X_norm_log.sum(axis=1)
array([0.90322304, 0.90322304, 0.7879869 ], dtype=float32)  # <<< Interested in this! Different counts for each cell
>>> X_norm_again.sum(axis=1)
array([1., 1., 1.], dtype=float32)  # Counts the same again

Versions
I'm not using the latest scanpy and anndata verions, but i don't think this will be different on the master branch

sc.logging.print_versions()
scanpy==1.4.5.post2 anndata==0.6.22.post1 umap==0.3.10 numpy==1.18.1 scipy==1.2.1 pandas==1.0.1 scikit-learn==0.22.1 statsmodels==0.11.0 python-igraph==0.8.0

An example with real data:

Code:
# Load the PBMC 3k data
adata = sc.read_10x_mtx(
    os.path.join(
        save_path, "filtered_gene_bc_matrices/hg19/"
    ),  # the directory with the `.mtx` file
    var_names="gene_symbols",  # use gene symbols for the variable names (variables-axis index)
)
adata.var_names_make_unique()

# Get counts
adata.obs["n_counts"] = adata.X.sum(axis=1).A1
sc.pp.normalize_per_cell(adata, counts_per_cell_after=1e4)
adata.obs["n_counts_normalized"] = adata.X.sum(axis=1).A1
sc.pp.log1p(adata)
adata.obs["n_counts_normalized_log"] = adata.X.sum(axis=1).A1

# Dim reduction
sc.tl.pca(adata, svd_solver="arpack")
sc.pp.neighbors(adata)
sc.tl.umap(adata)
sc.pl.umap(adata, color=["n_counts", "n_counts_normalized", "n_counts_normalized_log"])

@LuckyMD might be one of the people with the most informed opinion here

Hey @chris-rands,
This is a really interesting topic. You are absolutely correct that log transformation removes the perfect comparison of relative expression values that mean normalization provides. Aside from CPM normalization (as provided by sc.pp.normalize_total() not being a good normalization technique anyway (this is argued by any more advanced normalization methods paper, e.g., the scran pooling paper), there are a couple of things to consider here:

Do we even want relative expression counts?
What assumptions do downstream methods have on the distribution of expression values.

For the first question: relative gene expression values ignore differences in cell sizes/number of molecules in the cell. There are some molecules whose numbers scale with the size of the cell, and others that don't (e.g., many housekeeping genes). Choosing relative over absolute expression values to compare gene expression across cells would be helpful to compare expression of those genes that scale with size, but not the others.... so there's not really a perfect answer here. Thus, removing all effects of total counts may not be the desirable outcome.
Secondly, many downstream methods assume normally distributed expression data (e.g., DE methods like: t-tests, limma, MAST, or several batch correction/data integration methods). Log transformation is used as a variance stabilization to approximate a normal distribution (quite often poorly, but better than without). This leads to many methods performing better with log transformation.
IMO, the ideal approach is probably something like scVI, GLMPCA, or scTransform, where you fit a model directly to the count data and use the residuals to describe the data. This would address both steps of normalization and variance stabilization at the same time. If we have a good model to describe the data, the residuals should quantify the biological variance + normally distributed noise.
Overall, I would use other normalization approaches than CPM, and use log-transformation with anything that uses size factors that scale per-cell expression values.
Note also that the effect described in the second paper you mention (from Aaron Lun) will mainly be relevant when you have biased distributions of sequencing depth between two samples that you are comparing. If the size factors are similarly distributed between both conditions, then the DE effect will not be so dramatic (as far as I understood it anyway).

Thank you very much @LuckyMD for those insightful comments, that gives me plenty to think about beyond my original question about the mean normalisation being 'distorted' by log transformation. I had not considered how tricky a problem normalisation is. This actually makes me feel that perhaps the long term solution will be mostly experimental rather than computational, through developing better spike-ins. Anyway, I guess this discussion is not a Scanpy issue, so I will close this, but I appreciate your thoughts.

I'm not sure spike-ins will be the solution, as there will probably always be differences due to tissue preparation and cell lysis efficiency as well, which spike-ins can't model... at least as far as I understand the limitations here. I'd be curious if you think otherwise though :).

Good point that spike-ins may not always behave like endogenous transcripts. However, since spike-ins can account for the later biases in the workflow (e.g. variable sequencing depth, capture efficiencies, amplification etc.), I do think that they are still an important part of the solution and a step closer to absolute quantification- would you disagree?

I'm not sure spike-ins are so helpful if they only account for part of the technical effects that have to be normalized out. In the end you will have the same problem with the remaining effects you are not capturing on which you don't have a good experimental handle. Unless you can spike into a tissue directly somehow? I quite like the idea of spike-in cells though for batch effects, but those have similar limitations as well.
In general, I'm yet to see a spike-in approach that if modeled and used to normalize your data would compare well against a good normalization method. That doesn't mean it doesn't exist... but it seems that since droplet-based techniques took off spike-ins are being avoided as they just increase sequencing costs and the larger droplet-based datasets allow us to use model-based normalization techniques.

Unless you can spike into a tissue directly somehow?

You can transfect RNA into cells, but the amount of RNA taken up will not be very consistent, so unlikely to be useful for quantitative information. Do you have an idea for how much bias is actually introduced by tissue prep and cell lysis?
Anyway, all the scRNA-seq data I have actually worked with is droplet based and I certainly agree in the value of model-based normalisation techniques. I suspect most new scanpy users will just follow the tutorial and use sc.pp.normalize_total() so thanks again for highlighting the limitations of this


build for multiple dir/level project

a project web api
├── apps
│   ├── app.pyx
│   ├── libs
│   │   └── init.pyx
│   └── resources
│       ├── api.pyx
│       └── __init__.pyx
├── main.py
└── setup.py

main.pyx
from apps.app import create_app

app = create_app()

apps/app.pyx
from fastapi import FastAPI

from apps.libs.init import init

def create_app():
    app = FastAPI()
    init(app)
    return app

apps/libs/init.pyx
from apps.resources import register_routers

def init(app):
    register_routers(app)

apps/resources/__init__.pyx
from fastapi import FastAPI

from .api import router

def register_routers(app: FastAPI):
    app.include_router(router, '/api', tags=['test'])

apps/resources/api.pyx
from fastapi import APIRouter

router = APIRouter()

@router.get('/index')
def index():
    return {'success': True}

how to write a setup.py to make it and run this project demo?

Please ask support questions on the cython-users mailing list (or elsewhere) instead of the issue tracker.


[KYUUBI #2752] Kyuubi Spark TPC-DS Connector - configurable catalog's name by initialize method

Why are the changes needed?

fix #2752
How was this patch tested?

[x] Add some test cases that check the changes thoroughly including negative and positive cases if possible

[ ] Add screenshots for manual tests if appropriate

[x] Run test locally before make a pull request

Please rebase and resolve the conflicts

Codecov Report

Merging #2753 (21810f3) into master (ff1d7ec) will increase coverage by 0.02%.
The diff coverage is 50.00%.

@@             Coverage Diff              @@
##             master    #2753      +/-   ##
============================================
+ Coverage     61.74%   61.76%   +0.02%
- Complexity      385      390       +5
============================================
  Files           433      434       +1
  Lines         20345    20378      +33
  Branches       2760     2767       +7
============================================
+ Hits          12562    12587      +25
- Misses         6548     6554       +6
- Partials       1235     1237       +2

Impacted Files
Coverage Δ

...ubi/spark/connector/tpch/TPCHStatisticsUtils.scala
42.42% <0.00%> (+3.03%)
:arrow_up:

...pache/kyuubi/spark/connector/tpch/SparkUtils.scala
37.03% <37.03%> (ø)

...he/kyuubi/spark/connector/tpch/TPCHBatchScan.scala
79.10% <75.00%> (-0.27%)
:arrow_down:

...he/kyuubi/spark/connector/tpcds/TPCDSCatalog.scala
43.75% <100.00%> (+3.75%)
:arrow_up:

...ache/kyuubi/spark/connector/tpch/TPCHCatalog.scala
42.42% <100.00%> (+3.03%)
:arrow_up:

...mon/src/main/scala/org/apache/kyuubi/Logging.scala
39.72% <0.00%> (-1.37%)
:arrow_down:

.../org/apache/kyuubi/operation/KyuubiOperation.scala
68.00% <0.00%> (-1.34%)
:arrow_down:

...ache/kyuubi/operation/KyuubiOperationManager.scala
85.24% <0.00%> (+1.63%)
:arrow_up:

... and 2 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dbe315e...21810f3. Read the comment docs.

Thanks, merging to master


Use async interface.

Codecov Report
Attention: 83 lines in your changes are missing coverage. Please review.

Comparison is base (0abdf57) 60.34% compared to head (2c2948e) 45.00%.
Report is 3 commits behind head on main.

:exclamation: Your organization needs to install the Codecov GitHub app to enable full functionality.
Additional details and impacted files
@@             Coverage Diff             @@
##             main      #33       +/-   ##
===========================================
- Coverage   60.34%   45.00%   -15.35%
===========================================
  Files          19       19
  Lines         807      820       +13
  Branches       90       92        +2
===========================================
- Hits          487      369      -118
- Misses        270      413      +143
+ Partials       50       38       -12

Files
Coverage Δ

src/include/mesg_factory.hpp
100.00% <ø> (ø)

src/include/messaging_if.hpp
80.00% <100.00%> (-20.00%)
:arrow_down:

src/lib/data_service.hpp
100.00% <ø> (ø)

src/lib/grpc_server.cpp
82.05% <100.00%> (-5.13%)
:arrow_down:

src/lib/messaging.hpp
0.00% <ø> (ø)

src/lib/service.cpp
47.13% <100.00%> (-15.00%)
:arrow_down:

src/lib/service.hpp
25.00% <ø> (ø)

src/lib/mesg_factory.cpp
60.00% <0.00%> (-25.72%)
:arrow_down:

src/lib/messaging.cpp
43.39% <38.05%> (-20.13%)
:arrow_down:

... and 3 files with indirect coverage changes

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

@hkadayam additional we should review the NullAsyncResult; right now we return folly::Unit() which I think could provide more context.


The public wekan boards should be linked

The public wekan boards should be linked into the docs.

We switched to using GitHub Projects to manage our planning & refinement. See https://scs.community/tech/2022/04/07/github-projects/


[4.0.5-4.0.10] Static Semantics: IsValidSimpleAssignmentTarget for MemberExpression

This is related in #504.
The types which IsValidSimpleAssignmentTarget returns true is not only identifiers.
MemberExpression and SuperProperty returns true, too.
However, acorn 4.0.5-4.0.10 throws syntax error on the following cases:
(obj.a) = 1;
(obj[0]) = 1;
(f().a) = 1;
(f()[0]) = 1;
(new f().a) = 1;
(new f()[0]) = 1;
class A extends B {
    foo() {
        (super.a) = 1;
        (super[0]) = 1;
    }
}

Note: NewTarget is similar one but it returns false.
(new.target) = 1  // SyntaxError!

https://www.ecma-international.org/ecma-262/7.0/#sec-static-semantics-static-semantics-isvalidsimpleassignmenttarget

You're right that foo.bar should probably be a simple assignment target, but I don't see how that follows from the rule that you linked -- that only seems to talk about memberexpressions whose object is a call expression, another memberexpression, or super. Am I understanding something incorrectly?

I could change the code to simply accept every MemberExpression as a simple assignment target, but I'd like to first understand what on earth the standard is trying to say there.

@marijnh Basically yes - for pre-ES6 assignment targets you could use parenthesized lhs (identifier or member expression), while in ES6 object and array patterns are not allowed. This is one of complexities that we discussed a while back as hard cases in fixing #255 that kept that issue open for so long (and now they backfired).

All this discussion and cases were in #433 fwiw.

for pre-ES6 assignment targets you could use parenthesized lhs

Yes, agreed, that's what makes sense and what the various parsers seem to be doing, but how does that follow from the standard, and is there any reason the standard defines this in such an opaque way?

Pushed the simple fix and released 4.0.11. Still very interested in any clarifications regarding the relevant spec text.

Thank you very much!

that only seems to talk about memberexpressions whose object is a call expression, another memberexpression, or super.

The MemberExpression production can be a PrimaryExpression. Then the PrimaryExpression production can be a IdentifierReference. So MemberExpression . IdentifierName includes IdentifierReference . IdentifierName.

The MemberExpression production can be a PrimaryExpression.

Ah right, thanks for explaining that!


Parsing errors

Hello, thanks for the library.
Here are some errors I get for both playstore and appstore reviews.
appstore reviews:
Error parsing app store reviews
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at Request._callback (/usr/lib/node_modules/@trademe/reviewme/appstorereviews.js:82:24)
    at Request.self.callback (/usr/lib/node_modules/@trademe/reviewme/node_modules/request/request.js:185:22)
    at emitTwo (events.js:126:13)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (/usr/lib/node_modules/@trademe/reviewme/node_modules/request/request.js:1154:10)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
    at IncomingMessage.<anonymous> (/usr/lib/node_modules/@trademe/reviewme/node_modules/request/request.js:1076:12)
    at Object.onceWrapper (events.js:313:30)
INFO: Received no reviews from App Store for<PHONE_NUMBER>) (gw)
Error parsing app store reviews
SyntaxError: Unexpected end of JSON input
    at JSON.parse (<anonymous>)
    at Request._callback (/usr/lib/node_modules/@trademe/reviewme/appstorereviews.js:82:24)
    at Request.self.callback (/usr/lib/node_modules/@trademe/reviewme/node_modules/request/request.js:185:22)
    at emitTwo (events.js:126:13)
    at Request.emit (events.js:214:7)
    at Request.<anonymous> (/usr/lib/node_modules/@trademe/reviewme/node_modules/request/request.js:1154:10)
    at emitOne (events.js:116:13)
    at Request.emit (events.js:211:7)
    at IncomingMessage.<anonymous> (/usr/lib/node_modules/@trademe/reviewme/node_modules/request/request.js:1076:12)
    at Object.onceWrapper (events.js:313:30)

google playstore reviews:
UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'replace' of undefined
    at Object.descriptionText [as fun] (/usr/lib/node_modules/@trademe/reviewme/node_modules/google-play-scraper/lib/app.js:145:51)
    at R.map (/usr/lib/node_modules/@trademe/reviewme/node_modules/google-play-scraper/lib/utils/scriptData.js:22:19)
    at XWrap.f (/usr/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:5955:28)
    at XWrap.@@transducer/step (/usr/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:392:25)
    at _arrayReduce (/usr/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:5137:46)
    at _reduce (/usr/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:5167:24)
    at map (/usr/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:5954:20)
    at /usr/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:586:23
    at Object.f2 [as map] (/usr/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:473:22)
    at extractFields (/usr/lib/node_modules/@trademe/reviewme/node_modules/google-play-scraper/lib/utils/scriptData.js:16:14)
(node:3476) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)
(node:3476) [DEP0018] DeprecationWarning: Unhandled promise rejections are deprecated. In the future, promise rejections that are not handled will terminate the Node.js process with a non-zero exit code.

using the latest version, at 3.0.0

Also seeing the same error for the Google Play store. App Store seems to be posting reviews fine. I built the docker image with the latest from master (4a5c1f1d9a033604c412a8704bc1574de3d6ab69)
(node:1) UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'replace' of undefined\n    at Object.descriptionText [as fun] (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/google-play-scraper/lib/app.js:143:51)\n    at R.map (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/google-play-scraper/lib/utils/scriptData.js:19:19)\n    at XWrap.f (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:5955:28)\n    at XWrap.@@transducer/step (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:392:25)\n    at _arrayReduce (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:5137:46)\n    at _reduce (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:5167:24)\n    at map (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:5954:20)\n    at /usr/local/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:586:23\n    at Object.f2 [as map] (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/ramda/dist/ramda.js:473:22)\n    at extractFields (/usr/local/lib/node_modules/@trademe/reviewme/node_modules/google-play-scraper/lib/utils/scriptData.js:13:14)

(node:1) UnhandledPromiseRejectionWarning: Unhandled promise rejection. This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). (rejection id: 1)

Found the same errors here as well. Now google play reviews doesn't work


xcode version 15.0 (15A240d) crashes on phones younger than iOS17

Describe the bug
clash：

let options = NWProtocolTCP.Options()

Thread 1: EXC_BAD_ACCESS (code=1, address=0x0)

same issue

I changed the deployment target version to iOS 13 (remove IPHONEOS_DEPLOYMENT_TARGET) and it's resolve the issue

Make sure you're on version 4.0.6. That version increments the iOS target version from 9 to 12.
NWProtocolTCP.Options requires iOS 12+ as a target when compiling in Xcode15

Make sure you're on version 4.0.6. That version increments the iOS target version from 9 to 12.
NWProtocolTCP.Options requires iOS 12+ as a target when compiling in Xcode15

podspec https://github.com/CocoaPods/Specs/blob/master/Specs/2/2/f/Starscream/4.0.6/Starscream.podspec.json

Make sure you're on version 4.0.6. That version increments the iOS target version from 9 to 12.
NWProtocolTCP.Options requires iOS 12+ as a target when compiling in Xcode15

podspec https://github.com/CocoaPods/Specs/blob/master/Specs/2/2/f/Starscream/4.0.6/Starscream.podspec.json

still crash on version 4.0.6
ios version: 15.7.2
xcode version: 15.0

let options = NWProtocolTCP.Options()

EXC_BAD_ACCESS (code=1, address=0x0)

让选项 = NWProtocolTCP.Options()
线程 1：EXC_BAD_ACCESS（代码=1，地址=0x0）
这个问题怎么弄，项目现在已运行就闪退

描述错误
冲突：

让选项 = NWProtocolTCP.Options()

线程 1：EXC_BAD_ACCESS（代码=1，地址=0x0）

解决了吗？求方案

We are using SPM, but it appears to be the same issue. Minimum deployment target is set to iOS 13.

I try fix it
first
    public convenience init(request: URLRequest, certPinner: CertificatePinning? = FoundationSecurity(), compressionHandler: CompressionHandler? = nil, useCustomEngine: Bool = true) {
        if #available(macOS 10.15, iOS 13.0, watchOS 6.0, tvOS 13.0, *), !useCustomEngine {
            self.init(request: request, engine: NativeEngine())
        }
// don't use network transport engine
//        else if #available(macOS 10.14, iOS 12.0, watchOS 5.0, tvOS 12.0, *) {
//            self.init(request: request, engine: WSEngine(transport: TCPTransport(), certPinner: certPinner, compressionHandler: compressionHandler))
//        }
        else {
            self.init(request: request, engine: WSEngine(transport: FoundationTransport(), certPinner: certPinner, compressionHandler: compressionHandler))
        }
    }

last
// initial socket object with useCustomEngine: false
self.socket = WebSocket(request: request,useCustomEngine: false)

我将部署目标版本更改为 iOS 13（删除IPHONEOS_DEPLOYMENT_TARGET），它解决了这个问题

But every time I re-use 'pod install', it goes back to the way it used to be.😂

我将部署目标版本更改为 iOS 13（删除IPHONEOS_DEPLOYMENT_TARGET），它解决了这个问题

But every time I re-use 'pod install', it goes back to the way it used to be.😂

there pics not display , the link is private
if you change code in pods , you can get origin code when  you pod install.
because pod install will pull origin code , not you changed code


ls: reading directory '.': Invalid argument (share mounted via drvfs)

Microsoft Windows [Version 10.0.16176] 64-bit

Running in a Windows 10 VM in Parallels on Mac OS
Y: is a mapped network drive to a Parallels shared folder

$ sudo mkdir /mnt/y
$ sudo mount -t drvfs Y: /mnt/y
$ cd /mnt/y
$ ls
ls: reading directory '.': Invalid argument

Note that other operations like cd and cat do work, so the mount operation was successful.
Strace output:
(strace -ff ls)
execve("/bin/ls", ["ls"], [/* 16 vars */]) = 0
brk(NULL)                               = 0x8e9000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f53543f0000
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=20939, ...}) = 0
mmap(NULL, 20939, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f53543ea000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libselinux.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260Z\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=130224, ...}) = 0
mmap(NULL, 2234080, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5353dd0000
mprotect(0x7f5353def000, 2093056, PROT_NONE) = 0
mmap(0x7f5353fee000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1e000) = 0x7f5353fee000
mmap(0x7f5353ff0000, 5856, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f5353ff0000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\t\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=1864888, ...}) = 0
mmap(NULL, 3967392, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5353a00000
mprotect(0x7f5353bbf000, 2097152, PROT_NONE) = 0
mmap(0x7f5353dbf000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1bf000) = 0x7f5353dbf000
mmap(0x7f5353dc5000, 14752, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f5353dc5000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpcre.so.3", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000\25\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=456632, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f53543e0000
mmap(NULL, 2552072, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5353790000
mprotect(0x7f53537fe000, 2097152, PROT_NONE) = 0
mmap(0x7f53539fe000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x6e000) = 0x7f53539fe000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\240\r\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14608, ...}) = 0
mmap(NULL, 2109680, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5353580000
mprotect(0x7f5353583000, 2093056, PROT_NONE) = 0
mmap(0x7f5353782000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7f5353782000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260`\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=138696, ...}) = 0
mmap(NULL, 2212904, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7f5353360000
mprotect(0x7f5353378000, 2093056, PROT_NONE) = 0
mmap(0x7f5353577000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x17000) = 0x7f5353577000
mmap(0x7f5353579000, 13352, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7f5353579000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f53543d0000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f53543c0000
arch_prctl(ARCH_SET_FS, 0x7f53543c0800) = 0
mprotect(0x7f5353dbf000, 16384, PROT_READ) = 0
mprotect(0x7f5353577000, 4096, PROT_READ) = 0
mprotect(0x7f5353782000, 4096, PROT_READ) = 0
mprotect(0x7f53539fe000, 4096, PROT_READ) = 0
mprotect(0x7f5353fee000, 4096, PROT_READ) = 0
mprotect(0x61d000, 4096, PROT_READ)     = 0
mprotect(0x7f5354225000, 4096, PROT_READ) = 0
munmap(0x7f53543ea000, 20939)           = 0
set_tid_address(0x7f53543c0ad0)         = 2019
set_robust_list(0x7f53543c0ae0, 24)     = 0
rt_sigaction(SIGRTMIN, {0x7f5353365b50, [], SA_RESTORER|SA_SIGINFO, 0x7f5353371390}, NULL, 8) = 0
rt_sigaction(SIGRT_1, {0x7f5353365be0, [], SA_RESTORER|SA_RESTART|SA_SIGINFO, 0x7f5353371390}, NULL, 8) = 0
rt_sigprocmask(SIG_UNBLOCK, [RTMIN RT_1], NULL, 8) = 0
getrlimit(RLIMIT_STACK, {rlim_cur=8192*1024, rlim_max=8192*1024}) = 0
statfs("/sys/fs/selinux", 0x7fffe09b1510) = -1 ENOENT (No such file or directory)
statfs("/selinux", 0x7fffe09b1510)      = -1 ENOENT (No such file or directory)
brk(NULL)                               = 0x8e9000
brk(0x90a000)                           = 0x90a000
open("/proc/filesystems", O_RDONLY)     = 3
fstat(3, {st_mode=S_IFREG|0444, st_size=0, ...}) = 0
read(3, "nodev      sysfs\nnodev      root"..., 4096) = 367
read(3, "", 4096)                       = 0
close(3)                                = 0
open("/usr/lib/locale/locale-archive", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=1668976, ...}) = 0
mmap(NULL, 1668976, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5354228000
close(3)                                = 0
ioctl(1, TCGETS, {B38400 opost isig icanon echo ...}) = 0
ioctl(1, TIOCGWINSZ, {ws_row=56, ws_col=147, ws_xpixel=0, ws_ypixel=0}) = 0
open(".", O_RDONLY|O_NONBLOCK|O_DIRECTORY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFDIR|0777, st_size=512, ...}) = 0
getdents(3, 0x8eecd0, 32768)            = -1 EINVAL (Invalid argument)
open("/usr/share/locale/locale.alias", O_RDONLY|O_CLOEXEC) = 4
fstat(4, {st_mode=S_IFREG|0644, st_size=2995, ...}) = 0
read(4, "# Locale name alias data base.\n#"..., 4096) = 2995
read(4, "", 4096)                       = 0
close(4)                                = 0
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_US/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en.utf8/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en/LC_MESSAGES/coreutils.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, "ls: ", 4)                     = 4
write(2, "reading directory '.'", 21)   = 21
open("/usr/share/locale/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_US.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_US.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en_US/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en.UTF-8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en.utf8/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
open("/usr/share/locale-langpack/en/LC_MESSAGES/libc.mo", O_RDONLY) = -1 ENOENT (No such file or directory)
write(2, ": Invalid argument", 18)      = 18
write(2, "\n", 1)                       = 1
close(3)                                = 0
close(1)                                = 0
close(2)                                = 0
exit_group(2)                           = ?
+++ exited with 2 +++

Hi @jgoz
Could you please record detailed logs using the steps from https://github.com/Microsoft/BashOnWindows/blob/master/CONTRIBUTING.md#8-detailed-logs, and post them here?
Thanks!

logs-issue-1954.zip
@SvenGroot Zipped and attached. If there's anything else you need, just let me know.

@SvenGroot do you have any updates for this issue? Also see #1959 which is similar to this for VMware. Thanks!

All of these issues are caused by file systems that either do not support some of the functionality DrvFs is trying to use, or behave in unexpected ways. Officially, DrvFs currently support NTFS, ReFS, FAT, CDFS, and SMB.
We will try to accommodate other file systems when possible, but it's unlikely we will get to all of them.

@jgoz, @laktak - Thanks for your comments. If you would like to see support for more file systems (other than the ones mentioned by @SvenGroot), it would be good if you can open a User Voice ticket for that.

@sunilmut OK, posted here

Support for NFS would greatly be appreciated! I have a NFS drive mounted on my Windows system directly. I used
mount -t drvfs Z: /mnt/z
to mount the drive. I'm able to go to directories on the NFS I know exist (and get an error message for bad directories), and I'm able to read the contents of files I know to exist (and get error messages for those that do not)
However, just like @jgoz ls does not work because the getdent call fails

Hi @jgoz,
did you solve it? How?
Cheers

@enovella No, the issue is still open.

Same issue here when I enable folder sharing in remote desktop connection(RDS). In windows I can access "\tsclient\blah", and under Windows subsystem for Linux I can mount "mount -t drvfs '\TSCLIENT\blah' /mnt/blah" but I just cannot explore the content of  /mnt/blah.

I have a mount with type cifs and got the strange result that ls works on one sub-directory and gives the error discussed here on another.
The only real difference I can see is the number of files. It does not work in directories with more than about 70.000 files.

Now I found a strange solution: Option vers was on default which is 1.0 according to the man-page. But after I have set this to 1.0 explicitely it worked on all sub-directories, inlcuding the large ones.

@EmilObermayr What do you mean "option vers"?

My usb drive was working in the ubuntu subsystem in windows, and it suddenly stopped. I can see my files in Windows explorer, but when I try to access /mnt/d I get the same error. sigh...

I'm seeing the same issue on Windows 10 Build 18990 after joining the Insiders Program and installing said build about an hour ago. I did this specifically to enable WSL 2 but I seem to have gone 2.5 years back in time instead of 6 months forward! What gives?!

I as well am having this issue

Also have the same issue...

The problem (from the strace output) is in getdents ( getdents, getdents64 - get directory entries) system call, right? See "man getdents". Apparently the count parameter is too large (32768) whereas WSL only supports 10000?
getdents(3, 0x8eecd0, 32768)            = -1 EINVAL (Invalid argument)

I'm having the same issue :|

I have the same issue after updating windows to 2004 (may 2020 update).
I'm mounting a Samba network drive with drvfs that used to work before the update. Like the OP it fails on getdents().

Updated the disto (ubuntu 18.04) to wsl2: probem present
Rolled-back distro to wsl1: probem present
Rolled-back windows to 1909: problem absent

kernel info with issue present:
Linux HOST 4.4.0-19041-Microsoft #1-Microsoft Fri Dec 06 14:06:00 PST 2019 x86_64 x86_64 x86_64 GNU/Linux

kernel info with issue absent:
Linux HOST 4.4.0-18362-Microsoft #836-Microsoft Mon May 05 16:04:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux

So I guess there's a regression in drvfs somewhere. Hope this info can help.

Had the same issue on Windows 10 on an NFTS /mnt/e drive (with same privileges/configs as my /mnt/d and /mnt/c drives, where there was no issue). It started after I ran rm (no software updates since 5/16/2020, happened a few minutes ago on 6/6/2020).
Restart solved this issue. Kernel info:
uname -a
Linux WindowsFelix 4.4.0-18362-Microsoft #836-Microsoft Mon May 05 16:04:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux

I still have this issue when attempting to work within a Google File Stream directory. I imagine File Stream operates/emulates as a network drive so I imagine the issues are similar. cd works, ls doesn't, cd autocomplete doesn't. I'm unable to use make correctly.

Also here to confirm this is an issue with BUILD 2004....
I upgraded to it and one of my drives could not be seen when doing an LS (cd /mnt/x/ and then ls).
Uninstalled BUILD 2004 and boom, ls worked and was able to see files again.... Only took my 3 hours to find this thread to solve this issue.

restarting the whole desktop solves the problem until you physically disconnect the usb HDD again. we have to restart everytime we disconnect

I just updated Windows to 1909 (18363.1049) and now I'm getting this error with a mounted mapped network drive. Sigh. Typical Microsoft. Updates just fix things, they always have to break something as well...

Some more info:

the bug appeared on 1909 after updates installed KB4574727 which updated the wsl kernel
rolling that update back (replaced by KB4565351) fixed the issue (uname -a: Linux HOST 4.4.0-18362-Microsoft #836-Microsoft Mon May 05 16:04:00 PST 2020 x86_64 x86_64 x86_64 GNU/Linux)

Also I noticed that for me, when the bug is present for samba drives, it depends on smbd version at the other end:

bug present with smbd 3.6.25
bug absent with smbd 4.4.16 and 4.8.0

Don't know if that's true in general however.
This is now preventing me for updating my machine since I need network drives access in wsl. If there's a workaround somewhere while waiting for a fix that'd be great.
Thanks

Also I noticed that for me, when the bug is present for samba drives, it depends on smbd version at the other end:

bug present with smbd 3.6.25

bug absent with smbd 4.4.16 and 4.8.0

Don't know if that's true in general however.
This is now preventing me for updating my machine since I need network drives access in wsl. If there's a workaround somewhere while waiting for a fix that'd be great.

Digging a little further it looks like my drive that won't work uses SMB1 dialect and the ones that work use SMB2 or 3 (checked with Get-SmbConnection in powershell).
Turns out wsl2 supports directly mounting cifs shares where we can specify the version to use:
sudo apt install cifs-utils
sudo mount -t cifs -o user=pif,pass=paf,vers=1.0 //server/share /mnt/cifs

works as a workaround in my case.

@Clonkex I found this issue was caused for me by the offline file sync feature of the mapped network drive. If I click "work online" under the easy access dropdown in the explorer page I was able to access the files. I enabled offline files around the same time as the update so it is hard to say if it is related.

@alexbrodersen Unfortunately it looks like this for me:

Hving this issue when vim is open in the dir.

Having the same issue!
Restarting the whole system works for me, but you never know when is the next 'sudden stop'
Sigh

sudo umount -l /mnt/d

Running into the same issue: offline files are not available in WSL when mounted volume is actually offline.

Same issue still here - Win10PRO + WSL running inside VMWareWorkstation PRO on a WIN11 host, mounted VM shared drive (folder shared from host to virtual machine, then mounted in WSL Ubuntu) - root can't ls files, subdirectories are working OK. Mounting a subdirectory from the netdrive results in the same behavior - its subdirectories are accessible, can't stat mount root (so it is a problem just for a mount root, not a source FS).


[DO NOT MERGE] Conversational Analytics Dashboard referencing 4.2 telemetry

Description

Moving PowerBI sample dashboards to a centralized /Microsoft/AI/solutions/analytics directory that will be referenced in related README files. I'm keeping and renaming the old Example Power Bi Dashboard to Power Bi Dashboard, as it works with 4.1 telemetry. The Conversational Analytics Example Dashboard is using several tabs from the VPA work, allowing users to gain insights into their dialog health and conversation properties.

Overall Usage tab is refactored from the previous  dashboard.
❗All Dialogs Overview will show all a count of all dialogs captured, as well as their proportionate status (Abandoned/Canceled/Done/Started are provided by Waterfall SDK telemetry)
❗Dialog Overview shows the previous graphs per dialog, by date. You can also gain insights into the average bot/user turn counts each day.
❗Conversation Metrics will highlight the average numbers of conversations per unique users and average duration.
❗Conversation Drill Down tab shows how to drill down by conversation to view all related user utterances, bot responses, and dialogs triggered during the same session.
❗Transcript tab demonstrates how to show a defined From here, you can select a specific conversation to gain further insights on how a conversation went between your bot and it's users.
Demographics & Usage merges the Demographics & Users and Conversations tabs from the previous  dashboard.
Word Cloud merges the Word Cloud and Common Words tabs from the previous dashboard.
Sentiment remains from the previous dashboard.
QnAMaker Insights remains from the previous dashboard, plus new Answer Found filter.

❗
New tab

Related Issue

#134
#354  - new telemetry tracked here
Microsoft/BotBuilder#5148
Testing Steps

Checklist

[x] I have commented my code, particularly in hard-to-understand areas - n/a
[x] I have added or updated the appropriate unit tests - n/a
[x] I have updated related documentation - coming in related ET/CST/VA Middleware PRs.

If this contains changes that needs to be replicated between the Enterprise Template <-> Virtual Assistant

[X] A duplicate issue is filed to track future  work. #134 #135 #354

If you have updated responses or .lu files:

[x] All languages have been updated - n/a
[x] You have tested deployment with your new models - n/a

I'm open to feedback on if there's a need to keep the old dashboard?

@darrenj .pbit file provided instead - it will now ask for an application ID.


Teach TypeReconstruction how to look through associated types of the …

Code to make TypeReconstruction able to discover associated types of Self protocols, e.g. Iterator in
extension Collection { func foo(_ x: Iterator) {...} }

…Self protocol

@swift-ci please smoke test

Failed in llbuild:
":0: error: missing input 'input' and no rule to build it
/bin/sh -c true
:0: error: build had 1 command failures
note: command had no output on stdout or stderr
error: command failed with exit status: 1"
I don't think this is related to my change...

@swift-ci please smoke test Linux platform

Looks good for CCC.


install script stopped working because of broken symlink

Describe the bug
install.sh is not correctly grabbing the release tag, which makes the download of the correct jar file and symlink to fail.
Any okta command from that point fails with this error.
Okta Password:
Error: Could not find or load main class com.okta.tools.WithOkta
Caused by: java.lang.ClassNotFoundException: com.okta.tools.WithOkta
Error: Could not find or load main class com.okta.tools.WithOkta
Caused by: java.lang.ClassNotFoundException: com.okta.tools.WithOkta

To Reproduce
Steps to reproduce the behavior:

Download the install.sh file, wget https://github.com/oktadev/okta-aws-cli-assume-role/blob/master/bin/install.sh
Run the install script, sudo bash ./install.sh
check the output or run okta commands after installation.
You will see the broken symlink on the jar file.

❯ sudo bash install.sh -i
Installing into ~/.okta
Latest release JAR file: https://github.com/oktadeveloper/okta-aws-cli-assume-role/releases/download//okta-aws-cli-.jar
Fetching JAR file → ~/.okta/okta-aws-cli-.jar
Symlinking ~/.okta/okta-aws-cli.jar → okta-aws-cli-.jar
Creating example ~/.okta/config.properties

Notice the missing release tag after okta-aws-cli-
Expected behavior
Install script should install the correct jar.
Additional context
https://github.com/oktadev/okta-aws-cli-assume-role/blob/master/bin/install.sh#L83
this line here seems to be returning latest instead of the release tag.

👍

Seeing the same issue. Is there a workaround?

workaround is to manually download the install script, and replace the  releaseTag definition (which  does not parse properly) with the latest tag... currently,  it is v2.0.5. So:
# download
$ curl https://raw.githubusercontent.com/oktadeveloper/okta-aws-cli-assume-role/master/bin/install.sh > install.sh

# replace definition of releaseTag w/ following in install.sh:
releaseTag=v2.0.5

# proceed w/ install
$ PREFIX=~/.okta bash install.sh -i


where to switch "dark" and "light" model

after i download the lastest version, i can't find where to switch the “dark" and "light" theme, please kindly advise.
originally, it should located in "Appearance Settings", but now only have logo and system messages setup.

it is not implemented as far as I have seen it in the code (only for the modals)

We do not support a light mode anymore since the UI redesign. Simplifies the design of new screens and components. It will return in the future

I just came here to comment that I would greatly appreciate a light mode.  I searched through all github issues, and there are several that mention lightmode, but no open issues involving light mode.
It's an accessibility issue -- for me personally I have vision issues that make it so I find dark mode very difficult to see.
(Obviously, this is open source and an amazing product, and I'm just putting in a humble tiny vote for light mode and that's all.)


Fix: text_on_image Creator - Web design #102

#602

Creating a new pull request by creating a branch. I will submit a new PR now


⚠ Bofrost.de has degraded performance

In 0cc28d2, Bofrost.de (https://www.bofrost.de) experienced degraded performance:

HTTP code: 200
Response time: 1449 ms

Resolved: Bofrost.de performance has improved in 94d8dac after 6 minutes.


WIP: Federated following

This pull request implements ActivityPub-federated following so users on a Gitea instance can follow users on any other ActivityPub server (Gitea, Mastodon, PeerTube, Pleroma, etc) and vice versa.
Since #19133 hasn't been merged yet, this PR is currently meant for discussing high-level changes like remote users instead of reviewing the code. (If you want to view the code changes, I made a PR on my personal Gitea repository with that)
Why federated following?
Federated following seems like an odd choice for the first federated behavior for Gitea, but it is actually a good place to start:

Following uses only plain AP (ActivityPub) and doesn't require ForgeFed (which is unsupported by go-ap).
Federated following only requires implementing remote federated users. I say "only" because things like federated pull requests and issues also require handling remote repos which is a whole other tricky issue to resolve.
Federated following requires most of the core ActivityPub components like inboxes and outboxes to work, so this PR already does much of the most difficult work and supporting other activity types should be a lot easier.

Inboxes and outboxes
The first commit implements basic inbox and outbox handling. Since we are not implementing C2S ActivityPub for Gitea, the inbox is not persisted to the database and the outbox reuses the Action table for database storage.
Currently, go-ap does not yet support any of the ForgeFed activity types. As a result, this PR does not implement any type-specific processing (so it currently just returns an empty outbox).
In the future, we will add more ActionTypes for the various AP activity types that we wish to support.
Additional information: #5
Remote users
This PR currently stores remote users in the database using the new LoginType Federated, and stores the full<EMAIL_ADDRESS>as the username and the ActivityPub IRI as the user's website (which is very hacky and needs to be changed). When viewing a remote user's profile in the web UI, you are redirected to their instance.
Additional information: #2
UI
Currently, this PR only adds the backend code for federated following, so there is no UI (yet) for federated following. From the end user's perspective, that means they will at the moment only be able to follow Gitea accounts using non-Gitea AP software like Mastodon, since there is no UI to follow an AP account remotely from a Gitea instance.
TODO

[ ] Fix lint errors

[ ] Write integration tests

[ ] Implement a federated following UI

[ ] Depends on #19133

Current code review is happening at: https://gitea.com/Ta180m/gitea/pulls/6

I'm closing this pull request since I need to rebase the changes onto the Gitea main branch. I'll submit a new PR tomorrow.

(And that is not possible here, with an git push --force-with-lease )


[Bug]: it is not always safe to explicitly pass api_key to OpenAI

Bug Description
Calling VectorStoreIndex.from_documents() with a ServiceContext that uses an instance of OpenAI as the llm that explicitly passes the api_key rather than setting the OPENAI_API_KEY  environment variable, raises a nasty traceback complaining: "LocalProtocolError: Illegal header value b'Bearer '".
I note that even though this works:
llm = OpenAI(model="gpt-4", api_key="XXX")

response = llm.complete("Paul Graham is ")

print(response)

Continuing on with:
service_context = ServiceContext.from_defaults(llm=llm)

reader = SimpleDirectoryReader(input_dir="/some/directory")

docs = reader.load_data()

index = VectorStoreIndex.from_documents(docs, service_context=service_context)

Reliably fails with a very long stack trace (mine contains 121 LocalProtocolErrors, 6 APIConnectionErrors, a CancelledError and a RetryError).
The crux of which is the complaint about the header.
I haven't spelunked far enough to find the offending code. It may be in the openai client library. The errors being thrown are from the http-related dependencies that enforce a rule against trailing spaces in header values.
It seems that some code path, potentially a retry path, is (a) not getting passed the api key passed to OpenAI, and (b) then relying on the environment variable being set. When it is not, the resulting header is simply "Bearer ".
If I set the environment variable, this problem goes away.
Version
0.8.64.post1
Steps to Reproduce
See above.
Relevant Logs/Tracbacks
[[Too long]]

...

---------------------------------------------------------------------------
LocalProtocolError                        Traceback (most recent call last)
File ~/dev/company/project/.venv/lib/python3.11/site-packages/httpcore/_exceptions.py:10, in map_exceptions(map)
      9 try:
---> 10     yield
     11 except Exception as exc:  # noqa: PIE786

File ~/dev/company/project/.venv/lib/python3.11/site-packages/httpcore/_sync/http11.py:142, in HTTP11Connection._send_request_headers(self, request)
    141 with map_exceptions({h11.LocalProtocolError: LocalProtocolError}):
--> 142     event = h11.Request(
    143         method=request.method,
    144         target=request.url.target,
    145         headers=request.headers,
    146     )
    147 self._send_event(event, timeout=timeout)

File ~/dev/company/project/.venv/lib/python3.11/site-packages/h11/_events.py:96, in Request.__init__(self, method, headers, target, http_version, _parsed)
     94 else:
     95     object.__setattr__(
---> 96         self, "headers", normalize_and_validate(headers, _parsed=_parsed)
     97     )
     98 if not _parsed:

File ~/dev/company/project/.venv/lib/python3.11/site-packages/h11/_headers.py:164, in normalize_and_validate(headers, _parsed)
    163     validate(_field_name_re, name, "Illegal header name {!r}", name)
--> 164     validate(_field_value_re, value, "Illegal header value {!r}", value)
    165 assert isinstance(name, bytes)

File ~/dev/company/project/.venv/lib/python3.11/site-packages/h11/_util.py:91, in validate(regex, data, msg, *format_args)
     90         msg = msg.format(*format_args)
---> 91     raise LocalProtocolError(msg)
     92 return match.groupdict()

LocalProtocolError: Illegal header value b'Bearer '

...

File ~/dev/company/project/.venv/lib/python3.11/site-packages/openai/_base_client.py:890, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    888 except Exception as err:
    889     if retries > 0:
--> 890         return self._retry_request(
    891             options,
    892             cast_to,
    893             retries,
    894             stream=stream,
    895             stream_cls=stream_cls,
    896         )
    897     raise APIConnectionError(request=request) from err
    899 return self._process_response(
    900     cast_to=cast_to,
    901     options=options,
   (...)
    904     stream_cls=stream_cls,
    905 )

File ~/dev/company/project/.venv/lib/python3.11/site-packages/openai/_base_client.py:925, in SyncAPIClient._retry_request(self, options, cast_to, remaining_retries, response_headers, stream, stream_cls)
    921 # In a synchronous context we are blocking the entire thread. Up to the library user to run the client in a
    922 # different thread if necessary.
    923 time.sleep(timeout)
--> 925 return self._request(
    926     options=options,
    927     cast_to=cast_to,
    928     remaining_retries=remaining,
    929     stream=stream,
    930     stream_cls=stream_cls,
    931 )

File ~/dev/company/project/.venv/lib/python3.11/site-packages/openai/_base_client.py:897, in SyncAPIClient._request(self, cast_to, options, remaining_retries, stream, stream_cls)
    889     if retries > 0:
    890         return self._retry_request(
    891             options,
    892             cast_to,
   (...)
    895             stream_cls=stream_cls,
    896         )
--> 897     raise APIConnectionError(request=request) from err
    899 return self._process_response(
    900     cast_to=cast_to,
    901     options=options,
   (...)
    904     stream_cls=stream_cls,
    905 )

APIConnectionError: Connection error.

@mattgiles I am unable to reproduce
>>> from llama_index.llms import OpenAI
>>> llm = OpenAI(model="gpt-4", api_key="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")
>>> from llama_index import Document, ServiceContext, VectorStoreIndex
>>> service_context = ServiceContext.from_defaults(llm=llm)
>>> index = VectorStoreIndex.from_documents([Document.example()], service_context=service_context)
>>> response = index.as_query_engine().query("What can you tell me about LLMs?")
>>>

Versions:
llamaindex=0.8.63.post1
openai=1.1.1

ah wait, my env might be set, one sec

I can confirm I still can't reproduce (although I think you need to set embed_model to be something local, otherwise it uses openai and fails for embeddings)
service_context = ServiceContext.from_defaults(llm=llm, embed_model="local")

I can confirm that your example reproduces the issue for me:

Your example reproduces for me:

@mattgiles even if you set the embed model to be local? The error is happening on from_documents() the the LLM is not actually the issue.
We used to automatically default to local embeddings, but it seems like somehow that broke

After installing transformers and torch, that does resolve the issue as well.

In my experience, "LocalProtocolError: Illegal header value b'Bearer '" points to having an invalid API KEY for OpenAI.

I think this issue is old enough to be considered fixed :) Going to close this one out.


[TACHYON-442] Apply PluginManagement to cleanup pom.xml in each module

https://tachyon.atlassian.net/browse/TACHYON-442
Further cleanup the pom files, focusing on maven-surefire-plugin and maven-compiler-plugin.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/3236/
Test PASSed.

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/Tachyon-Pull-Request-Builder/3237/
Test PASSed.

+1 LGTM, much cleaner than before.

Thanks!


Dailymotion

Hello! Thank you for your effort!
 Not sure what type of videos it blocks on dailymotion. Ads-videos? Could be a false positive too.

player.ooyala.com - The Ooyala Player. Example, used on accuweather.com to play videos.

Hi there! Thanks for reporting. I removed
player.ooyala.com
s1.yimg.com
s2.dmcdn.net
s.yimg.com
static2.dmcdn.net
vid3.ec.dmcdn.net
api.dailymotion.com
Now yahoo.com, dailymotion.com and AccuWeather videos works fine.
Update your adlist to apply the changes.

Images on 1337x.to loads with lx1.dyncdn.cc

api.ipify.org , dl.dropboxusercontent.com , login.live.com , m.hotmail.com , msftncsi.com , pricelist.skype.com , s.gateway.messenger.live.com , ui.skype.com
These domains (so far) are on your adservers list and on your whitelist. Is this intentionally, or? To avoid confusion, it would be good to remove the false positives from the adservers list.

I removed them, most of the Microsoft related sites were added without checking by this commit.

@HellboyPI  Thanks again for finding out these domains!


win: enable uv_thread_{get,set}name on MinGW

It supports the API: https://github.com/mirror/mingw-w64/blob/93f3505a758fe70e56678f00e753af3bc4f640bb/mingw-w64-headers/include/processthreadsapi.h#L358

Potential alternative, if we want to go the NT API route: NtSetInformationThread.

I think you had to squash haha https://github.com/libuv/libuv/commit/7752218db2cd3de28a2924d2e9b0f6e062b441c7

We should potentially disable that UI option. We have it enabled so that if you carefully craft the commits to add the PR number for a series, then you can rebase+merge that. But that is very rare and can be pushed directly to v1.x when that ever comes up.

Opsie, my bad 😅


Fonts changing to custom one.

Hey. I've tried to replace css vars
  Font styles

  /* Preview mode */

  --text:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;

  /* Editor mode */

  --text-editor:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;

  /* Code */

  --font-monospace:Menlo,SFMono-Regular,Consolas,"Roboto Mono",monospace;

  /* UI */

  --font-ui:-apple-system,BlinkMacSystemFont,"Segoe UI",Roboto,Oxygen-Sans,Ubuntu,Cantarell,"Helvetica Neue",sans-serif;

to my preferable font Ubuntu Mono but it didn't apply.
May i ask to give me a hint how to change the fonts?

I would recommend using the Minimal Theme Settings plugin to change font. Make sure that the name of the font exactly matches the name on your system, you may also need to add quotes around it.

Oh, that's interesting. Theme is configuring by a plugin. Ok, thank you, i'm trying it.