pmFromFile and matching problems

Server: Debian Wheezy
libapache2-mod-security2:             v2.8.0-2~bpo70+1
modsec rules: last from git

A simple test.php file including this code:
test
<img src="wp-content/themes/expound/screenshot.png">

I have a rule to catch all static files and for them skip my next custom rules.
This one works:
SecRule REQUEST_FILENAME "\.(ico|png|jpg|jpeg|gif|tiff|ods|fods|odt|fodt|odp|fodp|odg|doc|docx|xls|xlsx|rtf|csv|ppt|pptx|pps|ppsx|pdf|txt|css|js|ogg|ogm|mp4|flac|ape|wav|mkv|mpg|mpeg|swf|flv|mov|avi|wma)$"  "phase:1,id:2100,log,pass,skipAfter:END_CUSTOM_MARKER"

This one doesn't:
SecRule REQUEST_FILENAME "@pmFromFile /etc/modsecurity/liste_extensions.txt"    "phase:1,id:2100,nolog,pass,skipAfter:END_CUSTOM_MARKER"

Well, it works only when I put the complete filename in it (screenshot.png), but doesn't work when I try to put only file extensions:
png
or
.png
or
.*.png
or
png/
or
.png/

From what I read in the reference manual (https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#pmFromFile -> point 4), it should be able to match all files finishing by png.. or am I mistaken.
Thanks

I'll start with the thing I first noticed and maybe it will help. This is in Phase 1, too my understanding it shouldn't be since your inspecting file data. https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual#phase-request-body

Thanks for your help @csanders-git
From my understanding, modsecurity has the filenames at phase 1. It should or I don't understand how it can know what to process. Maybe not all arguments as the manual says, but the filename requested yes.
The fact that my first example works in phase 1 shows that.
The difference with my second example is the use of the action pmFromFile instead of putting all extensions directly in the rule.
And if I put the complete filename in the file, it works too.
The problem seems that modsecurity doesn't take each line from the file and check if it can find that pattern in the filename but only do a string comparison.
But from the manual and the section about false positives, putting 'png' on a line should trigger a 'return 1' for filenames like 'anything.png' or 'png.blabla'.
And putting 'png/' in the file should trigger a 'return 1' for filenames finishing by png, what I would like to have.

i'm thinking (but not entirely sure) that you are thinking about this backwards. For example. if the INPUT from modsecurity was <IP_ADDRESS> (REMOTE_ADDR for instance) and we did a pmf such as:
SecRule REMOTE_ADDR "@pmf blacklist.conf" "id:1"
then if <IP_ADDRESS> was in the FILE it would match.
To restate if the input is a subset of an element in the file it will ALSO match (in addition to being exact)
My thought is that you are trying to go the other way if the element in the file is a subset of the input your are expecting a match. Not sure that works... but without testing i don't know for sure.

I see your point.
But I don't it's correct. Not from what I understand from the manual and the example given (but I agree it's a bit confusing):

Because this operator does not check for boundaries when matching, false positives are possible in some cases. For example, if you want to use @pm for IP address matching, the phrase <IP_ADDRESS> will potentially match more than one IP address (e.g., it will also match <IP_ADDRESS> or <IP_ADDRESS>). To avoid the false positives, you can use your own boundaries in phrases. For example, use /<IP_ADDRESS>/ instead of just <IP_ADDRESS>. Then, in your rules, also add the boundaries where appropriate. You will find a complete example in the example.

For me, "the phrase <IP_ADDRESS>" means <IP_ADDRESS> in the file.
And "match more than one IP address (e.g., it will also match <IP_ADDRESS> or <IP_ADDRESS>)" means if the INPUT from modsec (the ip address that does the request) is <IP_ADDRESS>...
What gives me some certainty is the the § explains boundaries with the <IP_ADDRESS> ip address. So /<IP_ADDRESS>/ can only be in the file
Then... <IP_ADDRESS> -> trigger <IP_ADDRESS> & <IP_ADDRESS>
But as the § says, boundaries can be used on both sides (/<IP_ADDRESS>/). We can then assume the <IP_ADDRESS> would trigger also the rule.
Then:

png should trigger png & xxx.png & png.doc
/png should trigger png & png.doc
png/ should trigger png & xxx.png
/png/ should trigger only png

As I said, all of this is a bit confusing.

I did several tests to see where the problem is.
The difference with before is now I'm using the CRS v3 (instead of v2.2.9).
But libapache2-mod-security2 is the same version, so it should have the same result.
The rule:
SecRule REQUEST_FILENAME "@pmFromFile /etc/modsecurity-conf/liste_extensions.txt"    "phase:1,id:2100,nolog,pass,skipAfter:END_CUSTOM_MARKER"
  .. do xxxx
SecMarker END_CUSTOM_MARKER

In my file, I tried this:

aascreenshot.png
/aascreenshot.png
aascreenshot.png/
/aascreenshot.png/
screenshot.png
/screenshot.png
screenshot.png/
/screenshot.png/
eenshot.png
/eenshot.png
eenshot.png/
/eenshot.png/

And here are the results:

do xxxx -> correct with my theory / incorrect with yours
do xxxx -> correct with both our theories
do xxxx -> correct with my theory / incorrect with yours
do xxxx -> correct with both our theories
do NOT xxxx -> correct with both our theories
do NOT xxxx -> correct with both our theories
do NOT xxxx -> correct with both our theories
do xxxx -> incorrect with both our theories
do NOT xxxx -> correct with my theory / incorrect with yours
do xxxx -> correct with both our theories
do xxxx -> correct with your theory / incorrect with mine
do xxxx -> correct with both our theories

These results are very very confusing.
Unfortunately for me, the only one I want to use (11) is the only one where my theory is wrong :)
But I'd say there's something wrong with the action pmFromFile, whatever theory is the right one.

I was just wondering if there were any news on this.
Despite the wording of the Reference Manual, I have likewise noted that, when using @pmFromFile, an entry of 'aaa' in the file will not match 'aaabbb' in input.
Indeed, CRS3.0 seems to assume that phrases in a file will be subject to exact match rather than substring.  For example, REQUEST-933-APPLICATION-ATTACK-PHP.conf uses php-function-names-933151.data which has numerous examples of successive phrases where one is a substring of the others, for example:
date_parse
data_parse_from_format
If the @pmFromFile actually worked the way the Manual suggests (not checking for boundaries and thus acting like a substring checker) then there would be no point to having both lines (the first of the two would be sufficient).

I was just wondering if there were any news on this.
Despite the wording of the Reference Manual, I have likewise noted that, when using @pmFromFile, an entry of 'aaa' in the file will not match 'aaabbb' in input.
Indeed, CRS3.0 seems to assume that phrases in a file will be subject to exact match rather than substring. For example, REQUEST-933-APPLICATION-ATTACK-PHP.conf uses php-function-names-933151.data which has numerous examples of successive phrases where one is a substring of the others, for example:
date_parse
data_parse_from_format
If the @pmFromFile actually worked the way the Manual suggests (not checking for boundaries and thus acting like a substring checker) then there would be no point to having both lines (the first of the two would be sufficient).

I couldn't reproduce this on the latest code in v2.9 branch master:
[01/Oct/2018:19:50:51 --0400] [localhost/sid#7fbacf5d9820][rid#7fbac40350a0][/index.html][5] Rule 7fbacf5e20f0: SecRule "ARGS:pm" "@pmf /tmp/files.txt" "phase:4,status:403,tag:'DROP on phase 4!',id:25,deny,log,auditlog"
[01/Oct/2018:19:50:51 --0400] [localhost/sid#7fbacf5d9820][rid#7fbac40350a0][/index.html][4] Transformation completed in 0 usec.
[01/Oct/2018:19:50:51 --0400] [localhost/sid#7fbacf5d9820][rid#7fbac40350a0][/index.html][4] Executing operator "pmf" with param "/tmp/files.txt" against ARGS:pm.
[01/Oct/2018:19:50:51 --0400] [localhost/sid#7fbacf5d9820][rid#7fbac40350a0][/index.html][9] Target value: "aaabbb"
[01/Oct/2018:19:50:51 --0400] [localhost/sid#7fbacf5d9820][rid#7fbac40350a0][/index.html][4] Operator completed in 1 usec.
[01/Oct/2018:19:50:51 --0400] [localhost/sid#7fbacf5d9820][rid#7fbac40350a0][/index.html][4] Rule returned 1.
[01/Oct/2018:19:50:51 --0400] [localhost/sid#7fbacf5d9820][rid#7fbac40350a0][/index.html][9] Match, intercepted -> returning.
[01/Oct/2018:19:50:51 --0400] [localhost/sid#7fbacf5d9820][rid#7fbac40350a0][/index.html][1] Access denied with code 403 (phase 4). Matched phrase "aaa" at ARGS:pm. [file "/etc/modsecurity/modsecurity.conf"] [line "567"] [id "25"]

Same for v3.x:
[153843777148.107283] [] [4] Initializing transaction
[153843777148.107283] [] [4] Transaction context created.
[153843777148.107283] [] [4] Starting phase CONNECTION. (SecRules 0)
[153843777148.107283] [] [9] This phase consists of 0 rule(s).
[153843777148.107283] [] [4] Starting phase URI. (SecRules 0 + 1/2)
[153843777148.107283] [/?pm=aaabbb] [4] Adding request argument (GET): name "pm", value "aaabbb"
[153843777148.107283] [/?pm=aaabbb] [4] Starting phase REQUEST_HEADERS.  (SecRules 1)
[153843777148.107283] [/?pm=aaabbb] [9] This phase consists of 1 rule(s).
[153843777148.107283] [/?pm=aaabbb] [4] (Rule: 25) Executing operator "PmFromFile" with param "/tmp/files.txt" against ARGS:pm.
[153843777148.107283] [/?pm=aaabbb] [9] Target value: "aaabbb" (Variable: ARGS:pm)
[153843777148.107283] [/?pm=aaabbb] [9] Matched vars updated.
[153843777148.107283] [/?pm=aaabbb] [4] Running [independent] (non-disruptive) action: log
[153843777148.107283] [/?pm=aaabbb] [9] Saving transaction to logs
[153843777148.107283] [/?pm=aaabbb] [4] Rule returned 1.
[153843777148.107283] [/?pm=aaabbb] [4] Running (disruptive)     action: deny
[153843777148.107283] [/?pm=aaabbb] [8] Running action deny
[153843777148.107283] [/?pm=aaabbb] [4] Running (non-disruptive) action: auditlog
[153843777148.107283] [/?pm=aaabbb] [8] Skipping this phase as this request was already intercepted.
[153843777148.107283] [/?pm=aaabbb] [4] Starting phase RESPONSE_HEADERS. (SecRules 3)
[153843777148.107283] [/?pm=aaabbb] [9] This phase consists of 0 rule(s).
[153843777148.107283] [/?pm=aaabbb] [9] Appending response body: 116 bytes. Limit set to:<PHONE_NUMBER>00
[153843777148.107283] [/?pm=aaabbb] [9] Appending response body: 170 bytes. Limit set to:<PHONE_NUMBER>00
[153843777148.107283] [/?pm=aaabbb] [4] Starting phase RESPONSE_BODY. (SecRules 4)
[153843777148.107283] [/?pm=aaabbb] [9] This phase consists of 0 rule(s).
[153843777148.107283] [/?pm=aaabbb] [4] Starting phase LOGGING. (SecRules 5)
[153843777148.107283] [/?pm=aaabbb] [9] This phase consists of 0 rule(s).
[153843777148.107283] [/?pm=aaabbb] [8] Checking if this request is suitable to be saved as an audit log.
[153843777148.107283] [/?pm=aaabbb] [8] Checking if this request is relevant to be part of the audit logs.
[153843777148.107283] [/?pm=aaabbb] [5] Saving this request as part of the audit logs.
[153843777148.107283] [/?pm=aaabbb] [8] Request was relevant to be saved. Parts: 6014

Please let us know if the issue persist on the current versions and we can investigate further. Thanks!


pkgsMusl.game-music-emu: fix build

Description of changes

Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.11 Release Notes (or backporting 22.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 192228 run on x86_64-linux 1

Isn't this a library instead of an application?

I confirm build is failing at master:

And this PR indeed fixes: nix build .#pkgsMusl.game-music-emu

It is a library.

It is a library.

Would you mind fixing the package location from application to library?

Done.

Result of nixpkgs-review pr 192228 run on x86_64-linux 1

Done.

Thanks!


New layout for hybrid US-Arabic

The keyboard layout is for people who primarily use US keyboard but want to switch to Arabic from time to time.
Based on the linux Arabic keyboard layout.
The font Noto Kufi Arabic is used - https://fonts.google.com/noto/specimen/Noto+Kufi+Arabic

The keyboard layout is for people who primarily use US keyboard but want to switch to Arabic from time to time.
Based on the linux Arabic keyboard layout.

Thanks for your contribution! :partying_face:


Client Credentials are invalid

I am facing the issue of client credentials.

not yet succeed to changed it to http://localhost/

Hi,
I'm also encountering an error in user/login endpoint.
Please see below screenshots:


STM32 F105 cannot use USBX CDC ACM normally

I have successfully ported ThreadX and USBX to STM32F105, and USBX HID can work normally. When I try to use USBX CDC ACM, the problem occurs. There is VCOM virtualized by STM32 in the device manager of windows, but when I try to use this serial port STM32 will freeze and stop running during communication. Why is there such a problem? I do the same operation on STM32 H7 and there is no problem

Are you running the application with debugger? To see where the program goes.

I tried to add breakpoints to observe in Debug mode, but unfortunately, as soon as VCOM sends data, the system will be stuck and MDK has not found the stuck position.、

I tried to upload the code here, but it failed to upload. Can you leave your email? I will communicate with you via email

The current problem has been solved. CDC can be used to communicate normally. But a new problem has arisen, that is, I can't use high frequency to send data to VCOM, otherwise the software will be stuck.Where will the CDC transceiver performance be restricted?


Bug: Infinite loop using Microsoft Provider

Hi guys.
Using Microsoft as a login provider, after the login we have a infinite loop when the page is redirected, even using the "cleanURL" param as true.
Steps To Reproduce

Set https://login.microsoftonline.com/organizations/oauth2/v2.0/authorize/ as authorizationEndpoint
Set https://login.microsoftonline.com/organizations/oauth2/v2.0/token as tokenEndpoint

Configure your own clientId, and after the login you will see the infinite loop refresh, between microsoft and your application.

The problem was resolved adding !loginInProgress in the IF statement, file AuthContext.tsx, line 225:

Hi @wesley-monaro
Thank you for reporting this :slightly_smiling_face:
However, I belive this was fixed with PR #93, available in version 1.13.5.
Are you sure you are running this latest version?

Hey @soofstad
Perfect, the problem was really solved on this PR. Using the 1.13.5 works fine.
Thank you so much.


Stream has null value on emulator but not on real device

I am having some very strange behaviour with StreamControllers in my flutter app on Android Virtual Devices with Android Studio. When I connect a real android device the app works fine.
The app did run fine on AVDs but seemed to stop after I downloaded a new device and OS (Android 9.0).
I am using a bloc for an auth form that listens for on change, validates the values then on submit sends to server.
The issue I am having is that both the email and the password are correctly being added to the stream and are validating as expected.
However, when I hit submit in submitForm() if the last field I have touched is password, then the _email.value value is null. If the last field I touched is email then the value of _password.value is null. Again, this only happens in AVDs. When I use my real phone, the value of both _email and _paswsword are as expected.
I completely removed Android Studio and reinstalled but the result is the same. I am running Hyper V as I use docker but there were no warnings from Android about HAMX.
System:
OS Name	Microsoft Windows 10 Pro
Version	10.0.17763 Build 17763
Other OS Description 	Not Available
OS Manufacturer	Microsoft Corporation
System Manufacturer	MSI
System Model	MS-7A68
System Type	x64-based PC
System SKU	Default string
Processor	Intel(R) Core(TM) i7-7700 CPU @ 3.60GHz, 3600 Mhz, 4 Core(s), 8 Logical Processor(s)
BIOS Version/Date	American Megatrends Inc. 1.30, 28/06/2017
SMBIOS Version	3.0
Embedded Controller Version	255.255
BIOS Mode	UEFI
BaseBoard Manufacturer	MSI
BaseBoard Product	Z270 TOMAHAWK (MS-7A68)
BaseBoard Version	1.0
Platform Role	Desktop
Secure Boot State	Off
PCR7 Configuration	Binding Not Possible
Windows Directory	C:\WINDOWS
System Directory	C:\WINDOWS\system32
Boot Device	\Device\HarddiskVolume3
Locale	United States
Hardware Abstraction Layer	Version = "10.0.17763.404"
Time Zone	GMT Daylight Time
Installed Physical Memory (RAM)	16.0 GB
Total Physical Memory	16.0 GB
Available Physical Memory	4.49 GB
Total Virtual Memory	21.5 GB
Available Virtual Memory	5.44 GB
Page File Space	5.50 GB
Page File	C:\pagefile.sys
Kernel DMA Protection	Off
Virtualization-based security	Running
Virtualization-based security Required Security Properties
Virtualization-based security Available Security Properties	Base Virtualization Support, UEFI Code Readonly, Mode Based Execution Control
Virtualization-based security Services Configured
Virtualization-based security Services Running
Device Encryption Support	Reasons for failed automatic device encryption: TPM is not usable, PCR7 binding is not supported, Hardware Security Test Interface failed and device is not InstantGo, Un-allowed DMA capable bus/device(s) detected, TPM is not usable
A hypervisor has been detected. Features required for Hyper-V will not be displayed.

Code:
bloc:
import 'dart:async';
import 'dart:convert';
import 'package:rxdart/rxdart.dart';
import 'package:http/http.dart' as http;
import './auth_validator.dart';

class AuthBloc with AuthValidator {
  final _email = BehaviorSubject<String>();
  final _password = BehaviorSubject<String>();
  final _isLoading = BehaviorSubject<bool>();
  final _loginSucceded = BehaviorSubject<bool>();

  // stream getters
  Stream<String> get email => _email.stream.transform(validateEmail);
  Stream<String> get password => _password.stream.transform(validatePassword);
  Stream<bool> get isLoading => _isLoading.stream;
  Stream<bool> get loginSuccess => _loginSucceded.stream;
  Stream<Map> get redirect => Observable.combineLatest2(_loginSucceded,
      _isLoading, (log, load) => {'loginSuccess': log, 'isLoading': load});
  Stream<bool> get submitValid =>
      Observable.combineLatest2(email, password, (e, p) => true);

  // add data to sink onChange
  Function(String) get emailChanged => _email.sink.add;
  Function(String) get passwordChanged => _password.sink.add;

  login(jsonUser) async {
    try {
      // submit to server
      final http.Response response = await http.post(
        'http://<IP_ADDRESS>:5000/api/users/signin',
        body: jsonUser,
        headers: {'Content-Type': 'application/json'},
      );

      final Map<String, dynamic> decodedRes = await json.decode(response.body);

      if (response.statusCode == 200) {
        _loginSucceded.sink.add(true);
      }

      return decodedRes;
    } catch (e) {
      _isLoading.sink.add(false);
    }
  }

  void submitForm() async {
    try {
      final Map user = {
        'email': _email.value,
        'password': _password.value
      };
      final jsonUser = json.encode(user);
      print(jsonUser);

      await login(jsonUser);

      void dispose() {
        _email.close();
        _password.close();
        _isLoading.close();
        _loginSucceded.close();
      }
    } catch (e) {
      print('error: $e');
    }
  }
}

validator:
import 'dart:async';

class AuthValidator {
  final validateEmail = StreamTransformer<String, String>.fromHandlers(handleData: (email, sink) {
    if (!email.contains(RegExp(r"[a-z0-9!#$%&'*+/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+/=?^_`{|}~-]+)*@(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?\.)+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?"))) {
      sink.addError('Please enter a valid email address');
    } else {
      sink.add(email);
    }
  });

  final validatePassword = StreamTransformer<String, String>.fromHandlers(handleData: (password, sink) {
      final goodLength = password.length >= 8;
      final hasNumber = password.contains(RegExp(r'[0-9]'));

      if (!goodLength) {
        sink.addError('Must be at least 8 characters and contains a number');
      } else {
        sink.add(password);
      }
    });
}

page:
import 'package:flutter/material.dart';
import '../blocs/auth_bloc.dart';

class LoginPage extends StatelessWidget {
  final authBloc = AuthBloc();

  @override
  Widget build(BuildContext context) {
    // TODO redirect would not work in StreamBuilder block
    // manually listening for redirect conidtions here
    // once a response is received from the server
    authBloc.loginSuccess.listen((data) {
      if (data) {
        Navigator.pushReplacementNamed(context, '/dashboard');
      }
    });

    return StreamBuilder(
        stream: authBloc.isLoading,
        initialData: false,
        builder: (context, snapshot) {
          print(snapshot.data);
          return Scaffold(
            body: _buildPage(authBloc, snapshot.data),
          );
        });
  }

  Widget _buildPage(AuthBloc authBloc, bool isLoading) {
    return Container(
      margin: EdgeInsets.all(20.0),
      child: Center(
        child: SingleChildScrollView(
          child: Column(
            children: <Widget>[
              _emailField(authBloc),
              _padding(),
              _passwordField(authBloc),
              _padding(),
              _submitButton(authBloc, isLoading)
            ],
          ),
        ),
      ),
    );
  }

  Widget _circularSpinner() {
    return Center(
      child: CircularProgressIndicator(),
    );
  }

  Widget _emailField(AuthBloc authBloc) {
    return StreamBuilder(
      stream: authBloc.email,
      builder: (BuildContext context, snapshot) {
        return Container(
          padding: EdgeInsets.only(top: 10.0),
          child: TextField(
            onChanged: authBloc.emailChanged,
            autofocus: true,
            keyboardType: TextInputType.emailAddress,
            decoration: InputDecoration(
              hintText<EMAIL_ADDRESS>              labelText: 'Email Address',
              errorText: snapshot.error,
              border: OutlineInputBorder(),
            ),
          ),
        );
      },
    );
  }

  Widget _passwordField(AuthBloc authBloc) {
    return StreamBuilder(
      stream: authBloc.password,
      builder: (BuildContext context, snapshot) {
        return TextField(
          onChanged: authBloc.passwordChanged,
          obscureText: true,
          keyboardType: TextInputType.emailAddress,
          decoration: InputDecoration(
            hintText: '8 characters or more with at least 1 number',
            labelText: 'Password',
            errorText: snapshot.error,
            border: OutlineInputBorder(),
          ),
        );
      },
    );
  }

  Widget _padding() {
    return Padding(
      padding: EdgeInsets.only(top: 20.0),
    );
  }

  Widget _submitButton(AuthBloc authBloc, isLoading) {
    final spinner = CircularProgressIndicator();
    final button =  RaisedButton(
            child: Text('Login'),
            color: Colors.blue,
            onPressed: authBloc.submitForm,
          );
    return StreamBuilder(
        stream: authBloc.submitValid,
        builder: (context, snapshot) {
         return isLoading ? spinner : button;
        });
  }
}

Update:
I also downloaded Geny Motion and this works fine so the issue does seem to be with Flutter + AVD.
This is frustraing as Geny Motion is not Hyper V compatible which means that I cannot run Docker while using it.

@jhc1982 is it the same with current flutter stable?


ory-elements loads a 2.4 MB CSS file, does critical CSS overrides

Preflight checklist

[X] I could not find a solution in the existing issues, docs, nor discussions.
[X] I agree to follow this project's Code of Conduct.
[X] I have read and am following this repository's Contribution Guidelines.
[X] This issue affects my Ory Cloud project.
[X] I have joined the Ory Community Slack.
[X] I am signed up to the Ory Security Patch Newsletter.

Describe the bug
Using ory-elements loads a 2.4 MB CSS file (minified size).
This file includes:

the full Font Awesome icon collection
the Inter font in 3 font weights
the JetBrainsMono font in 2 font weights

Most importantly, it also includes normalize.css which is a full CSS reset, basically conflicting with every single project which doesn't use the exact same file.
It also does global style overrides like textRendering: "geometricPrecision" etc.
I'm sorry to say but this way it's not usable in any project. This might work on a separate subdomain hosted auth screens, but then the whole point of this project is to do it inside an app, isn't it?
Reproducing the bug
Build a project, look at the bundled 2.4 MB CSS file.

Thank you, you’re absolutely right. We will reduce the amount of data we load here and also do away with excessive font loading :)

@jfcurran maybe we can reduce the number of fonts we load?

The more critical problem is the global scoped CSS styles and the reset. It can totally break an existing website's CSS if those are loaded.

Got it, we’ll sort that first. Our UI team is busy shipping our new website this team but we’ll get to this right after. @kmherrmann

@Benehiko for visibility
Thanks for the report, @hyperknot . This is definitely not an acceptable size for CSS and we'll work to clean it up!

The interesting thing is that I've removed everything (fonts, icons, CSS) and nothing broke. Only the font family had to be commented out as it was hardcoded.
I don't know what's the tiny icon which was used from FontAwesome but I haven't even seen it.

Hey @hyperknot
Sorry about this, we will address it soon!

This should now be resolved with the latest release (v0.0.1-beta.1)

Thanks a lot for fixing this! There is still 3 hard-coded font-families.
  font-family: "Font Awesome 6 Free";
  font-family: Inter;
  font-family: JetBrainsMono;

For Inter I think you might have forgotten one line.
For JetBrainsMono I guess you might need to introduce a var like --ory-theme-font-family-mono.
For the Font Awesome, normally for React there is an official React package. But it's quite a complicated setup, with some special case for Next.js. If it was me I'd inline the SVG icon and remove the FA fonts and packages.
I think this is the whole icon:
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512"><!--! Font Awesome Pro 6.2.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2022 Fonticons, Inc. --><path d="M470.6 105.4c12.5 12.5 12.5 32.8 0 45.3l-256 256c-12.5 12.5-32.8 12.5-45.3 0l-128-128c-12.5-12.5-12.5-32.8 0-45.3s32.8-12.5 45.3 0L192 338.7 425.4 105.4c12.5-12.5 32.8-12.5 45.3 0z"/></svg>

Thanks a lot for fixing this! There are still 3 hard-coded font-families.
  font-family: "Font Awesome 6 Free";
  font-family: Inter;
  font-family: JetBrainsMono;

For Inter I think you might have forgotten one line.
For JetBrainsMono I guess you might need to introduce a var like --ory-theme-font-family-mono.
For the Font Awesome, normally for React there is an official React package. But it's quite a complicated setup, with some special case for Next.js. If it was me I'd inline the SVG icon and remove the FA fonts and packages.
I think this is the whole icon:
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
<!--! Font Awesome Pro 6.2.1 by @fontawesome - https://fontawesome.com License - https://fontawesome.com/license (Commercial License) Copyright 2022 Fonticons, Inc. -->
<path d="M470.6 105.4c12.5 12.5 12.5 32.8 0 45.3l-256 256c-12.5 12.5-32.8 12.5-45.3 0l-128-128c-12.5-12.5-12.5-32.8 0-45.3s32.8-12.5 45.3 0L192 338.7 425.4 105.4c12.5-12.5 32.8-12.5 45.3 0z"/>
</svg>

Thank you for finding these, @mszekiel and I discussed reducing the total css this week. We will be tackling that soon, which would mean removing a lot of font awesome, but this will be handled in a new PR :)
I'll create a new issue to track the css changes :)

Hi there, these fonts are causing problems for us using ory/elements overriding our styles, what's the update on this - I couldn't find the new issue?


Pointless page

[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: ba0a1d89-3377-deef-faf6-8d73cba667e7
Version Independent ID: 0d322b99-0aeb-8dcd-41be-0cadbd07d96b
Content: ClientApplicationBase.UserTokenCache Property (Microsoft.Identity.Client) - Azure for .NET Developers
Content Source: xml/Microsoft.Identity.Client/ClientApplicationBase.xml
Service: active-directory
GitHub Login: @rloutlaw
Microsoft Alias: routlaw

//cc: @nonik0, @anaismiller

Thank you for your feedback.  Tagging and routing to the team members best able to assist.

@jsquire I took a look at this and looks like it's an MSAL page. I don't have a recent contact with them, can you cc that team please?

Thanks, @nonik0.   I appreciate the assist.

This issue has been transferred to the MSAL repository as #2872.   Closing this out.


Csc task has many undocumented parameters

The documentation of the Csc task omits several parameters that are however set by the "CoreCompile" target in Microsoft.CSharp.Core.targets:

AdditionalFiles
Analyzers
ChecksumAlgorithm
CodeAnalysisRuleSet
EmbedAllSources
EmbeddedFiles
ErrorEndLocation
ErrorLog
Features
HighEntropyVA
Instrument
PathMap
PublicSign
Prefer32Bit
PreferredUILang
ProvideCommandLineArgs
ReportAnalyzer
RuntimeMetadataVersion
SharedCompilationId
SkipCompilerExecution
SourceLink
SubsystemVersion
UseSharedCompilation
VsSessionGuid

See also https://github.com/dotnet/docs/issues/20061 about documenting the options of csc.exe.

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: caef0578-7b6a-6b5a-8cc4-268ddb2fb743
Version Independent ID: a99db5b2-2ff5-a675-95c2-333db10231ff
Content: Csc Task - Visual Studio
Content Source: docs/msbuild/csc-task.md
Product: visual-studio-windows
Technology: msbuild
GitHub Login: @ghogen
Microsoft Alias: ghogen

We added all of the parameters that are documented in the C# compiler docs. Some parameters are not intended for use, and  remain undocumented.


disableWebSecurity -> allowDisplayingInsecureContent

Allow mixed content does not use disableWebSecurity anymore.
We don't use a sledgehammer to crack a nut when this gets merged. Disabling security features is usually a bad idea - in this case we do not need to disable all security features to reach the goal.
As of electron 1.4.5 we can disable only certain security features as we need to.
This also fixes the youtube preview issue which was related to the disabled CORS security feature.

[x] Complete Mattermost Contributor Agreement
[x] Execute npm run prettify to format codes
[x] Update CHANGELOG.md and/or doc/*.md if it's necessary.
[x] Write about environment which you tested

Ubuntu 16.10 Unity

Tested on macOS 10.12. The feature works well!
On the https server (pre-release.mattermost.com), they appear:

"http"(insecure) images
youtube preview

You can find the artifacts to test here: https://circleci.com/gh/mattermost/desktop/768#artifacts

Tested on 64-bit Windows installer and works great. Big thanks @jnugh!
Test cases:

Mixed content enabled: http, https images and YouTube videos render
Mixed content disabled: http images don't render, YouTube videos and https images do

--
I assume the build failure is due to the test script Yuya mentions above.

One note: Anything else we should test with the Electron dependency bump from 1.4.2 --> 1.4.5?
I noticed that the zoom in/out issue @yuya-oc described here no longer reproduces. If I click on the tab bar or the chat area, zooming in and out works properly.
The zoom rendering issues only reproduce when restarting the app as described in https://github.com/mattermost/desktop/issues/334.

@jasonblais As far as reading changelog, I can't find significant change for our app.
I tested the app on Windows 10 again. I think that the issue still reproduces when clicking the tab bar. Would you check again?

I tried again on Windows 10, and I was able to reproduce it after clicking on the tab bar twice.

After clicking on tab bar once

After clicking on tab bar twice

Just rebased and pushed multiple changes:

Updated the tests - no failures anymore on my machine
Changed some comments inside the code, as requested in the review, I didn't change the code itself
Updated Changelog to be more specific about the security features disabled previously

https://circleci.com/gh/mattermost/desktop/780#artifacts :)

In package.json, Electron is not 1.4.5 probably due to rebasing. Please check.

You are right, seems like I confused the prebuilt package version I removed during merge with the resulting one :disappointed:
Sorry for that - https://circleci.com/gh/mattermost/desktop/781#artifacts should have a working version in a few seconds.

Sorry @jnugh, it seems like the build is failing, can you help take a look?

Seems like CircleCI (or the the Docker Repo) is having issues right now.
Err https://apt.dockerproject.org ubuntu-trusty/main amd64 Packages
gnutls_handshake() failed: Handshake failed
I restarted twice by pushing a new ref - will try again later.

I restarted CircleCI, and the test is passing.
https://circleci.com/gh/mattermost/desktop/784#artifacts

Test cases:

Mixed content enabled: http, https images and YouTube videos render
Mixed content disabled: http images don't render, YouTube videos and https images do

All passed on 64-bit Windows installer :+1:
Thanks @jnugh :)

Tested on macOS, the test cases passed!

Thanks @jnugh! :)

@jnugh Thanks many times!


[Feature]: Nested Routes for settings and nav bar

Contact Details
No response
What do you like to be improved?
Some awesome stuff!

[ ] Implement #215

[ ] Implement #156

[ ] Implement nested routes for top nav bar


Silly Putty is the same game as Putty

Let's leave "Putty", so it groups with Putty Squad (the sequel) — unless there's a difference I am unaware of. I think Silly Putty was an earlier version, but e.g. Lemon Amiga only has a listing for Putty.

Yes, I suppose Silly Putty is an early/unreleased/leaked or somesuch version. 👍


Update schema figures

I recently committed (c160106) a new future schema diagram to the docs-update branch. Please review the diagram and provide comments here.

Done, finally, in 5879aa9.


feat: Add OStim VR addresses

So I see what you've been working on recently. ;)

wait this is finally happening??!!


Socket.IO server version support

Error: It seems you are trying to reach a Socket.IO server in v2.x with a v4.x client. Switch the client version from request settings.

it's now 4.0 version compatible. https://github.com/mrniko/netty-socketio/pull/897


Refresh token each time start app.

Hi, is it normal that each time i start the app i register a new token?
I set: FirebasePushNotificationManager.Initialize(this,false);

Token will refresh:
The app deletes Instance ID
The app is restored on a new device
The user uninstalls/reinstall the app
The user clears app data.
If you change the app version

If you change the app version
wow..this is a usefull feature i didn't know. Thanks

Hello @rdelrosario ,
What do you mean by "The app deletes Instance ID" ? I want to force renew of the token when a user disconnects from the app and reconnects with another account w/o closing the app.
Thanks.

On Android project use FirebasePushNotificationManager.Reset();

Hello @rdelrosario
Thank you for your answer. So I've tried .Reset(), it works, but it does not seems to trigger a renew for the token. It is empty when I log in again after calling this method. Thanks


Update testnets config: add Paseo, remove Wococo

Add Paseo config: https://forum.polkadot.network/t/the-new-polkadot-community-testnet/4956
Remove Wococo config as the network is now shut down: https://wiki.polkadot.network/docs/maintain-networks#wococo-test-network-inactive

@PierreBesson will you or someone else look after the paseo balances?

@mordamax I believe that we should be responsible for it, for the sake of consistency. At least, setting up the alerts and reacting to them.


Display notification when layer processing has completed

Presently, when layer processing is complete, the line item disappears from the processing widget. We should let it remain in the widget until the user dismisses/archives the line item. In addition, the user should be notified that layer processing has been completed.

I'm unable to reproduce this.

Currently, the layer stays in the processing block if it completes during the session.
I think that the layer should also show up in the Imports list in the UI. We could call fetch when the status of the LayerModel changes to complete.


[Podspec] Add missing RLMPlatform to public_header_files

Fixes #1400.

Verified as working in a project using it via CocoaPods 0.35.0, master generating static libraries, and master generating frameowkrs.


osutil: pid 1 should exit directly instead of trying to kill itself

/cc @crawford @marineam

@crawford Fixed. PTAL.

It occurred to me that you could also implement this by releasing interruptExitMu. Either way works.
LGTM

@crawford Yea. I thought it for a minute and thought it would make tricky thing even more trickier by addressing it implicitly. But it does help to prevent the deadlock in potential bad cases anyway...


Add support for github labels ignore list

Adds the option to configure github labels which, if present on the pull request, will cause the build to be skipped. This is useful, for example, when working on a pull request that is still in progress, and you don't want commits to trigger a build.
https://github.com/jenkinsci/ghprb-plugin/issues/210

👍

👍 👍

Udi - I'm planning to give this a look in the coming days.

Hi @udisch overall looks pretty good. Could you please make a couple of modifications?

Can you change labelslist to labelsIgnoreList or something along those lines? Basically I'd like to leave room for creating a labels whitelist later.
I see you've updated tests which is great, but I'd like a test that asserts a build is/is not triggered depending on the label that was found on the pull request. I didn't see a test for that but it would help prevent regressions later.

Thanks for the pull request!

@benpatterson Thanks for the feedback!

@benpatterson I added a test and renamed the labels field. Also fixed some small bugs in my PR
which I found while testing:

Labels weren't checked when adding comments
Labels read from the jenkins config assumed whitespace as the separator, but github allows spaces labels, so now newline is used.

Also note that I didn't add any handling for hooks - Removing a label won't trigger anything, but it's possible to use "retest this please".
Thanks!

Updated.

thanks @udisch
Psyched

@benpatterson Do you know when the next plugin release is due?

Hi @udisch we have released this now. :)

@benpatterson Thanks! I don't see it in the jenkins plugin repository yet..

@benpatterson Thank you for the great feature. The version is already out but I'm not able to use it as GhprbTriggerContext doesn't support it yet. Any ideas when would that happen? Thanks.

Unfortunately it looks like this has caused #441. I'm planning on reverting unless I get a workaround to the underlying issue in the github-api plugin.

@udisch Unfortunately this change has caused a regression in functionality. That's described in the thread for issue 441. I'm reverting this change in order to fix the plugin, then we can look at label options after that, likely in tandem with another contributor.

@benpatterson Thanks, and sorry. What happened exactly? Was the api removed from the github-api lib?

Hi @udisch label support is still available in the github-api library; however, it turns out that when calling getLabels, it uses GitHubs issues API instead of its pull request API. When making that call, it rewrites the pr object, including the root url to use for other calls (e.g., .../issues/... in the url, instead of .../pulls/...). Any subsequent call by the plugin would return a 404 because those calls use the pull request API endpoints.
If you want, you can walk through it on the issue 441 thread. I really like your feature and would like to see it re-implemented. There is another PR that you'll see was referenced. I'll do some rebasing, etc, and CC you on it when I'm done. From there perhaps we can tackle the underlying issue. (Or we will benefit from a fix on the underlying github-api  😎 )

On the github api, labels are only available on the /issues endpoint: https://developer.github.com/v3/pulls/#labels-assignees-and-milestones

@udisch Yes... I'm not arguing that we use some other API. The issue is that the underlying implementation on the other plugin ends up re-writing the pr object as an issue object (basically).
I think re-implementing this would require a workaround to that problem...unless we get more details on the other issue. I'm not sure why it was implemented the way it was....it could be a  bug, or it could be by design.


test(playwright): shortcuts-modal improvements

Checklist:

[x] I have read and followed the contribution guidelines.
[x] I have read and followed the how to open a pull request guide.
[x] My pull request targets the main branch of freeCodeCamp.
[x] I have tested these changes either locally on my machine, or GitPod.

Related to #53895

Added check for two dialogs on modal,  removed .all() method checking the count of dialogs using .toHaveCount()

@huyenltnguyen  thanks for your notice, no problem guys, I haven't worked worked on other modals, that would be my last pr on modal tests.


Add delete/clear messages functionality

It would be useful to be able to delete or clear messages (individual and all).

Dan just added an option to delete all messages with commit: 49e86ef
It will be in the next release.


Close stale PRs

This will follow #42438 by a week.
After publishing new guidelines, I'll close PRs that are past stale.

Associated WorkItem - 307293

Language on closing stale PRs is published here, in the 3rd and 4th paragraph.
Moving this to October to give time for people to respond.

I've reviewed and closed or merged all stale PRs except for 2, where I need some product team review.

Everything that hasn't been updated in the last 30 days has been merged or closed.


MacroCalls in if and while lose IDE functionality

macro_rules! foo {
    ($e:expr) => {$e};
}
let foo = true;
if foo!(foo) {}
     // ^^^ hovering this doesn't resolve, highlighting, completions and anything else basically doesn't work, expanding the macro via the command still works

I assume this might've been caused by https://github.com/rust-analyzer/rust-analyzer/pull/11375
Regression happened inbetween these commits https://github.com/rust-analyzer/rust-analyzer/compare/02904e99a...0b53744f2

cc @ChayimFriedman2, maybe you know whats going on here, if not that is also fine :) .
I'm also not 100% sure whether your PR was the cause but it looks like it was the only one in that commit span that touched those areas.


audit cardinality properties

Closes #97

Looks good, except maybe one thing.  P114 (cardinality $\aleph_1$) has the alias "first-uncountable".  I have never seen that phrase to mean "cardinality equal to the first uncountable cardinal", have you?  Unless it's meant as a convenient shortcut for use in the search box, which then is fine (we have other instances of that).

Yeah my primary motivation was to get it to show up when people search for "uncountable". I could say "cardinality equal to the the first uncountable cardinal" I suppose, but I don't think anyone is looking for that phrase. I don't think "first uncountable cardinal" is a valid descriptor (it's describing a space, not a cardinal).

Ok, I'll approve.


Never Seen NearDrop's menu bar icon

I've been using NearDrop for about a month and from the first run, I never saw any icon for it in the menubar.  Since it just worked, I never even knew it had a menu bar icon.  But, now I'm trying to figure out why my Android no longer detects my Mac and in that process realized I can't see the icon, so can't quit it to test, upgrade, etc.
My menu bar right side is quite full so I'm not sure where other icons get pushed off to.
How can I find NearDrop's control icon?
MacOS Sonoma 14.0

You should enable NearDrop sharing extension in Settings > Privacy & Security > Extensions > Added extensions


HxCEnchants simply not working

HxC core seems to be functioning fine but I can't get any enchants on swords working. I'm enchanting them using the xp enchanter and infusing them with levels so they are showing a charge but not one single enchant is working. I hit a mob with my sword with various hxc enchants and not one drop of charge is draining
MC 1.7.10
ForgeMultipart-1.7.10-<IP_ADDRESS>5-universal
HxCCore-1.7.10-1.11.0-universal
HxCEnchants-1.7.10-2.2.4
UPDATE 2
Well I've just been testing the enchants and the armor ones actually are working but nothing I put on a sword will work. I've tried modded swords and a basic vanilla iron sword. I can put charge in it. I can even enchant it in the xp enchanter. I cannot however get it to use any charge or use any of its abilities.
UPDATE 3
Still can't sword enchants working but I have noticed a strange bug with armor.

Sorry I forgot to upload the 2.2.5
I've had it for like 3 weeks
It's somewhere in the issues area

I didn't have internet for a few days, I'm poor ya know

I've updated to the 2.2.5 build. For some reason my sword enchants are still not working. I've found something quite puzzling though. Normally when testing new mods I use NEI to do a dump of the used/free/all enchants. When I install your mod the resultant dumped CSV is a zero byte file. Remove your mod (HxC Core and HxC Enchants) and all of a sudden it works and I get back a 102 KiB file as expected. I know your mod does some sort of jiggery pokery to expand the enchant array/list but I'm sure something is going wrong on my setup.
I'm running 1.7.10 client and server and HxC Enchants 2.2.5 along with the most recent HxCCore 1.110

I increase the max number of enchantments by 256 so that I can add lots of enchants on a whim(up to 256 enchants), NEI may not be to fond of that.
I'm currently too tired to code, but tomorrow or so I'll dig deep.

Try with latest 2.3.0 build I changed a bit

I am now using HxCEnchants-1.7.10-2.3.2 and HxCCore-1.7.10-1.12.0-universal after being on hiatus for some time. I've noticed the armour enchants seem to work but the sword ones do nothing. I've held the item in my hand and pressed shift to ensure there is sufficient charge but nothing on the swords. As I say the armour enchants appear to be working fine.

Fixed in version 2.3.3


[Feature](inverted index) support range predicate for inverted index

Proposed changes
Issue Number: close #xxx
Test with httplogs dataset, schema as following, we use inverted index for size column:
CREATE TABLE `httplogs` (
  `@timestamp` int(11) NULL,
  `clientip` varchar(20) NULL,
  `request` text NULL,
  `status` int(11) NULL,
  `size` int(11) NULL,
  INDEX size_idx (`size`) USING INVERTED COMMENT ''
) ENGINE=OLAP
DUPLICATE KEY(`@timestamp`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`@timestamp`) BUCKETS 24
PROPERTIES (
"replication_allocation" = "tag.location.default: 1",
"is_being_synced" = "false",
"storage_format" = "V2",
"compression" = "ZSTD",
"light_schema_change" = "true",
"disable_auto_compaction" = "false",
"enable_single_replica_compaction" = "false"
);

Total lines for httplogs:
mysql> select COUNT() from httplogs;
+------------+
| count(*)   |
+------------+
|<PHONE_NUMBER> |
+------------+
1 row in set (0.18 sec)

before optimization:
mysql> select COUNT() from httplogs where size > 20000 and size <= 20001;
+----------+
| count(*) |
+----------+
|      150 |
+----------+
1 row in set (1.33 sec)

after range query optimization:
mysql> select COUNT() from httplogs where size > 20000 and size <= 20001;
+----------+
| count(*) |
+----------+
|      150 |
+----------+
1 row in set (0.05 sec)

Without inverted index:
mysql> set enable_inverted_index_query=false;                                                                                Query OK, 0 rows affected (0.01 sec)

mysql> select COUNT() from httplogs where size > 20000 and size <= 20001;
+----------+
| count(*) |
+----------+
|      150 |
+----------+
1 row in set (0.62 sec)

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run buildall

run  clickbench-new

run clickbench

run buildall

run buildall

run buildall

run buildall


Loading multiple large files (>4GB) onto a lazy array produces deserialization error

Issue description
This is an issue regarding the question posted here: https://stackoverflow.com/questions/64565208/how-can-i-load-many-very-large-files-using-uproot
When loading 3 files of about 9 GB each onto a lazy array, the code crashes when the event loop reaches the borders between the first and second files producing a deserialization error. The code works flawlessly with small (~ 1 GB) files. Way of access (local vs XRootD) also doesn't seem to affect the issue. The error message is the following:
Traceback (most recent call last):
  File "scripts/plotEventShapes_lazy.py", line 44, in <module>
    tracks_x = events['Tracks.fCoordinates.fX'][ievt]
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/awkward1/highlevel.py", line 946, in __getitem__
    self._layout[where], self._behavior, cache=self._cache
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/awkward1/partition.py", line 366, in __getitem__
    return PartitionedArray.from_ext(self._ext.getitem_at(where))
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/uproot4/behaviors/TBranch.py", line 1621, in array
    _ranges_or_baskets_to_arrays(
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/uproot4/behaviors/TBranch.py", line 518, in _ranges_or_baskets_to_arrays
    uproot4.source.futures.delayed_raise(*obj)
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/uproot4/source/futures.py", line 37, in delayed_raise
    raise exception_value.with_traceback(traceback)
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/uproot4/behaviors/TBranch.py", line 444, in chunk_to_basket
    basket = uproot4.models.TBasket.Model_TBasket.read(
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/uproot4/model.py", line 128, in read
    self.read_members(chunk, cursor, context, file)
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/uproot4/models/TBasket.py", line 55, in read_members
    ) = cursor.fields(chunk, _tbasket_format2, context)
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/uproot4/source/cursor.py", line 168, in fields
    return format.unpack(chunk.get(start, stop, self, context))
  File "/Users/chrispap/Library/Python/3.8/lib/python/site-packages/uproot4/source/chunk.py", line 278, in get
    raise uproot4.deserialization.DeserializationError(
uproot4.deserialization.DeserializationError: while reading

    TBasket version None as uproot4.models.TBasket.Model_TBasket (? bytes)
        fNbytes: -1607368158
        fObjlen: -1243566277
        fDatime:<PHONE_NUMBER>
        fKeylen: -27664
        fCycle: 21409

attempting to get bytes 483015:483033
outside expected range 510698:538758 for this Chunk
in file /Users/chrispap/QCD/Autumn18.QCD_HT1500to2000_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root

Reproduction instructions
The three very large files (9 GB each) are the following:
root://cmseos.fnal.gov//store/user/kdipetri/SUEP/Production_v0.2/2018/NTUP/Autumn18.QCD_HT1000to1500_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root
root://cmseos.fnal.gov//store/user/kdipetri/SUEP/Production_v0.2/2018/NTUP/Autumn18.QCD_HT1500to2000_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root
root://cmseos.fnal.gov//store/user/kdipetri/SUEP/Production_v0.2/2018/NTUP/Autumn18.QCD_HT2000toInf_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root

Two smaller files (1 GB) for which the code works nicely:
root://cmseos.fnal.gov//store/user/kdipetri/SUEP/Production_v0.2/2018/NTUP/PrivateSamples.SUEP_2018_mMed-1000_mDark-2_temp-2_decay-darkPho_13TeV-pythia8_n-100_0_RA2AnalysisTree.root
root://cmseos.fnal.gov//store/user/kdipetri/SUEP/Production_v0.2/2018/NTUP/PrivateSamples.SUEP_2018_mMed-1000_mDark-2_temp-2_decay-darkPhoHad_13TeV-pythia8_n-100_0_RA2AnalysisTree.root

The following code reproduces the issue using the files that I cite above:
import uproot4 as uproot
import uproot_methods
import awkward1 as ak
import numpy as np
import matplotlib.pyplot as plt

# Get the file and import using uproot
base = 'root://cmseos.fnal.gov//store/user/kdipetri/SUEP/Production_v0.2/2018/NTUP/'
#base = '/Users/chrispap/QCD/'
datasets = {
    base + 'Autumn18.QCD_HT1000to1500_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root': 'TreeMaker2/PreSelection',
    base + 'Autumn18.QCD_HT1500to2000_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root': 'TreeMaker2/PreSelection',
    base + 'Autumn18.QCD_HT2000toInf_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root': 'TreeMaker2/PreSelection',
}

events = uproot.lazy(datasets)

multiplicity = np.zeros(len(events['Tracks.fCoordinates.fX']))
for ievt in range(99000,len(events['Tracks.fCoordinates.fX'])):
    if ievt%1000 == 0:
        print("Processing event %d. Progress: %.2f%%"%(ievt,100*ievt/len(events['Tracks.fCoordinates.fX'])))
    if events['HT'][ievt] < 1200:
        multiplicity[ievt] = -1
        continue
    tracks_x = events['Tracks.fCoordinates.fX'][ievt]
    tracks_y = events['Tracks.fCoordinates.fY'][ievt]
    tracks_z = events['Tracks.fCoordinates.fZ'][ievt]
    tracks_E = np.sqrt(tracks_x**2+tracks_y**2+tracks_z**2+0.13957**2)
    tracks = uproot_methods.TLorentzVectorArray.from_cartesian(ak.to_awkward0(tracks_x),
                                                               ak.to_awkward0(tracks_y),
                                                               ak.to_awkward0(tracks_z),
                                                               ak.to_awkward0(tracks_E))
    tracks_fromPV0 = events['Tracks_fromPV0'][ievt]
    tracks_matchedToPFCandidate = events['Tracks_matchedToPFCandidate'][ievt]
    tracks = tracks[(tracks.pt > 1.) & (tracks.eta < 2.5) &
                    (ak.to_awkward0(tracks_fromPV0) >= 2) &
                    (ak.to_awkward0(tracks_matchedToPFCandidate) > 0)]
    multiplicity[ievt] = tracks.size

# Plot results
fig = plt.figure(figsize=(8,8))
ax = plt.gca()

CrossSection = events['CrossSection'][events['HT'] > 1200]
multiplicity = multiplicity[events['HT'] > 1200]

ax.hist(multiplicity, bins=100, density=True, weights=CrossSection, histtype='step', color='b')

plt.show()

I don't have authorization to access these files:
% xrdcp root://cmseos.fnal.gov//store/user/kdipetri/SUEP/Production_v0.2/2018/NTUP/Autumn18.QCD_HT1000to1500_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root issue-131.root
[0B/0B][100%][==================================================][0B/s]
Run: [FATAL] Auth failed:  (source)

Ahh. Sorry about that. You probably need to have Kerberos initialized for FNAL. I am uploading them on CERNBox. I will post a link in a few minutes.

I'll only need one 9 GB and one 1 GB, so you can save space.

Alright, I think this should work: https://drive.google.com/drive/folders/1s_sX10H7pN2Xq1JK8QMuoS7i7gpSiOs2?usp=sharing

Actually, I ran it on both and couldn't find a bug. Specifically, what I ran was a translation of your code to its columnar equivalent:
import numpy as np
import awkward1 as ak
import uproot4

# f = uproot4.open("~/storage/data/uproot4-big/issue-131little.root")
f = uproot4.open("~/storage/data/uproot4-big/issue-131big.root")
t = f["TreeMaker2/PreSelection"]

events = t.arrays([
    "HT",
    "CrossSection",
    "Tracks.fCoordinates.fX",
    "Tracks.fCoordinates.fY",
    "Tracks.fCoordinates.fZ",
    "Tracks_fromPV0",
    "Tracks_matchedToPFCandidate",
])  #, entry_start=99000)

cut_events = events[events.HT > 1200]
HT, CrossSection, x, y, z, num_fromPV0, matched_to_candidate = ak.unzip(cut_events)

pt = np.sqrt(x**2 + y**2 + z**2)
eta = np.arcsinh(z / np.sqrt(x**2 + y**2))

track_cut = (pt > 1) & abs(eta < 2.5) & (num_fromPV0 >= 2) & matched_to_candidate
multiplicity = ak.sum(track_cut, axis=1)

which ought to be a significant boost in efficiency (both speed and memory use), but it shouldn't cause or fix a bug. What you showed in StackOverflow had an actual error in deserialization. Perhaps something went wrong in the XRootD source?
(I commented out the limit on entry_start and entry_stop because I could read all events into memory, for these 7 branches. It took about a minute: watching htop, a single Python thread popped up now and then while it was accessing disk, and then there was a burst of 10 threads while it decompressed. The math took almost no time after the I/O was done. Doing the HT cut early vs. late hardly mattered, either.)
(Also, I assume that you wanted to cut on abs(eta < 2.5), not eta < 2.5.)
Since this ran without troubles from a local file, we should suspect the XRootD source. I no longer think it was a "reading position" issue (interpreting the wrong bytes as the TBasket header), because such a thing would affect a local source the same way. Maybe all the memory you were using confused PyXRootD? (If so, this would be a bug, and a subtle one, in PyXRootD's C++.) Or maybe Uproot's XRootDSource has a bug in it? If it's the former, then using the low-resource-usage code I wrote above would at least fix the symptom; if it's the latter, you'd get the same error either way, so try switching to the above with XRootD—at least it would be a good diagnostic.

Oh, I should mention that this took 4.57 GB of memory; if you have less than that, you'll need to use entry_start/entry_stop to split it up. Actually, that's what uproot4.iterate is for: it repeatedly calls arrays with different entry_start/entry_stop values, though it's a little smarter than that in that it avoids re-reading TBaskets that are needed by more than one step in the iteration.

This real issue is that I don't think that I can go fully columnar. In fact, I need to recluster AK15 jets and to my knowledge the best approach so far is to use pyjet on an event loop. (Is there a way to pass jagged arrays with multiple events to pyjet or some equivalent?) My full code reproduces the error locally and looks like:
import uproot4 as uproot
import uproot_methods
import awkward1 as ak
import numpy as np
from math import pi
import matplotlib.pyplot as plt
import mplhep as hep
import pyjet
import matplotlib.colors as colors
import matplotlib.cm as cmx

plt.style.use(hep.style.ROOT)

def sphericityTensor(particles,r=2):
    s = np.zeros((3,3))
    s[0][0] = particles.x.dot(particles.x)
    s[0][1] = particles.x.dot(particles.y)
    s[0][2] = particles.x.dot(particles.z)
    s[1][0] = particles.y.dot(particles.x)
    s[1][1] = particles.y.dot(particles.y)
    s[1][2] = particles.y.dot(particles.z)
    s[2][0] = particles.z.dot(particles.x)
    s[2][1] = particles.z.dot(particles.y)
    s[2][2] = particles.z.dot(particles.z)
    s = s/particles.p.dot(particles.p)
    return s

def sphericity(s):
    s_eigvalues, s_eigvectors = np.linalg.eig(s)
    s_eigvalues = np.sort(s_eigvalues)
    sphericity = 1.5*(s_eigvalues[0]+s_eigvalues[1])
    return sphericity

def makeJets(tracks, R, p=-1):
    # Cluster AK(R) jets
    vectors = np.zeros(tracks.size, np.dtype([('pT', 'f8'), ('eta', 'f8'),
                                              ('phi', 'f8'), ('mass', 'f8')]))
    i = 0
    for track in tracks:
        vectors[i] = np.array((track.pt, track.eta, track.phi, track.mass),
                              np.dtype([('pT', 'f8'), ('eta', 'f8'),
                                        ('phi', 'f8'), ('mass', 'f8')]))
        i += 1
    sequence = pyjet.cluster(vectors, R=R, p=p)
    jets = sequence.inclusive_jets()
    return jets

def isrTagger(jets, warn=True, warnThresh=130):
    mult0 = len(jets[0])
    mult1 = len(jets[1])
    if (mult0 > warnThresh) & (mult1 > warnThresh) & warn:
        print("Warning: both multiplicities are above %d!"%warnThresh)
    elif (mult0 < warnThresh) & (mult1 < warnThresh) & warn:
        print("Warning: both multiplicities are below %d!"%warnThresh)
    if mult0 < mult1:
        return uproot_methods.TLorentzVectorArray.from_ptetaphim([jets[1].pt],
                                                                 [jets[1].eta],
                                                                 [jets[1].phi],
                                                                 [jets[1].mass])
    else:
        return uproot_methods.TLorentzVectorArray.from_ptetaphim([jets[0].pt],
                                                                 [jets[0].eta],
                                                                 [jets[0].phi],
                                                                 [jets[0].mass])

# Get the file and import using uproot
base = 'root://cmseos.fnal.gov//store/user/kdipetri/SUEP/Production_v0.2/2018/NTUP/'
#base = '/User/chrispap/QCD/'
datasets = {
    base + 'Autumn18.QCD_HT1000to1500_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root': 'TreeMaker2/PreSelection',
    base + 'Autumn18.QCD_HT1500to2000_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root': 'TreeMaker2/PreSelection',
    base + 'Autumn18.QCD_HT2000toInf_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root': 'TreeMaker2/PreSelection',
}

events = uproot.lazy(datasets)

evtShape = np.zeros(len(events['Tracks.fCoordinates.fX']))
for ievt in range(len(events['Tracks.fCoordinates.fX'])):
    if ievt%1000 == 0:
        print("Processing event %d. Progress: %.2f%%"%(ievt,100*ievt/len(events['Tracks.fCoordinates.fX'])))
    if events['HT'][ievt] < 1200:
        evtShape[ievt] = -1
        continue
    tracks_x = events['Tracks.fCoordinates.fX'][ievt]
    tracks_y = events['Tracks.fCoordinates.fY'][ievt]
    tracks_z = events['Tracks.fCoordinates.fZ'][ievt]
    tracks_E = np.sqrt(tracks_x**2+tracks_y**2+tracks_z**2+0.13957**2)
    tracks = uproot_methods.TLorentzVectorArray.from_cartesian(ak.to_awkward0(tracks_x),
                                                               ak.to_awkward0(tracks_y),
                                                               ak.to_awkward0(tracks_z),
                                                               ak.to_awkward0(tracks_E))
    tracks_fromPV0 = events['Tracks_fromPV0'][ievt]
    tracks_matchedToPFCandidate = events['Tracks_matchedToPFCandidate'][ievt]
    tracks = tracks[(tracks.pt > 1.) & (abs(tracks.eta) < 2.5) &
                    (ak.to_awkward0(tracks_fromPV0) >= 2) &
                    (ak.to_awkward0(tracks_matchedToPFCandidate) > 0)]
    jetsAK15 = makeJets(tracks, 1.5)
    isrJet = isrTagger(jetsAK15, warn=False)
    tracks_boosted_minusISR = tracks.boost(-isrJet.p3/isrJet.energy)
    s = sphericityTensor(tracks_boosted_minusISR)
    evtShape[ievt] = sphericity(s)

# Plot results
fig = plt.figure(figsize=(8,8))
ax = plt.gca()

CrossSection = events['CrossSection'][events['HT'] > 1200]
evtShape = evtShape[events['HT'] > 1200]

ax.hist(evtShape, bins=100, density=True, weights=CrossSection, histtype='step', color='b')

plt.show()

Perhaps uproot4.iterate could solve this issue?

As I said above, uproot4.iterate iterates over chunks of events, so it doesn't change the for-loop vs. columnnarness. You can certainly use columnar techniques in some parts of your script, even if a restrictive interface prevents you from using it everywhere.
For a long time now, I've been wanting to give pyjet an interface that takes jagged arrays of Lorentz vectors, so that it wouldn't become a bottleneck by having to do Python for-loops over events. As it is, you'll need to do for loops in that one spot. Even this is much better than FastJet's own Python interface, which makes you do for loops over all the particles!

Is this issue still happening?
uproot4.deserialization.DeserializationError: while reading

    TBasket version None as uproot4.models.TBasket.Model_TBasket (? bytes)
        fNbytes: -1607368158
        fObjlen: -1243566277
        fDatime:<PHONE_NUMBER>
        fKeylen: -27664
        fCycle: 21409

attempting to get bytes 483015:483033
outside expected range 510698:538758 for this Chunk
in file /Users/chrispap/QCD/Autumn18.QCD_HT1500to2000_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root

I can't test the XRootD access because of permissions, but we've narrowed it down to that: the bug does not occur with local files. If you can't reproduce it now; I'll close this issue.

The error message above has been produced with local files and the code that I posted when I created the issue. If you need the second large file I can upload it, too.

You can see that the file is local in the last line of the error message, too.

Is the 9 GB file (Autumn18.QCD_HT1000to1500_TuneCP5_13TeV-madgraphMLM-pythia8_RA2AnalysisTree.root) not large enough to cause the error? Could you put a file that is large enough to cause the error on Google Drive?
(This is good, because I'd much rather debug an Uproot issue than an XRootD one!)

In fact, I think that you need two of them since the issue happens exactly when the loop goes from ievt = 99999 to 100000. (i.e. when it goes to the next file). I will post a link a the second file in a few minutes.

That fact will likely be important in the solution; thanks!

Does it work if you try going from one file to itself, or from the small file to the large file, or from the large file to the small file? In any of those configurations, I would be able to reproduce it with the two files I have.

Here's the link https://drive.google.com/drive/folders/1s_sX10H7pN2Xq1JK8QMuoS7i7gpSiOs2?usp=sharing
The other file is the QCD with HT1500to2000. I haven't tried these configurations. I will do it now.

So I tested a few different configurations and it turns out that the issue only occurs when trying to access the first entry of the HT1500to2000 file. Could it be a bad file? That's confusing though because I have run code on that file before and it worked.

It's entirely possible that it's a bad file. It's easy to make mistakes in bookkeeping: more than half the time, when I think I've found a test case that demonstrates an error in the main codebase, it's the test case that's wrong, not the main codebase. I'm downloading it and I'll test it soon. I'm going through a huge backlog of issues—I might as well start with this one.

Yeah, there's something wrong with that file. The "Tracks.fCoordinates.fX" and "Tracks.fCoordinates.fZ" branches have this junk in the first TBasket, and the "Tracks.fCoordinates.fY" has a "count branch" that points to "Tracks.fCoordinates.fX". (It should be pointing to a branch of "number of items per entry", not "x" values.) I used TBranch.debug to search around nearby to see if the file pointer is just a little off, but I didn't see anything that looks like the beginning of a TBasket (which includes some easily recognizable strings, but I didn't see them).
So it's just that one file. Something went wrong while writing it. Also, your TBaskets are very small, like 7 entries each. Reading them would be more efficient if they were big enough that NumPy is doing more work than Python. For these 7 branches, loading 100 MB at a time (big, but can't possibly be a problem for the amount of RAM on your computer), you'd want 4000‒7000 entries per basket:
>>> branches = ["HT", "CrossSection", "Tracks.fCoordinates.fX", "Tracks.fCoordinates.fY", "Tracks.fCoordinates.fZ", "Tracks_fromPV0", "Tracks_matchedToPFCandidate"]
>>> t = uproot4.open("~/storage/data/uproot4-big/issue-131-big1.root:TreeMaker2/PreSelection")
>>> t.num_entries_for("100 MB", branches)
7657
>>> t = uproot4.open("~/storage/data/uproot4-big/issue-131-little.root:TreeMaker2/PreSelection")
>>> t.num_entries_for("100 MB", branches)
4689

Finally, for reference, you'll probably want a procedure like the following for gathering multiplicities (and anything else) from your set of files—it's what I used in debugging:
import numpy as np
import awkward1 as ak
import uproot4

multiplicity_chunks = []

for events, report in uproot4.iterate(
    [
        "~/storage/data/uproot4-big/issue-131-little.root:TreeMaker2/PreSelection",
        "~/storage/data/uproot4-big/issue-131-big1.root:TreeMaker2/PreSelection",
    ], [
        "HT",
        "CrossSection",
        "Tracks.fCoordinates.fX",
        "Tracks.fCoordinates.fY",
        "Tracks.fCoordinates.fZ",
        "Tracks_fromPV0",
        "Tracks_matchedToPFCandidate",
    ], step_size="1 GB", report=True
):
    print(report)

    cut_events = events[events.HT > 1200]
    HT, CrossSection, x, y, z, num_fromPV0, matched_to_candidate = ak.unzip(cut_events)

    pt = np.sqrt(x**2 + y**2 + z**2)
    eta = np.arcsinh(z / np.sqrt(x**2 + y**2))

    track_cut = (pt > 1) & abs(eta < 2.5) & (num_fromPV0 >= 2) & matched_to_candidate
    multiplicity_chunk = ak.sum(track_cut, axis=1)

    multiplicity_chunks.append(multiplicity_chunk)

multiplicity = ak.concatenate(multiplicity_chunks)
print(multiplicity)

I'm going to close this because I think it's your file that's wrong. If you have any other problems, ping this issue or open a new one!


Readme needs to be updated

After switching from RxJava 1.X ->2.X we are no longer using Subscription but Disposable

fixed by #11


Artifacts: Change name to display_name in rill.yaml

Rename name to display_name in rill.yaml and use that as a human readable title of the project displayed in the application.
Make the display_name editable within application, similar to dashboard display names.
During rill init the display name is inferred from the directory name as it is done currently.

@AdityaHegde : Can you look into it ? +cc @hamilton @begelundmuller

Closed by https://github.com/rilldata/rill-developer/pull/2132


chore: avoid runtime array bounds checks

This PR avoids runtime array bounds checks in places where it can trivially be done at compile time.
None of these changes are of particular consequence: I mostly wanted to learn how much we do this, and what the obstacles are to doing it less.

!bench

One papercut I've experience with this is that _ is not permitted as the hypothesis in a for loop (i.e., for _ :  a in as is illegal). This is why I tend to use the as[i]'h syntax even when h is inferred so it is clear where the named variable is actually being used.
Support for for _  : would also make it clear in this PR how many of these hypotheses are actually used in the source.

Oh! I wasn't expecting to see anything significant in the benchmark, but there are some non-trivial improvements there.


question about SUP internet required URL http://ntservicepack.microsoft.com

On machine 1, the TCP test connection is success and reply message from ntservicepack.microsoft.com in netmon trace.
We could see the response from ntservicepark.microsoft.com on port 80.
On machine 2 , customer found the tcp test connection was failed. We captured the network monitor trace and no any response from ntservicepack.microsoft.com.
Note: Machine 2 has proxy and cx add all the URLs list into the bypass list , and only the ntservicepack.microsoft.com couldn’t pass the tcp test on port 80/443.  Interesting that we can resolve the IP address of ntservicepack.microsoft.com on machine 2.
I can understand that ping/ICMP may be not allowed by ntservicepack.microsoft.com, but why the TCP port testing couldn’t pass also.

I tested this in my lab without proxy settings, but get the same symptom with customer on machine 2.
Test-NetConnection ntservicepack.microsoft.com -port 80
WARNING: TCP connect to ntservicepack.microsoft.com:80 failed
WARNING: Ping to ntservicepack.microsoft.com failed -- Status: TimedOut

ComputerName           : ntservicepack.microsoft.com
RemoteAddress          : <IP_ADDRESS>
RemotePort             : 80
InterfaceAlias         : Ethernet 2
SourceAddress          : <IP_ADDRESS>
PingSucceeded          : False
PingReplyDetails (RTT) : 0 ms
TcpTestSucceeded       : False
-NetConnection ntservicepack.microsoft.com -port 443
WARNING: TCP connect to ntservicepack.microsoft.com:443 failed
WARNING: Ping to ntservicepack.microsoft.com failed -- Status: TimedOut
ComputerName           : ntservicepack.microsoft.com
RemoteAddress          : <IP_ADDRESS>
RemotePort             : 443
InterfaceAlias         : Ethernet 2
SourceAddress          : <IP_ADDRESS>
PingSucceeded          : False
PingReplyDetails (RTT) : 0 ms
TcpTestSucceeded       : False
nslookup  ntservicepack.microsoft.com
Server:  ty-win2008r201.tatsuro.com
Address:  <IP_ADDRESS>
Non-authoritative answer:
Name:    ntservicepack.microsoft.nsatc.net
Address:  <IP_ADDRESS>
Aliases:  ntservicepack.microsoft.com
Question:
Does the endpoint ntservicepack.microsoft.com still use for updates ?  If yes, how to verify the TCP connection ?
If no, can we modify the document to avoid confuse ?
[Enter feedback here]

Document Details
⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

ID: 85eb6a21-2659-019a-71ab-66f2e3a7f474
Version Independent ID: ee2ab576-3c19-15a5-8c41-29a00275614b
Content: Step 2 - Configure WSUS
Content Source: WindowsServerDocs/administration/windows-server-update-services/deploy/2-configure-wsus.md
Product: windows-server
Technology: wsus
GitHub Login: @eross-msft
Microsoft Alias: lizross

See a very similar or the same question asked here:

https://docs.microsoft.com/en-us/answers/questions/121284/wsus-update-url.html

For troubleshooting, see the following documentation page:

https://docs.microsoft.com/en-us/troubleshoot/mem/configmgr/windows-server-update-services-best-practices

bing.com web search results related to "ntservicepack.microsoft.com":

https://www.bing.com/search?q="ntservicepack.microsoft.com"+site%3Amicrosoft.com

Suffice it to say that the host ntservicepack.microsoft.com may only be configured to respond to WSUS signals and traffic.
Also, feel free to read some notes I have accumulated during my time contributing to this Microsoft Docs repository:
Wish I could help, but this repository is not intended for regular support issues, support requests or support questions.
You may want to look elsewhere for answers, unless you have suggestions for improving one or more specific pages in the MS Docs repository.
Disclaimer: I am not affiliated with Microsoft or the Microsoft Docs teams here on GitHub.
The intention of this repository is to improve on the existing documentation by providing the users a way to report factual errors, improvement suggestions and request updates for new versions.
The MS Docs team members might not have time to answer your question, both due to the small size of the team and the big workload.
Try the following resources to get in touch with users who may be able to help you:

TechNet forums: https://social.technet.microsoft.com/Forums/en-US/home
MSDN: https://social.msdn.microsoft.com/Forums/en-US/home
Microsoft Community: https://answers.microsoft.com/

#assign:IngridAtMicrosoft

We hugely appreciate your contribution, feedback, and support to our technical documentation.
We’ve phased out GitHub Issues on this repository as the feedback mechanism for content and have replaced it with a new feedback system. For more information see, Content User Feedback. As part of this rollout, we are migrating your issue where our team will use your comments to improve the documentation experience. Unfortunately, this does mean you won't be able to access this issue after the migration is complete.
All issues will be migrated by Thursday 29th August 2024.
Once again, thank you for your valuable feedback, we hope you’ll continue to contribute using our new feedback system.


App getting crashed when scrolling extension option

Steps to reproduce
When I scroll extension option for extension.. App getting crashed/restart.
I don't why , my device's old, Android 6 arm v7a.
I faced the same issue in previous version
Expected behavior
...
Actual behavior
...
LNReader version
1.1.16
Android version
Android 6
Device
LG Stylus2 Dual
Other details
No response
Acknowledgements

[X] I have searched the existing issues and this is a new ticket, NOT a duplicate or related to another open or closed issue.
[X] I have written a short but informative title.
[X] If this is an issue with a source, I should be opening an issue in the sources repository.
[X] I have updated the app to version 1.1.16.
[X] I will fill out all of the requested information in this form.

Could you share logs of the crash? You can use logcat.

Closing this issue as its stale.
You can reopen if the issue still persists.


Should useCode use a callback as first argument?

I'm asking because this is the first time I see a hook with the deps parameter where the first parameter is not a callback. In fact, the eslint-hook plugin won't lint for exhaustive dependencies when the first argument is not a callback: https://github.com/facebook/react/blob/master/packages/eslint-plugin-react-hooks/src/ExhaustiveDeps.js#L64
And indeed now it feels a bit strange that node expression is always "evaluated" for each render even though it should only have effects depending on the deps parameter. What are your thoughts on this?

Hm, now that I think of it I'd say it should always be a callback and never just a list of nodes. Would it make sense to make a breaking change here and change the behavior?

I'd be ok with ok, should the PR return an error if the param if not of type function?


Multiple ruleset configuration per file set

Hello,
I have a question for the community of PHPMD :
The context :
I want to be able to define rulesets according to class usages.
For example :

Class type
Complexity
Public method count
Unit Test Coverage

Controllers
3
10
-

Entites
1
-
0

Services
10
10
90%

Factory
10
-
90%

DAO
5
10
60%

Numbers are just examples to understand, and are not real values I want.
 But, you concentrate your testing effort where complexity can be Services.
Doing this is only useful on heavy projects with many people on it. You cannot be sure that all developpers will be knight of open-source with code quality in mind.
Class Type is often easily identifiable by the fileName,className,namespace
What I do now:
By using Phing/Ant I base role on names of classes with fileset:
build.xml
<phpmd rulesets="MyCustomControllerRuleSet">
  <fileset dir=".">
    <patternset>
      <include name="src/**/*Controllers*"/>
    </patternset>
  </fileset>
  <formatter type="xml" outfile="${project.logdir}/pmd.xml"/>
</phpmd>

This do the job in Jenkins for CI. PhpMd cannot do it by itself.
What are limitation:

You must use Ant/Phing to run phpmd event when your on you dev machine (before commit ?)
You cannot integrate those rules into your IDE (SublimText,PhpStorm,...)
To see what rule is apply to what file, you must read/understand both build.xml and ruleset.xml files

What I suggest
In order to make ruleset more powerful, I suggest adding filters at different level  (Ruleset and Rule)
This is not a small code change
I'm ready to code it with or without help.
Filter I suggest is just custom defined class declared in rulesets with pattern of inclusion/exclusion
Filter can be defined/implemented in PHPMD directly or create by developers.
Filter can apply on fileName, className, NameSpace, Whatever PhpMD is capable of using as Token
What do you think ?

I totally see your problems, but I don't think we get there any soon.
If you are interested in providing a PR, you are welcome.

I think what you are looking for is multiple ruleset configuration per file set within a project.
This could be done in many ways. I see to viable ways:

Configuration per file/class, with inheritance from superclasses. For examle, you set the maximum allowed count of public methods to 12 in AppController, which all Controllers base upon. All Controllers now have this set to 12, except you overwrite it in some Controllers. It's a bit like https://github.com/phpmd/phpmd/issues/30
Being able to set configuration in phpmd.xml for certain file sets. A bit like PHPCS allows to do, e.g. https://github.com/cakephp/cakephp-codesniffer/blob/3.0.0/CakePHP/ruleset.xml#L116-L147. See https://github.com/squizlabs/PHP_CodeSniffer/wiki/Annotated-ruleset.xml.

In any case I don't see we are going to implement this in the near future.


Enable FileSystemWatcher to watch virtual file systems

When we implement a virtual file system, the API spec dictates that we need to implement a watch functionality. We have one in progress here in the azure-storage extension. Thanks @monalgarg!
Setting up a FileSystemWatcher does not keep track of URI's related to the virtual file paths. Details in this StackOverflow post.
One alternative I see is having a side-process that frequently polls the virtual file system for changes, and emits them on VSCode. We noticed that VSCode does a lot of stat calls to the File system (in building the file tree for the explorer), so us implementing this watch seems like there would be :

repeated logic/code across multiple explorer providers.
Inefficient, since VSCode's stat calls would return information that can be used here (for e.g., create and delete events on files and folders). Us making the similar calls again can be rate-limited by the service serving up the virtual files/folders.

Can my blocker be side-stepped by another approach?
If not,
Is there a way to make it easy to have a FileSystemWatcher watch virtual URI's without a lot of repeatable scaffolding?

@PrashanthCorp I do not understand the blocking issue implementing the watch() method for your file system provider?
But it seems to me you are asking for a different way of reporting file change events by switching to a polling based approach? That sounds very inefficient to me, and how would it work exactly? Should we stat each file that is opened?

I was wondering if there's a way to either make vscode.FileSystemWatcher work with virtual file URI's (since fs.watch will not). Else, if you have pointers on implementing our own watcher for our virtual FileSystemProvider, that would be great too.
I agree that polling would be pretty inefficient, so I'm looking to see if there are alternatives.

@PrashanthCorp to clarify, you are implementing watch functionality for your custom file system provider and you say vscode.FileSystemWatcher does not work even though you do that? That would be a pretty serious bug on our end.


fix: remove aws-lambda from expected npm version failures

See https://github.com/DefinitelyTyped/DefinitelyTyped/pull/68667 - I think I'm doing this right...

Yep, thank you!


integration/fixtures-expired: do not force 'rm'

To make gencerts.sh script safer.

Codecov Report

Merging #7874 into master will decrease coverage by 0.03%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #7874      +/-   ##
==========================================
- Coverage    75.7%   75.67%   -0.04%
==========================================
  Files         332      332
  Lines       26203    26203
==========================================
- Hits        19838    19830       -8
- Misses       4929     4939      +10
+ Partials     1436     1434       -2

Impacted Files
Coverage Δ

mvcc/watchable_store.go
68.99% <0%> (-17.06%)
:arrow_down:

pkg/adt/interval_tree.go
84.14% <0%> (-6.41%)
:arrow_down:

proxy/grpcproxy/watcher.go
85.41% <0%> (-4.17%)
:arrow_down:

etcdserver/api/v3rpc/lease.go
92.85% <0%> (-3.58%)
:arrow_down:

etcdserver/v3_server.go
70.07% <0%> (-0.24%)
:arrow_down:

lease/leasehttp/http.go
62.59% <0%> (ø)
:arrow_up:

clientv3/lease.go
92.24% <0%> (+0.77%)
:arrow_up:

rafthttp/transport.go
83.05% <0%> (+1.12%)
:arrow_up:

rafthttp/peer.go
91.6% <0%> (+1.52%)
:arrow_up:

proxy/grpcproxy/kv.go
95.23% <0%> (+1.58%)
:arrow_up:

... and 10 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 3b251b0...4edbae4. Read the comment docs.


Run workflow on @eth-bot ping (Fixes #49)

Completely untested

As a stop-gap, how about I open a pull that puts information on how to re-trigger the bot (e.g. closing and reopening the pull) in EIP-1?

I'm a bit hesitant because I suspect that no one reads the Readme (or EIP-1) and the few people who do probably are experienced EIP editors who already know they can close/reopen to kick things.  Adding more content to the readme/EIP-1 is likely to further reduce the number of people who actually read it.
Do we have reason to believe that this will be a net positive, even after the side effects?

Do we have reason to believe that this will be a net positive

You're probably right. Even if people read the readme and EIP-1 (I certainly did), it's long enough that such a detail will probably be forgotten. I'll close this for now.


Docs: Mixed languages

The English version of the getting started guide for the Pico kit has a mixed (what I presume is) Chinese and English navigation area.
http://esp-idf.readthedocs.io/en/latest/get-started/get-started-pico-kit.html
Update: It seems the mix is present in other pages as well:
http://esp-idf.readthedocs.io/en/latest/get-started/idf-monitor.html?highlight=jtag
This is how it looks for me

Hi @PerMalmberg,
Thank you for pointing this out.
I have submitted a pull request to get it resolved.

Languages can now be chosen using pop-up panel at the bottom of the navigation area.


Fail to debug C# code using System.Data.SqlClient in VSCode.

Does this issue occur when all extensions are disabled?: Yes/No

VS Code Version: 1.80.2
OS Version: Linux Mint x86_64

Debug C# code using System.Data.SqlClient and throws "System.PlatformNotSupportedException: System.Data.SqlClient is not supported on this platform". from SqlConnection .ctor.
The exception comes up when debugging in VSCode or running "dotnet run" CLI, but all right when running "dotnet run -a x64". So the problem seems to be the debugger can't select correct platform or load correct version of library. How to tell debugger of VSCode to use x64 mode and load 64bit version, not 32bit or other versions libraries?
Excuse me for posting the issue here though it may not be bug of vscode. I know the proper place should be StackOverflow, but I have no way to do that because StackOverflow needs Captcha validation before post action, which is blocked in China. And I don't know if I can access github tomorrow via unstable vpn, so please kind helpers send me a email with solution to<EMAIL_ADDRESS>besides sharing your ideas here.

Thanks for creating this issue! It looks like you may be using an old version of VS Code, the latest stable release is 1.82.0. Please try upgrading to the latest version and checking whether this issue remains.
Happy Coding!

This sounds like an issue with your code, or with .NET, not with VS Code


Company custom field page - Action is not available when we select multiple companies

Describe the bug

when we select multiple company action is not showing

https://www.berrycast.com/conversations/e9354922-6a9c-5cd6-b0df-2c2f9f5f6907

Adding Email / SMS / Notification to action
OR removing Action button


Error in running

After successful installation, I tried to run a few test cases, and none were running.
Current behavior
One of the test cases I tried is test/LowerColorado_TX.
Error
Traceback (most recent call last):
  File "/usr/lib64/python3.9/runpy.py", line 197, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib64/python3.9/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.9/site-packages/nwm_routing/__main__.py", line 2028, in <module>
    main_v04(v_args[1])
  File "/usr/local/lib/python3.9/site-packages/nwm_routing/__main__.py", line 54, in main_v04
    ) = _input_handler_v04(args)
  File "/usr/local/lib/python3.9/site-packages/nwm_routing/input.py", line 44, in _input_handler_v04
    troute_configuration = Config(**data)
  File "pydantic/main.py", line 341, in pydantic.main.BaseModel.__init__
pydantic.error_wrappers.ValidationError: 6 validation errors for Config
network_topology_parameters -> supernetwork_parameters -> geo_file_type
  extra fields not permitted (type=value_error.extra)
network_topology_parameters -> waterbody_parameters -> rfc
  extra fields not permitted (type=value_error.extra)
compute_parameters -> data_assimilation_parameters -> streamflow_da -> wrf_hydro_lastobs_file
  extra fields not permitted (type=value_error.extra)
compute_parameters -> data_assimilation_parameters -> reservoir_da -> gage_lakeID_crosswalk_file
  extra fields not permitted (type=value_error.extra)
compute_parameters -> data_assimilation_parameters -> reservoir_da -> reservoir_persistence_usace
  extra fields not permitted (type=value_error.extra)
compute_parameters -> data_assimilation_parameters -> reservoir_da -> reservoir_persistence_usgs
  extra fields not permitted (type=value_error.extra)

Originally, I needed to run the NextGen model with AWI Dataset, but we are getting Segmentation fault (issue CIROH-UA/NGIAB-CloudInfra#21). After debugging, it seems the problem could be related to the t-route.  In case the t-route is not a problem, then I am sure any guidance in other directions will help, too.

@TrupeshKumarPatel I believe we can close this issue with #688


Feature/rex add ansible devtools

这个 PR 实现什么功能/修复什么问题:
增加 ansible template 功能，批量创建、管理模板配置
是否需要 backport 到之前的 release 分支:

release/2.13

Codecov Report

Merging #3851 into master will decrease coverage by <.01%.
The diff coverage is 0%.

@@            Coverage Diff            @@
##           master   #3851      +/-   ##
=========================================
- Coverage    6.93%   6.93%   -0.01%
=========================================
  Files         499     500       +1
  Lines       84385   84418      +33
=========================================
- Hits         5856    5854       -2
- Misses      78034   78068      +34
- Partials      495     496       +1

Flag
Coverage Δ

#aFlag
6.93% <0%> (-0.01%)
:arrow_down:

Impacted Files
Coverage Δ

pkg/cloudcommon/db/db_dispatcher.go
0% <ø> (ø)
:arrow_up:

pkg/mcclient/options/devtooltemplate.go
0% <0%> (ø)

pkg/cloudcommon/db/lockman/fifo.go
80% <0%> (-10%)
:arrow_down:

pkg/baremetal/agent.go
0% <0%> (ø)
:arrow_up:

pkg/compute/models/managedresource.go
0% <0%> (ø)
:arrow_up:

pkg/compute/models/capabilities.go
0% <0%> (ø)
:arrow_up:

pkg/compute/guestdrivers/managedvirtual.go
0% <0%> (ø)
:arrow_up:

pkg/baremetal/manager.go
0.51% <0%> (ø)
:arrow_up:

pkg/compute/models/guests.go
0.59% <0%> (ø)
:arrow_up:

... and 3 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 6c63b0d...59f2268. Read the comment docs.

/lgtm

/lgtm

/approve


Fix automatic module name

Make sure database and communitiy modules have a differnt automatic module name then the core module
Fixes https://github.com/flyway/flyway/issues/3342

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

Can someone please merge it? We cannot upgrade to Flyway 8+ due to this issue and considering stop using Flyway

This change is probably not sufficient, as these modules also have split packages with each other (specifically, the package org.flywaydb.database), and therefor would not be loadable as a module if more than one of these is on the module-path.