Move VMs to HostPid Namespace

Short explanation: VM processes are now in pid namespace. Virt-launcher detects VM pid through a pid file. POD to VM process coupling is guaranteed and enforced through a watchdog informer
Host Pid Namespace Changes

qemu wrapper launches process in host pid namespace. Records pid to a pid file on a shared host mount
Virt-Launcher detects VM process through pid file on shared host mount, verifies pid is in virt-launcher's cgroup slice before monitoring

VM Process Cleanup Guarantees

virt-launcher creates watchdog file and periodically updates the modification time of that file by touching it.
virt-handler requires watchdog file from virt-launcher before allowing vm to be started.
virt-handler uses watchdog informer to learn about expired virt-launcher's
virt-handler destroys VMs with expired watchdog files.

Misc

pidfiles, watchdog files, and virt-launcher sockets have been moved to one flat shared host mount directory.

retest this please

retest this please

This change broke migrations. I'm still in the process of investigating how this happened.  It appears to have something to do with the qemu wrapper script not forwarding the signal properly.

I fixed the issue with migrations.  The qemu wrapper just needed to be updated to forward signals to the right process now that nsenter isn't used.  This only impacted migrations because shutdown originates from virt-launcher in all other situations our test suite covers.

@rmohr @stu-gott  @vladikr
We have a k8s level guarantee still that VM processes will not be orphaned by virt-launcher even when the host pid namespace is used. The k8s runtime kills all processes in a container's cgroup during termination regardless of the pid namespace. This means k8s enforces a VM process will not outlive a POD regardless of the availability of the KubeVirt runtime.
This is trivial to reproduce.
Step 1. launch a container in the k8s cluster on the master node.
cat << END > fake-virt-launcher.yaml
apiVersion: v1
kind: Pod
metadata:
  name: fake-virt-launcher
  namespace: default
spec:
  containers:
  - command:
    - "/bin/bash"
    - "-c"
    - "sleep 22"
    image: kubevirt/libvirt-kubevirt:devel
    imagePullPolicy: IfNotPresent
    name: virt-launcher
  hostPID: true
  securityContext:
    runAsUser: 0
  nodeName: master
  restartPolicy: Never
END

kubectl create -f fake-virt-launcher.yaml

Step 2. Run this script on the master node while the container is running.
export CONTROLLERS='cpu,cpuacct,cpuset,freezer,memory,net_cls,perf_event' &&
export SLICE=$(cat /proc/$(ps aux | grep '[s]leep 22' | grep -v docker | awk -F' ' '{print $2}')/cgroup | grep memory | awk -F':' '{print $3}')
echo "cgroup info ${CONTROLLERS}:$SLICE"
cgclassify -g ${CONTROLLERS}:$SLICE --sticky $$

When the container exits, the terminal on the host gets killed. I get the same immediate result when I force kill a virt-launcher process with SIGKILL. The VM process immediately "crashes"
Even with the k8s cgroup cleanup guarantee, I'm still hesitant to remove the watchdog logic. The watchdog is the only way we have from recovering if the k8s runtime fails in a way that results in cgroups not properly being cleaned up.

With this change, we have a k8s level guarantee that VM processes will not be orphaned by virt-launcher even when the host pid namespace is used. The k8s runtime kills all processes in a container's cgroup during termination regardless of the pid namespace. This means k8s enforces a VM process will not outlive a POD regardless of the availability of the KubeVirt runtime.

Very good! :+1: That will help us a lot!

I think we have everything we want now.

qemu is in host pid namespace
qemu process inherits the wrapper's pid as if the wrapper doesn't even exist. No [wrapper to qemu] signal forwarding required. no hidden process trees involved.
k8s runtime kills qemu process (if it still exists) when virt-launcher terminates
kubevirt runtime enforces qemu termination through the use of a watchdog (in the event k8s runtime fails to clean up cgroups)

On the first round it looks excellent. I want to have a closer look into the watchdog after the weekend.

@rmohr one thing to think about.  I'm using a pidfile now as the discovery mechanism for virt-launcher to learn about the qemu pid.... however, it's technically possible for us to just scan the process tree and find the qemu process that's in virt-launcher's cgroup slice. If I did that, we wouldn't have to have the pidfile at all.

@davidvossel just to clarify the point about the pidfile: virt-launcher is still in its own PID namespace, but qemu is outside of it (host PID namespace), correct? So this basically means virt-launcher has to blindly trust whatever the pidfile says?

just to clarify the point about the pidfile: virt-launcher is still in its own PID namespace, but qemu is outside of it (host PID namespace), correct? So this basically means virt-launcher has to blindly trust whatever the pidfile says?

@stu-gott  Both processes are in host pid namespace now.  Previously, virt-launcher and qemu lived in an isolated pid namespace, so it was safe for virt-launcher to assume any qemu process that it could see was the right one for it to monitor.
Now that virt-launcher and qemu are in host pid namespace, virt-launcher can see many qemu processes across the host. Somehow we have to assign pids to virt-launchers in the global host pid namespace. The pidfiles do this.  The other option I'm considering is to scan the proc directory for a qemu process in the virt-launcher cgroup. I'm already doing a validation check for the pid in the pidfile that guarantees virt-launcher will only monitor a qemu pid in its cgroup slice.

Minor, in the initial comment you state:

Short explanation: VM processes are now in pid namespace.

you probably want

Short explanation: VM processes are now in the host pid namespace.

retest this please

update

addresses latest round of feedback.
removes use of pidfile. Pids are now detected by scanning cgroups
virt-launcher now uses kubevirt logging package
functional tests verify VM process is stopped when virt-launcher is kill -9'ed

@davidvossel could you consider to add the pid file again? I was really happy that virt-launcher now had a way of seeing if the qemu process was already there. With the cgroup as the main mechanism, there is always the issue that we don't know if the qemu process disappeared (normally because of misconfiguration). The flow for virt-launcher is then very simple: If it is there, keep running, if it was there, stop. The same for virt-handler: For as long as the pod detector can detect the pod, continue trying to start. If it is gone, move the VM to failed.

could you consider to add the pid file again? I was really happy that virt-launcher now had a way of seeing if the qemu process was already there. With the cgroup as the main mechanism, there is always the issue that we don't know if the qemu process disappeared (normally because of misconfiguration). The flow for virt-launcher is then very simple: If it is there, keep running, if it was there, stop. The same for virt-handler: For as long as the pod detector can detect the pod, continue trying to start. If it is gone, move the VM to failed.

@rmohr  I agree that in the event that a VM fails during initialization that the virt-launcher container needs to exit early instead of waiting for the full timeout period. I don't think the pidfile is the right mechanism for this. A VM's process fast exiting is only one reason that prevents the VM from starting successfully. There are other things that can block the VM from starting that don't involve the VM process at all (Like a misconfigured k8s secret for example)  I want the virt-launcher pod to exit early if the vm reaches a finalized state for any reason regardless if the qemu process was actually invoked or not.
In general, under normal operation, the component that creates a resource should be in charge of it's lifecycle.
virt-handler creates the local VM process, so virt-handler should manage the entire lifecycle of the process including timeouts associated with startup and shutdown.
Similarly, virt-controller creates the virt-launcher pod and should manage the entire lifecycle of the pod. I think the virt-launcher timeout is just a safety mechanism that gives us a guarantee that a virt-launcher pod won't live forever without a corresponding VM process.
I want to continue this discussion, but I think it's outside of the scope of this patch. I've written some code in a separate PR to address this. I propose we continue this discussion over there. #512

latest update adds a functional test to verify virt-launcher actually attaches to the VM processes on initialization.

I want to continue this discussion, but I think it's outside of the scope of this patch. I've written some code in a separate PR to address this. I propose we continue this discussion over there. #512

Works for me :+1:

For reference: https://rfenkhuber.fedorapeople.org/jenkins/452/console.log

retest this please

retest this please

For reference: https://rfenkhuber.fedorapeople.org/jenkins/452/console.log

@rmohr PR #488 fixes the transient regression test failure.

rebased and addressed latest round of feedback.


Connection to anoher port doesn't supported

My DB is available through another port (!= 3306).
I can't found a way to use port as the parameter.
localhost:port doesn't working.

I was careless


Orientation in the observations

Hi
I was wondering if including the orientation or rotational speed in the observations would be make sense.
Currently it can be inferred from the velocity but when the agent is just rotating in one place there's only the visual information.
Thanks

Hi,
This is actually not feasible as the agent technically doesn't have an angular velocity. When you send a non zero turn action, it rotates the agent by 6 degrees which changes its orientation instantaneously. Your agent will not have any angular momentum either.

I see, thanks.


feat: add InMemoryMetricExporter

Which problem is this PR solving?
Introduces the InMemoryMetricExporter-class which collects metrics and stores it in memory
Fixes #3035 (issue)
Short description of the changes
Adds the InMemoryMetricExporter-class
Type of change
Please delete options that are not relevant.

[X] New feature (non-breaking change which adds functionality)

How Has This Been Tested?
Unit tests
Checklist:

[ ] Followed the style guidelines of this project
[ ] Unit tests have been added
[ ] Documentation has been updated

@open-telemetry/java-instrumentation-approvers ready to review I think :)

@weyert hope you don't mind I went ahead and made the changes so we can get this merged. @legendecas please take another quick look so we can get this merged.


Bumped mongoid from RC to stable

Coverage remained the same at 99.738% when pulling b274b372d98ff5dd2f0af2bc1eae7c628760ae36 on sivagollapalli:mongoid6_support into 4a69aec69b633cd0eaaee221fe7c4b1234434ff5 on mongoid:master.

We should add a CHANGELOG line like "compatibility with mongoid 6", please.

And thanks much!

Coverage remained the same at 99.748% when pulling bdbe0fd2ed2f54f3b6f8f52e0898c2b240bfa3c4 on sivagollapalli:mongoid6_support into b2e100fc6ca1d09545948a1cfd714d1369f26078 on mongoid:master.

Merged, thanks.


Cds 76640 git ex bidirectional sync

Harness Developer Pull Request
Thanks for helping us make the Developer Hub better. The PR will be looked at
by the CODEOWNERS.
What Type of PR is This?

[ x] Issue
[ ] Feature
[ ] Maintenance/Chore

If tied to an Issue, list the Issue(s) here:

https://harness.atlassian.net/browse/CDS-76640

House Keeping
Some items to keep track of. Screen shots of changes are optional but would help the maintainers review quicker.

[ ] Tested Locally
[ ] Optional Screen Shoot.

Please check the Execution Link of the Pipeline for the Website Draft URL. This is located in the Preview Step behind the Harness VPN and also is available in #hdh_alerts. E.g Website Draft URL: https://unique-id--harness-developer.netlify.app


Validate Response Body Schemas

See #33

I'm so looking forward to this getting eventually merged after more than two years. @Piefayth would you mind rebasing to resolve the conflicts?

This is updated (I had to include a small change, since the tests changed directories), but there is currently a failing test for an external spec that does not define the top level swagger property. Looks like an OAPI 3.0 spec, which makes sense.

I'm sorry but my PR that just got merged caused another conflict in your spec file 😦

@BigstickCarpet or @Piefayth would one of you be willing to resolve the conflicts so we could finally profit from the goodness this feature brings?

@marcelstoer  - I'm taking a look at it now


devDependencies are not put in the right place

React version: LTS
Steps To Reproduce
Run npx create-react-app my-app  --template typescript
Check package.json
The current behavior
Dependencies are shown like this:
  "dependencies": {
    "@testing-library/jest-dom": "^5.11.4",
    "@testing-library/react": "^11.1.0",
    "@testing-library/user-event": "^12.1.10",
    "@types/jest": "^26.0.15",
    "@types/node": "^12.0.0",
    "@types/react": "^17.0.0",
    "@types/react-dom": "^17.0.0",
    "react": "^17.0.1",
    "react-dom": "^17.0.1",
    "react-scripts": "4.0.3",
    "typescript": "^4.1.2",
    "web-vitals": "^1.0.1"
  }

The expected behavior
  "dependencies": {
    "react": "^17.0.1",
    "react-dom": "^17.0.1",
    "react-scripts": "4.0.3",
    "web-vitals": "^1.0.1"
  },
  "devDependencies": {
    "@testing-library/jest-dom": "^5.11.4",
    "@testing-library/react": "^11.1.0",
    "@testing-library/user-event": "^12.1.10",
    "@types/jest": "^26.0.15",
    "@types/node": "^12.0.0",
    "@types/react": "^17.0.0",
    "@types/react-dom": "^17.0.0",
    "typescript": "^4.1.2"
  }

Until this is fixed, to save future readers some time:
npm uninstall -S @testing-library/jest-dom @testing-library/react @testing-library/user-event @types/jest @types/node @types/react @types/react-dom typescript
npm install -D @testing-library/jest-dom @testing-library/react @testing-library/user-event @types/jest @types/node @types/react @types/react-dom typescript

I would also assume react-scripts are a devDependency.


Exception when using Reader.SetUpData3DPointsData

Hello,
I'm having problem using Reader.SetUpData3DPointsData included from E57SimpleReader.
Here is my simple code :
#include <iostream>
#include "E57SimpleReader.h"
#include "E57SimpleData.h"
#include "E57Exception.h"

using namespace std;
using namespace e57;

int main() {
    Reader reader("bunnyDouble.e57");
    E57Root fileHeader;
    Data3D data3DHeader;
    Data3DPointsData buffers;

    reader.GetE57Root(fileHeader);
    cout << "FormatName : " << fileHeader.formatName << endl;
    cout << "e57LibraryVersion : " << fileHeader.e57LibraryVersion << endl;
    cout << "data3DSize : " << fileHeader.data3DSize << endl;

    reader.ReadData3D(0, data3DHeader);
    cout << "name : " << data3DHeader.name << endl;

    if (reader.IsOpen()) {
        cout << "Open" << endl;
        try {
            cout << "Here A " << endl;
            CompressedVectorReader dataReader = reader.SetUpData3DPointsData (0, 2, buffers);
            cout << "Here B" << endl;
        } catch (E57Exception &e) {
            cout << "exception caught in (" << e.context() << "): " << Utilities::errorCodeToString(e.errorCode()) << '\n';
        }
    } else
        cout << "Close" << endl;

    return 0;
}

And here is the output :
FormatName : ASTM E57 3D Imaging Data File
e57LibraryVersion : InteliSum-LD3-Studio-V5.1-E57RefImpl-1.0.154-x86-windows
data3DSize : 1
name : bunny
Open
Here A
exception caught in (fileName=bunnyDouble.e57): bad API function argument provided by user (E57_ERROR_BAD_API_ARGUMENT)

Honestly, I don't know what went wrong. Everything seems fine.
If I trace the line that throw the exception, I guess the line that throw the exception is in E57FormatImpl.cpp, line 486-490 :
/// dbufs can't be empty
   if ( dbufs.empty() )
   {
      throw E57_EXCEPTION2( E57_ERROR_BAD_API_ARGUMENT, "fileName=" + destImageFile->fileName() );
   }

Should the buffer initialized first? How?
A correct example of usage is apreciated.
Note: I'm a C++ newbie. But have a little experience in Java. So please don't bully me if somehow I made an elementary mistake.

Should the buffer initialized first? How?

Yes - it seems that Data3DPointsData is a bunch of pointers to buffers that you are supposed to allocate. Since you aren't doing so, it gives an error because there's nothing to do. It looks like it needs count space (in your example, "2") for each item.
My guess is that if you just want the (x, y, z), for example, you would allocate space for those in buffers (cartesianX, cartesianY, cartesianZ) and the data will be filled in. Something like:
const int cBufferSize = 2;

buffers.cartesianX = new float[cBufferSize];
buffers.cartesianY = new float[cBufferSize];
buffers.cartesianZ = new float[cBufferSize];

(Don't forget to delete the pointers when you're done with them.)
@ptc-jhoerner Would know more about this than I do and can correct me if I'm wrong.

I see, it's all settled now. The code run without exception.
It seems I did make elementary mistake.
Thank you so much


Add finalize market to market page if I'm the market creator

Although the MC can finalise from the notification, if they decide to click on the market first it's missing the finalize button
Design:
https://www.figma.com/file/fLWVwmanAwetVZbujQquEi/Market-Page?node-id=3533%3A0

closing - not needed


TDS 326 - Visual Regression

As discussed during stand up. Here is what I have working:
yarn test:2e2
Should we be moving the command to the individual package.json instead?
I have also identified the pattern needed for the scaffolding script.

Should we be moving the command to the individual package.json instead?

Maybe? But would rather not have to duplicate that into every package.json, because then when we want to change something we have to do the same change 40 times!
Instead, we can try to run commands from the root package and scope them by component there.
Here is what I'm imaging would be cool
yarn test:e2e core-button

# Then, we could integrate lerna. The "lerna updated" command should return modified packages.
# So something like this could pipe the updated package names into the yarn test:e2e command to run the tests on all updated components. :)
# This is similar to how the "husky" precommit stuff works, it passes modified files into prettier/linter
lerna updated | yarn test:e2e

Oh, another request. Seems like you got a lot of good stuff from another repo, I think from Jeff Chang. May be good to include some comments telling future maintainers of this code where the code came from so that we can reference it if needed


Validation dataset

Im curious - can anybody help me understand why there's a separate validation dataset for the auto-encoder?
If the purpose of the autoencoder is to take us from (categorical, discreet) to (multinomial, continuous) and then back to (categorical discreet) then isn't it better if the network is trained on all of the samples, even to the point of overfitting? I'm saying this because we're not trying to train the network to generalize molecular structure - it's simply a transformer. Once we construct the latent space then it's up to us to analyse it, but shouldn't the autoencoder simply be trained to encode and decode, and not to generalize?
We still get predictive ability by manipulating transforms through the latent space (as a totally separate task), then the decoder can spit out the molecule that was the result of our latent transform.
No?

I don't think you want want your network to overfit (eg: memorize the training set) because eventually you want to use the model to encode molecules not seen in the training set, right? A validation dataset provides metrics on how well the model is doing for a single run, and across multiple runs can be used to tune hyperparameters like --latent_dim to an optimal value. Also, keep in mind a latent space can support all sorts of other operations, such as classification, interpolation, extrapolation, analogy, attribute vectors, etc. But to test these operations, it probably best to use data not in the training set.

I agree with you, @alainrichardt, that it is not totally necessary to have a separate validation set. @dribnet makes good points on how you might utilize a validation set. However, I want to point out that the VAE doesn't promise to give a consistent latent space representation for a particular input, nor a particular decoding of a latent space point, so "accuracy" is a fuzzy measure that depends on exactly how your input is autoencoded at that instance. More than just wanting to encode molecules outside of the training set, you would also like to decode sampled latent space points that aren't representations of your dataset, and verify that calculated characteristics are valid. Then, your "validation" set might not actually be molecules, but sampled regions near known points in the latent space.


Addresses v12.1 changes and incompatibilities in GTM sub-module

Fixes Issues: #719,  #679
Updated Files:
test/functional/gtm/test_pool.py
test/functional/test_server.py
f5/bigip/tm/gtm/pool.py
f5/bigip/tm/gtm/server.py

Updates:
As we started adding Partition to the selflinks in 12.1.0, some tests needed to be amended. There is a serious bug in 12.x branch with members, as described in #719, I have had to create custom create methods for 2 member classes in pools.py.
Corrected subcollection resource names to be singular not plural as per discussion with Tim, amended tests accordingly

will address the coverage of pool.py in separate PR, tracked under: #680

looks like test coverage dropped significantly with the necessary changes.

we can live with that for now :)


Updated GitBlade gui client: added Windows version and updated the price

Yes indeed, Mac should be available within a couple of days, but currently it’s just windows and Linux :)
Thanks for the merge!


ValueError: Dimensions {'bounds'} do not exist. Expected one or more of ('lon', 'bnds')

I've been trying to use xESMF to regrid an existing dataset to a coarser grid taken from a different example file. I've tried many variations but I have not managed to solve the problem... The original dataset has a non-coordinate dimension called "bnds" which has a value of 2, whereas the new dataset doesn't have it since no variable of interest uses "bnds". Here is my python3 script:
import matplotlib.pyplot as plt
import cartopy.crs as ccrs
import numpy as np
import xarray as xr
import xesmf as xe
from netCDF4 import Dataset
import h5py
import shutil
import subprocess as sub
import os
import glob
import sys
years_list = [2000,2001,2002,2003,2004,2005,2006,2007,2008,2009]
data_DIR_root = '/my_path1/'
output_DIR_root = '/my_path2/'
reference_file_for_regridding = '/my_path3/file_pluc_Newloc.nc'
reference_file_for_regridding_name_only = os.path.basename(reference_file_for_regridding)
data_DIR = data_DIR_root
list_of_files = glob.glob(data_DIR + 'gpm_3imerg_v06b_20*.nc')
if not os.path.exists('./xESMF_regrid/'):
os.makedirs('./xESMF_regrid/')
for file in list_of_files:
print( "WORKING ON FILE " + file)
file_name_only = os.path.basename(file)
file_name_only_no_ext = os.path.splitext(file_name_only)[0]

output_file = os.path.join(output_DIR_root, 'xESMF_regrid', '%s_xESMF_regridded.nc' % (file_name_only_no_ext) )
output_file_name_only = os.path.basename(output_file)

ds = xr.open_dataset(file)
dr = ds['precipitationCal']
ds.rename_dims({'bnds': 'bounds'})
print(ds)

ds_ref = xr.open_dataset(reference_file_for_regridding)
ds_ref_lat = ds_ref.lat
ds_ref_lon = ds_ref.lon
print(ds_ref_lat)

ds_out = xr.Dataset({'lat': (['lat'], ds_ref_lat),
                     'lon': (['lon'], ds_ref_lon),
                    }
                   )
print(ds_out)

regridder = xe.Regridder(ds, ds_out, 'conservative', periodic=True)
dr_out = regridder(dr)
dr_out.to_netcdf(output_file)

And I get the following error message:
[gadi-login-03.gadi.nci.org.au:2763401] shmem: posix: file name search - max attempts exceeded.cannot continue with posix.
[gadi-login-03.gadi.nci.org.au:2763702] shmem: posix: file name search - max attempts exceeded.cannot continue with posix.
Traceback (most recent call last):
File "regridding_dataset_with_xESMF_v1.py", line 63, in
regridder = xe.Regridder(ds, ds_out, 'conservative', periodic=True)
File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.01/lib/python3.8/site-packages/xesmf/frontend.py", line 689, in init
grid_in, shape_in, input_dims = ds_to_ESMFgrid(
File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.01/lib/python3.8/site-packages/xesmf/frontend.py", line 129, in ds_to_ESMFgrid
lon_b, lat_b = _get_lon_lat_bounds(ds)
File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.01/lib/python3.8/site-packages/xesmf/frontend.py", line 75, in _get_lon_lat_bounds
lon_b = cfxr.bounds_to_vertices(lon_bnds, 'bounds', order=None)
File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.01/lib/python3.8/site-packages/cf_xarray/helpers.py", line 53, in bounds_to_vertices
values = bounds.transpose(bnd_dim, *old_dims).data
File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.01/lib/python3.8/site-packages/xarray/core/dataarray.py", line 2210, in transpose
dims = tuple(utils.infix_dims(dims, self.dims, missing_dims))
File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.01/lib/python3.8/site-packages/xarray/core/utils.py", line 788, in infix_dims
existing_dims = drop_missing_dims(dims_supplied, dims_all, missing_dims)
File "/g/data3/hh5/public/apps/miniconda3/envs/analysis3-21.01/lib/python3.8/site-packages/xarray/core/utils.py", line 879, in drop_missing_dims
raise ValueError(
ValueError: Dimensions {'bounds'} do not exist. Expected one or more of ('lon', 'bnds')
The file to regrid can be downloaded from the hyrax server (https://gpm1.gesdisc.eosdis.nasa.gov/opendap/hyrax/GPM_L3/GPM_3IMERGHH.06/2020/001/contents.html), using the opendap protocol, in netcdf4 format instead of the original HDF5.
The reference file from which to take the new grid can be found at https://wetransfer.com/downloads/f017ea89c61431310b3889e4897c56f120210709035420/a8b5f112e6b89266597bc5415865112020210709035729/6be6f6

This version of the package is no longer maintained. Would you mind moving your issue to https://github.com/pangeo-data/xESMF?


Support IF statement in Transform function for post aggregation

This is for queries like:
SELECT
   IF AVG(met1) == 0 THEN 0 ELSE SUM(met0) / AVG(met1)
FROM "auctions"
WHERE event_time =<PHONE_NUMBER>000
LIMIT 1

@Jackie-Jiang does pinot support transform function on top of aggregations?

Yes, modeled as post-aggregation


Update compiler-interface to 1.5.1

Updates org.scala-sbt:compiler-interface from 1.4.4 to 1.5.1.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-sbt", artifactId = "compiler-interface" } ]

labels: library-update, semver-minor

Superseded by #41.


Designing "autograder"

The code is on the grading branch
To do:

[ ] Put in module -> Bram
[ ] Update example notebook -> Bram

Any ideas where we can let the students upload the finished notebooks? @MichielStock

We might also give a small homework notebook that they have to hand in lieu of everything? We only have to provide some evaluation.

I would suggest that they hand in all the notebooks and projects and that
we just automatically count all the correct answers too conclude if they
pass.
Op ma 7 dec. 2020 21:58 schreef Michiel Stock<EMAIL_ADDRESS>
We might also give a small homework notebook that they have to hand in
lieu of everything? We only have to provide some evaluation.
—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/Beramos/DS-Julia2925/issues/6#issuecomment-740175813,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACYUUXC2XQVVWIVLSJ5ORCTSTU6V3ANCNFSM4T327GEQ
.

Or is that not a good idea?

Seems good. I saw you made a tracker for the number of questions that were right, curious to try it out.
M

On 8 Dec 2020, at 10:06, BramDeJaegher<EMAIL_ADDRESS>wrote:
Or is that not a good idea?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/Beramos/DS-Julia2925/issues/6#issuecomment-740487292, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABN5PI3G4VWPSSTJTYMUYRDSTXUBZANCNFSM4T327GEQ.

Ok, I'll add an example asap then you can try it out :)
Op di 8 dec. 2020 10:09 schreef Michiel Stock<EMAIL_ADDRESS>
Seems good. I saw you made a tracker for the number of questions that were
right, curious to try it out.
M

On 8 Dec 2020, at 10:06, BramDeJaegher<EMAIL_ADDRESS>wrote:
Or is that not a good idea?
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub <
https://github.com/Beramos/DS-Julia2925/issues/6#issuecomment-740487292>,
or unsubscribe <
https://github.com/notifications/unsubscribe-auth/ABN5PI3G4VWPSSTJTYMUYRDSTXUBZANCNFSM4T327GEQ
.

—
You are receiving this because you were assigned.
Reply to this email directly, view it on GitHub
https://github.com/Beramos/DS-Julia2925/issues/6#issuecomment-740488663,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ACYUUXE7JYS2DURQ2SZUMKDSTXULDANCNFSM4T327GEQ
.


help

Traceback (most recent call last):
File "/Users/yuwei/Desktop/novel_analysis-master/test.py", line 5, in
model = hub.Module(path)
File "/Anaconda/anaconda3/lib/python3.6/site-packages/tensorflow_hub/module.py", line 151, in init
self._spec = as_module_spec(spec)
File "/Anaconda/anaconda3/lib/python3.6/site-packages/tensorflow_hub/module.py", line 36, in as_module_spec
return load_module_spec(spec)
File "/Anaconda/anaconda3/lib/python3.6/site-packages/tensorflow_hub/module.py", line 63, in load_module_spec
print(registry.loader(path))
File "/Anaconda/anaconda3/lib/python3.6/site-packages/tensorflow_hub/registry.py", line 48, in call
self._name, args, kwargs))
RuntimeError: Missing implementation that supports: loader(*('/Users/yuwei/Desktop/chinese_L-12_H-768_A-12',), **{})

@yw1991 So, The problem is with the path ('/Users/yuwei/Desktop/chinese_L-12_H-768_A-12',). Can you please take a look at the issue in stack overflow and let me know if that solves your problem. Thanks!

@gowtham-kp  I have already solved this problem. I find that files should be from Tensorflow-hub.
First my Bert model download from Bert website, so the problem happened;then I download the Bert model form Tensorflow hub, it works.

@yw1991 Thank you for sharing the information. Closing this issue as it has been solved!


Method replyKeyboardMarkup does not exist

There was a problem using the method ReplyKeyboardMarkup from Telegram Bot API.
$reply_markup = $bot->replyKeyboardMarkup(['keyboard' => $keyboard, 'resize_keyboard' => true, 'one_time_keyboard'                  => false]);                 $bot->sendMessage(['chat_id' => $chatId, 'text' => $reply, 'reply_markup' => $reply_markup]);
How to use a similar design?

I realized that I need to create a keyboard class.
$reply_markup = new Keyboard(['keyboard' => $keyboard, 'resize_keyboard' => true, 'one_time_keyboard' => true]);


Add mixin to build button with params

This pull request adds a new mixin for building a button with the following params:

Line height
Padding y
Padding x
Font size
Radius

In case you want to create a different type of basic button (a rounded button for instance) this is more scalable because you can create a new CSS class for this new button with that mixin.

Probably not going to do this for v4—if we do one mixin-based abstraction, we'll likely be asked to do it for many other components. Thanks though!


Weird bottom margin when trying to disable titlebar on maximize

I'm trying to hide the titlebar when the window is maximized so I can have full real state when I need it, or fancy titlebars when I don't.
In my rc.lua I have this code:
client.connect_signal("request::geometry", function(c)
  if c.maximized then
      awful.titlebar.hide(c)
  else
      awful.titlebar.show(c)
  end
end)

The undesired behavior:

Ok... this issue is related to the way I handle the hiding of the titlebar and not with nice.
Closing, but if anyone knows how to accomplish this without that weird bottom margin, let me know.

I have experienced this issue before and never really looked into it too much. It almost seems like the window geometry isn't updated to expand to fill the space available after removing the titlebar. You can see the space left at the bottom is the size of the titlebar. A hack I use to get rid of this issue is to forcibly set the window geometry when it is maximized. Connect a signal to the maximized property and set x and y to 0 and width and height to screen width and height accordingly.

@Pablo1107 I have experienced this issue before and never really looked into it too much. It almost seems like the window geometry isn't updated to expand to fill the space available after removing the titlebar. You can see the space left at the bottom is the size of the titlebar.

Yep, it's seems it runs the hide function but set the width and height assuming the titlebar is present. Maybe this is an issue which should be fixed upstream, but I wouldn't be qualified to try fix it. lol

A hack I use to get rid of this issue is to forcibly set the window geometry when it is maximized. Connect a signal to the maximized property and set x and y to 0 and width and height to screen width and height accordingly.

What do you mean with this? I already have a connect_signal on request::geometry. Should I add there the forced resizing?

@Pablo1107 Check out the latest commit. I added an option to disable titlebars for maximized windows. There are no gaps around the window.

Thanks @mut-ex. I notice you added that option, but as you used the screen resolution to set the width and height it goes over the statusbar. I actually end up using this on my rc.lua:
    awful.key({ modkey,           }, "m",
        function (c)
            if not c.maximized then
                awful.titlebar.hide(c)
            end
            c.maximized = not c.maximized
            c:raise()
            if not c.maximized then
                awful.titlebar.show(c)
            end
        end ,
        {description = "(un)maximize", group = "client"}),

Actually, I found that awesome offers the screen workarea to work around the statusbar, I made a PR to fix that as well.

Thanks for the fix! I merged it


Create .password-store on startup if not already present.

If you install autopass, configure it and just run it you will be greetet by this.
[user@host ~]$ autopass
/usr/lib/ruby/2.2.0/fileutils.rb:127:in `chdir': No such file or directory @ dir_chdir - /home/user/.password-store (Errno::ENOENT)
       from /usr/lib/ruby/2.2.0/fileutils.rb:127:in `cd'
       from /usr/bin/autopass:232:in `checksums'
       from /usr/bin/autopass:222:in `update_cache'
       from /usr/bin/autopass:161:in `initialize'
       from /usr/bin/autopass:251:in `new'
       from /usr/bin/autopass:251:in `<main>'

a simple
[user@host ~]$ mkdir .password-store

fixes this. Maybe instead of creating the dir a hint to "pass init" would be preferable. But with a complete config we also have all the data needed for creation.

But with a complete config we also have all the data needed for creation.

Not quite. pass init needs the key id to encrypt for. I could make assumptions that the cache_key config value should be used but I'd rather do it the other way around. (use the key id stored in .password-store .gpg-id for encrypting the cache)
I will provide a more meaningful error message for uninitialized password stores. But autopass will not create a password store for the user. It is designed to be a read only interface to pass.

meaningful error message

\o/


RIA-7464 Remove WA task for Judiciary-Prepare Decisions and reasons

JIRA link (if applicable)
https://tools.hmcts.net/jira/browse/RIA-7464
Change description
Remove WA task for Judiciary-Prepare Decisions and reasons dmn files and tests
Does this PR introduce a breaking change? (check one with "x")
[ ] Yes
[ x ] No

I am not progressing with this PR as only judges have this task and if they are not going to do it then it should not be initiated.


images: Prevent default browser behavior on paste.

This PR prevents the browser from dropping in a plaintext version of a file name when an image or other file is pasted into the compose box before being handled by the file-upload logic.
CZO discussion:
https://chat.zulip.org/#narrow/stream/191-kandra-support/topic/embedded.20links.20broken.3F/near/1635730
Screenshots and screen captures:

Before
After

Self-review checklist

[x] Self-reviewed the changes for clarity and maintainability
(variable names, code reuse, readability, etc.).

Communicate decisions, questions, and potential concerns.

[x] Explains differences from previous plans (e.g., issue description).
[x] Highlights technical choices and bugs encountered.
[x] Calls out remaining decisions and concerns.
[x] Automated tests verify logic where appropriate.

Individual commits are ready for review (see commit discipline).

[x] Each commit is a coherent idea.
[x] Commit message(s) explain reasoning and motivation for changes.

Completed manual review and testing of the following:

[x] Visual appearance of the changes.
[x] Responsiveness and internationalization.
[x] Strings and tooltips.
[x] End-to-end functionality of buttons, interactions and flows.
[x] Corner cases, error conditions, and easily imagined bugs.

The change makes sense but we should clarify the comment; I think it's missing the information that would safe a bit of research for anyone thinking about refactoring that code in the future.

Merged, thanks @karlstolley!


Avoid passing named keys into call_host_ functions.

In this PR I have removed named keys from the signatures of the following functions:

Runtime::call_host_mint()
Runtime::call_host_auction()
Runtime::call_host_handle_payment()

Now we look up and pass in a Contract  instance, which these functions take named keys from.

@darthsiroftardis I did it that way because we already have a contract in scope here, so if we pass in a contract, it'd only have to be looked up once in every case. If we encapsulate the logic that gets the contract, it'd be looked up every time a call_host_ method is called, and we'd waste the contract that we already have in execute_contract.
I am happy to make the requested change, but I'd like to hear from @mpapierski to see what he thinks.


Update readme instructions.

Change instructions order.
Add python3.9-dev requirement.

No Logic Change


No mib2dev from Free SNMP simulator in ubuntu 16.4

I have installed free SNMP simulator and I followed the installation instructions: https://github.com/etingof/snmpsim and everything seemed to be ok.
However, when I tried to mib2dev tool I get command not found.
When I tried to find if it exists i get:
    ~$ sudo find -name "mib2dev"
    ./.local/snmpsim/data/mib2dev

but still no mib2dev.py anywhere.
    ~/.local/snmpsim/data/mib2dev$ ls -a
    host-resources-mib.snmprec  ip-mib.snmprec  tcp-mib.snmprec  udp-mib.snmprec

I even tried reinstalling but nothing changed.
Any help would be appreciated.

The mib2dev tool is no different from the others shipped with snmpsim. How exactly you install it?
The most common way would be to set up a virtual environment and pip install snmpsim into it:
$ python3 -m venv venv
$ source venv/bin/activate
(venv) $ pip install snmpsim
(venv) $ which mib2dev.py
~/snmpsim/venv/bin/mib2dev.py

As the instructions say:
Just run:
$ pip install snmpsim
and it was installed.
Should I uninstall it and install it with a different method?

I installed as you suggested and seems to be ok only within the venv.


Add linux_devices test

Adds the linux_devices integration test.

Codecov Report

Merging #2708 (dce1122) into main (919ff4a) will not change coverage.
Report is 4 commits behind head on main.
The diff coverage is n/a.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2708   +/-   ##
=======================================
  Coverage   65.40%   65.40%
=======================================
  Files         133      133
  Lines       16942    16942
=======================================
  Hits        11081    11081
  Misses       5861     5861

Hey, due to a bug in our CI, currently the tests are not validated on runc. May I ask you to wait for https://github.com/containers/youki/pull/2707 to get merged and rebase on it, so we know that the added tests also work with runc? Thanks :)

It is merged now, So can you rebase and push?

Thank you. Should be rebased now

Hello, yes this is a straightforward port from the test in oci-runtime tools and there is another test validateDefaultDevices that seems to use this function as well (in oci-runtime tools).

Hello, yes this is a straightforward port from the test in oci-runtime tools and there is another test validateDefaultDevices that seems to use this function as well (in oci-runtime tools).

Ok, so do you think we should separate both of these, and have smaller, maybe more modular functions that do tests specific to their test cases? Or this is the best solution, and breaking up function would be worse code than this? This does not have to be done in this PR, just want to know your thoughts.

@omprakaash ping!

Hey sorry for the delay. Will get back to this tomorrow.

Hey I believe that this is fine, since these are used only for testing and the other test when added, could just call this function. Having separate functions for each type seems unnecessary to me since a lot of the logic seems to overlap for multiple device types, but happy to change this if you strongly feel otherwise.

Hey I believe that this is fine, since these are used only for testing and the other test when added, could just call this function. Having separate functions for each type seems unnecessary to me since a lot of the logic seems to overlap for multiple device types, but happy to change this if you strongly feel otherwise.

Fair enough. I don't have any better idea either, so if in future this proves to be an issue, we can fix ti then. Let's keep it as is now :+1:

Hey I mad a minor change where we don't return immediately from check function if permissions are not configured, it is similar to how uid/gid validation is done in our implementation.


@conda-forge-admin, rerender

Issue:

Environment (conda list):

$ conda list

Details about  conda and system ( conda info ):

$ conda info

Hi! This is the friendly automated conda-forge-webservice.
I just wanted to let you know that I rerendered the recipe in conda-forge/crosstool-ng-feedstock#17.


TypeError: '>' not supported between instances of 'IndividualValueList' and 'float'

https://github.com/rosin-project/mros1_reasoner/blob/ba6ec6c1d25bda7fd778adae293f1a4e3ede88ea/mros1_reasoner/scripts/mros1_reasoner.py#L74
This is error is usually related to to fd.fd_qa_tradeoff=[] because no value has been assigned by any SWRL rule, for example when the only QA we are considering is energy

fixed in https://github.com/rosin-project/mros1_reasoner/commit/08851437e85bcc23ce954f9ef17e5c421191fc85


Use a key-value store to coordinate ssh authentication for zfs peer-to-peer storage

Having to ensure that all nodes can access each other using private key ssh authentication can be extremely difficult to automate and keep the /root/.ssh/authorized_key in sync on every host, especially in large deployments where nodes can come and go quite often.
It would be extremely useful if flocker can use a key-value store to coordinate this. Maybe an automated workflow like so:

When the node flocker is on launches, flocker automatically registers with the k/v store.
When a node needs to contact another node, it uses the k/v store to discover the other node and uses that to broker access.

Something like docker/libkv can be used to easily provide support for consul, etcd, boltdb and zookeeper.
Users of zfs peer-to-peer storage most likely already have a k/v store running to support docker's overlay networking, so I think it's a feature that would fit in quite well.
Another thing that would be nice to have is to be able to have flocker use a user other than root to log into different nodes to transfer data.

@F21 thank you for your feedback. ZFS is currently experimental and not recommended for production and has many known bugs. I will bring your suggestion back to the team.

Closing as ZFS support is currently deprecated.


Trim collapse spaces 3240

Codecov Report

Merging #7 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master       #7   +/-   ##
=======================================
  Coverage   73.91%   73.91%
=======================================
  Files         199      199
  Lines       16533    16533
  Branches     2688     2688
=======================================
  Hits        12219    12219
  Misses       3881     3881
  Partials      433      433

Impacted Files
Coverage Δ

src/robot/libraries/BuiltIn.py
26.00% <ø> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 1c76861...dcd2f3b. Read the comment docs.


[examples] FXAA anti-alising example for RenderTexture

Issue description
I'm trying to get RenderTexture to be DPI aware. I'm using rlImGui (raylib DearImGui backend), and its very pixel-y no matter how big I make the underlying texture.
Regarding #65 which was posted almost 10 years ago, I tried the given shader solution and looked all over for examples of its usage in the repository, but none were found. The shader does not work as intended.
I think it would be nice to have a working solution for RenderTexture with hiDPI and antialiasing, whether there's a compile time option to let RenderTextures enable MSAA, or a runtime flag, a shader that gets most of the way there, or a hack/workaround within the raylib examples that works. I feel like this should be documented somewhere, no?
Issue Screenshot
Here's the render texture as an imgui image element:

^ This looks the same even without putting it an imgui element and rendering it directly on the main frame buffer. This is not DPI aware, but even if I scale it up, it still looks jagged and pixel-y.
Here's the same draw commands but within the main framebuffer:

The render texture is rendered at 1280 by 720, while hiDPI is enabled, I don't think it should look this low-quality... so I'm assuming this is a filtering issue?
I looked this up and found that RenderTexture does not use MSAA for performance reasons.
Is this why one looks pixelated and the other looks fine?
How do I get the best looking image for the RenderTexture?
Code Example
Here's the fragment shader I tried using:
#version 330

#define FXAA_REDUCE_MIN (1.0/128.0)
#define FXAA_REDUCE_MUL (1.0/8.0)
#define FXAA_SPAN_MAX 8.0

in vec2 fragTexCoord;
in vec4 fragColor;

out vec4 finalColor;

uniform sampler2D texture0;
uniform vec2 resolution;

void main()
{
    vec2 inverse_resolution = vec2(1.0/resolution.x,1.0/resolution.y);

    vec3 rgbNW = texture2D(texture0, fragTexCoord.xy + (vec2(-1.0,-1.0)) * inverse_resolution).xyz;
    vec3 rgbNE = texture2D(texture0, fragTexCoord.xy + (vec2(1.0,-1.0)) * inverse_resolution).xyz;
    vec3 rgbSW = texture2D(texture0, fragTexCoord.xy + (vec2(-1.0,1.0)) * inverse_resolution).xyz;
    vec3 rgbSE = texture2D(texture0, fragTexCoord.xy + (vec2(1.0,1.0)) * inverse_resolution).xyz;

    vec3 rgbM  = texture2D(texture0,  fragTexCoord.xy).xyz;

    vec3 luma = vec3(0.299, 0.587, 0.114);

    float lumaNW = dot(rgbNW, luma);
    float lumaNE = dot(rgbNE, luma);
    float lumaSW = dot(rgbSW, luma);
    float lumaSE = dot(rgbSE, luma);
    float lumaM  = dot(rgbM,  luma);
    float lumaMin = min(lumaM, min(min(lumaNW, lumaNE), min(lumaSW, lumaSE)));
    float lumaMax = max(lumaM, max(max(lumaNW, lumaNE), max(lumaSW, lumaSE)));

    vec2 dir;
    dir.x = -((lumaNW + lumaNE) - (lumaSW + lumaSE));
    dir.y =  ((lumaNW + lumaSW) - (lumaNE + lumaSE));

    float dirReduce = max((lumaNW + lumaNE + lumaSW + lumaSE) * (0.25 * FXAA_REDUCE_MUL),FXAA_REDUCE_MIN);
    float rcpDirMin = 1.0/(min(abs(dir.x), abs(dir.y)) + dirReduce);

    dir = min(vec2( FXAA_SPAN_MAX,  FXAA_SPAN_MAX),max(vec2(-FXAA_SPAN_MAX, -FXAA_SPAN_MAX),dir * rcpDirMin)) * inverse_resolution;

    vec3 rgbA = 0.5 * (texture2D(texture0,   fragTexCoord.xy   + dir * (1.0/3.0 - 0.5)).xyz + texture2D(texture0,   fragTexCoord.xy   + dir * (2.0/3.0 - 0.5)).xyz);
    vec3 rgbB = rgbA * 0.5 + 0.25 * (texture2D(texture0,  fragTexCoord.xy   + dir *  - 0.5).xyz + texture2D(texture0,  fragTexCoord.xy   + dir * 0.5).xyz);

    float lumaB = dot(rgbB, luma);

    if((lumaB < lumaMin) || (lumaB > lumaMax))
    {
        finalColor = vec4(rgbA, 1.0);
    }
    else
    {
        finalColor = vec4(rgbB,1.0);
    }
}

No, this wouldn't work. Even when the resolution is matched to the render width and height, the lines are still jagged, they don't seem to have any filter applied.
I'm not sure if the other issue is still accurate today (almost 10 years later) in that RenderTexture still doesn't use MSAA for performance reasons. However, that FXAA shader that helps with antialiasing in that issue seems outdated as well somehow.

Yes, I can see why that would be confusing, but I was mostly saying that to provide context. I think the rest of my issue makes it pretty clear though that I'm trying to get a smoother image using some type of anti-aliasing on the render texture.

I would be very glad if anti-aliasing on rendertextures were possible, I often use them to make videos by converting a rendertexture to an image and piping that to ffmpeg. It's a bit of a pity that the video isn't anti-aliased then.

A quick and easy way to get 4x SSAA is to make render texture 4x times bigger (2x on each axis) and then draw that texture to a regular size render texture ( don't forget to enable bilinear texture filtering on a big render texture). I assume additional overhead is not that important for an offline workflow that you have.

Yes, thanks, I was thinking of something like that, but now I found an even better (for this usecase at least) way: grab the actual (anti-aliased) framebuffer from the main window, similar to what TakeScreenshot does internally:
#include "rlgl.h"
unsigned char *imgData = rlReadScreenPixels(screenWidth, screenHeight);
Image img = { imgData, screenWidth, screenHeight, 1, PIXELFORMAT_UNCOMPRESSED_R8G8B8A8 };

(apologies for going a bit off-topic)

MSAA on RenderTextures is cumbersome to implement on OpenGL and it's not supported by raylib, it's not as easy as enabling a flag, like it is for main frmaebuffer.
About FXAA, it's just a shader and it can be used with RenderTextures BUT note that it is applied to all the RenderTexture buffer and not specific parts of the drawing, it could generate not expected results. Probably in many cases a small blur shader would give better results.
Drawing everything on a 4x RenderTexture is a good solution but it increases the render cost A LOT (not done by hardware like MSAA).
I'm closing this issue because it seems more a discussion about a topic than an issue with raylib library.


[Test] Investigate why Agent Java Versions tests fail randomly

After @hanahmily 's new MAL implementation in Prometheus fetcher, which relates to the self-observability, some e2e tests become very unstable. It could fail randomly.
We need to check what is going on.

Have been proved by #5955, this is a so11y e2e error.


Feature/add option to only store and retrieve csv files locally

I like the new feature to save as CSV but this only works with an Amazon S3 account.
Considering the file already gets saved to the server it'd be convenient if that file could be retrieved as a FileResponse directly without storing the intermediate result on the cloud. We for one are not interested in storing the created CSV files.
This feature adds a config ONLY_STORE_CSV_FILES_LOCALLY to set whether the user is fine only saving to S3.
Basically I would like it to work even if no S3_AWS credentials set up. This config is a quick means to check whether that can be bypassed.
Then the result can be retrieved using the a new api call get_response_local_file which just creates a FileResponse from the local file.

Appologies, I didn't rebase correctly. I will close this and create another one.


Create an intro

Add an intro about yourself: who are you, what is your strength, why are you here.

I am a master's student in Applied Mathematics, passionate about coding and solving complex problems. I love learning new technologies and am here to enhance my skills and gain hands-on experience. This opportunity will help me develop and apply my expertise to impactful projects.


fix auth bug

we need fix the auth bug
the bugs is remained for a long time . it is caused by Himanshu Varshney. I fix the auth bug

Please update your comments for this PR, you can refer to this format: This PR is designed(/proposed) to solve {introduce the problem}.


Plexing needed to help with production readiness...

For large scale use and production readiness however, it really wants plexing. You sort of achieve that with the mutxes, but only within a limited scope.
If I want to use this for battle ready scenarios and large scale, there are areas where it can block where it doesn't have to leading to things like hanging, lost performance gains, etc.
My particular use case is that I want to use this buried between layers of abstraction that let you program asyncronously but very easily. The result of that can be a surprising amount of different scenarios and consider and use cases and need to be handled out of site.
I think a really quick fix might be something like:
Thread {
    public static multiJoin(int $timeout, Thread ...$threads):?Thread
}

Though that still leaves a lot of questions unanswered. I may want to wait for joins and locks at the same time. Same issue with the mutexes, you might want to wait for the first to unlock. You don't really want things like interrupted exceptions everywhere either.
You could consider a Selectable interface for anything blocking (locks, join, etc).
If you consider stability, there's a lot to think about such as what happens when one thing putting on the queue dies?
The mutexes do have a limitation over sync. I can't see an obvious way to wait for data efficiently. In thread communication there is a need to syncronise as well as exchange data safely. In this case I think only the latter case is really directly supported.
Unless I'm not thinking straight it requires some odd roundabout ways to do some things.
$q = new Queue();

function producer() use($q) {
    for($i = 0; $i < 10**6; $i++) {
        $q->lock();
        $q->push($i);
        $q->unlock();
    }
}

function consumer() use($q) {
    while(true) {
        $q->lock();
        $items = $q->shiftAll();
        $q->unlock();
        foreach($items as $i)
            echo "$i\n";
    }
}

thread producer();
thread consumer();

It's subtle but if you look closely you'll notice that the consumer will use 100% CPU, doing nothing while the producer is doing something other than working on the queue. The obvious way to see that is to comment out starting the producer and watch the CPU go whoooop.
$q = new Queue();
// Wrong thread lock ownership!
$q->lock();

function producer() use($q) {
    for($i = 0; $i < 10**6; $i++) {
        $q->push($i);
        $q->unlock();
        if(rand(0, 10**5) === 0)
            throw new Exception();
        $q->lock();
    }

    $q->unlock();
}

$active = true;

function consumer() use($q, &$active) {
    // When does it end?
    while(true) {
        $q->lock();
        $items = $q->shiftAll();
        if(!$active && count($items) === 0)
            break;
        $q->unlock();
        foreach($items as $i)
            echo "$i\n";
    }
}

$c = thread consumer();
join thread producer();
$active = false;
join $c;

I'm not really sure if there are any data exchange cases mutexes can't handle and at this point I'll skip the cognitive effort of determining that as at this point I think there's already enough to generate questions and to give food for thought. This is also just based on a very quick glance of the library so I might have tripped up on something somewhere.
I believe you can use the mutex to achieve sync but really that implies pretty much that you actually want standalone mutexs. Appart from being ugly the main limitation seems to be the lack of a select on locks as well as the question if a thread can unlock someone else's lock.

Even pthread mutexes do have a syncronisation mechanism it turns out which you've not exposed, if you look at its signalling.

Example of something I'm up to on my fork:
// Note: Bool might not be enough.
zend_bool handle_thread_tasks(thread_obj_t *thread) {
	while(1) {
		pthread_mutex_lock(&thread->lock);

		if(thread->status == JOINED) {
			pthread_mutex_unlock(&thread->lock);
			return TRUE;
		}

		if(!thread->tasks->size && !pthread_cond_wait(&thread->has_tasks, &thread->lock)) {
			pthread_mutex_unlock(&thread->lock);
			return FALSE;
		}

		task_t *task = pht_queue_pop(&thread->tasks);
		pthread_mutex_unlock(&thread->lock);

		if (!task) {
			return FALSE;
		}

		switch (task->type) {
			case CLASS_TASK:
				handle_class_task(task);
				break;
			case FUNCTION_TASK:
				handle_function_task(task);
				break;
			case FILE_TASK:
				handle_file_task(task);
				break;
		}

		task_delete(task);
	}

	return TRUE;
}

If you compare that to the original that just locks and unlocks around the queue...
I'm not actually sure this is going to be enough as thread state and queue state are two different things though I've opted to leave that for later.

Wait a minute.
while (thread->status != JOINED || thread->tasks.size) {
How do you still run tasks when the thread has gone? Something there I'm missing (have only looked at a small bit of the implementation).

How do you still run tasks when the thread has gone?

You don't. When you join a thread, any tasks associated with that thread should be considered finished with (if they aren't finished with, then why are you joining the thread?).
I'll look through this more over this weekend (I'm too busy on week days to do this right now, unfortunately).

Not rush, it's not a set of issues that really can be quick fixed.

Just to make it less ambiguous. The tasks thing is a bit of a red herring, you're using that status to tell the thread if it should shut down or not, not to say that a join has finished but something wants it to finish (has joined into it).
There are two issues here: too much blocking in one place, not enough blocking in another place
Too much
Too much blocking is when calling join. Basically people want a notification of when a thread is finished. This can be done in userland to some degree but it's not entirely reliable and can waste a small amount of time blocking unnecessarily. I can only join one thread at a time. If I have two threads, the first one I join could take ages leaving the thread wasting time waiting for that one when it could be doing stuff after the second one finished.
This is actually related to a limitation in POSIX threads. Doing some research I'm not the only one complaining about that. However it's not necessarily a limitation you have to export when wrapping pthreads in a higher level wrapper. You can at least have thread detach and have each thread when they're about to exit send a signal. This means you'd want to also make sure you avoid any situation where a thread can get lost or not exit gracefully.
This should be quite reliable much of the time and minimise excess blocking though there's always the question of is it possible for the thread to fail in some other manner (an occasional sweep up might make sense). I'd assume things like OOM, segfault, etc will down out a whole process. Leaves me wondering under what condition a thread might unexpectedly pop off and if it can be causes by programming error.
Not enough blocking
It's not enough to just wrap a mutex around shared data. Threads need to wait (block) until that data enters certain states.
This situation is already well explained but a simple plain English explaination...
If you have one thread that pops from a stack and a thread that pushes to the stack, the thread that pops from the stack should go to sleep until there's something on the stack, so when the thread pushing to the stack does so, it needs to wake up the thread popping from the stack. The lock doesn't achieve that, all it does is to prevent them accessing the stack at the same time.


mapillary link down

Hi,
since mapillary change of API, the mapillaty URL call is unknown, see demo map for example.
Mapillary client Id fill (now pKey) has changed too with the new API.

Solved. I'll try to create a pull request


SoulDew - 2.1.0

Refactor state management
Add computedProperty handling
Added a flawless and efficient way to handle both async and sync tasks with no additional complexity.

Summary by CodeRabbit

New Features

Introduced methods for subscribing to state changes and adding computed properties.
Enhanced state management with batch updates to improve performance.
Added asynchronous event emission for better error handling.

Improvements

Refined event handling and state management structure for increased functionality.

Templating will be a future idea however utilising it will be completely optional. Templates can be used to register events and verify their integrity and also help prevent the error of emitting unknown events.


new contributor

Adds name to contributor list and incremented list count by 1.

Looks like vscode settings got mixed in. Would it be best to include vscode in our .gitignore to avoid this issue in the future?


python folder missing

trying to run the snakemake pipeline it fails searching for the python directory in misc/sprite-pipeline/
the "sprite-pipeline" github page (https://github.com/GuttmanLab/sprite-pipeline) has a python folder but it does not seem to contain all the needed files (e.g. "filter_all_tags.py")

The R folder is missing as well.

Uploaded. Thanks.


(SP:5) Update items fix

You need to write method that will help us to compare 2 states of item:

object for update;
object in database.
You don't need to compare full object, only fields that present in object for update.

For example:
object to update:
{
   text: [],
   title: [],
}

object in database:
{
   _id: '...',
  __v0: 0
   text: [],
   title: [],
   name: []
   images: []
}

Method or function need to compare text and title field in objects and return true if data in this field will be equal in 2 objects.


Clarify where using an expression in a test might fail

Description & motivation
We noticed that using the --store-failures flag would cause uniqueness tests that relied on the trick described in option 2 to fail, so I added a line to note this behavior. Hopefully somebody else finds it useful.

We appreciate your contributions to the dbt product documentation! Unfortunately, we are closing this PR as outdated with merge conflicts. We don't have permissions to edit this fork to fix the merge conflicts so we cannot update this content for publishing.
Feel free to open a new PR if you want to add this content to the docs! 🙇🏻


notion-refactor

ensure the tests pass on github actions - do this by skipping the test if the config file does not exist!

expand the use of seneca-doc - add descriptions for messages, and inject the options - see the seneca-doc docs


tick labels are too large (WPF only)

Hello!
First off, thank you for making this excellent library.
I am having an issue where the x-axis labels are being slightly cropped off. I'm sure this is user error, but I can't find a solution. I'm using the QuickStart example for the WPF on version 4.0.18.

Thank you for your help,
Zach

Thanks for reporting this @zrolfs! I'll take a closer look at this tonight

Hey @zrolfs, the quickstart here works fine on my machine: /doc/quickstart/WpfApp
https://github.com/swharden/ScottPlot/blob/c723b897e364314a9eec7905afa3ea8fa9e71d46/doc/quickstart/WpfApp/MainWindow.xaml#L10-L12
https://github.com/swharden/ScottPlot/blob/c723b897e364314a9eec7905afa3ea8fa9e71d46/doc/quickstart/WpfApp/MainWindow.xaml.cs#L23-L31
Is the code you're using different than this?

... and I have to ask, are you using display scaling? If you disable display scaling, is the problem fixed? GDI+ (ScottPlot's rendering system until version 4.2 is released) is known to play well with scaled displays.
I'll go ahead and close this issue for now because I think you can fix your issue with:
plt.Layout(yScaleWidth: 20, titleHeight: 20, xLabelHeight: 20, y2LabelWidth: 20);

https://github.com/swharden/ScottPlot/tree/master/cookbook#custom-padding

Thanks for your help and quick response @swharden!
When I download and run the quickstart solution, I get the same issue:

I'm not sure if I'm using display scaling or not... I can follow up if you're interested.
The line RatioComparisonPlot.plt.Layout(yScaleWidth:20, titleHeight: 20, xLabelHeight: 20, y2LabelWidth: 20);
fixed the x-axis but cropped the y-axis. However, RatioComparisonPlot.plt.Layout(titleHeight: 20, xLabelHeight: 20, y2LabelWidth: 20); worked great for both! This think this work around will be acceptable for me. Thanks again!

RatioComparisonPlot.plt.Layout(titleHeight: 20, xLabelHeight: 20, y2LabelWidth: 20); worked great for both! I think this work around will be acceptable for me. Thanks again!

Sounds good! Another thing to try could be plt.Ticks(fontSize: 10);
Best,
Scott

I ran across this problem myself. It looks like title and axis label size is normal, but ticks are huge. I'm using display scaling on this computer. My guess is that tick label font size isn't set, and being auto-detected somehow 🤔

An alternative temporary fix is:
wpfPlot1.plt.Ticks(fontSize: 8);

Yep, that works for me!

Maybe unrelated, but I'm getting two horizontal red lines on the perimeter of the plot. Are you familiar with this? They're visible in the first figure.

Maybe unrelated, but I'm getting two horizontal red lines on the perimeter of the plot. Are you familiar with this? They're visible in the first figure.

Yeah, this was fixed by 7e72265a624466585c5efe877b79f012999e1835
It'll be fixed in the next NuGet release

Fixed this issue with 4837053 - it'll be in the next NuGet release (hopefully today)

Spoke too soon. Ticks are still oddly larger than labels when display scaling is on.

Finally figured this out...
All fonts created like this
var fnt = new Font(fontName, fontSize);

should be like this to disable display scaling:
var fnt = new Font(fontName, fontSize, GraphicsUnit.Pixel);

or like this to support display scaling:
var fnt = new Font(fontName, fontSize, GraphicsUnit.Point);

A thorough review of the whole project should be conducted to find all new Font creations and ensure consistency. I bet something similar should be done with lines, circles, etc. The simplest solution for now is to disable display scaling across the whole project and work only in pixel units.


API to get information back from specific commands

At the moment we have attributes and some methods that let us query specific information from the vehicle (home position, location, gps etc). We can send MAVLink messages that "do" something - set Yaw, move to position, set servo value, etc.
What we don't have is a public method to get arbitrary information back from the vehicle. So for example I can sent a message to the API with the message ID requesting the camera status or the current value of a particular servo, but I can't then get the response back into my application.
In particular it would be great to be able to query the capabilities of a particular target platform.

Hi Hamish,
Thanks for updating the project! I'm new to dronekit..Initially I was trying to read altitude via "vehicle.location.alt" every 0.1 sec, but it popped out 6 same readings although the drone was climbing up fast. At what rate do the vehicle attributes get updated? Alternatively I could "ask" the drone for altitude readings from on-board sensors (ultrasonic sensor for example), but your thread suggests that retrieving message from vehicle has not been implemented yet (which kinda explains why I didn't find any example getting messages from vehicle..) Any updates on getting info back from the vehicle, sensor readings etc.?
Cheers,
csmiler

@csmiler This is more an autopilot question than a DroneKit question, and will be impacted by the autopilot implementation, hardware and the communication link (if you're over USB it will be potentially a lot faster than over an RF Telemetry link). DroneKit does process every message it receives. AFAIK there is nothing you can do about getting a faster rate than the comms link and hardware allow.
All that said, this post indicates someone was getting updates over USB around every 0.12seconds.
We still don't have a mechanism for getting "arbitrary" specific messages back from the system. If I "had" to do this I'd look at set_mavlink_callback().

@csmiler - check your stream parameters.  Especially SR0_Position:  http://copter.ardupilot.com/wiki/configuration/arducopter-parameters/#position_stream_rate_to_ground_station_sr0_position
The max stream rate is 10 hz, but yours may be set to a smaller number.  Many GCSes will set the stream rates to numbers smaller than 10 hz.

@eliao That's awesome - the wiki indicates that the firmware can use this information to automatically adapt the telemetry stream rates to the data rate that the radios can sustain - so does that mean the parameters just set the "maximum value"?
My gut feeling is that adding information on stream rates to DroneKit docs may not be appropriate, but perhaps we can do more on the wiki?

I'm not sure what happens if you set the stream rates to something that your radios cannot sustain.  I assume it will start dropping messages, but I am not sure which messages get dropped.

@hamishwillee Thanks for the reply! (didn't expect to hear back from you so soon!) I understand there's physical limit over RF link. I'll try set_mavlink_callback() tomorrow. In fact I was looking at the function earlier today, but didn't go with it because of the conservative warning..on the other hand, I use channel_override..
@eliao That's a good point! I'll check my param. Thanks for the suggestion. I noticed that the telemetry log has much fewer data points (< 3 Hz) compared to dataFlash log which records at 10 Hz. Maybe the stream rates was too low.

@csmiler Please don't use channel_override() - @rmackay9 will hunt you down :-) I've just added a bunch of documentation on the right way to do takeoff, position control and velocity control which are recommended. While you're there, note the lovely new addition of the page table of contents in the left-sidebar :-)

@eliao I checked my parameters today. SR0_POSITION was 3 Hz! But even if I changed it to 10 Hz, the stream rate seems to be the same ( 2 or 3 points per sec)..I also played with other params, but couldn't increase the rate..below is the params I am using. I turned off SR1_xxx (telemetry 2 on Pixhawk in my 3DR IRIS+ is connected to FrSky receiver) [oddly everytime I reconnect to IRIS via mavproxy SR1_xxx params change to 4 except SR1_PARAMS 10, so I had to manually set them to 0 again]... Do you think there is anything else I can do to increase the stream rate, or maybe it's capped due to transmission limit (3 Hz is kinda low though..)? Thanks so much for your help!
SR0_EXTRA1       10.000000
SR0_EXTRA2       10.000000
SR0_EXTRA3       2.000000
SR0_EXT_STAT     2.000000
SR0_PARAMS       10.000000
SR0_POSITION     10.000000
SR0_RAW_CTRL     4.000000
SR0_RAW_SENS     2.000000
SR0_RC_CHAN      2.000000
SR1_EXTRA1       0.000000
SR1_EXTRA2       0.000000
SR1_EXTRA3       0.000000
SR1_EXT_STAT     0.000000
SR1_PARAMS       0.000000
SR1_POSITION     0.000000
SR1_RAW_CTRL     0.000000
SR1_RAW_SENS     0.000000
SR1_RC_CHAN      0.000000
SR2_EXTRA1       0.000000
SR2_EXTRA2       0.000000
SR2_EXTRA3       0.000000
SR2_EXT_STAT     0.000000
SR2_PARAMS       0.000000
SR2_POSITION     0.000000
SR2_RAW_CTRL     0.000000
SR2_RAW_SENS     0.000000
SR2_RC_CHAN      0.000000

Are you using MAVProxy?  There's some logic in MAVProxy that automatically sets all the stream rates to 4 hz every few seconds.  You can pass it the "--streamrate=-1" option to disable this.

@eliao Yes I use MAVProxy. SR1_xx are set to 4 hz every time I reconnect via mavproxy, but after I manually set them to 0, they won't be reset to 4 after a few seconds, unless I disconnect and reconnect. But you are right about "--streamrate=-1"!! After I add the option, even if I reconnect the SR1_xx remains 0 except SR1_PARAMS automatically set to 10..Thanks!
The stream rate was the same though...Any idea would be appreciated~ :+1:

@csmiler - You've exhausted my knowledge.   Maybe the link is maxed out?  You might be able to turn off some other streams to see if that helps.

@hamishwillee Yes I see you have added so much more stuff which makes it more intuitive and helpful for newbies like me :). The cute table on the left is a huge plus! Good work! :100:
If I understand correctly, at the moment the only way to get info off the copter is to enable stream, and there is no such thing as "one-time talk" by sending one message asking for a current reading of one sensor and receiving the response message promptly which contains the info requested, am I right? So I cannot really "ask" for altitude every 0.1 sec (even if transmission is fast enough) and "receive" an answer, but can only take whatever is fed from the attribute "location.alt" which comes from VFR_HUD.alt. I checked the flood from set_mavlink_callback(), and messages for "ATTITUDE" and "VFR_HUD" are at ~ 2 hz, which is consistent with the limited stream rate..no luck there..
I learnt the functions "takeoff, position control and velocity control" earlier from your demo (thanks!!!). What I am trying to do here is to fly IRIS indoor (no GPS) with additional positioning system feeding my program with the copter's 3D location, and be able to fly to any coordinate within a say 2 m x 1 m x 1 m area within ONE second. The functions in GUIDED mode are not yummy since GPS is not locked. So the "naive" idea I had is to "mimic" a joystick by overriding channel 1~4 and write external PID controllers to provide channel values based on its current position and target position fed by additional positioning system (silly, right?). So all controls are off Pixhawk, and the only task for Pixhawk is to maintain stabilized...I am using "Stabilize" mode since it is least damped and most responsive to channel inputs given the one-second limit. Then it is crucial to "know" its position in a timely manner. Right now I'm just testing my PID controller using feedback from "vehicle.location.alt", but it turns out that the feedback is too slow because of the stream rate...any suggestion on indoor navigation problem Expert Willee?
Sorry if I am asking silly questions...I just started flying drones..not an easy thing but intersting!!

@eliao You've helped me a lot! Many thanks! It'll be unfortunate if the telemetry can only provide this much of stream rate...I'm getting a ultrasonic sensor for altitude, maybe I'd better hook it up to a raspberry pi to get its reading rather than connect to Pixhawk and stream it..

@csmiler Sorry, just got back from holiday. Yes, your understanding above is correct. Unfortunately you've hit the limit of my knowledge here too - no more ideas :-(

@mrpollo Looks like some stuff in the "right direction" happening in Android for this: https://github.com/DroidPlanner/DroneKit-Android/pull/188

@tcr3dr I think we can do this now, simply by defining a message listener right?

@hamishwillee That's correct. In the constructor of a Vehicle, that's effectively how we track state changes anyway.

The bandwidth can be determined by using the https://github.com/ArduPilot/pymavlink/blob/master/tools/mavtelemetry_datarates.py script


Sankey diagram

https://bost.ocks.org/mike/sankey/
Not sure this is possible using vega currently

:+1 for adding this


Several items have broken names.

Describe the bug
block.occultism.spirit_campfire
block.occultism.spirit_lantern
block.occultism.silver_ore_deepslate
block.occultism.spirit_torch
item.occultism.raw_iesnium
item.occultism.raw_silver
Also mining iesnium ore in the world drops iesnium ore, which must be placed and mined again to get the raw iesnium. This might be intentional on your part. Also once you have that iesnium ore it can be mined with a diamond pickaxe.
Expected behavior
Plaintext names.
System (please complete the following information):

Occultism Version: 1.18.1-1.26.9
OS: Windows 10
Minecraft Version:1.18.1
Forge-39.0.88
Modpack Link and Version, or list of mods: Patchouli, Occultism, JEI, JER, GeckoLib, and Curios API (Forge)

Could you re-test this on the latest version (on 1.18.2)?
I believe all of them are fixed there
As for the iesnium ore: that is indeed intended, although I might skip this in the future

Ouch, but understandable.
ATM all the YUNGs mods are on 1.18.1 except for his patreon supporters who get early access to 1.18.2 versions, so I bet a lot of people are still on <IP_ADDRESS> for that alone.

Closing this for now as, at least in dev, these item names work in latest


Fix updating an existing user's rank not returning the updated values

This pull request changes the following things:
Fixes issue where moebot wouldn't immediately announce when users could become veteran. The correct Rank and MessageSent values weren't being return by UserServerRankUpdateOrInsert if the user had an existing record in the user_server_rank table (which would probably happen always unless the Veteran Rank for the server could be reached before the points buffer filled once.)
Fixes #48
By submitting this Pull Request I agree to have done the following (check all that apply):

[x] Run go fmt on all the files I've changed
[x] Tested the bot on my own server to verify the feature works
[x] Verified with CamD67 that the feature is desired (Check out the Projects page, or submit an issue for new features)
[x] Submitted PR to the develop branch not master!

👍


Error installing xxhash

I got this error while trying to install xxhash:
Command "C:\Python27\python.exe -u -c "import setuptools, tokenize;file='c:\users\0196~1\appdata\local\temp\pip-build-dt9yim\xxhash\setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record c:\users\0196~1\appdata\local\temp\pip-pdfyfs-record\install-record.txt --single-version-externally-managed --compile" failed with error code 1 in c:\users\0196~1\appdata\local\temp\pip-build-dt9yim\xxhash\
I tried installing this:
https://www.microsoft.com/en-us/download/details.aspx?id=44266
but it didn't helped.
Can you help me please?

Same thing, can't install xxhash

I got it.
I don't know if it's related, but I my profile name in windows was in hebrew.
I changed it to english, restarted, uninstalled the c++ compiler for windows, installed it again and it worked.
Maybe this could help to other users as well.

Well... I really don't think it's going to help me lol

Install this: https://www.microsoft.com/en-us/download/details.aspx?id=44266
Then try to install xxhash again. Worked for me.

Either install C Compiler for python (if using windows), or make sure you have GCC and a development branch of python installed if you are building on a linux server!

For linux users, the command is
sudo apt-get install python-dev
as mentioned by @SysQo

This needs to be part of the instructions.
https://www.microsoft.com/en-us/download/details.aspx?id=44266
works perfectly after this.

@kara71 Please note that is Ubuntu only. If he were using RHEL (like I use) he would be using yum install python-devel

This seems to have resolved itself?


feat: Add support for specifying assume role conditions

Description

Add support for specifying assume role conditions

Motivation and Context

Resolves #11

Breaking Changes

No

How Has This Been Tested?

[x] I have updated at least one of the examples/* to demonstrate and validate my change(s)
[x] I have tested and validated these changes using one or more of the provided examples/* projects
[x] I have executed pre-commit run -a on my pull request

This PR is included in version 1.3.0 :tada:


Refactor index.js to allow multiple connections, async startup, cleaner bootstrapping

This is the refactoring changes from #295 with an improvement (and test) to loading db/query.sql and db/query/file.sql both.

Factors out the global self variable in index.js which allows you to massive.connect to multiple databases without as many issues as before
Refactors walkSqlFiles to be asynchronous for consistency
Splits connect into two phases: the collection phase: bootstrap (which queries the database, filesystem, etc); and the initialisation phase: initialize (which takes the data from the collection phase and applies it to the local instance)
Fixes the missing error handler here
Uses fs.stat rather than extension/error to see if paths are files/directories
Fixes the issues of having both db/query.sql and db/query/file.sql - now deterministically loads both queries rather than the previous behaviour which would have one or the other win based on the filesystem order
[BREAKING] Prevents SQL files from overwriting existing methods

Warns the user via either console.warn or a custom notifier if there's a clash
No longer deletes built in functions if they share a name with a script source folder

@robconery / @dmfay Any thoughts on this PR?

You add var self = this; at a few points and don't seem to use it, and I think you can take away the 'would love to be refactored' comment on walkSqlFiles! Other than that I'm good, but since the behavior when loading files is different I'll defer to Rob on pulling it in.

There is a lot going on with this and it makes me nervous. I'm wondering if this can be pushed to v3? I need to get that rolling (hopefully in the next month or so) and the main thing I want to avoid is a major change to v2 that will break things that we need to support.
Just to be clear: the driver for this change is multiple connections?

The main driver is to allow lightweight cloning of massive instances to enable #295 (connection wrapper to enable local variables in transactions for every query) and similar functionality others may wish to introduce; the multiple connections thing is just a side benefit that I've not tested yet.
Your response is definitely fair - the diffs do seem quite daunting, but thankfully that's just a façade!
Other than making walkSqlFiles asynchronous it's really just re-arranging the code you already have and splitting bootstrapping into two phases; e.g.

instead of loading each thing in turn, we fetch each thing and then apply our learnings all at once:

fetching is just the first half of loading; e.g. for functions:

applying the loaded functions is identical (but at a significantly reduced indentation level):

I'm happy to give this treatment to all the changes if it's helpful (or to split them into separate commits to make it easier to review, maybe?). Hopefully you can see from this that there's not much actual change going on, it's mostly just moving code around, so shouldn't break anything sensible.
The only breaking change that it has, as far as I'm aware, is that naming one of your SQL files in such a way as to overwrite a built in method is now no longer allowed - but I'm not sure that should have been allowed in the first place (I think it's an accidental feature?). I'm happy to revert this so that the functions can be overwritten if you want...
As you may have noticed, I'm invested in the future of Massive.js (since I'm using it for one of my projects) so I'm keen to keep my version as close to yours as possible - let me know how I can make your life easier with my changes and I'll do my best 😄 . I did dip my toes in the bluebird branch too (since I'd like to use async/await) but I got scared and ran away.

Cheers - thanks so much for the lengthy reply :). Let me download and stare at it for a bit. The biggest thing for me is losing touch with the codebase - which I suppose is already happening anyway.
I can get to this over next weekend.

@robconery I'm not sure if you got a chance to look at this PR in the end; but I've just rebased it on the latest master so it's merge-able again (hopefully).

I've rebased this on the latest master, @robconery have you had any further thoughts on this?

@benjie I like where you're going with this, but my first priority right now is wrapping up v3 so I'd rather not pull anything big into master in the mean time. If you have the time/inclination do check out the promises branch, though!

@benjie Thanks for adding support for multiple dbs -- it is exactly what I need.  What did you end up doing?

Closing out legacy PRs.


Remove pk col attribute on table when changing nullability

I don't see any reason to think that this would ever do anything useful.

@tgoyne No, not in the perfect world, but we have this issue: https://github.com/realm/realm-cocoa/issues/6724. The error occurs for a table where the primary key string column is changed from optional to required. I have not (yet) been able to find out what goes wrong, but the m_primary_key_col seems to be wrong. This change would just make sure it is reset. Well - anyway - I will keep looking for the root cause, so if you think this PR is a bad idea - no problem.

@jedelbo Do you want to close or keep this?

Outdated


Can't download youtube video - now with full log

Full log this time, sorry for the previous post:
C:\Downloads>youtube-dl https://www.youtube.com/watch?v=Pk_0kftCQdA -v
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: ['https://www.youtube.com/watch?v=Pk_0kftCQdA', '-v']
[debug] Encodings: locale cp1252, fs mbcs, out cp437, pref cp1252
[debug] youtube-dl version 2017.07.15
[debug] Python version 3.4.4 - Windows-7-6.1.7601-SP1
[debug] exe versions: ffmpeg N-83280-gcba4f0e, ffprobe N-83280-gcba4f0e
[debug] Proxy map: {}
[youtube] Pk_0kftCQdA: Downloading webpage
[youtube] Pk_0kftCQdA: Downloading video info webpage
[youtube] Pk_0kftCQdA: Extracting video information
[youtube] Pk_0kftCQdA: Downloading MPD manifest
WARNING: Requested formats are incompatible for merge and will be merged into mk
v.
[debug] Invoking downloader on 'https://r1---sn-ux3n588t-mjh6.googlevideo.com/vi
deoplayback/id/3e4ff491fb4241d0/itag/137/source/youtube/requiressl/yes/gcr/ca/in
itcwndbps/1635000/ei/7rVvWfa8NMrSDvSmgPAG/mm/31/mn/sn-ux3n588t-mjh6/pl/19/mv/m/m
s/au/ratebypass/yes/mime/video%2Fmp4/otfp/1/gir/yes/clen/3724712492/lmt/15003051
15633637/dur/13109.760/key/dg_yt0/signature/588428A4E2C870DFBF6B1DF1F61D524B35AD
1ACA.1013C866DDA665ABBB0893A331EC1440BBC3B712/mt/1500493222/ip/<IP_ADDRESS>/ipbi
ts/0/expire/1500514894/sparams/ip,ipbits,expire,id,itag,source,requiressl,gcr,in
itcwndbps,ei,mm,mn,pl,mv,ms,ratebypass,mime,otfp,gir,clen,lmt,dur/'
[dashsegments] Total fragments: 2562
[download] Destination: Turandot, live from The Royal Opera House-Pk_0kftCQdA.f1
37.mp4
[download]  36.7% of ~2.38GiB at  5.24MiB/s ETA 06:09          ERROR: unable to
download video data: [WinError 5] Access is denied: 'Turandot, live from The Roy
al Opera House-Pk_0kftCQdA.f137.mp4.part-Frag938'
Traceback (most recent call last):
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp9jcmgxq0\bu
ild\youtube_dl\YoutubeDL.py", line 1842, in process_info
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp9jcmgxq0\bu
ild\youtube_dl\YoutubeDL.py", line 1790, in dl
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp9jcmgxq0\bu
ild\youtube_dl\downloader\common.py", line 361, in download
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp9jcmgxq0\bu
ild\youtube_dl\downloader\dash.py", line 42, in real_download
File "C:\Users\dst\AppData\Roaming\Build archive\youtube-dl\rg3\tmp9jcmgxq0\bu
ild\youtube_dl\downloader\fragment.py", line 114, in _append_fragment
PermissionError: [WinError 5] Access is denied: 'Turandot, live from The Royal O
pera House-Pk_0kftCQdA.f137.mp4.part-Frag938'
C:\Downloads>

[WinError 5] Access is denied

The problem is most likely with your storage.


fixed PropTypes deprecation warning

This is fixing a deprecation warning about accessing PropTypes via the main React package, as PropTypes have been extracted to a separate package, see release notes: https://facebook.github.io/react/blog/2017/04/07/react-v15.5.0.html#migrating-from-react.proptypes

What's the hold-up on merging this in?

@alexhisen The main problem is my lack of motivation to spend time on the project, on top of lack of free time and perhaps a tiny bit of burnout but deep down I know it's still the motivation. Apologies for that.
When I am able to continue work on this I will be merging this and the other PRs but honestly I don't know when that will be so I cannot make promises.

I think the preferable method is using PropTypes from 'prop-types' package, not from 'react/lib/ReactPropTypes'

Thanks again for the PR.
Once I merge #187 (and see if I can update react/three even further) I'll update the prop-types package too.


bluetooth: host: conn: Check if *conn is not NULL

This commit adds a warning and a Kconfig option to bt_conn_le_create and bt_conn_le_create_synced functions which are meant to warn a user of a potential leakage of an active connection object.
This change is implemented due to frequent incorrect use of the connection pointer where a pointer to an existing connection object is overwritten by bt_conn_le_create and bt_conn_le_create_synced functions which in turns leads to sporadic critical bugs. See https://github.com/zephyrproject-rtos/zephyr/pull/78284#discussion_r1754304535 for more details.
The Kconfig option is introduced instead of always returning the error to not affect current implementations. However, it is recommended to keep this option enabled to avoid potential bugs.

I think it'd make sense to add a test case for this too, since right now you're not actually covering the new code branches at all in CI.

Added

I have a suggestion for the comment above the warning. I am happy to follow up on this with my own PR, but you may grab it if you like it.

This rule helps application developers prevent leaks of connection references. If a bt_conn variable is not null, it presumably holds a reference and must not be overwritten. To avoid this warning, initialize the variables to null, and set them to null when moving the reference.

I have a suggestion for the comment above the warning. I am happy to follow up on this with my own PR, but you may grab it if you like it.

This rule helps application developers prevent leaks of connection references. If a bt_conn variable is not null, it presumably holds a reference and must not be overwritten. To avoid this warning, initialize the variables to null, and set them to null when moving the reference.

Added. Please have a look.

There's a lot of mocks that seem unrelated to the tests. But I assume they are necessary.

Yes, unfortunately :( Otherwise I'd have to pull a lot of dependencies into the test. Same as tests/bluetooth/host/id/mocks for example. But then if anybody wants, they can expand the test to cover the rest behavior of the conn.c in unit test.


Feature: Player inventory to DBI CurrentCharacter

Description
Adding an Inventory instance of the Player directly to the ActiveCharacter
Type of Change

[ ] Bug fix
[X] New feature
[ ] Breaking change
[ ] Documentation update

Hi there, thanks a lot for your interest in my little project. I left a few comments for you.

Hello, you're welcome, your project can leads to an impressive DofusModding framework. With that, we will be able to add lot of QoL mods in the game.


WIP: DOM Binding Docs

This is still very much a WIP, and so is not yet in a condition to be merged, but I have opened this PR to provide a place for continued discussion.
Key pieces still missing:

[x] Breaking out the different sections in the _Guide_ into their own subfolder so that they appear correctly in the nav
[x] Completing the list of formatters on the Reference page (have completed three to give an indication of direction)
[ ] Better documentation of error and success callback handling
[ ] Add "for theme designers" section, giving some guidance and best practices for making Cart.js powered themes compatible and easily extensible (see #66).
[ ] Add references to cover usage described in #82

@gavinballard, I have left the unsquashed branch up for now, but had planned to clean up the commits before offering up for merging, though equally happy to leave full commit history in place if you would prefer.

This all looks really good @michaelrshannon! I made a few comments on various commits, keen to hear your thoughts. I'm happy to help out with this, so feel free to leave any grunt work (for example, documenting some formatters) to me.

Thanks for all the feedback - always great to get a second pair of eyes! Will iterate on this over the next week, and give you a shout if an extra pair of hands would be useful!

:+1: Thanks!

@michaelrshannon Just letting you know that I've updated your comment at the top of this PR to capture some of the other requirements for documentation.

Thanks :+1:

I know it's been a while, but it would be great to have these doc updates merged in, they're very helpful compared to what's currently being displayed for the Dom Binding section.

@michaelrshannon Any sense of whether this PR is good to go in? /cc @curiouscrusher

Ping @michaelrshannon :).

🪦


Generated XML file is missing the XML declaration

The generated XML file is missing the XML declaration:
<?xml version='1.0' encoding='UTF-8'?>

This results in an error, where gmail won't allow you to import the file. Make sure the generated file includes the XML declaration.

That's interesting. I've definitely used the XML export in the past, but I've used --sync for so long that Gmail may have changed their behavior in the meantime. I agree the declaration should be included; I'll fix that now.


Change YAML Parser to ignore Helm templates

Expected Behavior
Kics ignores helm templates for k8s
Actual Behavior
Kics doesn't ignore helm templates for k8s

Closed by #2348


Follow renames in file history

Originally reported on Google Code with ID 167
As mentioned by Matthias Sohn in http://groups.google.com/group/gitblit/browse_thread/thread/aa64c91db70e037e:

Have a look at RevWalkFollowFilterTest or EGit's GitHistoryPage for examples how to
use FollowFilter to configure a RevWalk to follow renames.

Reported by xeno.by on 2012-11-19 19:09:42

This is a must have feature for gitblit, because in case of file or even directory renames the user loses previous history for that file or directory and hence all the UI power of gitblit become useless if, for example, he wants to find some changes in a particular file.


Cert Manager Common Name example needs quotes

  # Enter a wildcard domain name
  # Examples: *.example.com or *.subdomain.example.com
  commonName: *.andrey.onepanel.io

The above failed, but wrapping it in quotes succeeded.
  # Enter a wildcard domain name
  # Examples: *.example.com or *.subdomain.example.com
  commonName: "*.andrey.onepanel.io"

Let's update the example to wrap the values in quotes.

Looks like this is already done in a newer commit.


New: Support Dashed line for Line Graph (fixes #29)

Fixes #29
What is the purpose of this pull request? (Check any that's applicable)

[x] Feature
[ ] Bug fix
[ ] Dependency updates
[ ] Documentation updates
[ ] Build related changes
[ ] Changes an existing functionality
[ ] Other, please explain:

Does this introduce a breaking change?

[ ] Yes
[x] No

What changes did you make? (Give an overview)
Introduced style property for the line data value to display the line in dashed format. User can pass the strokeDasharray as a string with a combination of dash and gap.
Add any screenshots

:tada: This PR is included in version 1.16.0 :tada:
The release is available on:

npm package (@latest dist-tag)
GitHub release

Your semantic-release bot :package::rocket:


Allow custom protocols to participate in RPC

Custom protocols are currently second class citizens because they can't participate in RPC. We have already done some steps to modularize RPC in quic-rpc and elsewhere.
Not sure if this can be done without making the default case more unwieldy, but ideally custom protocols should get the option to participate in RPC. Basically each custom protocol handler would also get a look at each RPC message and have the option to handle it.
With this we could thin down the giant switch statement that is our current RPC handler, so that only really node internal RPCs would be handled directly by the node.

RPC is going to fully move into the protocols


allow to remove thread replies

originally from https://github.com/jgengo/slack-cleaner/commit/cb321c046b22f55371b04c29b5be860370e5dd00

I forgot I did that already: https://github.com/sgratzl/slack-cleaner/pull/55 :-D