use for own dataset

hello everyone, can you show me any guide to run augmentation on my own dataset.

You should update the dataloader, the Dataset class in pba/data_utils.py to load your dataset. It should work for any image size.
You can see load_test() in pba/data_utils.py or scripts/test_search.sh for an example.

Hello @arcelien Thank for your respond.
Because your input data is resized in fix size. like below:
train_data = train_data.reshape(-1, 3, 32, 32)

How to work with a folder that contain different size of images.

I was looking for this and I am pretty sure many people will be, An example notebook might be extremely helpful in that case. Maybe an end-to-end demo covering augmenting your own dataset using PBA and train a model using them. It will really be immensely helpful.
I ran the pba.ipynb demo notebook and it went pretty smoothly.

Note that the line train_data = train_data.reshape(-1, 3, 32, 32) is in the load_cifar() function which loads CIFAR data. You should define a new function that loads your dataset instead with the correct size.
See load_test() for an example of loading 224 x 224 data. The image size is passed around in a self.image_size variable in the dataloader not constrained to any fixed size.
I will look into such a notebook, but may not have the bandwidth for that very soon.

@arcelien @sayakpaul  thank for your help.

Hi @arcelien, i have word image dataset, i want to use your code to augment this dataset for my OCR system.
Each images have width in range(26 - 150), height in range(26 - 40) that is not fix.
Total number of classes is 1, because dataset just contain word images.
I don't know how to get start with this dataset. Do you have any guide to help me augment this dataset?
Thank you!

Hi @arcelien , can we only load all data in data_loader. But it's not friendly to large data sets. And I want to know whether we can build a data generator.

Hi @arcelien , can we only load all data in data_loader? But it's not friendly to large data sets. And I want to know whether we can build a data generator. Looking forward your reply. Thanks.

The way the current dataloader is implemented only supports loading all data at once, but it should not be too difficult to adapt this to generate data on demand or use a new dataloader which does so.
To adapt this, I'd imagine we could use something like the torchvision.datasets.SVHN data generator to load data in the next_batch() function and apply all the augmentation/transformations then instead of in the initialization.

Hi @arcelien , how can we get " MEAN" and "STD"  in files of augmentation_transforms.py  while using PBT in my own database.

If you're using the default dataloader you can pass the flag --recompute_dset_stats while training. Otherwise, you can just compute the per-channel mean and standard deviation of your dataset (i.e. call mean and std on your data tensors).

@arcelien thank you for your help!

The way the current dataloader is implemented only supports loading all data at once, but it should not be too difficult to adapt this to generate data on demand or use a new dataloader which does so.
To adapt this, I'd imagine we could use something like the torchvision.datasets.SVHN data generator to load data in the next_batch() function and apply all the augmentation/transformations then instead of in the initialization.

Hi @arcelien , Hvae a doubt on this, probably did not understand it clearly.

Even if we load the data in the 'next_batch()' function, aren't we still loading the whole dataset into memory? Or did you mean to create a kind of data iterator which loads only a batch inside the next_batch function?
I am currently using a dataset with imagenet like images of size 224x224x3 with a resnet50 model. But i see it takes a lot of GPU memory for each of the workers and hence other workers have to wait for the first few workers to finish. I am not clear how so much GPU memory (roughly 13GB) is being used per worker. Am i missing something in the data_loader side?

Thanks!

It could be that with ImageNet, you just need to use much smaller batches or distribute over multiple GPUs or run fewer trials concurrently. For reference, with a (32x32x3) input like CIFAR/SVHN, the size per trial is ~2GB with a network like ResNet 50.

I actually changed the dataloader so that it loads the data on the fly. Unfortunately could not get very good results. Have you tried getting some numbers for Imagenet by any chance?

Unfortunately, I haven't tried ImageNet. The original method from AutoAugment requires some special handling (Section 4.2 in https://arxiv.org/pdf/1805.09501.pdf):

we use a reduced subset of the ImageNet training set, with 120 classes (randomly chosen) and 6,000 samples, to search for policies. We train a Wide-ResNet 40-2 using cosine decay for 200 epochs...

AutoAugment hadn't released their code for ImageNet at the time and I unfortunately did not have the compute to redo their hyperparameter tuning to achieve good results. I'd imagine you wouldn't get great results if you didn't carefully tune things like LR/WD/BS in both the search phase and the re-training phase.

Hi @arcelien while using tfrecords as my database I have met the problem that I can't start the "coords" and "threads" to get my data in data_loader.next_batch . Can you give me some advice ?
Thanks!

Hi @tienduchoang , I have the same problem as you. Have you solved it now? Can you give me some advice ?

Hi @arcelien , while I have run the scripts search.sh in my own datasets and I have get some files in "results" ,the question is that how  to get the schedules like show in file "schedules/rcifar10_16_wrn.txt", which can directly used in training procedures.
Looking forward to your reply！

Hey, there should be a set of files in the results folder with the schedules corresponding to each worker. You can take a look at the tensorboard to select which worker schedule to pick.

@arcelien
Your help is much appreciated！

Hey, guys.
I think I've successfully applied search to my own dataset which need to make some changes to the data_utils.py.
The core idea is save img paths in self.train_images but not load the real data into memory as the original file. On this basis, the image is actually read into memory in function next_batch()
You can do the preprocess the way you want in next_batch().
So the problem that want to work with a folder that contain different size of images asked by @tienduchoang has been solved. And you can also run on a large dataset and don't have trouble with memory.
Don't forget to do normalization after load images in next_batch(), and "MEAN" and "STD" need to be computed by your self(I used a separate script to calculate it) and add into augmentation_transforms.py, like this:
MEANS = {
    'cifar10_50000': [0.49139968, 0.48215841, 0.44653091],
    'cifar10_4000': [0.49056774, 0.48116026, 0.44726052],
    'cifar100_50000': [0.50707516, 0.48654887, 0.44091784],
    'svhn_1000': [0.45163885, 0.4557915, 0.48093327],
    'svhn-full_604388': [0.43090966, 0.4302428, 0.44634357],
    'test_10000': [0.458078, 0.4862745, 0.530588]  # TODO:
}
STDS = {
    'cifar10_50000': [0.24703223, 0.24348513, 0.26158784],
    'cifar10_4000': [0.24710728, 0.24451308, 0.26235099],
    'cifar100_50000': [0.26733429, 0.25643846, 0.27615047],
    'svhn_1000': [0.20385217, 0.20957996, 0.20804394],
    'svhn-full_604388': [0.19652855, 0.19832038, 0.19942076],
    'test_10000': [0.16196, 0.15887, 0.1668]  # TODO:
}

I put the detailed modification below, which may be a little long.
modify test_search.sh to:
#!/bin/bash
export PYTHONPATH="$(pwd)"

python pba/search.py \
    --local_dir "$PWD/results/" \
    --model_name wrn_40_2 \
    --dataset test \
    --train_size 10000 --val_size 3000 \
    --checkpoint_freq 0 \
    --name "test_search" --gpu 0.49 --cpu 2 \
    --num_samples 2 --perturbation_interval 3 --epochs 6 \
    --explore cifar10 --aug_policy cifar10 \
    --lr 0.1 --wd 0.0005 --bs 2 --test_bs 2

modify data_utils.py to:
# Copyright 2018 The TensorFlow Authors All Rights Reserved.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# ==============================================================================
"""Data utils."""

from __future__ import absolute_import
from __future__ import division
from __future__ import print_function

import copy
try:
    import cPickle as pickle
except:
    import pickle
import os
import random
import numpy as np
from PIL import Image
import tensorflow as tf
import torchvision

from autoaugment.data_utils import unpickle
import pba.policies as found_policies
from pba.utils import parse_log_schedule
import pba.augmentation_transforms_hp as augmentation_transforms_pba
import pba.augmentation_transforms as augmentation_transforms_autoaug

# pylint:disable=logging-format-interpolation

def parse_policy(policy_emb, augmentation_transforms):
    policy = []
    num_xform = augmentation_transforms.NUM_HP_TRANSFORM
    xform_names = augmentation_transforms.HP_TRANSFORM_NAMES
    assert len(policy_emb
               ) == 2 * num_xform, 'policy was: {}, supposed to be: {}'.format(
                   len(policy_emb), 2 * num_xform)
    for i, xform in enumerate(xform_names):
        policy.append((xform, policy_emb[2 * i] / 10., policy_emb[2 * i + 1]))
    return policy

def shuffle_data(data, labels):
    """Shuffle data using numpy."""
    np.random.seed(0)
    perm = np.arange(len(data))
    np.random.shuffle(perm)
    data = data[perm]
    labels = labels[perm]
    return data, labels

class DataSet(object):
    """Dataset object that produces augmented training and eval data."""

    def __init__(self, hparams):
        self.hparams = hparams
        self.epochs = 0
        self.curr_train_index = 0

        self.parse_policy(hparams)
        self.load_data(hparams)

        ## Apply normalization
        #self.train_images = self.train_images.transpose(0, 2, 3, 1) / 255.0
        #self.val_images = self.val_images.transpose(0, 2, 3, 1) / 255.0
        #self.test_images = self.test_images.transpose(0, 2, 3, 1) / 255.0
        #if not hparams.recompute_dset_stats:
        #    mean = self.augmentation_transforms.MEANS[hparams.dataset + '_' +
        #                                              str(hparams.train_size)]
        #    std = self.augmentation_transforms.STDS[hparams.dataset + '_' +
        #                                            str(hparams.train_size)]
        #else:
        #    mean = self.train_images.mean(axis=(0, 1, 2))
        #    std = self.train_images.std(axis=(0, 1, 2))
        #    self.augmentation_transforms.MEANS[hparams.dataset + '_' + str(hparams.train_size)] = mean
        #    self.augmentation_transforms.STDS[hparams.dataset + '_' + str(hparams.train_size)] = std
        #tf.logging.info('mean:{}    std: {}'.format(mean, std))

        #self.train_images = (self.train_images - mean) / std
        #self.val_images = (self.val_images - mean) / std
        #self.test_images = (self.test_images - mean) / std

        assert len(self.test_images) == len(self.test_labels)
        assert len(self.train_images) == len(self.train_labels)
        assert len(self.val_images) == len(self.val_labels)
        tf.logging.info('train dataset size: {}, test: {}, val: {}'.format(
            len(self.train_images), len(self.test_images), len(self.val_images)))

    def parse_policy(self, hparams):
        """Parses policy schedule from input, which can be a list, list of lists, text file, or pickled list.

        If list is not nested, then uses the same policy for all epochs.

        Args:
        hparams: tf.hparams object.
        """
        # Parse policy
        if hparams.use_hp_policy:
            self.augmentation_transforms = augmentation_transforms_pba

            if isinstance(hparams.hp_policy,
                          str) and hparams.hp_policy.endswith('.txt'):
                if hparams.num_epochs % hparams.hp_policy_epochs != 0:
                    tf.logging.warning(
                        "Schedule length (%s) doesn't divide evenly into epochs (%s), interpolating.",
                        hparams.num_epochs, hparams.hp_policy_epochs)
                tf.logging.info(
                    'schedule policy trained on {} epochs, parsing from: {}, multiplier: {}'
                    .format(
                        hparams.hp_policy_epochs, hparams.hp_policy,
                        float(hparams.num_epochs) / hparams.hp_policy_epochs))
                raw_policy = parse_log_schedule(
                    hparams.hp_policy,
                    epochs=hparams.hp_policy_epochs,
                    multiplier=float(hparams.num_epochs) /
                    hparams.hp_policy_epochs)
            elif isinstance(hparams.hp_policy,
                            str) and hparams.hp_policy.endswith('.p'):
                assert hparams.num_epochs % hparams.hp_policy_epochs == 0
                tf.logging.info('custom .p file, policy number: {}'.format(
                    hparams.schedule_num))
                with open(hparams.hp_policy, 'rb') as f:
                    policy = pickle.load(f)[hparams.schedule_num]
                raw_policy = []
                for num_iters, pol in policy:
                    for _ in range(num_iters * hparams.num_epochs //
                                   hparams.hp_policy_epochs):
                        raw_policy.append(pol)
            else:
                raw_policy = hparams.hp_policy

            if isinstance(raw_policy[0], list):
                self.policy = []
                split = len(raw_policy[0]) // 2
                for pol in raw_policy:
                    cur_pol = parse_policy(pol[:split],
                                           self.augmentation_transforms)
                    cur_pol.extend(
                        parse_policy(pol[split:],
                                     self.augmentation_transforms))
                    self.policy.append(cur_pol)
                tf.logging.info('using HP policy schedule, last: {}'.format(
                    self.policy[-1]))
            elif isinstance(raw_policy, list):
                split = len(raw_policy) // 2
                self.policy = parse_policy(raw_policy[:split],
                                           self.augmentation_transforms)
                self.policy.extend(
                    parse_policy(raw_policy[split:],
                                 self.augmentation_transforms))
                tf.logging.info('using HP Policy, policy: {}'.format(
                    self.policy))

        else:
            self.augmentation_transforms = augmentation_transforms_autoaug
            tf.logging.info('using ENAS Policy or no augmentaton policy')
            if 'svhn' in hparams.dataset:
                self.good_policies = found_policies.good_policies_svhn()
            else:
                assert 'cifar' in hparams.dataset
                self.good_policies = found_policies.good_policies()

    def reset_policy(self, new_hparams):
        self.hparams = new_hparams
        self.parse_policy(new_hparams)
        tf.logging.info('reset aug policy')
        return

    def load_cifar(self, hparams):
        train_labels = []
        test_labels = []
        num_data_batches_to_load = 5
        total_batches_to_load = num_data_batches_to_load
        train_batches_to_load = total_batches_to_load
        assert hparams.train_size + hparams.validation_size <= 50000
        # Determine how many images we have loaded
        train_dataset_size = 10000 * num_data_batches_to_load

        if hparams.dataset == 'cifar10':
            train_data = np.empty(
                (total_batches_to_load, 10000, 3072), dtype=np.uint8)
            datafiles = [
                'data_batch_1', 'data_batch_2', 'data_batch_3', 'data_batch_4',
                'data_batch_5'
            ]
            datafiles = datafiles[:train_batches_to_load]
            test_data = np.empty((1, 10000, 3072), dtype=np.uint8)
            datafiles.append('test_batch')
            num_classes = 10
        elif hparams.dataset == 'cifar100':
            assert num_data_batches_to_load == 5
            train_data = np.empty((1, 50000, 3072), dtype=np.uint8)
            datafiles = ['train']
            test_data = np.empty((1, 10000, 3072), dtype=np.uint8)
            datafiles.append('test')
            num_classes = 100

        for file_num, f in enumerate(datafiles):
            d = unpickle(os.path.join(hparams.data_path, f))
            if f == 'test' or f == 'test_batch':
                test_data[0] = copy.deepcopy(d['data'])
            else:
                train_data[file_num] = copy.deepcopy(d['data'])
            if hparams.dataset == 'cifar10':
                labels = np.array(d['labels'])
            else:
                labels = np.array(d['fine_labels'])
            nsamples = len(labels)
            for idx in range(nsamples):
                if f == 'test' or f == 'test_batch':
                    test_labels.append(labels[idx])
                else:
                    train_labels.append(labels[idx])
        train_data = train_data.reshape(train_dataset_size, 3072)
        test_data = test_data.reshape(10000, 3072)
        train_data = train_data.reshape(-1, 3, 32, 32)
        test_data = test_data.reshape(-1, 3, 32, 32)
        train_labels = np.array(train_labels, dtype=np.int32)
        test_labels = np.array(test_labels, dtype=np.int32)

        self.test_images, self.test_labels = test_data, test_labels
        train_data, train_labels = shuffle_data(train_data, train_labels)
        train_size, val_size = hparams.train_size, hparams.validation_size
        assert 50000 >= train_size + val_size
        self.train_images = train_data[:train_size]
        self.train_labels = train_labels[:train_size]
        self.val_images = train_data[train_size:train_size + val_size]
        self.val_labels = train_labels[train_size:train_size + val_size]
        self.num_classes = num_classes

    def load_svhn(self, hparams):
        train_labels = []
        test_labels = []
        if hparams.dataset == 'svhn':
            assert hparams.train_size == 1000
            assert hparams.train_size + hparams.validation_size <= 73257
            train_loader = torchvision.datasets.SVHN(
                root=hparams.data_path, split='train', download=True)
            test_loader = torchvision.datasets.SVHN(
                root=hparams.data_path, split='test', download=True)
            num_classes = 10
            train_data = train_loader.data
            test_data = test_loader.data
            train_labels = train_loader.labels
            test_labels = test_loader.labels
        elif hparams.dataset == 'svhn-full':
            assert hparams.train_size == 73257 + 531131
            assert hparams.validation_size == 0
            train_loader = torchvision.datasets.SVHN(
                root=hparams.data_path, split='train', download=True)
            test_loader = torchvision.datasets.SVHN(
                root=hparams.data_path, split='test', download=True)
            extra_loader = torchvision.datasets.SVHN(
                root=hparams.data_path, split='extra', download=True)
            num_classes = 10
            train_data = np.concatenate(
                [train_loader.data, extra_loader.data], axis=0)
            test_data = test_loader.data
            train_labels = np.concatenate(
                [train_loader.labels, extra_loader.labels], axis=0)
            test_labels = test_loader.labels
        else:
            raise ValueError(hparams.dataset)

        self.test_images, self.test_labels = test_data, test_labels
        train_data, train_labels = shuffle_data(train_data, train_labels)
        train_size, val_size = hparams.train_size, hparams.validation_size
        if hparams.dataset == 'svhn-full':
            assert train_size + val_size <= 604388
        else:
            assert train_size + val_size <= 73257
        self.train_images = train_data[:train_size]
        self.train_labels = train_labels[:train_size]
        self.val_images = train_data[-val_size:]
        self.val_labels = train_labels[-val_size:]
        self.num_classes = num_classes

    def load_test(self, hparams):
        """Load random data and labels."""
        test_size = 3000
        self.num_classes = 2
        img_root_dir = 'the root img dir of yours'

        labels_dir_list = []
        for label_name in os.listdir(img_root_dir):
            labels_dir_list.append(os.path.join(img_root_dir, label_name))

        total_imgs_path_list = []
        total_imgs_labels = []
        for idx, label_dir in enumerate(labels_dir_list):
            for img_name in os.listdir(label_dir):
                img_path = os.path.join(label_dir, img_name)
                label = idx
                total_imgs_path_list.append(img_path)
                total_imgs_labels.append(label)

        # shuffle
        perm = np.arange(len(total_imgs_labels))
        np.random.shuffle(perm)
        total_imgs_path_list = np.array(total_imgs_path_list)
        total_imgs_labels = np.array(total_imgs_labels)
        total_imgs_path_list = total_imgs_path_list[perm]
        total_imgs_labels = total_imgs_labels[perm]

        train_size, val_size = hparams.train_size, hparams.validation_size
        assert train_size + val_size + test_size <= len(total_imgs_labels)
        self.train_images = total_imgs_path_list[:train_size]
        self.train_labels = total_imgs_labels[:train_size]
        self.val_images = total_imgs_path_list[train_size:train_size+val_size]
        self.val_labels = total_imgs_labels[train_size:train_size+val_size]
        self.test_images = total_imgs_path_list[-test_size:]
        self.test_labels = total_imgs_labels[-test_size:]

    def load_data(self, hparams):
        """Load raw data from specified dataset.

        Assumes data is in NCHW format.

        Populates:
            self.train_images: Training image data.
            self.train_labels: Training ground truth labels.
            self.val_images: Validation/holdout image data.
            self.val_labels: Validation/holdout ground truth labels.
            self.test_images: Testing image data.
            self.test_labels: Testing ground truth labels.
            self.num_classes: Number of classes.
            self.num_train: Number of training examples.
            self.image_size: Width/height of image.

        Args:
            hparams: tf.hparams object.
        """
        if hparams.dataset == 'cifar10' or hparams.dataset == 'cifar100':
            self.load_cifar(hparams)
        elif hparams.dataset == 'svhn' or hparams.dataset == 'svhn-full':
            self.load_svhn(hparams)
        elif hparams.dataset == 'test':
            self.load_test(hparams)
        else:
            raise ValueError('unimplemented')

        self.num_train = self.train_images.shape[0]
        self.image_size = 160

        # make one-hot label
        self.train_labels = np.eye(self.num_classes)[np.array(self.train_labels, dtype=np.int32)]
        self.val_labels = np.eye(self.num_classes)[np.array(self.val_labels, dtype=np.int32)]
        self.test_labels = np.eye(self.num_classes)[np.array(self.test_labels, dtype=np.int32)]
        assert len(self.train_images) == len(self.train_labels)
        assert len(self.val_images) == len(self.val_labels)
        assert len(self.test_images) == len(self.test_labels)

        # convert img_path to real train data
        val_imgs = []
        for img_path in self.val_images:
            cur_img = self._process_single_img(img_path)
            val_imgs.append(cur_img)
        self.val_images = np.array(val_imgs)
        test_imgs = []
        for img_path in self.test_images:
            cur_img = self._process_single_img(img_path)
            test_imgs.append(cur_img)
        self.test_images = np.array(test_imgs)

    def _process_single_img(self, img_path):
        img = Image.open(img_path, 'r')

        # make sure channel order is RGB
        if img.mode != 'RGB':
            img = img.convert("RGB")

        # resize(maintain aspect ratio)
        width, height = img.size
        long_edge_size = self.image_size
        if width > height:
            height = int(height * long_edge_size / width)
            width = long_edge_size
        else:
            width = int(width * long_edge_size / height)
            height = long_edge_size
        img = img.resize((width, height), Image.BILINEAR)

        # padding
        rgb_mean = self.augmentation_transforms.MEANS[self.hparams.dataset + '_' + str(self.hparams.train_size)]
        rgb_std = self.augmentation_transforms.STDS[self.hparams.dataset + '_' + str(self.hparams.train_size)]
        back_color = tuple([int(x * 255.0) for x in rgb_mean])
        img_padd = Image.new('RGB', (long_edge_size, long_edge_size), back_color)
        if width > height:
            h_st = int((long_edge_size - height)/2)
            img_padd.paste(img, (0, h_st))
        else:
            w_st = int((long_edge_size - width)/2)
            img_padd.paste(img, (w_st, 0))

        # convert PIL to numpy
        img = np.array(img_padd)

        # normalization
        img = img / 255.0
        img = (img - rgb_mean) / rgb_std
        return img

    def next_batch(self, iteration=None):
        """Return the next minibatch of augmented data."""
        next_train_index = self.curr_train_index + self.hparams.batch_size
        if next_train_index > self.num_train:
            # Increase epoch number
            epoch = self.epochs + 1
            self.reset()
            self.epochs = epoch

        # convert img_path to real train data
        batch_train_imgs = []
        for img_path in self.train_images[self.curr_train_index:self.curr_train_index + self.hparams.batch_size]:
            cur_img = self._process_single_img(img_path)
            batch_train_imgs.append(cur_img)

        batched_data = (np.array(batch_train_imgs),
            self.train_labels[self.curr_train_index:self.curr_train_index +
                              self.hparams.batch_size])
        final_imgs = []

        dset = self.hparams.dataset + '_' + str(self.hparams.train_size)
        images, labels = batched_data
        for data in images:
            if not self.hparams.no_aug:
                if not self.hparams.use_hp_policy:
                    # apply autoaugment policy
                    epoch_policy = self.good_policies[np.random.choice(
                        len(self.good_policies))]
                    final_img = self.augmentation_transforms.apply_policy(
                        epoch_policy,
                        data,
                        dset=dset,
                        image_size=self.image_size)
                else:
                    # apply PBA policy)
                    if isinstance(self.policy[0], list):
                        # single policy
                        if self.hparams.flatten:
                            final_img = self.augmentation_transforms.apply_policy(
                                self.policy[random.randint(
                                    0,
                                    len(self.policy) - 1)],
                                data,
                                self.hparams.aug_policy,
                                dset,
                                image_size=self.image_size)
                        else:
                            final_img = self.augmentation_transforms.apply_policy(
                                self.policy[iteration],
                                data,
                                self.hparams.aug_policy,
                                dset,
                                image_size=self.image_size)
                    elif isinstance(self.policy, list):
                        # policy schedule
                        final_img = self.augmentation_transforms.apply_policy(
                            self.policy,
                            data,
                            self.hparams.aug_policy,
                            dset,
                            image_size=self.image_size)
                    else:
                        raise ValueError('Unknown policy.')
            else:
                final_img = data
            if self.hparams.dataset == 'cifar10' or self.hparams.dataset == 'cifar100':
                final_img = self.augmentation_transforms.random_flip(
                    self.augmentation_transforms.zero_pad_and_crop(
                        final_img, 4))
            elif 'svhn' in self.hparams.dataset:
                pass
            else:
                tf.logging.log_first_n(tf.logging.WARN, 'Using default random flip and crop.', 1)
                final_img = self.augmentation_transforms.random_flip(
                    self.augmentation_transforms.zero_pad_and_crop(
                        final_img, 4))
            # Apply cutout
            if not self.hparams.no_cutout:
                if 'cifar10' == self.hparams.dataset:
                    final_img = self.augmentation_transforms.cutout_numpy(
                        final_img, size=16)
                elif 'cifar100' == self.hparams.dataset:
                    final_img = self.augmentation_transforms.cutout_numpy(
                        final_img, size=16)
                elif 'svhn' in self.hparams.dataset:
                    final_img = self.augmentation_transforms.cutout_numpy(
                        final_img, size=20)
                else:
                    tf.logging.log_first_n(tf.logging.WARN, 'Using default cutout size (16x16).', 1)
                    final_img = self.augmentation_transforms.cutout_numpy(
                        final_img)
            final_imgs.append(final_img)
        batched_data = (np.array(final_imgs, np.float32), labels)
        self.curr_train_index += self.hparams.batch_size
        return batched_data

    def reset(self):
        """Reset training data and index into the training data."""
        self.epochs = 0
        # Shuffle the training data
        perm = np.arange(self.num_train)
        np.random.shuffle(perm)
        assert self.num_train == self.train_images.shape[
            0], 'Error incorrect shuffling mask'
        self.train_images = self.train_images[perm]
        self.train_labels = self.train_labels[perm]
        self.curr_train_index = 0

Appreciate the detailed investigation @monkeyDemon! I'll see if I can check in something along these lines to be helpful for future users as well.


python3Packages.python-lsp-server: make some dependencies optional

Motivation for this change
Some dependencies of python-lsp-server can be optional
https://github.com/python-lsp/python-lsp-server/blob/v1.2.1/setup.py#L47
cq-editor fails to build with nixpkgs-review because it is already broken
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[x] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[x] Tested execution of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module

[x] Fits CONTRIBUTING.md.

This is a semi-automatic executed nixpkgs-review with nixpkgs-review-checks extension. It is checked by a human on a best effort basis and does not build all packages (e.g. lumo, tensorflow or pytorch).
Result of nixpkgs-review pr 135276 run on x86_64-linux 1

1 package failed to build and already failed to build on hydra master:

cq-editor: log was empty

13 packages built:

python38Packages.pyls-flake8
python38Packages.pyls-isort
python38Packages.pyls-spyder
python38Packages.pylsp-mypy
python38Packages.python-lsp-black
python38Packages.python-lsp-server
python38Packages.spyder
python39Packages.pyls-flake8
python39Packages.pyls-isort
python39Packages.pyls-spyder
python39Packages.pylsp-mypy
python39Packages.python-lsp-black
python39Packages.python-lsp-server


please give me some reference for add new row functionality

HI,
Please provide me some sample code to add new row functionality as am not getting anywhere by searching with react bootstrap table2. As am new to react js am not able to debug it properly. Please help me.
Thnk you in advance

@PriyaSahi maybe the editor sub-package can fulfill your use case: https://react-bootstrap-table.github.io/react-bootstrap-table2/docs/basic-celledit.html

@PriyaSahi we don't have add row feature. you have to add new data and render the table.


search engine isnt working anymore

Description
https://iptv-org.github.io/

I don't see any problem.

Strange. WhAt is the link again?
On Thu, Nov 24, 2022 at 8:30 AM Aleksandr Statciuk @.***>
wrote:

Closed #1393 https://github.com/iptv-org/epg/issues/1393 as completed.
—
Reply to this email directly, view it on GitHub
https://github.com/iptv-org/epg/issues/1393#event-7886104161, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AD5X34UIFUPC3LCGPWD4O7TWJ5UX5ANCNFSM6AAAAAASKMU3ZA
.
You are receiving this because you authored the thread.Message ID:
@.***>

--
In Memory of Spot (FOREVER)

I see.  I used Microsoft Edge and its okay now. Oh boy.


BUG:payment integration

Is there an existing issue for this?

[X] I have searched the existing issues

What happened?
when i click on order now button then nothing to show anything
how to solved :
user click on that button then open payment page or modal for pay that
Record

[X] I agree to follow this project's Code of Conduct
[X] I'm a GSSOC'24 contributor
[X] I want to work on this issue

@RitiChandak ,please assign this

Hey @Varsani2520 !
You can start working on this issue.
happy coding ✨


Automatic conjunct/disjunct simplification

Not sure if this should be optional or the default, but a common pattern in building theories has raised an issue...
T = true
T &= <constraint 1>
T &= <constraint 2>
...

This ends up creating a theory T such as And(<constraint 1>, And(<constraint 2>, And ( ... ))). With any moderately sized theory, the recursion depth is hit in a variety of ways (checking height, simplifying, etc). The workaround was to just store the constraints in a list of conjuncts and then wrap as And(conjuncts), but something like the following might be reasonable behaviour:

If taking the And of two theories with And as their root, return the And of their children. I.e., And(T1.children, T2.children)
If just one theory (wlog, assume T1) is an And, add the other theory as a child of it. I.e., And(T1.children, T2)
If neither theory is an And, do as normal. I.e., And(T1, T2)
Same for disjunctions.

This is tricky.
Making &= combine nodes would give terrible performance. It's currently O(1), but combining nodes is O(n) in the number of children (because it has to construct a whole new frozenset), so repeated use of &= would go from O(n) to O(n²). It's the same problem as repeatedly using += on strings.
(It's in principle possible to solve this at a low level, see e.g. efficient mutation in the frozenmap proposal. But I don't think that's reasonable here.)
Merging nodes can also drastically increase the sentence size, at least in contrived cases and possibly even in useful cases. Consider this sentence:

If we run .simplify() on it the single Or node at the bottom is merged with each of the five other Or nodes and we get this:

That increases the sentence size from 20 to 35.
For the same structure with 100 variables it increases the size from 400 to 10200.
(.simplify() takes a merge_nodes=False argument for this reason.)
So I'm hesitant to combine nodes automatically. It can make things worse, and the current behavior is at least very easy to analyze.
But it may be reasonable and feasible (though tricky) to restructure .simplify() to use some kind of queue instead of the call stack so that it never hits the recursion limit. Then you could use that after constructing the sentence to "fix" it.
Maybe other methods could be adjusted as well. It would definitely be nice to be able to handle sentences of unlimited depth.

Fair enough -- this issue, along with some of the others that have come up, stem from my first medium scale encoding task with the library. It occurs to me that we might want custom functionality for building theories destined for a SAT solver.

custom_object.add_constraint(<NNF>) instead of T &= <NNF>
custom_object.to_CNF(demorgan=True)

warns if loads of Aux vars are being created
special assumptions given the makeup of a set of constraints
etc.

I'm also thinking of a set of decorators to turn arbitrary object oriented code into SAT encodings, that would make sense in such a library.
Think it makes sense as part of python-nnf or a separate library that builds on it?

Unless there's major objections, I think we'll move forward with a companion library that focuses on advanced techniques for modelling. It will build heavily on nnf, and the student (👋 @KarishmaDaga ) will likely contribute fixes/enhancements to this library too.

Making simplify() work with these tall sentences turned out to be harder than I thought, because the intermediate results take up too much memory even if you work around the recursion limit.
A higher level companion library sounds like a great idea.


21110 Non-functional code to delete unsaved Minio documents when UI exits

Issue #: bcgov/entity#21110
Description of changes:

non-functional code to delete unsaved Minio documents when UI exits
this is not a working solution -- do not merge

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of the bcrs-entities-create-ui license (Apache 2.0).

All - this is a non-functional solution but I learned some things along the way and wanted to share.
Here are some comments I added to the ticket:

Part 2 Summary
After a fair bit of research and testing, I believe this cannot be implemented, as there is no way to delay the app from unloading while any unsaved documents are deleted.

Specifically, window.onbeforeunload can only be used to display a generic (synchronous) "unsaved messages" dialog. The next event that we can catch is window.onpagehide (because onunload is deprecated) and it's synchronous, too. So, we cannot wait for any network calls (deletions) to complete, and in fact any pending network calls are cancelled when the app terminates.

Basically, asking the client to clean up before exiting is not a good design (and it doesn't handle all cases, such as power interruptions and some mobile scenarios). According to what I've researched, a better design is to have the API auto-delete any documents whose keys we haven't stored in the filing JSON. I have created #21728 for this.

One potential client-side solution is to use a Service Worker to delete unsaved files. This is a separate thread and apparently can be made to outlive the initial web page thread. However, this still doesn't handle all cases and it's complex/overkill for just this purpose.

So what's the alternative here Sev? Do you have something in mind?

So what's the alternative here Sev? Do you have something in mind?

We can often detect when the web app ends (eg, close tab, close browser, navigate away) but there are some undetectable conditions (like power loss, or some mobile scenarios).
In the end, it's not a good design for the client (web app) to clean up what are essentially temporary documents on the (Minio server). We should assume the client could just "disappear" and the servers need to protect themselves, including their data integrity, which includes cleaning up orphaned documents. (I call them temporary documents because they are not "locked in" until the filing is submitted.)
A better design would have the server do the cleanup. Legal API is already involved in getting a key for the documents that the UI uploads. Legal API could hold on to those keys and delete documents that don't appear in a filing after, say, a year. (Or also store the filing id and then purge obsolete documents when that filing is submitted.)

So what's the alternative here Sev? Do you have something in mind?

We can often detect when the web app ends (eg, close tab, close browser, navigate away) but there are some undetectable conditions (like power loss, and some mobile scenarios).
In the end, it's not a good design for the client (web app) to clean up what are essentially temporary documents on the (Minio) server. We should assume the client could just "disappear" and the servers need to protect themselves, including their data integrity, which includes cleaning up orphaned documents. (I call them temporary documents because they are not "locked in" until the filing is submitted.)
A better design would have the server do the cleanup. Legal API is already involved in getting a key for the documents that the UI uploads. Legal API could hold on to those keys and delete documents that don't appear in a filing after, say, a year. (Or also store the filing id and then purge obsolete documents when that filing is submitted.)
cc: @vysakh-menon-aot


Repo no longer deployed because of a library error

The error is the following:

Reopening since the issue is not yet fixed

@AnnaLeinfelt now I see why it still didn't work. I've fixed the requirements.txt in my PR so that installs the right packages (and in fact it works when you try it locally), but the github action that deploys the docs instead install [other packages] https://github.com/Clinical-Genomics/meatballs/blob/bddb11d8abf8b0a0ec669b0eeb9923e2ced04891/.github/workflows/auto-deploy-docs.yml#L21C1-L21C1 of its choice before deployment, and that's why it still fails. I'll fix!


⚡(api) Use Ralph async HTTP client to improve API performances

Purpose
Use Ralph async HTTP client to improve API performances.
Proposal
Upgrade ralph-malph to its latest version, 3.9.0, to benefit from @Leobouloc works. Ralph has now an async HTTP client to query xAPI Statements. Thus, resulting in better resource utilization in the warren API and improved performances.
Note: Indicators are going to be merged in #49, some of these current updates might be duplicated/redundant.

[X] Upgrade ralph-malph==3.9.0
[X] Fix updated ralph-malph dependency paths.
[X] Refactor indicators to be async.

@Leobouloc


[usbdev] Random, Minimum & Maximum length out transaction sequences

This PR contains Random, Minimum & Maximum length out transaction sequences

@jdonjdon kindly merge this PR.


Feature/wkwebview

Fixes https://github.com/readium/SDKLauncher-iOS/issues/48

Also see: https://github.com/readium/SDKLauncher-iOS/pull/67


RSP-1623: Receipt code missing

Receipt page

Add CPMS reference
Add missing create date/time

Multi penalty non-card payment pages

Added create date/time to cash, cheque and postal order payment pages

Please confirm that DVSA want this change @jameshighfieldbjss.

Please confirm that DVSA want this change @jameshighfieldbjss.

@johnmorleybjss
Confirmed with DM who has confirmed with DVSA.


How to get the number of clients of HLS viewer?

The stat.xsl can not see the number of clients of HLS viewer.
So, how can i get this number?

@WL-hohoho did you make any progress with hls-watch? Any chances it will be merged with the official nginx rtmp module?

Interested in this as well to get a generic idea of hls viewers

What about using javascript in the website to track hls viewers? I am doing this successfully with jwplayer, checking the player status of the player, to confirm that the stream is running.

That's how I do it currently, but I prefer to measure those stats serverside regardless of what player the user wants to use. By doing it clientside with javascript I will have to be restricted to whatever player I end up using. Also on Android devices loading the .m3u8 file directly by using the android's native player tends to be far more stable than through jwplayer or flowplayer, so javascript in that case is not an option.

@misiek08 I understand you cannot share your code. Could you at least describe a high level overview of how you are calculating HLS stats? I wouldn't mind implementing this as a lua script or something similar, as long as the statistics are calculated server-side and not client-side with javascript :) Thanks.

I'm able to control most of websites using server so I give token to every
user. Then I use lua shared memory with on_abort and add users to array by
channel. Many connections from one user are reduced by token.
29 sty 2015 15:16 "Cha0s"<EMAIL_ADDRESS>napisał(a):

@misiek08 https://github.com/misiek08 I understand you cannot share
your code. Could you at least describe a high level overview of how you are
calculating HLS stats? I wouldn't mind implementing this as a lua script or
something similar, as long as the statistics are calculated server-side and
not client-side with javascript :) Thanks.
—
Reply to this email directly or view it on GitHub
https://github.com/arut/nginx-rtmp-module/issues/420#issuecomment-72031118
.

I'm able to control most of websites using server so I give token to every
user. Then I use lua shared memory with on_abort and add users to array by
channel. Many connections from one user are reduced by token. .
29 sty 2015 15:16 "Cha0s"<EMAIL_ADDRESS>napisał(a):

@misiek08 https://github.com/misiek08 I understand you cannot share
your code. Could you at least describe a high level overview of how you are
calculating HLS stats? I wouldn't mind implementing this as a lua script or
something similar, as long as the statistics are calculated server-side and
not client-side with javascript :) Thanks.
—
Reply to this email directly or view it on GitHub
https://github.com/arut/nginx-rtmp-module/issues/420#issuecomment-72031118
.

Thank you for your reply. That was close to what I had in mind. I am looking for a solution that does not depend in anyway on the client. I was thinking maybe using sessions to differentiate the viewers (so they could load the .m3u8 index directly without any tokens). I am still researching. I'll update this post when I find a solution.

Try cookies. Or reduce by IP, but it work bad for people sharing IP.
1 lut 2015 15:58 "Cha0s"<EMAIL_ADDRESS>napisał(a):

Thank you for your reply. That was close to what I had in mind. I am
looking for a solution that does not depend in anyway on the client. I was
thinking maybe using sessions to differentiate the viewers (so they could
load the .m3u8 index directly without any tokens). I am still researching.
I'll update this post when I find a solution.
—
Reply to this email directly or view it on GitHub
https://github.com/arut/nginx-rtmp-module/issues/420#issuecomment-72367865
.

Send access_log to elasticsearch (via Logstash, there are many tutorial on that).

@ebuildy and? Just like any other basic databse he will need to reduce log entries by some value to get the number.
@Cha0sgr Depending if your player supports 301 redirects for playlists you have 2 simplest ways here:
yes) You generate random token for user, put it into cookie, redirect it to url with token included. Then you can server playlist for .ts files with token included. Adding token to set in Redis allows you to do just SCARD and get viewer number.
no) You do same thing like above but with cookies. That's worse way because some HLS players don't save cookies (just discards them) so you get double count for same people if player forgots cookie.
If you have access to all websites serving your HLS stream you can add cookie and token at website side and you are 100% protected against any problem. Even if player won't tell you cookie you can point it to playlist_abcdef123.m3u8 and in playlist make .ts fragments with _abdefg123 suffix. Simple rewrites in code and you get counting without even single error.

Hi All,
I'm glad to see that you people already did something for list of HLS views/viewers long ago. I just started a month ago to one of our clients. I was able to do all the setup and we are able to push the live to server using "arut/nginx-rtmp-module". Is there anything included in this module itself to get the views statistics. ?
Are there any ready made scripts to get the viewers statistics ?
It would be great if anybody can help us to move forward.
@WL-hohoho @aluminum711 @misiek08

lines_to_check has to be adjusted based on the amount of requests you receive for 1min.
This also assumes this type of log format:
<IP_ADDRESS> - - [04/Feb/2022:21:57:38 +0100] "GET /mtest-20220204158391.ts HTTP/1.1" 200 423376 "https://widgets-fosdem.ems.host/" "Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:90.0) Gecko/20100101 Firefox/90.0"
#!/bin/bash
log=access.log
lines_to_check=20000
formats='ts|hls|dash'
date_prefix=$(date +'%d/%b/%Y:%H')
cur_min=$(date +'%M')
cur_min_decimal=${cur_min:0:1}
if [[ $cur_min_decimal == 0 ]]; then
    prev_min_decimal=5
else
    prev_min_decimal=$(($cur_min_decimal - 1))
fi
cur_minuted_date="${date_prefix}:${cur_min_decimal}"
prev_minuted_date="${date_prefix}:${prev_min_decimal}"

tail -n $lines_to_check $log \
	| grep -E "(${cur_minuted_date}|${prev_minuted_date}).*GET \/.*\.($formats)" \
	| awk '{print $7}'|sort|uniq|wc -l

@hackman
there is a bug in time operation when cur_min_decimal is 0
for example :
if current time is 12:38 prev_minuted_date will be 12:28 (which is fine)
if current time is 12:08 prev_minuted_date will be 12:58 (which is not fine)
instead you can do :
...
prev_minuted_date=$(date +'%d/%b/%Y:%H:%M' -d '-10 minutes')
cur_minuted_date=$(date +'%d/%b/%Y:%H:%M')
...


Improvement: Add integration tests for other databases

The current integration tests only use SQLite. Different databases handle a lot of things differently, so we should add integration tests for these as well, specifically MySQL and Postgres.

Partially fixed via #122

fixed via #144


[WARN] YCSB libraries not found.

Hello.
I get this message when trying to load the ycsb. It says "mvn is not recognized"  but as you can see on the print screen below I have maven installed and the environment variables created aswell. Also have "maven\bin" on PATH.

@busbey No, the 'basic' datastore also doesn't work.
I compiled the ArangoDB3 binding with success before loading the workload with
mvn -pl com.yahoo.ycsb:arangodb3-binding -am clean package
I analysed the 'ycsb.bat' and it crashes on the line 172, but just call MAVEN on line 173.
What can I do?

@busbey when trying to execute
./bin/ycsb.bat load basic -P workloads/workloada
I get this error

does doing a full maven build from the top work? that error with the basic looks like you can't connect to maven central.

By "full maven build" you mean run this code?
mvn clean package
Should I delete the repository and reinstall it before doing that?

this step from the windows set up guide:
https://github.com/brianfrankcooper/YCSB/wiki/Prerequisites-for-Windows#first-build
mvn package should suffice. you might need to tell Maven to check for things it thinks are missing via mvn -U package

Running mvn -U package it successfully builds CORE YCSB and ArangoDB3 (the one I want to benchmark).

When I try to load basic or arangodb3 datastore it fails building CORE YCSB

the build from the top level failing is a red flag that something is wrong in the build environment. what's the error in the first case?

looks like you should first create a database 'ycsb' with table 'usertable' in your ArangoDB instance.

I already did, same problem...

On ArangoDb3 README it says to "drop the existing collection: "usertable" under database "ycsb"" with this code db._collection("usertable").drop() but it doesn't recognise "usertable".
On my comment above it says that failed to create table ycsb and collection usertable so my instance shouldn't have those created before (?).
Maybe I have to tell the default user login is root?

It's hard to say. The ArangoDB3 binding hasn't made it to our supported list yet, so it's not surprising that these kinds of edge cases could come up.
The error you got does look like a case of "I can't verify that you're authorized to do that", but I don't see anything in the current docs to say how to specify a user/password.
Does the older ArangoDB binding fail in the same way? It looks like it was tested in the 0.12.0 release.

Is it possible to benchmark ArangoDB3 with 0.12.0?

I don't think so, it was added in the 0.13.0 release. Does the ArangoDB binding work with different server versions? If so, we should document that.

I get the message "Unauthorized" when trying the ArangoDB binding.

I run the GitBash with admin privileges and set "Administrate" one ArangoDB root user for all databases.

@busbey , Once I get the error 401 Unauthorized, I searched for it and I found this:
https://httpstatuses.com/401

The request has not been applied because it lacks valid authentication credentials for the target resource.

This means that maybe it neads the user login (root) on parameters?

SOLVED
I had to turn off authentication parameter on the arangod.conf file located on C:\ArangoDB\etc\arangodb3\arangod.conf
authentication = false

Also, I'd like to have an open issue that tracks this gap so I can reference it in release notes. Would you prefer I reopen this one, or make a new one? (if I reopen this one you'll get github notifications when folks comment here or a PR fixes it, unless you unsubscribe from the issue.)

Not add, but change, it's value is "true" by default. Also, that information should be on ArangoDB3 README before the command db._collection("usertable").drop() One note: the file name is arangod.conf and not arangodb.conf.
The ArangoDB client bindings are not set up to provide authentication details to the ArangoDB instance, and as such you must ensure authentication is off for the system under test.

You can do this by altering your arangod.conf file and modify authentication parameter to "authentication=false".

You can reopen this ticket, no problem.


Time Weight > 0 result in tank levels greater than maximum tank levels

Hello,
Can you provide an example? [INP]
Best regards.

This is a simple sample file to illustrate the problem. After 7:00 a.m. tank T-1 continuously fills up even if the maximum level is reached.
Sample_TankMaxLevel.zip

This same issue was discovered when trying to implement Dynamic Tanks into the legacy EPANET 2.2 C-based code (see this link). Because of this, and the fact that they pose issues when trying to evaluate level controls, this feature was not implemented in EPANET 2.2.


[2.0]Implement GUI for more arcfutil generators in AFF Toolbox

Functions to be implemented (until arcfutil 0.10.1):

[x] timing_easing_by_disp()
[x] arc_envelop()
[x] arc_straighten() (in postprocess)
[x] arc_interlace() (in postprocess)

Implemented.


bug(tabs): Buttons no longer work in tab labels after MDC alignment.

Is this a regression?

[X] Yes, this behavior used to work in the previous version

The previous version in which this bug was not present was
14
Description
Prior to the MDC alignment, icon buttons could be added to mat-tab-labels. This allowed for removing tabs, just like how chrome tabs works. With the new MDC tabs, and mat-tab-nav-bar the whole tab takes the click, instead of being able to add an icon.
Reproduction
Here's a stackblitz:
https://stackblitz.com/edit/angular-yewshy?file=src%2Fapp%2Ftab-nav-bar-basic-example.html,src%2Fapp%2Ftab-nav-bar-basic-example.ts
You can see that there's a tab for adding new tabs, which works well, basically because it just looks like the user is clicking the icon, when it's just the tab.
But none of the close buttons work. I've tried changing the z-index and adding the mat-tab-nav-bar directive to a div instead of an anchor, and a few other things to no avail.
Expected Behavior
Adding a mat-icon-button should be clickable on a mat tab. Alternatively it would be great to have a feature to be able to remove the tab from the tab itself.
Actual Behavior
Nothing happens when clicking on the icon in either a mat-tab or an anchor tag with the mat-tab-nav-bar directive.
Environment

Angular: 15
Material:
Browser(s): Chrome
Operating System (e.g. Windows, macOS, Ubuntu): Windows 11

workaround: https://stackoverflow.com/questions/74712872/angular-material-v15-not-able-to-click-on-icon-inside-mat-tab/74713693#74713693

Digging a little further, it looks like pointer events: none has always been a part of material components web, according to what I believe is the source file here.
There's no mention of closing tabs in the material design spec, however pointer-events will bubble, so applying pointer-events: auto to the icon itself resolves this issue.
So with that being set is it more appropriate to:
a) leave it as it is, maybe update the docs to point out this functionality
b)
It looks like this is the file where the import takes place in this repo and it already has another override in there to handle the background-color on tabs.
Would it be possible to add in another override to handle this case?
or
c) change the mat-icon icons.scss to have pointer-events: auto?
I guess all that is a discussion for the actual Angular team though and not some random amateur haha.

Also I can't find a way to add a close icon to a mat-tab-link?
<nav mat-tab-nav-bar [tabPanel]="tabPanel">
  <a mat-tab-link
     *ngFor="let tab of tabs; let $index=index"
     (click)='activeIndex=$index'
     [active]="$index==activeIndex">
     ????
  </a>
</nav>
<mat-tab-nav-panel #tabPanel>
  <router-outlet></router-outlet>
</mat-tab-nav-panel>

Also I can't find a way to add a close icon to a mat-tab-link?
<nav mat-tab-nav-bar [tabPanel]="tabPanel">
  <a mat-tab-link
     *ngFor="let tab of tabs; let $index=index"
     (click)='activeIndex=$index'
     [active]="$index==activeIndex">
     ????
  </a>
</nav>
<mat-tab-nav-panel #tabPanel>
  <router-outlet></router-outlet>
</mat-tab-nav-panel>

Here's a stackblitz.
<a mat-tab-link *ngFor="let link of links"
  (click)="activeLink = link"
  [active]="activeLink == link"> {{link}}
  <mat-icon style="pointer-events: auto;" (click)=closeTab()>close</mat-icon>
</a>


Add ipaddress library needed by the geolocation code

Also fix the README and remove the tarball since its contents are
version controlled
@jasoncpatton if the latest tarball works, this PR is safe to merge

Works! Merging.


fix(sql-udf): correctly binding arguments for inner calling sql udfs

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.
What's changed and what's your intention?
Fixed a subtle but critical bug that causing the incorrect update of the udf_context in the previous implementation.
When I'm trying to support valid recursive sql udf in https://github.com/risingwavelabs/risingwave/pull/14392#discussion_r1444187650 (and eventually found it seems impossible without modification at present😅), I found that the calling like foo($1 + 1) could be error-prone and the inner binding could not give the correct result.
In the previous implementation, the udf_context will NOT be updated along the way, using the anonymous parameters from the root calling sql udf all the way down, this will lead to incorrect result if users try to call other sql udfs and change the input parameters.
Thus in the current implementation, we will update the udf_context whenever we meet a binding for sql udf (i.e., inside bind_function), after which we will restore / clear the udf_context depending on if the current binding is the root sql udf.
See the updated test cases for details.
P.S. I think introducing a new binder could not resolve this problem, thus the implementation is still based on udf_context.

Checklist

[x] I have written necessary rustdoc comments
[x] I have added necessary unit tests and integration tests
[ ] I have added test labels as necessary. See details.
[ ] I have added fuzzing tests or opened an issue to track them. (Optional, recommended for new SQL features #7934).
[ ] My PR contains breaking changes. (If it deprecates some features, please create a tracking issue to remove them in the future).
[x] All checks passed in ./risedev check (or alias, ./risedev c)
[ ] My PR changes performance-critical code. (Please run macro/micro-benchmarks and show the results.)

[ ] My PR contains critical fixes that are necessary to be merged into the latest release. (Please check out the details)

Documentation

[ ] My PR needs documentation updates. (Please use the Release note section below to summarize the impact on users)

Release note
If this PR includes changes that directly affect users or other significant modifications relevant to the community, kindly draft a release note to provide a concise summary of these changes. Please prioritize highlighting the impact these changes will have on users.

This PR maybe reviewed & merged first, since it's an fix & preparation for recursion support. (i.e., correctly handle the case like foo($1 + 1) by correctly manage udf_context)
cc @wangrunji0408 @BugenZhao @xxchan.

why is this closed :eyes:


Make app deployable

Per https://github.com/carawarner/pantheon-demo/issues/13
Goals
Code

[x] Have an Apache config for the demo so it can be served
[x] Have a .wsgi file for the demo so it can run
[x] Have a Dockerfile that generates a valid image for the demo

Not Code

[x] Have a remote Ubuntu/Xenial server with enough RAM to run the demo
[x] Give the server a static IP address
[x] Configure a DNS zone mapping carawarner.com and subdomains to my server's static IP address
[x] Update my domain registrar to use the nameservers provided by my host
[x] Push a valid Docker image to DockerHub in the private carawarner/pantheon-demo repo.
[x] Pull that Docker image onto the remote server and run it

Challenges
I encountered a number of bugs along the way. Most are on the Issue. Here are the three most frustrating:
WSGI can't load the app
The pantheon-generator package requires python3+. During development I used a python3 virtualenv. On my first launch attempt, mod-wsgi couldn't load the app. I got a 500 in browser and error messages in pantheondemo/logs/error.log. The problem was a mismatch between the version of Python against which mod-wsgi was compiled and the version of Python my virtualenv used. I had to install a Python3-specific version of mod-wsgi AND modify my Apache config file to tell Python that python-home=<location of virtualenv>.
WSGI can't find global modules
I removed the virtualenv when I started using Docker. In the absence of a virtualenv, the global pip install in my Dockerfile installed dependencies under user/local/lib/python2.7. I had to install a Python3-specific version of pip.
Insufficient memory on the remote server
I built a new Docker image and ran it in a local container, confirmed it was working, and pushed it to Docker Hub. When I pulled it onto my remote server and ran it and went to the browser, I got an endless spinner. There was nothing in the container's logs for apache or for pantheon-demo. So I looked in the server's syslog. My process was terminated for using too much memory. I had to spin up a new Lightsail server with 2GB RAM available.

Note for future self: remember that apache2 isn't running when you run the container. You have to manually start it. Fix this in the Dockerfile.

Posting a congratulatory GIF on my own PR in a personal repo: new high or new low?


Write with 'Krayons'

Another way to write, would be to embed colour infomation inside text eg
"Something to say: {blue:-}, I've got chills, {red:-} and now I'm feeling hot".
Where the tokens inside {} are called Krayons and control the colour being rendered.
This functionality represents 'unstructured' writes.

With this new api, do NOT use any sort of multi-dimensional arrays. They are too risky to use and are subject to the dreaded flattening which just breaks code too easily. Either we use an array of PSCustomObjects or we use some kind of compound structed string that can be split into constituent parts.

Actually, take a look at NodeJs chalk module

Implemented new writer class which replaces all other writer methods. Existing writer mehods are now considered deprecated in favour of th new writer, but will be left in until client modules have made the transition.

funcctions New-Writer, KP and KL not being exported properly.
It looks like the issue of class references in ps1 files without the import statment has been fixed in PSScriptAnalyzer, which means we can reference a class without having to import it. Becasue of this, we can extract out New-Writer, KP and KL functions into their own Public files and insert the alias exports in the source psd1 file.

The above statement about the classes is not correct. Its just that I've started to understand the finer details of classes in PowerShell

The above statement about the classes is not correct. Its just that I've started to understand the finer details of classes in PowerShell


[questions][feat] add questions models

Schema link: https://sp2fcymivn.larksuite.com/docs/docusmlmfDD140gpFXkE9n6xn0c
Implemented until answer comment vote table for first iteration

Thanks Hong Po, I'll have a look tmr morning
On Fri, 7 Oct 2022, 7:14 pm hpkoh, @.***> wrote:

Schema link:
https://sp2fcymivn.larksuite.com/docs/docusmlmfDD140gpFXkE9n6xn0c
Implemented until answer comment vote table for first iteration
You can view, comment on, or merge this pull request online at:
https://github.com/yangshun/tech-interview-handbook/pull/323
Commit Summary

df48c77
https://github.com/yangshun/tech-interview-handbook/pull/323/commits/df48c77b9945019dd4378d996984b09e86657229
[questions] [feat] add questions models

File Changes
(2 files
https://github.com/yangshun/tech-interview-handbook/pull/323/files)

A
apps/portal/prisma/migrations/20221007110225_add_questions_model/migration.sql
https://github.com/yangshun/tech-interview-handbook/pull/323/files#diff-08738981c2ac6d4f2166edefc2e78c464359812f11be3ce6de31eb97370eff08
(171)
M apps/portal/prisma/schema.prisma
https://github.com/yangshun/tech-interview-handbook/pull/323/files#diff-d4d4a199a945aa2004056933865a4990015ba72b5770fefc24abd6633aa06db5
(165)

Patch Links:

https://github.com/yangshun/tech-interview-handbook/pull/323.patch
https://github.com/yangshun/tech-interview-handbook/pull/323.diff

—
Reply to this email directly, view it on GitHub
https://github.com/yangshun/tech-interview-handbook/pull/323, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AAKBCHMNUMY3PR6XT5LRQC3WCAAXZANCNFSM6AAAAAAQ7PW7R4
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

@keanecjy what do you think of aligning on the way we do upvotes across the projects? I quite like the QuestionsVote enum, maybe we should just create a generic Vote enum

I think that sounds good, we can go with it


python3Packages.pyduke-energy: init at 1.0.0

Motivation for this change
Python module for the Duke Energy AP
https://github.com/mjmeli/pyduke-energy
Things done

Built on platform(s)

[x] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
22.05 Release Notes (or backporting 21.11 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[x] Fits CONTRIBUTING.md.

Result of nixpkgs-review pr 151323 run on x86_64-linux 1

2 packages built:

python38Packages.pyduke-energy
python39Packages.pyduke-energy


GH-40825:  [Java] Adding Spotless to Flight module

Rationale for this change
Adding code style and formatting options for Flight module.

[X] flight-core
[X] flight-integration-tests
[X] flight-sql
[X] flight-sql-jdbc-core
[X] flight-sql-jdbc-driver

What changes are included in this PR?
Code formatting spotless plugin has been added.
Some explicit involvement required for the following
[WARN] /home/vibhatha/github/fork/arrow/java/flight/flight-core/src/main/java/org/apache/arrow/flight/FlightProducer.java:94:3: Missing a Javadoc comment. [MissingJavadocMethod]
Audit done.
[WARNING] src/main/java/org/apache/arrow/flight/FlightProducer.java:[94,3] (javadoc) MissingJavadocMethod: Missing a Javadoc comment.

[WARN] /home/vibhatha/github/fork/arrow/java/flight/flight-sql-jdbc-core/src/main/java/org/apache/arrow/driver/jdbc/utils/AvaticaParameterBinder.java:58:3: Missing a Javadoc comment. [MissingJavadocMethod]
Audit done.
[WARNING] src/main/java/org/apache/arrow/driver/jdbc/utils/AvaticaParameterBinder.java:[58,3] (javadoc) MissingJavadocMethod: Missing a Javadoc comment.
AvaticaParameterBinder constructor java docs manually added

These docs have been explicitly added.
Are these changes tested?
Yes, but doesn't involve test cases, the plugin itself corrects.

GitHub Issue: #40825

@lidavidm this diff is too large, should we go for per sub-module?


Upgrade Rust toolchain to 2023-08-14

Description of changes:
Relevant upstream commits/PRs:
https://github.com/rust-lang/rust/pull/114382: this required handling the new compare_bytes intrinsic
https://github.com/rust-lang/rust/pull/114622: crate_types is no longer a method in Session
Resolved issues:
Resolves #ISSUE-NUMBER
Related RFC:

Optional #ISSUE-NUMBER.
Call-outs:

Testing:

How is this change tested? Current regressions

Is this a refactor change? No

Checklist

[X] Each commit message has a non-empty body, explaining why the change was made
[X] Methods or procedures are documented
[X] Regression or unit tests are included, or existing tests cover the modified code
[X] My PR is restricted to a single feature or bugfix

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 and MIT licenses.

I'm a bit confused about the fmt_i8 test because in #2576, it was reported that its verification time increased from ~6s to 10 minutes. Looks like the time went down at some point, and then went up again with this toolchain upgrade? The increase is not as substantial as before though as it only went up to ~44 seconds (and solver runtime went up to ~19 seconds).
@celinval do you have any more information on the performance trend of this test?

Ah, looks like changing the default solver to cadical brought down the time from ~1059 seconds to ~42 seconds:
https://github.com/model-checking/kani/actions/runs/5744858062
But based on the perf-benchcomp action runs, there is some variance in the runtime of this benchmark, so we can safely ignore the increase in runtime seen on this PR.


Apply style guides to "Upgrade to Corporate ToS"

Why:
I believe it's using boldface for UI element that cannot be interacted with.
Closes: N/A

What's being changed (if available, include any code snippets, screenshots, or gifs):
I don't have an organization to test it, but I believe this is referring to a section that cannot be interacted with (please double check 🙏):

Reference: User interface text

Use quotation marks to surround UI text that cannot be interacted with.

Check off the following:

[x] I have reviewed my changes in staging, available via the View deployment link in this PR's timeline (this link will be available after opening the PR).

For content changes, you will also see an automatically generated comment with links directly to pages you've modified. The comment won't appear if your PR only edits files in the data directory.

[x] For content changes, I have completed the self-review checklist.

@itecompro Thanks so much for opening a PR with another fix! This looks great ✨ I'll update the branch and get this merged once tests are passing 🍏


Dubbo GSoC 2023 - Metrics on Dubbo Admin

Dubbo Admin is a console of Dubbo. Today, Dubbo's observability is becoming more and more powerful. We need to directly observe some indicators of Dubbo on Dubbo Admin, and even put forward suggestions for users to improve problems.
https://issues.apache.org/jira/browse/DUBBO-110

https://qedzyx.axshare.com/#id=2pqh0k&p=admin__&g=1

I am very familiar with and interested in this task, I have submitted many PRs for related aspects of dubbo admin, and I will submit a proposal soon.


Injectors

Just a placeholder so I can attach this -- will fill out more later.
TL;DR -- we have materialize(to.xyz), this aims to add materialize(from_.xyz)
TODO -- fill out the lines below:
Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]
Describe the solution you'd like
A clear and concise description of what you want to happen.
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.

This is completed


Update ES translation

Coverage remained the same at 59.166% when pulling 3c08db382981f8af1e602556652b52957a3ff960 on Roboron3042:master into e88f5c64222d83cf6cca725b88ddf952ffe4308d on newsboat:master.

Also, would you like to be added to "translators" group

Sure. I have been using Newsboat for about a year: Just recently I switched to Miniflux, looked for fresh alternatives, and disenchanted went back to Newsboat. So I'll probably keep myself around for a while.

Thank you @Roboron3042! I just sent you an invite to the Translators team.


feat:CXAR-921 Agent Plane Code Overtake

Description
Overtakes code from catenax-ng/product-agents-edc
Fixes https://jira.catena-x.net/browse/KA-324
Pre-review checks
Please ensure to do as many of the following checks as possible, before asking for committer review:

[x] DEPENDENCIES are up to date. Dash license tool. Committers can open IP issues for restricted libs.
[x] Copyright and license header are present on all affected files

Additional information

To request a review, check out who's involved to see a list of contributors and committers
Use the mailing list if you are unsure, who to contact, or if want to engage in a general discussion
Check out our guide on how to contribute
Check out our Release Guidelines
Check out the Eclipse Foundation contribution guidelines

For the IP check
maven/mavencentral/org.apache.jena/jena-fuseki-core/4.8.0, , restricted, clearlydefined
see https://github.com/apache/jena/blob/main/LICENSE

Initial Contribution check ticket is created here: https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/9836

IP ticket is through. We need a rerun of DEPENDENCIES file for this restricted library maven/mavencentral/org.apache.jena/jena-fuseki-core/4.8.0, , restricted, clearlydefined and potentially an IP ticket before we can merge it.
All other comments from our side will be revalidatedin the QG4

Hi @drcgjung,
I was just trying to create the automated IP issues using Dash licence tool. When generating the input for dash via
`mvn --settings settings.xml dependency:list -DskipTests -Dmaven.javadoc.skip=true -DappendOutput=true -DoutputFile=maven.deps', I get the following error (same without the custom settings.xml from the repo):
[ERROR] Failed to execute goal on project agentplane-azure-vault: Could not resolve dependencies for project org.eclipse.tractusx.agents.edc.agent-plane:agentplane-azure-vault:jar:1.9.5-SNAPSHOT: The following artifacts could not be resolved: org.eclipse.tractusx.agents.edc.agent-plane:agent-plane-protocol:jar:1.9.5-SNAPSHOT (absent), org.eclipse.tractusx.edc:auth-jwt:jar:1.9.5-SNAPSHOT (absent): Could not transfer artifact org.eclipse.tractusx.agents.edc.agent-plane:agent-plane-protocol:jar:1.9.5-SNAPSHOT from/to edc (https://maven.pkg.github.com/eclipse-tractusx/tractusx-edc): status code: 401, reason phrase: Unauthorized (401)

could you either investigate the issue or provide a maven.deps file for dash input, that I can create the automated IP issues?

Hi @drcgjung,
I was just trying to create the automated IP issues using Dash licence tool. When generating the input for dash via `mvn --settings settings.xml dependency:list -DskipTests -Dmaven.javadoc.skip=true -DappendOutput=true -DoutputFile=maven.deps', I get the following error (same without the custom settings.xml from the repo):
[ERROR] Failed to execute goal on project agentplane-azure-vault: Could not resolve dependencies for project org.eclipse.tractusx.agents.edc.agent-plane:agentplane-azure-vault:jar:1.9.5-SNAPSHOT: The following artifacts could not be resolved: org.eclipse.tractusx.agents.edc.agent-plane:agent-plane-protocol:jar:1.9.5-SNAPSHOT (absent), org.eclipse.tractusx.edc:auth-jwt:jar:1.9.5-SNAPSHOT (absent): Could not transfer artifact org.eclipse.tractusx.agents.edc.agent-plane:agent-plane-protocol:jar:1.9.5-SNAPSHOT from/to edc (https://maven.pkg.github.com/eclipse-tractusx/tractusx-edc): status code: 401, reason phrase: Unauthorized (401)

could you either investigate the issue or provide a maven.deps file for dash input, that I can create the automated IP issues?

Since the build has not yet run on the target maven repo, you need to "install" in addition. Since its a multi-module pom, I guess you need to make the target file absolute:
' mvn --settings settings.xml dependency:list -DskipTests -Dmaven.javadoc.skip=true -DappendOutput=true -DoutputFile=$(pwd)/maven.deps'
works on my Mac.
I'll send you the file (and a fresh DEPENDENCIES) via email, ok?

Hi @drcgjung,
ok ok I indeed didn't run a install first. With the maven.deps you sent me only the following issue was created:

https://gitlab.eclipse.org/eclipsefdn/emo-team/iplab/-/issues/9841

so as soon as this is processed, you can generate the DEPENDENCIES again. Using the maven plugin should produce the same output as running Dash plainly

[INFO] Querying Eclipse Foundation for license data for 89 items.
[INFO] Found 71 items.
[INFO] Querying ClearlyDefined for license data for 18 items.
[INFO] Found 18 items.
[INFO] Vetted license information was found for all content. No further investigation is required.

@carslen could you please merge this initial contribution pre-QG4 ... otherwise we will not be able to run some of the required QG4 checks (the cat and her tail ...) Thx.

ip check and dep are fixed. merging it for pre qg4


hubot compatibility to mattermost

Hi,
I'm running mattermost version 4.8.0 on centos 7 server also I installed hubot to connect to mattermost.
I also completely run hubot help and its work on my terminal. But, I having trouble connecting it to mattermost. do you have any suggestion? Please help

I recommend hubot-matteruser adapter.


adding a check for the .dockerenv file to know we are in docker

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have signed the Adobe Open Source CLA.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

+1


add some build commands

For those who are unfamiliar, it may not be obvious that building is quite simple.  The script may give newcomers a clearer understanding of what it means.


Exist in editable range

There should be a component setting to state whether or not a component can exist in an editable region.
For example, you don't want a web editor adding a <block name="MainMenu"/> to a
 that is editable.
Not sure where this setting should reside. Probably within the Component itself. Perhaps ACL concerning the Component would suffice.
$component->allow_in_editable_region = false;
This would check if any parents featured an attribute that indicated whether or not it was editable. Perhaps
 or
, etc.

Outside scope of LHTML processing


Fixed Suse groupadd/useradd

I have removed fixed guid/uid number, letting system decide.
A problem could happend though, during an upgrade: already created folder could not be updated, if guid/uid is different from the previous one.
The %pre script still checks for RHEL5, otherwise uses default script for everything else.

This pull request is now managed with CodeReviewHub. Refresh.

Looks good. I'm merging this so we can have some feedback from users using the SNAPSHOT version. I'll wait a bit before doing a release ...
I'm still not convinced we have the right solution for managing the UID ... but I do not have a bettre proposal ...

Yeah I know, I will keep on looking for other solutions around, if I find one which better suits jmxtrans.
Thanks


How do I register INotification with generics?

Is there any way to use notifications with generics?  I have this:
public sealed record ApprovedNotification<T>(T Checklist) : INotification
    where T : IChecklist;

public sealed class NotificationHandler<T> : INotificationHandler<ApprovedNotification<T>>
    where T : IChecklist

When I send the notification, the handler is never called.  I'm guessing I need to do some type of explicit registration?

You can use generics out of the box in almost the same way:
public record ApprovedNotification<T>(T Checklist) : INotification
    where T: IChecklist;

public sealed class NotificationHandler<T> : INotificationHandler<T>
    where T : ApprovedNotification<IChecklist>
{
    public Task Handle(T notification, CancellationToken cancellationToken)
    {
        Console.WriteLine("Handled");
        return Task.CompletedTask;
    }
}

The differences:

The notification type can't be sealed due to boxing/unboxing constraints
The handler needs to the type of the notification, not the type inside the notification, though you can specify that it handles only notifications of the specified type you're looking for

Thank you, @MattWood21 !


🐛 [Bug]: 数据源批量删除后，按钮状态没有更新问题

Environment
chrome
Version
latest
Version
latest
Link to minimal reproduction
none
Step to reproduce
如图所示，删除静态数据后，"批量删除"按钮没有置灰

What is expected
No response
What is actually happening
No response
What is your project name
tiny-engine develop
Any additional comments (optional)
No response

感谢反馈 @langyjn


Try adb wait_for_device

Please make sure these boxes are checked before submitting your pull request - thanks!

[ ] Run the unit tests with gradlew app:connectedGmsDebugAndroidTest to make sure you didn't break anything (with an emulator running or phone connected)

[ ] If you have multiple commits please combine them into one commit by squashing them.

Note you may see the Cirrus CI system failing on your PR. It's pretty flaky right now so don't
be alarmed.

Yeah, that didn't work.


enet_search_cv

Hello,
This issue will be long. There are 2 problems.
First :
This function don't run if the vdataframe have been modified. The problem is "shema_relation" function in bayesian_search_cv function.

so when I run :
enet_search_cv(input_relation = titanic,                 X = ["age", "fare", "pclass",],                 y = "survived",                 metric = "auto",                cv = 3,                 estimator_type = "enet",                cutoff = -1,                cursor=cur,                print_info = True,)
I have :

If the vdataframe haven't been modified :

it's ok.
Second problem :
When enet_search_cv use bayesian_search_cv, the prameter "prams_gs" is empty.
To show the problem rapidly, when we are :

and we use :

I think it's because we haven't access to public shema :

So, when we create "estimator" variable in enet_search_cv, we must give a shema parameter to solve it.
Regards,
Alexandre

Thanks a lot Alexandre.
This problem will be solved in the next PR thanks to the "set_option" function.


node stream

old issue but... To be less competing with node vs whatwg streams and being browser/deno compatible then i think (async) iterators would be a more viable options.

We've switched to using cursors instead of streams directly in 4.0, so we can close this issue.


CM-79 Enable https in user module

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/25/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/26/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/27/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/29/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/28/

test this please

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/30/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/31/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/33/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/34/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/35/

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/36/

test this please

Refer to this link for build results (access rights to CI server needed):
http://utils.brazilsouth.cloudapp.azure.com:8080/job/ci/job/continuous-integration-user/37/


improvement

Allow to chain commands
Allow to call the sh instead of sh.command

Thanks for the pull request, I'm likely to merge it soon.
First, I like the idea to make sh() a function via __call metatable method.
Chaining is a cool feature, too, as it resembles normal shell pipelines.
Can you please explain when in line 72 self does not equal to the last command result table? Is there any need in that condition check?
Other that this, I see some lines are added being commented out, so I either merge this and remove them, or you might want to remove them and update the pull request.

Fixed ;-)


added new resource mergeomics

This is a test PR to add mergeomics as a resource

This PR is failing because the test requires https and this is a http link


Create binary release

Create a binary release for Windows and Ubuntu

Will have to wait until I have a more stable release, but will be done in the near future.

I just created binary releases for both Win10 and Ubuntu 18/20 and shared them. However, as I felt that the platform is still too unstable, I have only given it the version tag v0.1.0. Probably, in the near future, I can say that I am satisfied enough with the platform to give it the oh-so-holy v1.0.0, but that's for future André to enjoy.


Support ESLint 8.x

ESLint v8.0.0 is released 🎉
It would be awesome to have official ESLint 8 support. 👊
I'm happy to help where I can of course 🙂

Related issue: https://github.com/standard/standard-engine/issues/234
Is there is anything to change for this eslint configuration to support eslint@8 ?

ESLint v8.0.0 is officially released

@Divlo I created #41 to make sure this plugin is working against ESLint 8


Documentation please: Exclude category

Please provide us with the following information:

This issue is for a: (mark with an x)
- [ ] bug report -> please search issues before submitting
- [ ] feature request
- [x] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

NA

Any log messages given by the failure

NA

Expected/desired behavior

To exclude a set/category of sources from search

OS and Version?

Windows 10.

azd version?

azd version 1.1.0

Versions

NA

Mention any other details that might be useful
I would love to use the "Exclude category" option but can't find it mentioned in the documentation. My data includes PDF and HTML files, so let's say I want to ask a question and tell the chatbot to search in PDFs only (or files with a specific predefined category), how can I use the "Exclude category" functionality?
Your time and efforts to help are highly appreciated.
regards
Mo

Thanks! We'll be in touch soon.

I use something like this.
filter = "category ne '{}'".format(exclude_category.replace("'", "''")) if exclude_category else None
example
"category ne 'Configuring_the_Server' and category ne 'Features'  and category ne 'Casino_Games'  and category ne 'Administration_Panel'  and category ne 'Affiliate_program'"

Thank you very much @krumkrum

I've gone into the Azure portal, to blob storage and added index tags to my files, as M$ docs said categorys will be generated by index tags, can we use those tags to exclude? I see in the gtpkb fields category, sourcepage, and sourcefile are filterable, so in theory we should be able to filter by category, sourcepage or sourcefile right? It hasn't seemed to work for me yet, but i'm probably missing something. Would love a bit more clarification on how it works.

The ingestion guide discusses categories now, and there's an Include category feature, so I think we can close this:
https://github.com/Azure-Samples/azure-search-openai-demo/blob/main/docs/data_ingestion.md#enhancing-search-functionality-with-data-categorization


in operator doesn't work correctly with var

I believe this is a bug, but please tell me if it's just my expectation that needs to be corrected:
irb(main):028:0> data = {"code": "45736123"}
=> {:code=>"45736123"}
irb(main):029:0> rule = {"in": ["123",{"var": "code"}]}
=> {:in=>["123", {:var=>"code"}]}
irb(main):030:0> JSONLogic.apply(rule,data)
=> false

Works fine when hardcoded:
irb(main):031:0> rule = {"in": ["123","45736123"]}
=> {:in=>["123", "45736123"]}
irb(main):032:0> JSONLogic.apply(rule,data)
=> true
irb(main):033:0> JSONLogic.apply(rule,nil)
=> true

bump

I am happy to take a look into what would be needed to fix this, but before I do I'd like to know that a reasonable pull request will be accepted. Please let me know.

bump

Hey just got this

Yeah I need to adjust ntoification settings on this repo

@l4cr0ss Thanks for contacting us on this. It does indeed look to be a bug. As far as submitting a PR goes, you just need to fix the code and introduce a test to make sure it stays fixed. Nothing fancy here, this is not a highly used repository.

Has there been any update to this? @bhgames @l4cr0ss

Not since I last replied to him. I would be happy to accept a PR from you @mjuszczak, if you're interested!

@bhgames We wrote a test and @eropple from our team satisfied it. Can you review the PR?

Reviewing.

Soon to be resolved by https://github.com/bhgames/json-logic-ruby/pull/13


Bug report: Invalid bytes values when sending QueryMsg

Prerequisites

[X] I am running the latest CosmPy version.
[X] I checked the documentation and found no answer to my problem.
[X] I checked the existing issues to make sure my problem has not already been reported.
[X] I have read the code of conduct before creating this issue.

Expected Behavior
I'm sending a signed message to a smart contract and checking if the signer matches a recorded public key.
I expect the query to return true.
Current Behavior
It returns false when using cosmpy, but is true when using FetchStation.
The signer is also a match in Rust, so I'm guessing the issue is on the Cosmpy side when encoding the QueryMsg
To Reproduce
Contract is deployed on Dorado at address "fetch1c5ltwskxhua9v78d5gutcgptsst237k3skn87xgx8p79d5a3gkcs9t5rs0"
Send the following query:
{"check_signed_payload": {
	"message": [84, 104, 105, 115, 32, 105, 115, 32, 97, 32, 116, 101, 115, 116, 32, 111, 102, 32, 116, 104, 101, 32, 116, 115, 117, 110, 97, 109, 105, 32, 97, 108, 101, 114, 116, 32, 115, 121,
115, 116, 101, 109, 46],
"payload": [231, 229, 98, 94, 71, 205, 242, 111, 155, 109, 14, 207, 120, 39, 229, 213, 10, 11, 5, 42, 252, 34, 0, 150, 69, 223, 242, 178, 108, 142, 162, 255, 84, 122, 75, 147, 213, 46, 131, 189, 32, 114, 238, 230, 241, 57, 101, 20, 220, 95, 255, 86, 116, 15, 123, 143, 211, 51, 206, 255, 155, 185, 151, 1]
}}

And on FetchStation: https://fetchstation.azoyalabs.com/dorado/explorer/contracts/fetch1jvsc0cpvfn998mt8mwj892gd7w8qy07ejg09wcmfmzlwe6raznnqgfzcnh
Context
OS: Windows 10
Python: 3.10.7
Cosmpy: 0.5.1
Failure Logs
No response

Actually disregard this issue I got mixed up between contract deployments, my bad!


Benchmark against native Go http.HandleFunc

Hi. Please include the native Go http.HandleFunc in your benchmarks. I know it lacks a lot of features but it would be cool to see how httprouter compares against it. Thank you

Wrong repository. Here is the correct issue: https://github.com/julienschmidt/go-http-routing-benchmark/issues/76


Link to GUI .bin editor in README as additional tool.

I had recently worked on a GUI tool to edit .bin file. So I thought it might be helpful to link here.

Approved.
Thanks for creating a great tool.

Approved.
Thanks for creating a great tool.

Thanks for the approval.
And, since you have way more knowledge in this stuff, I was hoping maybe you could go through the code (it's messy) and validate it ( I might have been wrong somewhere while parsing the file. )

Thanks for the approval. And, since you have way more knowledge in this stuff, I was hoping maybe you could go through the code (it's messy) and validate it if it's ok with you. ( I might have been wrong somewhere while parsing the file. )

Looking at the source code, I think the interpretation of the pmb.bin file is correct.
I also tried the mod using GT3PMBBINEditor and the images were displayed correctly and worked without any problems, so I think this will work.

Cool!
Also, do you have any knowledge about the .inf music files? Me and my friend are working on it and we have mostly figured it out except one piece of data. So, maybe you can help if you know about it?

Sorry, I have no idea about the structure of .inf files, and if I were to do it, I would want to analyze mbl and pmb, so I won't.