Consider adding extraction of roman-numberated and arabic-numberated pages

qpdf extract pages by absolute page number.
But a PDF-Document might have roman-numerated (in the frontmatter) and arabic-numerated pages.
Example:
The file srcfile.pdf has 3 roman-numerated pages (i, ii, iii) and 10 arabic-numerated pages (1, 2, 3, ..., 10).
The following command extracts pages i, ii, iii, 1, 2,  , 7
qpdf --empty --pages srcfile.pdf 1-10 -- targetfile.pdf

Is it possible to extract only arabic-numerated pages 1 to 10?
This is possibly related to issue #939

This will fit into the pages work I'm planning on doing.

These are very good news.
FYI: I had a question posted in SO

I think I ran into this when I had a few recent .pdf files. Arabic numbers are often at the beginning, right? Kind of like a sub-index or something. I find this very confusing; I'd wish page numbers would only be 1, 2, 3 and so forth, but I understand the rationale given here.

Well, one of the things I'm trying to formalize is lots of different ways to refer to pages. People have asked for it by page label (this kind of user-supplied page number), also based on article threads, outlines, or other factors. I'm going go through all my notes and all the issues and try to come up with some framework. I don't know what it's going to look like. I'm concerned about adding even more complexity to qpdf's CLI, which is already very complex.

Maybe this can help to get some ideas (if not already known):
PDF 1.7 - Page Labels
EXAMPLE The following example shows a document with pages labeled i, ii, iii, iv, 1, 2, 3, A-8, A-9, …
1 0 obj
    << /Type /Catalog
        /PageLabels << /Nums [ 0 << /S /r >> % A number tree containing
                               4 << /S /D >> % three page label dictionaries
                               7 << /S /D
                                    /P ( A- )
                                    /St 8
                                   >>
                             ]
                    >>
        …
    >>
endobj

And how other libraries manage page labels. For instance cpdf (Coherent PDF) - Page labels or pagelabels-py.

@me-kell Thanks. qpdf actually has pretty good support for page labels in the code and at the API level, just not so much on the CLI. The problem is that page labels can repeat and be ambiguous, especially when files are split and merged. If someone says they want pages i through iii but there is more than one set of pages numbered that way in a document because of concatenation of some other purpose, then it gets tricky. I have some notes about this since it can occur in other settings as well. For example, if you duplicate a page that an outline points to, which of the duplicate pages should the outline point to? These are just some of the issues I'm thinking about. I think most likely I will have qpdf default to simple behavior and pick the first match when there is ambiguity but provide some mechanism to be more specific and to detect matches, perhaps with a warning. When I get a little further along in my planning, I'll create a public discussion item to talk about the edge cases. Granted, these edge cases will not affect most users, so my goal will be to make the normal cases pretty simple.
One thing that I am likely to do is to provide some kind of JSON syntax that can be used to specify an unambiguous specification. I'm still working that out, and I'm still undecided as to what it should look like. I've considered other syntaxes as well (for the advanced/ambiguous case), but I'm reluctant to add yet another way to specify things.

Just an idea: how about passing the number of the label prepended to the page number?
For getting from "page 1 of pagelevel 3" to "page 2 of page level 4" one could pass something like [3:1]-[4:2] (see the page labels below).
{'/Nums': ['0', {'/S': '/D'},   # 1st pagelabel from page 0
           '1', {'/S': '/R'},   # 2nd pagelabel from page 1
           '5', {'/S': '/r'},   # 3rd pagelabel from page 5
           '9', {'/S': '/D'},   # 4th pagelabel from page 9
           '13', {'/S': '/a'},  # 5th pagelabel from page 13
           '17', {'/S': '/A'},  # 6th pagelabel from page 17
           '21', {'/S': '/r'}]} # 7th pagelabel from page 21

There is though one problem. Some documents might have different numberings without any pagelabels at all.

Good thoughts to keep in mind. Also note that you can get a dump of the page label information from a PDF file with qpdf using qpdf --json --json-key=pagelabels file.pdf. It's possible that qpdf could generate information to disambiguate that a caller could use, though it would be nice to have something that can be done without interaction for scripting. I think it can work though.

@jberkenbilt I didn't yet know the json options. Thanks.
With this option there is a workaround using a json parser like e.g. jq:
$ INPUTFILE=inpufile.pdf
$ PAGELABELSLENGHT=$(qpdf --json ${INPUTFILE} | jq -r '.pagelabels | length')
$ echo "file '${INPUTFILE}' has ${PAGELABELSLENGHT} pagelabels"
$ for i in $(seq 0 $((${PAGELABELSLENGHT}-1)))
    do echo "pagelevel #$i starts at index #$(qpdf --json ${INPUTFILE} \
       | jq -r .pagelabels[$i].index)";
    done

file 'inpufile.pdf' has 2 pagelabels
pagelevel #0 starts at index #0
pagelevel #1 starts at index #4

And with a little bash script pages in a pagelabel can be converted to absolute pages:
cat <<'EOF' | tee relpage.sh
#!/bin/bash
PDFFILE=$1
STR=$2
PAGELABENR=${STR%:*}
PAGENRINLABEL=${STR#*:}
PAGELABELINDEX=$(qpdf --json ${PDFFILE} | jq -r .pagelabels[$PAGELABENR].index)
ABSPAGENR=$(($PAGELABELINDEX+$PAGENRINLABEL))
echo $ABSPAGENR
EOF
chmod +x relpage.sh

Usage: ./relpage.sh inputfile.pdf 1:39
qpdf \
    --empty \
    --pages \
    ${INPUTFILE} \
    $(./relpage.sh ${INPUTFILE} 1:17)-$(./relpage.sh ${INPUTFILE} 1:39) \
    -- \
    output.pdf

@me-kell Yes, thanks for posting that. You're right -- this does provide a way to do it for now.

I think there's nothing left to do here, so I'm closing. Please reopen if that is not the case.


feat(enc): let user specify number of channels

First of all, thanks for making bindings to fdk-aac.
I'd like to be able to encode single-channel audio so I added an enum to the encoder config. My VS Code auto-formats everything so I broke this into two commits. The real change is here: b3eb13b10a654ed162de91190a546212dd9c832b

Thanks!! This looks great. If you can rebase your branch to retain only b3eb13b I'm happy to merge :)

Whoops, I did some repo cleaning including renaming the master branch to main and it looks like it made a bit of a mess on this branch.
I've cherry picked your commit as https://github.com/haileys/fdk-aac-rs/commit/f13a0f2b755fd7b65967db9e1c3a2286dd0a3316, which is now on main. I'll tag this up and release it now for you.

Released as 0.5.0 https://crates.io/crates/fdk-aac/0.5.0


Search & Replace is behaving strange for readonly files

We support readonly file system providers and indicate this typically with a lock icon and a readonly editor. When you do a search & replace operation from the view over these files, the edits are still applied, even though the underlying model is not writeable.
When you close and reopen the editor, these changes are gone.
Steps:

open a workspace and take note of the path
do a search & replace over a subfolder
it will work fine
edit the isReadonly condition of text models to return true when the resource is in the subfolder
repeat the steps after restart
you should see a lock icon when opening files of the sub folder
the edits are applied but lost when you close and reopen the editor

I ran this test using the file-readonly branch from PR #161716;
(makes it easy to "edit the isReadonly condition of text models to return true")
With the Editors marked as isReadonly:
-- the S&R asks confirmation then modifies the buffer/display contents
-- it  appears to have written the contents: the 'dirty/modified' flag is cleared
-- but the contents are not written to the file (even with explicit 'File: Save')
After changing the editor/buffer to 'editable' (clears the lock icon), panel still shows 'unmodified'
but now 'File: Save' (Cmd-S) writes the contents to the file.
Two issues for S&R to address:

Modifying a non-editable buffer (although it does ask confirmation/permission)
Clearing the 'modified' indicator, even though the write to file has not happened

Note:

"We support readonly file system providers..."

I did the test using diskFileSystemProvider, which does not advertise FileSystemProviderCapabilities.Readonly
IDK what happens with an actual Readonly provider.

There is 2 different readonly states:

configured by file system or setting
permissions on the OS

I think above is for permissions on the OS where its possible to overwrite these permissions. This would not appear for the former.


The again button in /px isn't working

The command works fine but pressing the "Again!" button shows an error message

n fact, all buttons in commands generate error messages


Add aria-describedby to elements with hints

This adds an aria-describedby to elements that have hints. In some cases the hints describe a group of elements (such as, for example a bunch of radio buttons), in this case, the aria-describedby element is added to the enclosing fieldset, which will also have a role of group (which is a pattern I've seen elsewhere, but hasn't until now, been consistent)

 Add aria-describedby to input elements with hints


Link title of a theme to the official marketplace site

At
https://github.com/builders-club/onlythemes/blob/94e0ab11e0861a14cf0d4f391d2c4be766b88be9/extension/src/onlyThemesView.ts#L133
we could probably link to the theme's official marketplace page. Or at least putting the direct link to the marketplace somewhere in the web view to reduce the amount of manual copy & pasting theme name around.
What do you think?

@all-contributors please add @andi1984 for ideas

Good suggestion.
There's now a 🔗 icon that links the user to the extensions page on the marketplace.


Update dependencies and workflows

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (d21f180) 100.00% compared to head (230e883) 100.00%.

Additional details and impacted files
@@            Coverage Diff            @@
##              main      #299   +/-   ##
=========================================
  Coverage   100.00%   100.00%
=========================================
  Files           94        94
  Lines         3197      3197
  Branches       249       241    -8
=========================================
  Hits          3197      3197

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Something went wrong, We could not fetch your achievements

Restart the app
Click on 3 lines button on the bottom-right
Select the "user name"
Go to leaderboards tab
Click on any user.
Sometimes it works, but most often the spinner spins for a few minutes, then a network error appears: Something went wrong, We could not fetch your achievements

Related: #2691

I am not able te replicate this issue , I tried 20 times and its working perfectly

I am able to reproduce right now (at the first try) on a very fast WiFi network (when posting the issue above I was on fast 4G).

@nicolas-raoul I also face such problem. But isn't this issue of backend?

Yes, possibly a backend problem. The backend's source code is at https://github.com/commons-app/commonsmisc

@nicolas-raoul on further debugging issues i found that three are probably 2 databases in the backend 1 for beta users/developers and 1 for normal users.  when we log in from development version our data is probably stored in a separate database. to prove this I tried logging in from the normal version (playstore version) with beta version credentials and I can't log in. In fact, in the forgot password screen I get an error that the account does not exist.  the same issue Is in the web version. I Can't say for sure without taking a look at the database structure.  And I checked the backend link you sent to me but it seems to have only a few scripts. I couldn't find the SQL schema file in that repo. It would be helpful if you can redirect me to database structure file. I will keep updated if i find anything else. Thanks

The database for achievements is probably only for prod.
Here is some more documentation:

https://github.com/commons-app/commons-app-documentation/blob/master/DB_Schema/leaderboard.md
https://github.com/commons-app/commons-app-documentation/tree/master/API

Working well recently 🙂


Sublime text 3 not support rails 2.2.2

Try running the debugger in three sublime text on mac.
The result: Stopping...
Ruby version: 2.2.2 is not supported.

I'm getting this:
Ruby version: 2.2.3 is not supported.


nixos/test-driver: add polling_condition

Motivation for this change
Allows tests to fail early when certain conditions are met. Example with vscodium test provided.
Things done

Built on platform(s)

[ ] x86_64-linux
[ ] aarch64-linux
[ ] x86_64-darwin
[ ] aarch64-darwin

[ ] For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test
made sure NixOS tests are linked to the relevant packages

[ ] Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
[ ] Tested basic functionality of all binary files (usually in ./result/bin/)
21.11 Release Notes (or backporting 21.05 Release notes)

[ ] (Package updates) Added a release notes entry if the change is major or breaking
[ ] (Module updates) Added a release notes entry if the change is significant
[ ] (Module addition) Added a release notes entry if adding a new NixOS module
[ ] (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes

[ ] Fits CONTRIBUTING.md.

With this rewrite I'm a lot more confident that this is actually a good idea. What should still happen before this can be merged:

[ ] Documentation
[ ] Rewrite the context manager logic so that it can be entered multiple times as follows:@polling_condition
def foo():
    ...

with foo:
      ...

...

with foo:
      ...

Currently this throws a "weird" error, but it's conceptually perfectly valid. Thus, it should Just Work™

(would really like to merge this to avoid future merge conflicts)

(would really like to merge this to avoid future merge conflicts)

@ofborg test vscodium.wayland

@ofborg test vscodium.wayland

@ofborg test vscodium.wayland
(since I force-pushed this didn't happen I think)

@ofborg test vscodium.wayland vscodium.xorg

Any further objections? Wanna merge this

thank you very much for your patience. the code looks good.
as it makes the test driver more complicated with another public interface, i would feel much better having a test that tries to test the polling condition behavior for correctness. can you write one for that, too? we can merge this pull request before and then have another pr for the test to relieve you from the burden of potential rebasing.

A test suite (both unit tests and integration texts) to assure that the
test driver behaves as expected would be nice to have anyways. Definitely
willing to make a start with that
On Sat, 8 Jan 2022, 13:14 Jacek Galowicz, @.***> wrote:

thank you very much for your patience. the code looks good.
as it makes the test driver more complicated with another public
interface, i would feel much better having a test that tries to test the
polling condition behavior for correctness. can you write one for that,
too? we can merge this pull request before and then have another pr for the
test to relieve you from the burden of potential rebasing.
—
Reply to this email directly, view it on GitHub
https://github.com/NixOS/nixpkgs/pull/146905#issuecomment-1007972037,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABV7PJ7RAIR7HYFVQHCY2HDUVATARANCNFSM5IPL2RHA
.
Triage notifications on the go with GitHub Mobile for iOS
https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675
or Android
https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.
You are receiving this because you authored the thread.Message ID:
@.***>


Dynamic form content

This doesn't appear to be working correctly for fields that are added to the page dynamically after load.
I don't see why or how this could be, but the only inputs which aren't working are added in this way.

I searched through the code (script.js) and it seems the extension looks for <input fields etc only on initial load. I guess the function "init_virtualKeyboardChromeExtension" should be loaded every time the DOM changes.
My application needs this case too (in the future) - maybe I'll improve the extension in a few weeks.
All the best
Georg

It does call on interval though, so I am thinking it may actually work but I was affected by another problem with how this initializes and marks inputs.
I have been meaning to return after I confirm, and test a bit more. Both of my issues may be related.
What happened to me was the dynamic form content was generated by cloning a hidden element. That element was still seen by virtual keyboard though and initialized with event listeners and an attribute. That attribute was cloned but the event listener wasn't so it created problems.
As far as I can tell I just need to remove the attribute or deep clone. Cloning all the event listeners may be a bad choice so I have had some success with this on my clone:
.removeAttr('_vkenabled')


How to predict and evaluate on a self-made test set

:question: Question

Hello, I have my own test set and need to predict and evaluate it. When I evaluate it, the following error occurs. How do I need to preprocess the labels?

The labels need to be placed into "raw_splitted/labelsTs" and the preprocessing needs to be rerun.
The code in scripts/preprocess.py can also be modified to only run the preprocessing on the labels. A more convenient solution will be provided in our next release.

The labels need to be placed into "raw_splitted/labelsTs" and the preprocessing needs to be rerun. The code in scripts/preprocess.py can also be modified to only run the preprocessing on the labels. A more convenient solution will be provided in our next release.

Ok, I have put the data and labels of the test set under “raw_splitted/”, how do I modify scripts/preprocess.py to preprocess the labels separately?

Do you process the label of the test by entering if cfg["data"]["test_labels"]: this condition? Or directly change test=False to test =True，as follows?

It is not that straight forward, a potential function to achieve this would be this one:
@env_guard
def main_prep_labels():
    """
    Prepare (non preprocessed) labels
    """
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "tasks",
        type=str,
        nargs="+",
        help="Single or multiple task identifiers to process consecutively",
    )
    parser.add_argument(
        "-np",
        "--num_processes",
        type=int,
        default=4,
        required=False,
        help="Number of processes to use for croppping.",
    )

    args = parser.parse_args()
    tasks = args.tasks
    num_processes = args.num_processes

    for task in tasks:
        task_path = get_task(task)
        create_labels(
            source_dir=task_path / "raw_splitted",
            preprocessed_output_dir=task_path / "preprocessed",
            num_processes=num_processes,
        )

Here is a code snipped to preprocess the labels:
@env_guard
def main_prep_labels():
    """
    Prepare (non preprocessed) labels
    """
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "tasks",
        type=str,
        nargs="+",
        help="Single or multiple task identifiers to process consecutively",
    )
    parser.add_argument(
        "-np",
        "--num_processes",
        type=int,
        default=4,
        required=False,
        help="Number of processes to use for croppping.",
    )

    args = parser.parse_args()
    tasks = args.tasks
    num_processes = args.num_processes

    for task in tasks:
        task_path = get_task(task)
        create_labels(
            source_dir=task_path / "raw_splitted",
            preprocessed_output_dir=task_path / "preprocessed",
            num_processes=num_processes,
        )

Ok, thank you very much, I have successfully predicted my self-made test set, and the prediction results have many _boxes.pkl files under “test_predictions/”, what are these files? Is it possible to use these files to visualize the detection  results?

Hi,
you can find information about the box format here: https://github.com/MIC-DKFZ/nnDetection/issues/156
and visualisation here: https://github.com/MIC-DKFZ/nnDetection/issues/16
Best,
Michael

Hi,
you can find information about the box format here: #156 and visualisation here: #16
Best, Michael

Hi Michael，
Thank you for your patience in answering my question. Okay, I understand, I have successfully visualized my detection results and generated the corresponding json file. I read issues 156, and I still want to confirm with you that my understanding is correct. "score": represents the object contained in the box Confidence probability, "box": the order from top to bottom is x_min, y_min, x_max, y_max, z_min, z_max, is my understanding correct?

Yes, it is correct :) x,y,z refer to the array loaded via SimpleITK.

Yes, it is correct :) x,y,z refer to the array loaded via SimpleITK.

Ok, I would also like to ask about the detection results are now displayed through the box as the picture one, whether there is a seg display?as the picture two?

nnDetection is currently only used for bounding box detection and does not include Instance Segmentation (which would be second image).

nnDetection is currently only used for bounding box detection and does not include Instance Segmentation (which would be second image).
Hi Michael，
Ok, next, I will analyze the detected results carefully, thank you for your patience in helping me solve the problem！

Hi,
you can find information about the box format here: #156 and visualisation here: #16
Best, Michael

Hi Michael
I have a few questions when analyzing the test results

Which script does nndet_boxes2nii run?
Does the --threshold specified by nndet_boxes2nii refer to the score?
When using the nndet_boxes2nii command, is the IoU of the detection box and the real box 0.1 by default?
If I want to calculate the false positive rate of the self-made test set when the score is 0.3, 0.4, 0.5, 0.6, which analysis file should I use to get it?

Hi,

the script executes scripts/utils.py/boxes2nii
The threshold specifies the score from which the boxes should be shown
boxes2nii does not use any IoU thresholds, all predictions which lie above the score threshold are shown (except predictions which might overlap, e.g. if one box fully encloses another one only the bigger one will be visible if it has an higher score
nnDetection currently does not include False Positive Rate for specific score thresholds. You could draw inspiration from the FROC implementation: https://github.com/MIC-DKFZ/nnDetection/blob/main/nndet/evaluator/detection/froc.py

Do not use the analysis file for any kind of quantitative evaluation! It uses a simplified algorithm for matching and is not tested.
Hi Michael
Humm,  I just read the .pkl file under /test_predictions/ and found that this file contains items such as "pred_boxes" and "pred_scores". I can use nndet_boxes2nii --threshold to filter the detection boxes with scores greater than the threshold, right? If I need to get the number of TP, FP, FN, TN under different scores (score>0.3, 0.4, 0.5, 0.6), how do I calculate it?I have obtained this information from the .pkl file under /test_predictions/:

Dear @MOMOANNIE ,
you can check the scripts for the FROC and the Histogram computation in the evaluation folder of nnDetection for some initial clues. We are currently not providing utilities to compute FP, FP, FN separately (TN do not exist in detection).
Best,
Michael

Dear @MOMOANNIE ,
you can check the scripts for the FROC and the Histogram computation in the evaluation folder of nnDetection for some initial clues. We are currently not providing utilities to compute FP, FP, FN separately (TN do not exist in detection).
Best, Michael

ok, thank you very much, I will close the issue


BLT Crash

Fri Sep 15 17:12:45 2017
Application has crashed: access violation

Callstack:
     payday2_win32_release  (???)     ???
     payday2_win32_release  (???)     ???
     payday2_win32_release  (???)     ???
     payday2_win32_release  (???)     ???
                       ???  (???)     ???

Current thread: Main
Script stack:
load_packages()  lib/setups/menusetup.lua:61
core/lib/setups/coresetup.lua:303

System information:
Application version : 1.76.327.
CPU : Intel(R) Celeron(R) CPU G540 @ 2.50GHz (1 core); SSE; SSE2; SSE3; SSSE3; SSE4.1; SSE4.2
DirectX : 12.0
GPU : Intel(R) HD Graphics / ddumd32.dll[<IP_ADDRESS>05]
Language : @ID94cca2fdaccfc2cd@
Memory :     1745MB 80KB
OS : 6.2.9200 () 0x100-0x1 (64 bits)
Physics : threaded
Renderer : DX9 threaded
Sound : Microsoft (Speakers (High Definition Audio Device))

i even tried alot of times to start it and reinstalled blt :(


Optimize TTData ramping

I noticed there is useless cpu consuming because we stop the Ramp unit even if there is no ramping.
As there is a RampStatus flag I guess we could optimize this to save more cpu.

@theod : Once I have pushed the changes to implement dataspace + ramping, maybe you can check again to see if this is still an issue?

On feature/325-dataspace-ramp branch, for decimal type it seems you test the ramp status flag before to stop the ramp so I would say ok ! thanks !
I keep the issue opened until you do the same for integer and array type.

They all share the same method now, so I'm closing this issue,


move dirty_attributes feature to separated helper module

@Fivell convince me that dirty attributes feature should be optional so i've moved it to it's own helper.
so we can add dirty tracking like this:
class BaseResourceWithDirty < BaseResource
  include JsonApiClient::Helpers::DirtyAttributes # this row will turn on dirty tracking for attributes
  self.relationship_linker = JsonApiClient::Relationships::RelationsWithDirty # this row will turn on dirty tracking for relationships
end

@chingor13 what do you think about it?

@senid231 , if you're talking about this comment https://github.com/chingor13/json_api_client/pull/96#issuecomment-124829810. I mean that if there is possibility to use ActiveModel::Dirty it's better to use it, (like you did with AM::Callbacks)

ActiveModel::Dirty use ActiveModel::AttributeMethods but it want all attributes to be statically defined. At now all attributes (except id and type) are defined dynamically.
i think we have to choices:

simple one (to copy this feature to json_pi_client, as i did)
to define attributes in class definition and use ActiveModel models (it's related with #45 where @bvandenbos propose to include ActiveModel to JsonApiClient::Resource ).
it means that we need to rewrite instance initialiazation and add attributes definitions

@chingor13 do you have thoughts about adding ActiveModel to JsonApiClient::Resource ?

checks failed cause in earlier version there is no clear_changes_information method and @changed_attributes is a Hash (in current version it's HashWithIndifferentAccess).
should i fix JsonApiClient::Helpers::DitryAttributes to have same behaviour for all versions of ActiveModel (it matches 1-to-1 with ActiveSupport version) or rewrite travis to test only lastest versions ?


Note that repo is outdated

Summary of the changes / Why this is an improvement
Checklist

[ ] Link to issue this PR refers to (if applicable):
[x] CLA is signed

Of course, thanks!


Add bevy-components feature

After https://github.com/bevyengine/bevy/pull/2254 there is a requirement that anything used as a component must derive Component explicitly.
From what I can tell, most of the data structures that are used as components in bevy_rapier are defined in rapier, but SharedShape is re-exported from parry, so the annotation has to go here.
I attempted to minimize the effect on parry by:

putting it behind a feature flag
minimizing the number of lines of code referring to bevy in parry
turning off default features in bevy to reduce the effect on compile time

One thing I'm unsure of is specifying the version of bevy. I've included it as 0.5 here, because that's what's on crates.io right now, but this code won't actually work with that version, only with a particular branch of bevy in this PR (which will hopefully land soon):
https://github.com/bevyengine/bevy/pull/2305

I don't think a feature like this should belong in the main branch. It's only a temporary fix until we get better syntax to do it properly from only the interop crate.

@MiniaczQ I'm not sure there is much of an alternative to this currently (maybe doing just wrapper types in the interop crate?). I don't think we are getting better syntax for doing this anytime soon based on https://github.com/rust-lang/rfcs/pull/2393.

@MiniaczQ I'm not sure there is much of an alternative to this currently (maybe doing just wrapper types in the interop crate?). I don't think we are getting better syntax for doing this anytime soon based on looking at https://github.com/rust-lang/rfcs/pull/2393.

It sounds like wrappers in the interop crate is the way to go, but that requires the most knowledge of the innards. This change was relatively easy to make as an outsider without changing how bevy_rapier works too internally.

Thank you for this PR @deontologician! I’m closing it since I ended up using wrapper types instead: https://github.com/dimforge/bevy_rapier/pull/102.


Add window event

Why needed these changes?

Model should be reset on window click

Have you tested these changes?

Yes

Any updates require?

No

What about the save modal?

Settings, about and save modal. Did I miss something?

What about the save modal?

Settings, about and save modal. Did I miss something?

Yes we've to add model reset for save also, I'll update it

Don't we do the same thing here:
const AboutOnClick = ({
    target
}) => {
    if (target === About) {
        OpenCloseAbout();
    }
};

Don't we do the same thing here:
const AboutOnClick = ({
    target
}) => {
    if (target === About) {
        OpenCloseAbout();
    }
};

Window events how we can handle in this function call?

@kevinmmansour  Let me know if anything need to update


Update Xcode projects to 8.2.0

This prevents project re-validation on initial Xcode launch.

Fixes #8390 for Xcode 8.2.1.


Return value for eventDestroy

The eventDestroy method takes an array of event IDs to be deleted, but returns nothing. Because of this, it's not clear if the passed events are deleted (and if they are, which ones exactly).
This method may return useful data, but returns nothing. Can you please improve the implementation?

The sc-server returns a boolean value for this type of request. In this client method a small inaccuracy was made. The return type can be fixed, this will not affect compatibility with previous versions.


100,000 lines returns this error Win32

Environment data

Steps to reproduce or exception report
Customer has over 100,000 mailboxes and we get a count so I run $a = get-mailbox -resultsize unlimited and I get this error.
I then follow up with $a.count, but so my question is there a limit to the amount it can put or a better way?

Not a PSReadLine issue.

Izybkr,
PowerShell actually tells us to come here to open a new issue:
https://github.com/lzybkr/PSReadLine/issues/new
Exception:
System.IO.IOException: Illegal operation attempted on a registry key that has been marked for deletion.
at Microsoft.Win32.RegistryKey.Win32Error(Int32 errorCode, String str)
at Microsoft.Win32.RegistryKey.GetValueNames()
at System.Management.Automation.Utils.GetGroupPolicySetting(String groupPolicyBase, String settingName, RegistryKey[] preferenceOrder)
at System.Management.Automation.ScriptBlock.ScriptBlockLoggingExplicitlyDisabled()
at System.Management.Automation.ScriptBlock.LogScriptBlockCreation(ScriptBlock scriptBlock, Boolean force)
at System.Management.Automation.ScriptBlock.LogScriptBlockStart(ScriptBlock scriptBlock, Guid runspaceId)
at System.Management.Automation.DlrScriptCommandProcessor.DoBegin()
at System.Management.Automation.Internal.PipelineProcessor.Start(Boolean incomingStream)
at System.Management.Automation.Internal.PipelineProcessor.SynchronousExecuteEnumerate(Object input)
at System.Management.Automation.Runspaces.LocalPipeline.InvokeHelper()
at System.Management.Automation.Runspaces.LocalPipeline.InvokeThreadProc()
at System.Management.Automation.Runspaces.LocalPipeline.StartPipelineExecution()
at System.Management.Automation.Runspaces.PipelineBase.CoreInvoke(IEnumerable input, Boolean syncCall)
at System.Management.Automation.Runspaces.PipelineBase.Invoke(IEnumerable input)
at System.Management.Automation.PowerShell.Worker.ConstructPipelineAndDoWork(Runspace rs, Boolean performSyncInvoke)
at System.Management.Automation.PowerShell.CoreInvokeHelper[TInput,TOutput](PSDataCollection1 input, PSDataCollection1 output, PSInvocationSettings settings)
at System.Management.Automation.PowerShell.CoreInvoke[TInput,TOutput](PSDataCollection1 input, PSDataCollection1 output, PSInvocationSettings settings)
at System.Management.Automation.PowerShell.Invoke(IEnumerable input, PSInvocationSettings settings)
at Microsoft.PowerShell.PSConsoleReadLine.ReadKey()
at Microsoft.PowerShell.PSConsoleReadLine.InputLoop()
at Microsoft.PowerShell.PSConsoleReadLine.ReadLine(Runspace runspace, EngineIntrinsics engineIntrinsics)

I see. Based on that stack, this seems like a more general PowerShell issue that could affect any running script, not just PSReadLine.


Erro

INFO:root:IN : ''                                                                                INFO:root:OUT : ''                                                                                INFO:root:Hit ctrl-c to exit                                                                                MIDI IN Device name is incorrect. Please use listed device name.                                                                                MIDI OUT Device name is incorrect. Please use listed device name.

Can you describe your environment such as OS?
What command line did you put? It seems like there are no MIDI devices on your computer.

Eu uso o Windows 10. E ultilizo o loopMidi para criar as portas MIDI.
O cofigo que useri foi:
python.exe .\serialmidi.py --serial_name=COM4 --midi_in_name="mnin" --midi_out_name="mnout"
mnin e mnout foram os nomes que dei as portas no loop midi

I don't understand the language you use so the answer will be just a guess.
When you do
python.exe .\serialmidi.py --serial_name=COM4
And if "mmin" or "mnout" is not shown in the list, it means the MIDI loopback device is not recognized to the program.
I don't know why. Do you see those device names in your DAW?

Yes, in my DAW the devices are recognized normally

I'm not sure what the problem is, if the device is not listed when you do
python.exe .\serialmidi.py --serial_name=COM4
Then it means Python doesn't recognize any MIDI devices. It could be drive issue, or Python installation issue.

What would be the ideal version of python to run the code?

I opened the device manager and there are some midi cables that I already deleted. Maybe it's something on the computer. I'll try to use it on another one and come back to tell you if it worked

Test on another notebook and it worked normally, it's a problem with my notebook. But as I will be using it on the computer that is inside the pipe organ I am building, there is no problem, I believe that on his computer it will work normally.


Adding overloaded $ and $$ method that accepts By for Android and IosLocators

Feature Request
Currently if there is a need for dynamic XPath or there is a need to use By locator for android and ios app, then we have to create a wrapper method called click(By androidBy, By iosBy)
public static void click(By androidBy, By iosBy){
  $(getByBasedOnDriver()).shouldBe(visible).click();
}

private static By getByBasedOnDriver(){
  return androidBy if android and iosBy otherwise
}

public static void click(By androidBy, By iosBy, Condition condition, Duration duration){
  $(getByBasedOnDriver()).shouldBe(condition, duration).click();
}

The case gets even tougher when we have to also pass the Condition like visible, enabled and time to wait
Details
If we have a $ , element, $$, elements method overloaded - we can accept androidBy, iosBy and can automatically use the appropriate by based on the check instance of AndroidDriver. It will make life so easy for testers.
PS: What if we create SelenideAppiumElement class instead of reusing SelenideElement? @asolntsev @BorisOsipov

We choose an approach how we will handle such cases when we have to use different selectors\conditions for the same SelenideElement with different app driver under hood. Will we have build-in support or not?
The same question for https://github.com/selenide/selenide-appium/issues/10

@BorisOsipov @amuthansakthivel I added method $: https://github.com/selenide/selenide-appium/pull/131
Initially, this PR doesn't solve the problem with two selectors (Android and iOS selectors). It just extends classical SelenideElement with additional methods.


Bug: not possible to show only the categories from the config

It is possible to override the categories in the config.php, but its not possible to remove 1 of the 5 categories for example.
Example
We only use functionality, the rest we dont need and we dont want to show.
This line will need to be tuned:
 'categories' => kirby()->option('zephir.cookieconsent.categories'),

I'm not sure if this is the same issue but I'm having trouble hiding categories. I only want to show necessary and marketing but when adding this to my config file:
'measurement' => ['enabled' => false, 'readOnly' => false ],
'functionality' => ['enabled' => false, 'readOnly' => false ],
 'experience' => ['enabled' => false, 'readOnly' => false ],

Nothing changes on the website.

Same issue

To disable a category, you need to override the language files, as these are essentially the ones that define the categories. The categories section in the config is only for the defaults (enabled means the category is enabled by default).
It's not intuitive this way, but I'm basically just mirroring the functionality of the JS plugin. They mention that the translations are used to generate the modal content here: https://cookieconsent.orestbida.com/advanced/language-configuration.html#language-config.
I have updated the readme to more prominently mention how to disable a category.

Why not sync the categories with the translations? Their playground works that way. Basically, the translations are updated/included according to the selected categories.
I can make a pull request if you want.

@andrepimpao That sounds like a good idea! I basically took the functionality of the JS plugin and "mapped" it as best I could into a Kirby plugin. But it would be more intuitive if a category set to false was automatically excluded.
A PR would be very welcome 🙂

Done #7.

Release 3.2.0


Add h1 spider to ילין לפידות

#4

@nirgn975 Hi Nir,
I added basic spider for ילין לפידות,
but, for some reason, when I'm trying to response headers with xpath It's working in 'scrapy shell', but not in real. = 'response.xpath("//h1/text()")[0].extract())'
However 'response.body', works for real.
Do you know why?

I'll look at that, but first, re-roll from master. I think you checkout from a previous branch you had locally.
(There is too much changes for this PR, .travis.yml has a conflict, and GitHub shows code from previous PRs as new).

@nirgn975 I did checkout from master because i did pull before that and then checkout to branch 4, add Spider file and then commit and push.
If i open new local branch (etc 10), and I commit and push to origin 4, It's not visibele in that pull request.
Why?

@nirgn975 Now i trying to commit from new branches and it's not updating my master..

@daniellevi1
You can't push a new branch to this PR. Every PR is related to a branch, so This PR is for branch 4, if you create a new branch (eg 10) you'll need to create a new PR for him.
I don't quite sure whats up with your local branches, I can be more help if I was near you, or if you upload a log of the branches. Anyway, try to delete your local project, and clone again. Then checkout to branch 4 and the do git merge master (eg merge master into 4, you don't need to pull in master before because you clone and it should be update).
After you'll merge you'll see you have conflict in .travis.yml file, fix it, and then commit and push.

@nirgn975 I did that, firstly I had error in push to origin, so I overwrite the remote commit when I did 'push -f origin 4'.
This is the log for 'push origin 4', do you know what's wrong?:
git push origin 4
Username for 'https://github.com': daniellevi1
Password for<EMAIL_ADDRESS>To https://github.com/nirgn975/open_pension_scraper.git
! [rejected]        4 -> 4 (non-fast-forward)
error: failed to push some refs to 'https://github.com/nirgn975/open_pension_scraper.git'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Integrate the remote changes (e.g.
hint: 'git pull ...') before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.
daniel@daniel-virtual-machine://home/daniel/open_pension_scraper/open_pension_scraper$ git push -f origin 4
Username for 'https://github.com': daniellevi1
Password for<EMAIL_ADDRESS>Counting objects: 6, done.
Compressing objects: 100% (6/6), done.
Writing objects: 100% (6/6), 748 bytes | 0 bytes/s, done.
Total 6 (delta 2), reused 0 (delta 0)
remote: Resolving deltas: 100% (2/2), completed with 2 local objects.
To https://github.com/nirgn975/open_pension_scraper.git

22fce04...f7c2778 4 -> 4 (forced update)

@daniellevi1 It looks good now. There is only 1 file changed on this PR. But you still get an error on travis..

@nirgn975 Yes, I'm trying to understand what is the error

@daniellevi1
You delete line on .travis.yml file, revert them, the travis file is fine.
Your problem is on the file you created, travis just check that file and report. Notice the error travis write (on blank line).

@nirgn975 I don't really understand the corrections of Travis

So as I remember you had couple of blank lines in the end of the file, and this is an error.
Now travis say:

E302 expected 2 blank lines, found 0. You can see the arrow is at the start of the line, so he expect 2 blank lines before the class line.
E301 expected 1 blank line, fount 0. Again the arrow is at the start of the line, so he expect 1 blank line before the def` line.
W292 no newline at the end of the file. He expect a newline (blank newline) at the end of the file.

@daniellevi1 Travis works, great! I'll look at the code later..

@nirgn975 I'll just mention again that the order 'response.xpath("//h1/text()")[0].extract())' didn't work well for real. it worked in scrapy shell.

I run the crawler with the command: scrapy crawl h1 inside the scraper directory.

@daniellevi1 You don't need to save it to a file, we don't need it. It gonna be files that we download so there is no point in that. Try get the header like:
header = scrapy.Selector(response).xpath('//h1/text()')
print(header)

Just don't leave the print(..) and stuff when you finish the PR.

Also, re-roll (because GitHub says this branch is out of date).


Get playlist data from API

    public String getData(String token, String id) throws Exception {
        URL url = new URL("https://api.spotify.com/v1/playlists/" + id);

        HttpURLConnection connection = (HttpURLConnection) url.openConnection();

        connection.setDoOutput(true);
        connection.setRequestProperty("Accept-Charset", "UTF-8");
        connection.setRequestProperty("Authorization", "Bearer " + token);

        InputStream response = connection.getInputStream();
        String result = new BufferedReader(new InputStreamReader(response)) .lines().collect(Collectors.joining("\n"));

        return result;
    }


load pe cause error

Hello
I have some problem running :load pe command after build.
scala> :load pe
Loading pe...
import scala.language.postfixOps
locParam: String = null
location: String = /Users/kwangin/app
useAuthorization: Boolean = false
java.lang.NoSuchMethodError: scala.Product.$init$(Lscala/Product;)V
  at DataDirs.<init>(ctl.scala:131)
  ... 61 elided
<console>:17: error: not found: value dataDirs
                          dataDirs = dataDirs,
                                     ^
<console>:15: error: not found: value pe
val $ires1 = pe.mappingFile
             ^
<console>:13: error: not found: value pe
       pe.mappingFile = "mapping-pe.json";
       ^

Is there something I'm missing?

Hi there,
Was the packageCmwell task in sbt successful?
What Java version are you running? What OS?

@bryaakov thanks for response.

Yes, packageCmwell  built successfully.
Java8 / MacOS

Please let me know if you know something.

What Scala do you have in PATH?
Cm-Well needs the latest of today which is 2.12.6.

@e-orz thanks for help!
$ scala -version
Scala code runner version 2.11.8 -- Copyright 2002-2016, LAMP/EPFL

I'll update and try again.

Any time!
Please share if it worked and this issue can be closed.

@e-orz you're right! load pe passes after scala update.
But I got another problem:
$ pe.install
...
96.24%	at java.lang.Thread.dumpStack(Thread.java:1333) Failed to upload eli2 http://<IP_ADDRESS>:9000/meta/docs/examples/visualizer/permid.org/1-5000057859

	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
java.lang.Exception: Stack trace
	96.62%a.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	96.99%	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157) 97.37%
java.lang.Exception: Stack trace
	97.74%	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	98.50%la.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	98.12%la.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
	98.87%java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)   99.62%
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	99.25%	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
	100.0%
Info:   uploading basic userInfotons (if not exist)
	*** FAILED to upload built-in UserInfotons due to Tcp command [Connect(<IP_ADDRESS>:9000,None,List(),Some(10 seconds),true)] failed because of Connection refused ...Moving on!
Info:   updating version history
 Failed to upload eli2 http://<IP_ADDRESS>:9000/meta/logs/version-history/1532871950156
java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Info:   starting dc controller
Info: finished initializing cm-well

Could you find problem?

WS couldn't start.
go to ~/app/cm-well/app/ws/logs and look in application.log what happened.

@e-orz I found:
2018-07-29 13:37:25,413 application-akka.actor.default-dispatcher-3] INFO  akka.event.slf4j.Slf4jLogger         - Slf4jLogger started
2018-07-29 13:37:27,470 [main]                                       INFO  org.elasticsearch.plugins            - [Sleeper] loaded [], sites []
2018-07-29 13:37:29,191 [main]                                       INFO  cmwell.fts.FTSServiceNew             - starting es transport client [/<IP_ADDRESS>:9301]
2018-07-29 13:37:29,326 [main]                                       INFO  cmwell.fts.FTSServiceNew             - waiting for ES green status
2018-07-29 13:37:29,520 [main]                                       INFO  cmwell.fts.FTSServiceNew             - got green light from ES
2018-07-29 13:37:29,545 [main]                                       INFO  c.d.driver.core.GuavaCompatibility   - Detected Guava >= 19 in the classpath, using modern compatibility layer
2018-07-29 13:37:30,003 [main]                                       INFO  c.datastax.driver.core.ClockFactory  - Using native clock to generate timestamps.
2018-07-29 13:37:30,425 [main]                                       WARN  com.datastax.driver.core.NettyUtil   - Found Netty's native epoll transport, but not running on linux-based operating system. Using NIO instead.
2018-07-29 13:37:31,275 [main]                                       INFO  com.datastax.driver.core.Cluster     - Cannot connect with protocol version V4, trying with V3
2018-07-29 13:37:31,743 [main]                                       INFO  c.d.d.c.p.DCAwareRoundRobinPolicy    - Using data-center name 'DC1' for DCAwareRoundRobinPolicy (if this is incorrect, please provide the correct datacenter name with DCAwareRoundRobinPolicy constructor)
2018-07-29 13:37:31,762 [main]                                       INFO  com.datastax.driver.core.Cluster     - New Cassandra host /<IP_ADDRESS>:9042 added
2018-07-29 13:37:31,889 [main]                                       ERROR cmwell.driver.NativeDriver           - Keyspace 'data2' does not exist
com.datastax.driver.core.exceptions.InvalidQueryException: Keyspace 'data2' does not exist
	at com.datastax.driver.core.exceptions.InvalidQueryException.copy(InvalidQueryException.java:50)
	at com.datastax.driver.core.DriverThrowables.propagateCause(DriverThrowables.java:37)
	at com.datastax.driver.core.Cluster.connect(Cluster.java:282)
	at cmwell.driver.NativeDriver.$anonfun$session$1(NativeDriver.scala:67)
	at scala.util.Try$.apply(Try.scala:209)
	at cmwell.driver.NativeDriver.<init>(NativeDriver.scala:67)
	at cmwell.driver.Dao$.apply(NativeDriver.scala:95)
	at logic.CRUDServiceFS.irwService$lzycompute(CRUDServiceFS.scala:71)
	at logic.CRUDServiceFS.irwService(CRUDServiceFS.scala:71)
...

Is this project request Cassandra?

This project uses Cassandra. But it should be downloaded automatically and used by "pe".
You can look at Cassandra logs in ~/app/cm-well/log/cas.
Also, you can do pe.debug = true before doing the install and see if something went wrong.

@e-orz okay, I'll check and let you know.

@e-orz I found this:
Failed to upload eli2 http://<IP_ADDRESS>:9000/meta/docs/examples/visualizer/permid.org/1-5000057859	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)

	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	96.99%	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
 Failed to upload eli2 http://<IP_ADDRESS>:9000/meta/docs/_Images/health-md-icon.png
java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	97.37%	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
 Failed to upload eli2 http://<IP_ADDRESS>:9000/meta/docs/examples/iterate/iterate.sh	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)

	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
	97.74% Failed to upload eli2 http://<IP_ADDRESS>:9000/meta/docs/examples/visualizer/permid.org/1-34413413063
java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
	98.50%java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
	98.87%java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
	99.25%java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
	100.0%
Info:   uploading basic userInfotons (if not exist)
	*** FAILED to upload built-in UserInfotons due to Tcp command [Connect(<IP_ADDRESS>:9000,None,List(),Some(10 seconds),true)] failed because of Connection refused ...Moving on!
Info:   updating version history
 Failed to upload eli2 http://<IP_ADDRESS>:9000/meta/logs/version-history/1533042066129
java.lang.Exception: Stack trace
	at java.lang.Thread.dumpStack(Thread.java:1333)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:314)
	at DataInitializer$$anonfun$httpPost$5.applyOrElse(DataInitializer.scala:309)
	at scala.runtime.AbstractPartialFunction.apply(AbstractPartialFunction.scala:34)
	at scala.util.Failure.recover(Try.scala:230)
	at scala.concurrent.Future.$anonfun$recover$1(Future.scala:391)
	at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
	at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
	at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
	at java.util.concurrent.ForkJoinTask$RunnableExecuteAction.exec(ForkJoinTask.java:1402)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)
Info:   starting dc controller
Info: finished initializing cm-well

I have cassandra stderr.log like:
objc[2425]: Class JavaLaunchHelper is implemented in both /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/bin/java (0x10d44e4c0) and /Library/Java/JavaVirtualMachines/jdk1.8.0_102.jdk/Contents/Home/jre/lib/libinstrument.dylib (0x10e4744e0). One of the two will be used. Which one is undefined.

Can you find something?

@e-orz before showing error log above, it shows like:
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/kwangin/utils/apache-hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/kwangin/app/cm-well/app/kafka/kafka_2.12-<IP_ADDRESS>/libs/slf4j-log4j12-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Info:   starting bg
Info:  starting cw
Info:   starting web service
Info:     Web Service boot in progress ()
failed to check Web Service boot
Info:   waiting for ws...
	*** FAILED to wait for ws due to Tcp command [Connect(<IP_ADDRESS>:9000,None,List(),Some(10 seconds),true)] failed because of Connection refused ...Moving on!
Info:   inserting meta data
Info:     Web Service boot in progress ()

sorry for the delayed response, I was on a vacation.
I added some log prints to the installation package. Now it will print the actual exception occurred.
I have a feeling it will be "connection refused" which means WS didn't start properly. If this is the case please look at the ws logs and paste the error.

@e-orz hello, thanks for support!

Info:   updating version history
 Failed to upload eli2 http://<IP_ADDRESS>:9000/meta/logs/version-history/1534255024642
akka.stream.StreamTcpException: Tcp command [Connect(<IP_ADDRESS>:9000,None,List(),Some(10 seconds),true)] failed because of Connection refused

Info:   starting dc controller
Info: finished initializing cm-well

Could you find something?
Also, can it be related with elasticsearch, or python version?

It seems WS didn't start.
Go to ~/app/cm-well/app/ws/logs/application.log and paster here the reason.
and yes, in case of an error in some depenecies, WS won't start. The simplest way to find out what was the reason is to look in the logs.

Thanks!
application.log
Here's the log. Could you get something?

what python do you have in your machine?
During installation Cassandra's cqlsh is run.
The version that shipped with cm-well requires Python 2.7.

@e-orz you're right! It seems it was the problem of python version.
It works find after I changed to 2.7. . It would be good to add on document later.
Thanks for help 👍

It's documented in:
https://github.com/thomsonreuters/CM-Well/blob/master/docs/Tutorial.Building.md
Which has a link to it from the main page.
Maybe you are right and it should be in the main page.


Add experimental useRequestContext

Description
Closes #1429
This adds a small wrapper around useServerRequest / useEnvContext to provide a public (and experimental) API for 3p integrations.
It also exposes HydrogenRequest and HydrogenResponse types.
Additional context

Before submitting the PR, please make sure you do the following:

[x] Read the Contributing Guidelines
[x] Provide a description in this PR that addresses what the PR is solving, or reference the issue that it solves (e.g. fixes #123)
[x] Update docs in this repository according to your change
[x] Run yarn changeset add if this PR cause a version bump based on Keep a Changelog and adheres to Semantic Versioning

Thanks very much @frandiox!
We rolled this in to our 3p component library and it worked perfectly.
Most appreciated ❤

Hi @frandiox
We're experiencing an issue with release 1.0.0
We're referencing: import { useRequestContext } from '@shopify/hydrogen'; - for use in our server components.
In build prod on the latest demo store it's trying to resolve that as:
<EMAIL_ADDRESS>But in actual fact it resides here:
<EMAIL_ADDRESS>


Issue trying to run with command_block.

This line may cause an issue when I trying to run the command through the command_block.
https://github.com/Thatsmusic99/HeadsPlus/blob/7f77512fe602e44f0fcc68973f01889c8d86f35d/main/java/io/github/thatsmusic99/headsplus/commands/MyHead.java#L32

Commandblocks in 1.13 can only target players (@p, etc.) for built-in commands, but not for plugin commands. There is an issue on Jira (https://hub.spigotmc.org/jira/browse/SPIGOT-4295 ) but its not likely to get fixed any time soon according to md_5's comments.
I would recommend the OP look at using the CommandHook plugin as it seems to add all the original functionality:
https://www.spigotmc.org/resources/commandhook.61415/

OP hasn't responded for 2 months and I'm sure this is a suitable solution. Closing now.


Disallow authentication code once verified

This is a suggestion for small improvement. RFC 6238 requires the implementation to disallow (MUST NOT) multiple submissions of the same authentication code. This is apparently to detect MITM and over the shoulder attacks.

Thanks for pointing this out! I'll fix it when I have a chance.

Submitted to ROTP gem as mdp/rotp#44 as its their domain to comply with HOTP and TOTP specs in their respective RFCs.

I'm not sure the ROTP library is a good candidate. You have a good point that it is supposed to follow RFC (and at least document clearly where it does not). However, since you have to store somewhere the last accepted code or timestamp, ROTP would need to implement access layer to a database or other storage. That seems out of scope for such simple (purely algorithm) library.

@vilda I'm inclined to agree with you and the ROTP maintainer at this point. I'd hope that he'll provide extendable hooks and clear documentation in future work.
I'll probably dabble in implementing a solution in your library and submitting a PR, but I have little (read: zero) experience in working with Rails gems.

Would an approach like this work?

Add a column consumed_otp to the devise two-factor model table.
Server, upon supplied a valid OTP, hashes a concat of the current timestep and valid OTP and compares it to the value stored in consumed_otp.
If there is no match, permit login and update consumed_otp value for the row. If there is a match, deny login as it's a "burned" OTP code for that time step.

I feel like it's acceptable because:

The concatenation of the time step ensures the burned code applies only for that given time step
There's no master list of N-many "burned codes" since we only need to know the one code issued for a given timestep.
A valid OTP is inherently unique to a given user, therefore the resulting hash is unique to a given user.

I don't think the hashing step is necessary. Storing the timestep of the most recently successful authentication should be sufficient -- we don't care what the OTP was, we just care that a given OTP is only used once, and since each timestep has only one OTP, we can avoid needing to store the OTP entirely.
Other than that, your approach looks fine to me.
It's also important to ensure that we work with the timestep used to generate the OTP -- not the timestep associated with the request. Because of the drift period, these may not be the same.

I'm happy with your suggestion.
But with your approach, if you want to be anal about it, a user could theoretically login (thereby burning that timestep) disable & renable OTP, log out and try to login with a new OTP for that same timestep. Now with the default ROTP timestep interval being 30 seconds, and tinfoil's gem offering no option to override it, this becomes a very unpractical & narrow issue.

Ha, I hadn't thought of that!
That does sound like a valid bug, though it's enough of an edge case that I think I'd prefer sticking with the simpler implementation. If anyone raises their pitchforks in the future because of it, you can point them in my direction :P

Fine by me :wink:


Can't Stop the sound

Hello!
I think I found a bug. I was trying to stop the sound using the command explained at the documentation of the speakerCb callback.

I run the project and then pub a message to that topic as is shown in the next image.

The logic is fine since I enter the else case (where the variable m_pause should update and stop the ambient sound). However, the sound never stops.

I'm not sure if maybe I didn't understood the documentation. Because the update and logic makes sense. Basically the ambient sound will play as long as m_pause is 0 and m_multi_sound is 1

Thanks.

That's the challenge 😄 and why pause the ambientSound has +15%.
Extra points doesn't have extended documentation or something like that, because you need to force your logic and capabilities to solve it. That documentation just indicate when "Pause" action should be triggered.

There're several ways to do it 👍🏽

@Voldivh  Good luck!

Oh ok! So just to clarify, If i sent the 0 message the sound should stop. The thing is that I have to implement a way to make it stop. Is that right?

That's right, you could change some blocks of code if you want. Just don't broke your code and save a copy of the "homework" part to show us later.

Ok perfect! And for the homework part, I created 2 branches. The develop (with all the obligatory parts) and the feature/extra_homework. I will leave the develop branch as it is and I will keep adding the extra_points on the second branch if that's ok.

That's the way @Voldivh


Add support of 2021 rust edition

error: failed to parse manifest at `/home/*/projects/sandbox/Cargo.toml`
Caused by:
  feature `edition2021` is required
  consider adding `cargo-features = ["edition2021"]` to the manifest

at project with Cargo.toml
[package]
name = "sandbox"
version = "0.1.0"
edition = "2021"

[dependencies]

and empty lib.rs
Adding a feature does not help, since it should be not only in crates of workspace, but in in all dependencies wiith 2021 edition

I suspect you likely have an old version of nightly installed.  Can you say which version you have?

Yes, you are right, running via rustup run helped.


How can I fine-tune the model in chinese question gereration(number 9)

I am new in the period and I found your project is quite interesting.I am wondering if I could fine-tine your model in number ix (I got your model quite a long time ago and at that time you didnt ask for qualification).If I could ,what kind of dataset should I prepare and how can I do it(maybe there is script in your project?).Thank you for your help!

I’m very happy this repo can help someone interested in Chinese text generation.  Currently I have not time to complete a well-written documentation for model fine-tuning. Basically, you need to
(1) Prepare a  fine-tune config file. The config file is like model config with several sections, which contains the learning rate, dataset path, etc.  You can find args need to be specified in single trainer file.
(2) Prepare your dataset. The dataset content is just follow your task input and output. Each line of the dataset file is a json string.
{"src":input, "trg":output}
{"src":input, "trg":output}
...
(3) Run the code. python run.py --model_conf model_conf_path --train_conf train_conf_path.

thank you!


Fixnormalize the behavior of from_unixtime() according to Nereids planner

Proposed changes
Issue Number: close #xxx

Further comments
If this is a relatively large or complex change, kick off the discussion at<EMAIL_ADDRESS>by explaining why you chose the solution you did and what alternatives you considered, etc...

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 51.08 seconds
stream load tsv:          504 seconds loaded<PHONE_NUMBER>9 Bytes, about 141 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          64 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size:<PHONE_NUMBER>9 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711184248_clickbench_pr_176461.html

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 47.41 seconds
stream load tsv:          447 seconds loaded<PHONE_NUMBER>9 Bytes, about 159 MB/s
stream load json:         21 seconds loaded<PHONE_NUMBER> Bytes, about 107 MB/s
stream load orc:          56 seconds loaded<PHONE_NUMBER> Bytes, about 18 MB/s
stream load parquet:          29 seconds loaded 861443392 Bytes, about 28 MB/s
insert into select:          26.0 seconds inserted 10000000 Rows, about 384K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711115805_clickbench_pr_176460.html

run p0

run buildall

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 54.38 seconds
stream load tsv:          502 seconds loaded<PHONE_NUMBER>9 Bytes, about 142 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select:          29.1 seconds inserted 10000000 Rows, about 343K ops/s
storage size:<PHONE_NUMBER>3 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711205025_clickbench_pr_176550.html

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 50.77 seconds
stream load tsv:          499 seconds loaded<PHONE_NUMBER>9 Bytes, about 142 MB/s
stream load json:         20 seconds loaded<PHONE_NUMBER> Bytes, about 112 MB/s
stream load orc:          65 seconds loaded<PHONE_NUMBER> Bytes, about 16 MB/s
stream load parquet:          31 seconds loaded 861443392 Bytes, about 26 MB/s
insert into select:          28.5 seconds inserted 10000000 Rows, about 350K ops/s
storage size:<PHONE_NUMBER>9 Bytes
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711205026_clickbench_pr_176557.html

TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 49.1 seconds
stream load tsv:          447 seconds loaded<PHONE_NUMBER>9 Bytes, about 159 MB/s
stream load json:         19 seconds loaded<PHONE_NUMBER> Bytes, about 118 MB/s
stream load orc:          56 seconds loaded<PHONE_NUMBER> Bytes, about 18 MB/s
stream load parquet:          30 seconds loaded 861443392 Bytes, about 27 MB/s
insert into select:          25.8 seconds inserted 10000000 Rows, about 387K ops/s
https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230711135335_clickbench_pr_176548.html


refactor tests.

write tests in the style of react-testing-library. test behavior, not implementation and also use the actual/expected pattern

this ill be done together with #487


Error while installing tsne library

$ pip install tsne
Downloading/unpacking tsne
  Downloading tsne-0.1.1.tar.gz (51kB): 51kB downloaded
  Running setup.py (path:/home/salil/.virtualenvs/ipynb/build/tsne/setup.py) egg_info for package tsne

Requirement already satisfied (use --upgrade to upgrade): Cython>=0.19.1 in ./.virtualenvs/ipynb/lib/python2.7/site-packages (from tsne)
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.7.1 in /usr/lib/python2.7/dist-packages (from tsne)
Requirement already satisfied (use --upgrade to upgrade): scipy>=0.12.0 in /usr/lib/python2.7/dist-packages (from tsne)
Installing collected packages: tsne
  Running setup.py install for tsne
    skipping 'tsne/bh_sne.cpp' Cython extension (up-to-date)
    building 'bh_sne' extension
    x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include -Itsne/bh_sne_src/ -I/usr/include/python2.7 -c tsne/bh_sne_src/quadtree.cpp -o build/temp.linux-x86_64-2.7/tsne/bh_sne_src/quadtree.o -msse2 -O3 -fPIC -w
    cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]
    tsne/bh_sne_src/quadtree.cpp:12:19: fatal error: cblas.h: No such file or directory
     #include <cblas.h>
                       ^
    compilation terminated.
    error: command 'x86_64-linux-gnu-gcc' failed with exit status 1
    Complete output from command /home/salil/.virtualenvs/ipynb/bin/python -c "import setuptools, tokenize;__file__='/home/salil/.virtualenvs/ipynb/build/tsne/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-qO42Vo-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/salil/.virtualenvs/ipynb/include/site/python2.7:
    running install

running build

running build_py

creating build

creating build/lib.linux-x86_64-2.7

creating build/lib.linux-x86_64-2.7/tsne

copying tsne/__init__.py -> build/lib.linux-x86_64-2.7/tsne

running build_ext

skipping 'tsne/bh_sne.cpp' Cython extension (up-to-date)

building 'bh_sne' extension

creating build/temp.linux-x86_64-2.7

creating build/temp.linux-x86_64-2.7/tsne

creating build/temp.linux-x86_64-2.7/tsne/bh_sne_src

x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/lib/python2.7/dist-packages/numpy/core/include -I/usr/local/include -Itsne/bh_sne_src/ -I/usr/include/python2.7 -c tsne/bh_sne_src/quadtree.cpp -o build/temp.linux-x86_64-2.7/tsne/bh_sne_src/quadtree.o -msse2 -O3 -fPIC -w

cc1plus: warning: command line option ‘-Wstrict-prototypes’ is valid for C/ObjC but not for C++ [enabled by default]

tsne/bh_sne_src/quadtree.cpp:12:19: fatal error: cblas.h: No such file or directory

 #include <cblas.h>

                   ^

compilation terminated.

error: command 'x86_64-linux-gnu-gcc' failed with exit status 1

----------------------------------------
Cleaning up...
Command /home/salil/.virtualenvs/ipynb/bin/python -c "import setuptools, tokenize;__file__='/home/salil/.virtualenvs/ipynb/build/tsne/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-qO42Vo-record/install-record.txt --single-version-externally-managed --compile --install-headers /home/salil/.virtualenvs/ipynb/include/site/python2.7 failed with error code 1 in /home/salil/.virtualenvs/ipynb/build/tsne
Traceback (most recent call last):
  File "/home/salil/.virtualenvs/ipynb/bin/pip", line 11, in <module>
    sys.exit(main())
  File "/usr/lib/python2.7/dist-packages/pip/__init__.py", line 185, in main
    return command.main(cmd_args)
  File "/usr/lib/python2.7/dist-packages/pip/basecommand.py", line 161, in main
    text = '\n'.join(complete_log)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 42: ordinal not in range(128)

You need blas, open blas or alternatives. You should be able to install those using the system package manager. Or you could try the anaconda python distribution: http://continuum.io/downloads

thanks @danielfrg

Hi,
I have Fedora 22 [4.2.5-201.fc22.x86_64] and following rpm are installed.
I have similar issue as mentioned above by salilnavgire i.e. /bin/ld: cannot find -lcblas
while atlas, blas, openblas all are installed as shown below:
atlas-3.10.1-21.fc22.x86_64
atlas-devel-3.10.1-21.fc22.x86_64
blas-3.5.0-12.fc22.x86_64
blas-devel-3.5.0-12.fc22.x86_64
openblas-0.2.14-2.fc22.x86_64
openblas-devel-0.2.14-2.fc22.x86_64
openblas-openmp64-0.2.14-2.fc22.x86_64
liblastfm-1.0.9-2.fc22.x86_64
openblas-threads64-0.2.14-2.fc22.x86_64
openblas-serial64-0.2.14-2.fc22.x86_64
openblas-threads-0.2.14-2.fc22.x86_64
openblas-openmp-0.2.14-2.fc22.x86_64
Installation Logs
pip install tsne
Collecting tsne
Using cached tsne-0.1.5.tar.gz
Requirement already satisfied (use --upgrade to upgrade): Cython>=0.19.1 in /usr/lib64/python2.7/site-packages (from tsne)
Requirement already satisfied (use --upgrade to upgrade): numpy>=1.7.1 in /usr/lib64/python2.7/site-packages (from tsne)
Requirement already satisfied (use --upgrade to upgrade): scipy>=0.12.0 in /usr/lib64/python2.7/site-packages (from tsne)
Installing collected packages: tsne
Running setup.py install for tsne
Complete output from command /usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip-build-yuLQEB/tsne/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-lI6xVj-record/install-record.txt --single-version-externally-managed --compile:
/usr/lib/python2.7/site-packages/setuptools/dist.py:285: UserWarning: Normalizing 'v0.1.5' to '0.1.5'
normalized_version,
!!!!!!!!!!!!! ['tsne', 'tsne.tests']
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-2.7
creating build/lib.linux-x86_64-2.7/tsne
copying tsne/_version.py -> build/lib.linux-x86_64-2.7/tsne
copying tsne/init.py -> build/lib.linux-x86_64-2.7/tsne
creating build/lib.linux-x86_64-2.7/tsne/tests
copying tsne/tests/test_iris.py -> build/lib.linux-x86_64-2.7/tsne/tests
copying tsne/tests/init.py -> build/lib.linux-x86_64-2.7/tsne/tests
UPDATING build/lib.linux-x86_64-2.7/tsne/_version.py
set build/lib.linux-x86_64-2.7/tsne/_version.py to 'v0.1.5'
running build_ext
building 'bh_sne' extension
creating build/temp.linux-x86_64-2.7
creating build/temp.linux-x86_64-2.7/tsne
creating build/temp.linux-x86_64-2.7/tsne/bh_sne_src
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/local/include -Itsne/bh_sne_src/ -I/usr/include/python2.7 -c tsne/bh_sne.cpp -o build/temp.linux-x86_64-2.7/tsne/bh_sne.o -msse2 -O3 -fPIC -w
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/local/include -Itsne/bh_sne_src/ -I/usr/include/python2.7 -c tsne/bh_sne_src/quadtree.cpp -o build/temp.linux-x86_64-2.7/tsne/bh_sne_src/quadtree.o -msse2 -O3 -fPIC -w
gcc -pthread -fno-strict-aliasing -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -DNDEBUG -O2 -g -pipe -Wall -Werror=format-security -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/lib64/python2.7/site-packages/numpy/core/include -I/usr/local/include -Itsne/bh_sne_src/ -I/usr/include/python2.7 -c tsne/bh_sne_src/tsne.cpp -o build/temp.linux-x86_64-2.7/tsne/bh_sne_src/tsne.o -msse2 -O3 -fPIC -w
g++ -pthread -shared -Wl,-z,relro build/temp.linux-x86_64-2.7/tsne/bh_sne.o build/temp.linux-x86_64-2.7/tsne/bh_sne_src/quadtree.o build/temp.linux-x86_64-2.7/tsne/bh_sne_src/tsne.o -L/usr/local/lib -L/usr/lib64 -lpython2.7 -o build/lib.linux-x86_64-2.7/bh_sne.so -lcblas
/bin/ld: cannot find -lcblas
collect2: error: ld returned 1 exit status
error: command 'g++' failed with exit status 1
----------------------------------------

Command "/usr/bin/python -c "import setuptools, tokenize;file='/tmp/pip-build-yuLQEB/tsne/setup.py';exec(compile(getattr(tokenize, 'open', open)(file).read().replace('\r\n', '\n'), file, 'exec'))" install --record /tmp/pip-lI6xVj-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-yuLQEB/tsne

I had a similar issue on Ubuntu 14.04. For me the following Ubuntu packages resolved it:
sudo apt-get install libblas-dev libatlas-base-dev

I have same error on mac.

@mohitkaushikj133  This solution works for me

Sorry for digging up this old thread. This problem still exists as far as I see it. As of when I last benchmarked these in early 2017, Atlas was slow, OpenBLAS was faster, MKL was fastest (on cpu, because they also make the HW). cuBLAS is also an option. This installer (build script) is not finding what other installers are. I can see that openblas and MKL are both installed on my Ubuntu node and it keeps complaining about blas.h. LD_LIBRARY_PATH has it defined. I fixed this problem in the past with other libraries, but I don't remember how or maybe it's just not working this time. I'm giving up on this and taking the solutions above for now, but maybe I can help the next person get closer to a better answer.
I've tried one of the hacks that usually works, but not here.
export LD_PRELOAD=/opt/intel/mkl/lib/intel64/libmkl_core.so
FYI, you can use this to show where a program searches for libraries. You will have to Google the options. Set it back to empty to stop it.
export LD_DEBUG=libs
When I tried this, I couldn't see that it was even searching for any libs with cblas in the name.

This solution also works if libcblas is not able to be found.


Show held item in visualizer

After #34 now that the world DB shows robots holding items, have the visualizer pass that on and visualize it

Fixed with 43cca1c


Initialization of the oauth2 directive before endpoint values are established

The problem I had was that since the init for the oauth2.endpoint was being called when the directive was loaded, my values for the urls were not being used -- my bootstrap process determines the urls and so they are not known right away.
For example:
    <oauth2 authorization-url="{{loginEndpoint}}"
            sign-out-url="{{logoutEndpoint}}"
            sign-out-append-token="true"
            client-id="{{appKey}}"
            redirect-url="{{redirectEndpoint}}"
            sign-out-redirect-url="{{redirectEndpoint}}"
            sign-in-text="Log In"
            sign-out-text="Log Out"
            response-type="id_token token"
            scope="openid profile email roles all_claims">
    </oauth2>

In order to fix it, I just re-called init for the oauth2.endpoint module in the directive's login method. I hope that makes sense -- it doesn't seem like it would be very expensive...
This is just an FYI -- not sure if you want to take any action on it.

Why not using an ng-if on the oauth2 tag with a flag (or even a value) ?
The ng-if disable the component and prevent interpretation of directive.


Add the ability to generate routes

Consider generating the initial app integrated with component router.
Add the ability to generate new routes, with something like:
ng generate route 'heroes/:id'  hero-details-component

How do you see the routes getting created? Where would they live?

hmm.. I was expecting a single, top-level routes file.
I can see how projects may want to register their routes in a sub-folder, but it seems like that would be more of an advanced user case.  For the casual user wanting to start a new Angular2 project, it would be nice if routing worked "right-out-of-the-box."

My current thinking is that routes should really be routable components. These unlike regular components get generated in as a nested directory structure that maps to the route directory structure.

I can pick this up soon.

@filipesilva I am happy to help out with this too. I'm still learning ng2 but this project is great - the ember-cli is super useful.

@samjulien thanks! I think it's going to be easy enough, but adding routes will need these two PRs in as well https://github.com/angular/angular-cli/issues/66 https://github.com/angular/angular-cli/pull/100
@IgorMinar Routable components seems the way to go, like described here in angular.io. Maybe even doing the routable component + list component + detail component + service all at once.
Thus, running
ng generate route heroes

Would generate the following folder structure
...
|-- app
|   |-- ...
|-- heroes
|   |-- heroes.component.ts
|   |-- heroes-detail.component.ts
|   |-- heroes-list.component.ts
|   |-- heroes.service.ts
|-- app.ts
...

And would add the child route definition to the router in app.ts:
...
  {
    path: '/heroes/...',
    name: 'Heroes',
    component: HeroesComponent
  },
...

I've a PR proposal for route generation in https://github.com/angular/angular-cli/pull/139. It does somewhat more than adding a single route.


Implement automatic cookie for flows

We will use standard cookie behavior and a body entity to match flows and delete them.


feat/cleanup: Separate GM/Server Settings from Client Settings

The world and client settings live in a dialog together. This has been the case since original Tidy.
Because we now offer a shortcut for getting to that dialog more quickly via the sheet menu, this made an issue much clearer to see: GM settings and client settings should not be together in a dialog. There is no clear, quick delineation between the two, so it would be easy for a GM to change global settings for all sheets by mistake when making sheet changes.
Additionally, calling world or client settings "Sheet Settings" within the sheet menu implies that the settings apply to the current sheet and not all sheets. This is very confusing.
For the short term,

[x] Separate the GM only settings to a dialog all their own, under the title "World Settings"
[x] Organize the World Settings content into tabs
[x] Add GM only "World Settings" dialog button to Config Settings tab for Tidy 5e Sheets
[x] Rename the regular sheet settings to "User Settings"
[x] In the user settings dialog, ensure the title reads "User Settings - {currentUser}" or the like, make it clear that someone is editing their own settings
[x] Ditto for Theme Settings
Update the save logic for settings so that they only process the settings which are editable from the target dialog. This is to resolve #232 where unintentional differences are preventing low-permission users from saving changes
[x]  Separate the Info tab to its own dialog called "About Tidy 5e Sheets"
[x] Add an issue for Sheet-specific settings : a sheet might want its own theme, for example, which is user-agnostic and overrides any user- / world-defaults. There are likely some other settings for customizing sheets which make sense to provide, but this will need to be evaluated on a case-by-case basis.

Done: #327

[x] Add an issue for Client Settings culling : client settings pose a number of problems and make for more confusion. Review the client settings that currently exist and begin the process of phasing out superfluous ones.

Done: #328

Closing as completed.