Questions regarding Simulation and Network Parallelism

Hello,
I want to train a DAPG+PPO agent using 5 GPUs. I want to use the same hyperparameters as specified in the ManiSkill2 paper in Appendix F.8 in Table 9: Hyperparameters for DAPG+PPO.
My question is how I need to modify the hyperparameters given that I want to train on multiple GPUs.
Additionally I would like to get advice on whether is is more efficient to train and simulate on 5 GPUs or whether it is more efficient to separate training and simulation so that I train on 2 GPUs and simulate on 2 different GPUs. In the code it is required that len(args.sim_gpu_ids) == len(args.gpu_ids) meaning I could not make use of the fifth GPU that I have if I were to separate training and simulation.
I would like to train for 25e6 steps (total_steps), with 2e4 number of samples per PPO step (n_steps), number of samples per minibatch 300 (batch_size), number of critic warm up epochs 4 (critic_warmup_epoch), number of PPO updates epochs 2 (num_epoch), replay buffer capacity of 2e4 (I believe it must be the same size as n_steps, correct be if I am wrong), a model checkpoint every 1e6 steps, a final evaluation after the entire training is over.
Also I want to use a demonstration buffer with dynamic loading. From the Readme I got the following configuration. Do I need to make changes to any of the hyperparameters, like capacity and cache_size because I am training on 5 GPUs? Does the capacity of the demonstration replay buffer need to match the capacity of the experience replay buffer?
demo_replay_cfg=dict(
    type="ReplayMemory",
    capacity=int(2e4),
    num_samples=-1,
    cache_size=int(2e4),
    dynamic_loading=True,
    synchronized=False,
    keys=["obs", "actions", "dones", "episode_dones"],
    buffer_filenames=[
        "PATH_TO_DEMO.h5",
    ],
),

In the Readme the following information is provided. I am wondering if the Readme already named all of the hyperparmeters that are affected by simulation and training parallelism?

"Note that during training, if multiple simulation GPUs are used, then some arguments in the configuration file (e.g. train_cfg.n_steps, replay_cfg.capacity) are effectively multiplied by the number of simulation GPUs. Similarly, if multiple network-training GPUs are used, then some arguments (e.g. agent_cfg.batch_size) are effectively multiplied by the number of network-training GPUs."

whether is is more efficient to train and simulate on 5 GPUs or whether it is more efficient to separate training and simulation so that I train on 2 GPUs and simulate on 2 different GPUs

I think it's better to train and simulate on all 5 GPUs; it doesn't cost much extra GPU memory. The bottleneck is mainly on the CPU.
Do I need to make changes to any of the hyperparameters, like capacity and cache_size because I am training on 5 GPUs?

For demo_replay_cfg, you don't need to change its configs. However, you do need to change other agent configs and the main replay buffer config.
For 5-GPU training, I also recommend to use a larger (#gpus x batch size) and (#gpus x n_steps) compared to a single-gpu case, otherwise the multi-GPU training setting isn't really advantageous over a single-GPU training setting.
Example config for 5-GPU training:
agent_cfg = dict(
    type="PPO",
    gamma=0.95,
    lmbda=0.95,
    critic_coeff=0.5,
    entropy_coeff=0,
    critic_clip=False,
    obs_norm=False,
    rew_norm=True,
    adv_norm=True,
    recompute_value=True,
    num_epoch=2,
    critic_warmup_epoch=4,
    batch_size=256, # this will be effectively multiplied by # GPUs
    detach_actor_feature=False,
    max_grad_norm=0.5,
    eps_clip=0.2,
    max_kl=0.2,
    dual_clip=None,
    shared_backbone=True,
    ignore_dones=True,
    dapg_lambda=0.1,
    dapg_damping=0.995,
    actor_cfg=dict(
        type="ContinuousActor",
        head_cfg=dict(
            type="GaussianHead",
            init_log_std=-1,
            clip_return=True,
            predict_std=False,
        ),
        nn_cfg=dict(
            type="Visuomotor",
            visual_nn_cfg=dict(type="PointNet", feat_dim="pcd_all_channel", mlp_spec=[64, 128, 512], feature_transform=[]),
            mlp_cfg=dict(
                type="LinearMLP",
                norm_cfg=None,
                mlp_spec=["512 + agent_shape", 256, 256, "action_shape"],
                inactivated_output=True,
                zero_init_output=True,
            ),
        ),
        optim_cfg=dict(type="Adam", lr=3e-4, param_cfg={"(.*?)visual_nn(.*?)": None}),
    ),
    critic_cfg=dict(
        type="ContinuousCritic",
        nn_cfg=dict(
            type="Visuomotor",
            visual_nn_cfg=None,
            mlp_cfg=dict(
                type="LinearMLP", norm_cfg=None, mlp_spec=["512 + agent_shape", 256, 256, 1], inactivated_output=True, zero_init_output=True
            ),
        ),
        optim_cfg=dict(type="Adam", lr=3e-4),
    ),
    demo_replay_cfg=dict(
        type="ReplayMemory",
        capacity=-1,
        num_samples=-1,
        keys=["obs", "actions", "dones", "episode_dones"],
        buffer_filenames=[
            "PATH_TO_DEMO.h5",
        ],
    ),
)

train_cfg = dict(
    on_policy=True,
    total_steps=int(5e6), # this will be effectively multiplied by # GPU, so 5GPUs = 25e6
    warm_steps=0,
    n_steps=int(8e3), # this will be effectively multiplied by # GPU
    n_updates=1,
    n_eval=int(1e6), # this will be effectively multiplied by # GPU
    n_checkpoint=int(5e5), # this will be effectively multiplied by # GPU
    ep_stats_cfg=dict(
        info_keys_mode=dict(
            success=[True, "max", "mean"],
        )
    ),
)

env_cfg = dict(
    type="gym",
    env_name="PickCube-v0",
    obs_mode='pointcloud',
    ignore_dones=True,
)

rollout_cfg = dict(
    type="Rollout",
    num_procs=3, # this will be effectively multiplied by # GPU
    with_info=True,
    multi_thread=False,
)

replay_cfg = dict(
    type="ReplayMemory",
    capacity=int(8e3), # this will be effectively multiplied by #GPUs, and should keep the same as train_cfg.n_steps
    sampling_cfg=dict(type="OneStepTransition", with_replacement=False),
)

eval_cfg = dict(
    type="Evaluation",
    num_procs=5, # evaluation is always performed by the first GPU, so configs will not be multiplied by the #GPUs
    num=100,
    use_hidden_state=False,
    save_traj=False,
    save_video=True,
    log_every_step=False,
    env_cfg=dict(ignore_dones=False),
)

Then when initiating experiments, use the arg gpu-ids 0 1 2 3 4


Error when the cache folder doesn't exist

The PHP version you are using ($ php -v):
=> 7.3.14
PHP CS Fixer version you are using ($ php-cs-fixer -V):
=> 2.16.1
The command you use to run PHP CS Fixer:
=> vendor/bin/php-cs-fixer fix --dry-run
The configuration file you are using, if any:
<?php

use PhpCsFixer\Fixer\Import\OrderedImportsFixer;

$finder = PhpCsFixer\Finder::create()
    ->in(__DIR__.'/src')
    ...
;

return PhpCsFixer\Config::create()
    ...
    ->setCacheFile(__DIR__.'/var/cache/.php_cs.cache')
;

When the folder var/cache/ doesn't exist, there is this error:
Loaded config default from "/home/user/.php_cs".

Checked all files in 493.477 seconds, 30.000 MB memory used

In FileHandler.php line 91:

  Failed to write file "/home/user/var/cache/.php_cs.cache", "no reason available".

fix [--path-mode PATH-MODE] [--allow-risky ALLOW-RISKY] [--config CONFIG] [--dry-run] [--rules RULES] [--using-cache USING-CACHE] [--cache-file CACHE-FILE] [--diff] [--diff-format DIFF-FORMAT] [--format FORMAT] [--stop-on-violation] [--show-progress SHOW-PROGRESS] [--] [<path>]...

Shouldn't PHP-CS-Fixer be able to create the folder? or at least provide a better reason for the error?
Thank you

I don't think creating the directory is the responsibility of PHP CS Fixer though I agree the error message is not helpful.
Now, making PHP CS Fixer create the directory probably needs an amount of work that is similar to improving the error message so... I'd say let's create the directory then.

Creating the directory can be risky too. Perhaps it doesn't exist because someone has typoed it.

That's also true for directories that do exist though. Creating an unexpected directory would likely be spotted with a VCS such as Git anyway.

Not if it was made outside of that by mistake. Like someone accidentally putting "" which would be interpreted as /, perhaps? I've had this happen before. Deleting / is never good. 😆

Indeed, it is risky to make the directory on typos like '' and '/'.
IMHO, it is the responsibility of the user that the tool can write the cache where they want it. This also prevents including more logic and maintenance on the code base, for example when dealing with files on network space etc.
We can improve the error message though, currently we display what PHP itself set as error when the write failed.

Ok let's improve the message :+1:

fixed in https://github.com/FriendsOfPHP/PHP-CS-Fixer/pull/5161


rustfmt puts where on same line as impl for empty impl even if long

Given the following code:
impl<T: ?Sized, Idx> IndexRanges<Idx> for T
where
    T: Index<Range<Idx>, Output = Self::Output>
        + Index<RangeTo<Idx>, Output = Self::Output>
        + Index<RangeFrom<Idx>, Output = Self::Output>
        + Index<RangeInclusive<Idx>, Output = Self::Output>
        + Index<RangeToInclusive<Idx>, Output = Self::Output>
        + Index<RangeFull>,
{
}

rustfmt moves the where onto the same line as the impl. However, if you put a fn func() {} inside, it won't.
This may need to wait for a style edition to fix.

I believe this is related https://github.com/rust-lang/rustfmt/issues/5990.
We could version gate the fix before we get style edition rolled out.


Fix UI bug where the TestListItem accordian would expand to the wrong tab

Summary
A useEffect that was concerned with setting the tab index when the tab was empty of content was conflicting with another function which would set the tab index to the clicked on "ProblemBadge".  In theory, the old useEffect should not have affected the badge clicking but it clearly is.
Changing from the hook that fires on a data dependency to one that fires on the accordian expansion click solves the problem.  I've added tests to cover the previous situations.
I did not move the TabProps interface out of TestRunDetail.  I think that's a good place for it but I am using the type in the parent element.
Testing Guidance
Run and or inspect the tests that are added in this PR.  ;)
You can manually test too if you want.  Find test messages that have a variety of states:

Click on a test message with a warning.
Click on a test message with a request.
Click on a test message with nothing (infrastructure test suite).

But these should be covered by the tests.

Codecov Report
Base: 73.64% // Head: 95.38% // Increases project coverage by +21.74% :tada:

Coverage data is based on head (1e30ffc) compared to base (e77e5a9).
 Consider uploading reports for the commit 3c27d86 to get more accurate results

Additional details and impacted files
@@             Coverage Diff             @@
##             main     #276       +/-   ##
===========================================
+ Coverage   73.64%   95.38%   +21.74%
===========================================
  Files         184      113       -71
  Lines        8871     2992     -5879
  Branches      329        0      -329
===========================================
- Hits         6533     2854     -3679
+ Misses       2012      138     -1874
+ Partials      326        0      -326

Impacted Files
Coverage Δ

...ite/TestSuiteDetails/TestListItem/RequestsList.tsx

...ite/TestSuiteDetails/TestListItem/TestListItem.tsx

...te/TestSuiteDetails/TestListItem/TestRunDetail.tsx

client/src/api/infernoApiService.ts

client/src/components/Footer/Footer.tsx

client/src/components/LandingPage/index.ts

...nt/src/components/TestSuite/TestSuiteUtilities.tsx

client/src/store/app.ts

...mponents/RequestDetailModal/RequestDetailModal.tsx

...mponents/TestSuite/TestSuiteTree/TestSuiteTree.tsx

... and 61 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

:umbrella: View full report at Codecov.
:loudspeaker: Do you have feedback about the report comment? Let us know in this issue.


Create Contributing.md file with steps to follow for website calendar

Ref : https://github.com/Real-Dev-Squad/website-members/pull/145/files

To create a good developer experience for the contribution, it would be better we create a documentation that helps with the steps for first contribution

I want to take up this task and please assign it to me.


Is there a way to format currencies by their ISO codes?

My application shows different currencies, but it still needs to be in a given language such as English. I know you can set the entire application to a certain locale, but I couldn't find a way to just format the currency alone.

There is no easy way. You will need to specify the currency symbol whenever you are using the currency filter.

@lgalfaso but the symbol is only part of the problem, right? We still need to format a number correctly and stick the symbol at the "right" place.

this might be somehow off-topic, but if a page is language X, then I would expect that money amounts to be formatted using language X currency format (regardless of the currency being used).

@lgalfaso You aren't wrong, but there is also the common use case in finance where there are always multiple currencies shown on a given page. Bank accounts are a good example.

If at the same time, you need to display multiple currencies, then
specifying the currency symbol is a must. Btw, the current symbol can be an
expression

@lgalfaso Aside from the symbol, what about the precision as in the significant digits? Then there's other localisation factors such as $400k which would be 40万円. Are there future plans for these things?

@lgalfaso https://github.com/lgalfaso Aside from the symbol, what about
the precision as in the significant digits? Then there's other localisation
factors such as $400k which would be 40万円. Are there future plans for
these things?

Not that I am aware of, I would look for a third party module if this is a
requirement

Only slightly off topic: How about reimplementing the filters (currency, number, maybe date) based on the ECMAScript Internationalization API
Browser support is decent but not great. There's a polyfill, however it's not load and forget as you have to actually load the data you need.
If this is intresting, I'd be happy to create a pull request. Or maybe this should be a separate repository, due to the polyfill?

https://github.com/zwacky/isoCurrency


Incorrect host status label

Just wanted to bring this to your attention. Seems like Rancher is marking these hosts as disconnected when they are indeed connected and working.

Useful
Info

Versions
Rancher v1.1.4 Cattle: v<IP_ADDRESS> UI: v1.1.14

Access
github admin

Orchestration
Cattle

Route
hosts.index

Did you see any logging on the rancher/server? You are running a fairly old release, can you try to reproduce and file a new issue with more information how this is happening (random intervals, after adding x nodes, proxy in front yes/no?)


Update Email: Implement Authentication process state feedback

In order to provide robustness an adequate user experience, the authentication needs to provide feedback about the process.
Add login process feedback
We need to leverage/reuse the generic process state management feature.
For info, see "Process State Management" in GH-1580
Add "update email flow" functions to the facade
What we can do is then provide facade functions that will "orchestrate" the update email flow:
We would add to the facade these functions:
updateEmailFlow() {} // Truggers the update email + trigger the authentication on success.
getupdateEmailFlowSuccess() // Combines the update email + authentication streams
getupdateEmailFlowError() // Combines the update email + authentication streams
getupdateEmailFlowLoading() // Combines the update email + authentication streams
Adjust the UI component.
Next we need to tweak to UI to use the update email flow instead and leave the authentication trigger to the email flow.

This is an example how we could implement the combining part using combineLatest rxjs operator: https://stackblitz.com/edit/combine-latest-example

PR: GH-1927

Yesterday, we discussed that the ticket is done, but somewhat blocked. There is a problem with the occ calls that is happening which throws a global error message that says the user is anonymous. This happens because that call is called before the new token was loaded, therefore failing.
In conclusion, we discussed 4 possible solutions:

Leave it as-is with the error message.
Hide the error message + create a followup ticket to find a cleaner solution.
Find a cleaner solution before we release: Refactor the “CartData” implementation that seems to conflict with the email change.
Find a cleaner solution before we release: Investigate further to find a different solution than the CartData refactor.


WARN in ./nuxt-dist/utils.js (v2.15.0)

Hi.
I have a problem with the new version of Nuxt, v2.15.0
When I compile my project there is a warning that shown in the console:
 WARN  in ./nuxt-dist/utils.js  "export 'isSamePath' (imported as '_isSamePath') was not found in 'ufo'

@ms-fadaei Try refreshing your lockfile or running yarn upgrade ufo or npm up ufo. Does that help?


Added jupyter notebook examples for nn_translator and visualizer

Pull Request Test Coverage Report for Build 106

0 of 0 (NaN%) changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage remained the same at 56.642%

Totals

Change from base Build 104:
0.0%

Covered Lines:
226

Relevant Lines:
399

💛  - Coveralls


ISSUE NO. #9522 : "Added Doing math with python"

What does this PR do?
Added Book "Doing math with python" by Amit Saha Issue No. #9522.
For resources
Description
Why is this valuable (or not)?
How do we know it's really free?
For book lists, is it a book? For course lists, is it a course? etc.
Checklist:

[ ] Read our contributing guidelines.
[ ] Search for duplicates.
[ ] Include author(s) and platform where appropriate.
[ ] Put lists in alphabetical order, correct spacing.
[ ] Add needed indications (PDF, access notes, under construction).
[ ] Used an informative name for this pull request.

Follow-up

Check the status of GitHub Actions and resolve any reported warnings!

if it's not your book, you don't have the right to upload to Archive.org.


[flatbuffers] disable -Werror for gcc8

The warnings is:
/usr/bin/g++-8   -I".../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/include" -I".../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/grpc" -fPIC -std=c++0x -Wall -pedantic -Wextra -Werror=shadow -faligned-new -Wunused-result -Werror=unused-result -Wunused-parameter -Werror=unused-parameter -fsigned-char -MD -MT CMakeFiles/flatc.dir/src/flatc_main.cpp.o -MF CMakeFiles/flatc.dir/src/flatc_main.cpp.o.d -o CMakeFiles/flatc.dir/src/flatc_main.cpp.o -c '.../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/src/flatc_main.cpp'
In file included from .../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/include/flatbuffers/flexbuffers.h:24,
                 from .../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/include/flatbuffers/idl.h:26,
                 from .../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/include/flatbuffers/flatc.h:21,
                 from .../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/src/flatc_main.cpp:17:
.../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/include/flatbuffers/util.h: In function ‘int flatbuffers::FromUTF8(const char**)’:
.../vcpkg/buildtrees/flatbuffers/src/v1.9.0-693a89daa3/include/flatbuffers/util.h:324:45: warning: type qualifiers ignored on cast result type [-Wignored-qualifiers]
   if ((static_cast<const unsigned char>(**in) << len) & 0x80) return -1;  // Bit after leading 1's must be 0.

Thank you for the submission, it looks good.  You may want to also report this to the flatbuffers project.

I think they fixed it on master https://github.com/google/flatbuffers/pull/4666
I didn't test it but maybe upgrading to 1.10 also fixed the problem.


Remove unused istio componenets

As we are not using all the components provided by the istio, it is good to disable those unwanted components when installing istio via helm

Resolved with https://github.com/wso2-cellery/distribution/pull/185


no fatal when some normal fail

Coverage decreased (-0.1%) to 84.796% when pulling 16c822f347d403a8079d58b371920eaae1b28151 on xyl_no_fatal_on_grant_fail into 9e284613dd7c30e3895e58c13e2d9325e2411d1c on master.

Coverage decreased (-3.01%) to 81.931% when pulling c6b499dea5e01ae1f9a4d8d4129337197ad54a94 on xyl_no_fatal_on_grant_fail into 9e284613dd7c30e3895e58c13e2d9325e2411d1c on master.

Coverage increased (+0.2%) to 85.102% when pulling bf8deb45774428d38b5329dfcee3dc1e1f2f15f8 on xyl_no_fatal_on_grant_fail into 9e284613dd7c30e3895e58c13e2d9325e2411d1c on master.


Allow to use front or rear image of device

NetBox version
v3.7.5
Topology Views version
v3.9.0
Feature type
New functionality
Proposed functionality
I would like the topology view to automatically pull the front or rear image of each device (if it exists) and default to the icons if none is available.
Use case
I think this would make it easier to understand at first glance.
External dependencies
No response

I'd say draw an invisible box and resize the image (while preserving the aspect ratio) until it fits in the box. Another way to go about this might be to show the front and rear image in the tooltip instead

I'd say draw an invisible box and resize the image

So you would resize it in a way that it's the same size as the icons?

Another way to go about this might be to show the front and rear image in the tooltip instead

Okay, but which one would you prefer?

How much effort would it be to make a POC of both?
I think ma mix of both: front image in place of the icons, rear image in the tooltip. Or maybe a button to switch between the two.
Do you know the UniFi network application? They have device images implemented in their topology. Let me know if you need some screenshots of that.

How much effort would it be to make a POC of both?

Not a big deal. Here's what it could look like: Front image in place of the icons and both (front and rear) in the tooltip.

Personally I don't see any benefit from rear images in the topology view at all. So I'd suggest to just use front images.
As I played around a bit I found it quite useless to have images in the tooltip. Nobody wants to hover over each node in order to see images, IMO. If you don't mind I'd llike to stick to front images that overrides the role icons if the displayed device type has got an image. So basically like the screenshot above but without images in the tooltip. Any complaints?

Do you know the UniFi network application?

Yes, I'm familiar with it.

One last note: If you display a few more nodes, it might look like this:

You can see that there are 2 different pictures, one for patch panels and one for switches. This is hard to see at first sight (and this topology hasn't got very much nodes btw). But since assume to make this feature an option, that doesn't seem to be a big problem. I think the real benefit comes into play when non-racked devices with different shapes are shown.

Looks good, although i think there should be a way to limit the size of the images. Basically you could have an invisible square around the image and have an option to specify the maximum size. That would mean that the longer dimension (length or height) of the image would be affected.

I already limited the size in the code (just compare to the screenshot in my first post). We can make it even smaller of cause, but what's the benefit if you're not able to identify the device type anymore?
Just for the record: NetBox itself limits the size. I used images of different size and NetBox made them equal in width.
Which size (width) would you suggest compared to the current images in the screenshot?
Feel free to post some images or links to images. I can use them and show a concrete example.

I think this is good as a default, but you should make the configurable in the settings. Some people might like it smaller or bigger.

I think you're right. Unfortunately I am not able to set the icons to a certain size in the topology view. The engine (vis.network) seems to resize objects itself. When I played around with some images of different sizes, some images has been resized while others hasn't. I don't see any algorithm here.
This is a screenshot of some devices with the following dimensions:
SW019, SW021: 3046 x 313
SW005: 313 x 3046
SW008: 535 x 535

The dimensions of SW008 doesn't fit the other images at all. I also tried with a full HD picture which where displayed bigger than all other images together.
I'm afraid it leads to a very bad user experience if images with different dimensions are used.
It's not a problem to give the images a certain size for the tooltip because that's HTML and each image can be modified with CSS.

Closing this because it's not possible to define an image size.


fix lease does not renew #853

The etcd leaseId does not renew in keepAliveForever. Because leaseId is always greater than zero after startMember.
 It will print error: requested lease not found.
Fix:
Remove the statement: if leaseId <= 0 {}.

Fix the #853


SqlAzureDacpacDeployment: service principal authentication doesn't work for export action

Required Information
Question, Bug, or Feature?
Type: Bug
Enter Task Name: SqlAzureDacpacDeployment@1
Environment

Agent - Private: Windows_NT, 2.213.2

Issue Description
"SqlAzureDacpacDeployment" is not working when using service principal authentication for Export action, while it correctly works for Import action. Running the same command ("C:\Program Files\Microsoft SQL Server\150\DAC\bin\SqlPackage.exe" /Action:Export /TargetFile:"C:\XXXX.bacpac" /SourceServerName:"tcp:XXXX.database.windows.net,1433" /SourceDatabaseName:"XXXX" /AccessToken:"********" ) from Azure CLI works fine too.
This is the task configuration:
- task: SqlAzureDacpacDeployment@1
   displayName: Export DB
   inputs:
     azureSubscription: ${{ parameters.azureSubscription }}
     AuthenticationType: "servicePrincipal"
     ServerName: "tcp:$(databaseFQDN),1433"
     DatabaseName: $(databaseName)
     deployType: DacpacTask
     DeploymentAction: Export

Error logs
"C:\Program Files\Microsoft SQL Server\150\DAC\bin\SqlPackage.exe" /Action:Export /TargetFile:"C:\agent-01\_work\127\s\GeneratedOutputFiles\XXXX.bacpac" /SourceServerName:"tcp:XXXX.database.windows.net,1433" /SourceDatabaseName:"XXXX" /AccessToken:"********"

Connecting to database 'XXXX' on server 'tcp:XXXX.database.windows.net,1433'.
Time elapsed 00:00:00.79
##[error]*** An unexpected failure occurred: Exception has been thrown by the target of an invocation..
##[error]The Azure SQL DACPAC task failed. SqlPackage.exe exited with code 1.Check out how to troubleshoot failures at https://aka.ms/sqlazuredeployreadme#troubleshooting-

I'm experiencing the same issue. Are there any valid workarounds?

Adding a comment to keep the issue open
On Mon, 11 Sept 2023, 01:20 github-actions[bot], @.***>
wrote:

This issue is stale because it has been open for 180 days with no
activity. Remove the stale label or comment on the issue otherwise this
will be closed in 5 days
—
Reply to this email directly, view it on GitHub
https://github.com/microsoft/azure-pipelines-tasks/issues/17820#issuecomment-1712986139,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AANRND52QEAWSMIXDXN374LXZZKNXANCNFSM6AAAAAAVG35LOA
.
You are receiving this because you commented.Message ID:
@.***>

Adding a comment to keep the issue open

From: github-actions[bot] @.>
Sent: 10 March 2024 15:06
To: microsoft/azure-pipelines-tasks @.>
Cc: Jakub Konecki @.>; Comment @.>
Subject: Re: [microsoft/azure-pipelines-tasks] SqlAzureDacpacDeployment: service principal authentication doesn't work for export action (Issue #17820)
This issue is stale because it has been open for 180 days with no activity. Remove the stale label or comment on the issue otherwise this will be closed in 5 days
—
Reply to this email directly, view it on GitHubhttps://github.com/microsoft/azure-pipelines-tasks/issues/17820#issuecomment-1987262869, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AANRND5VSEF5DE4CDAJBJATYXRZFVAVCNFSM6AAAAAAVG35LOCVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSOBXGI3DEOBWHE.
You are receiving this because you commented.Message ID: @.***>


Add Language option during install

I've searched around but i didn't find anything related. By default scoop install the english version of the specified app: is it possible to change this behaviour adding an option to specify which languages version you want to download/install? It would be very usefull with multilanguage's apps like firefox. Something like
scoop install firefox --lang xx-XX
or whatever you like... thank you

Most apps only distribute a single installer/ZIP archive containing all languages. However, some large applications like Firefox distribute one installer per language. It's very difficult to support both in a consistent manner, so I don't think this would be worthwhile to implement.


Initial commit of Neutron Configuration Overrides

What is the purpose of this pull request?:
Initial Commit of Neutron Configuration Overrides
This commit is an evolution of the initial keystone, and nova  configuration
overrides work:
https://github.com/att-comdev/openstack-helm/pull/319
https://github.com/att-comdev/openstack-helm/pull/262
This also resolved a bug with nova bootstrap.
Unlike the other two pull requests, for the sake of expediency, this PR
lacks the usual mount append effort involved in configuration override
work for each chart.  That can be done as a separate pull request
against neutron.
What issue does this pull request address?: Fixes #
Notes for reviewers to consider:
This initial pull request adds in several configuration files for services not yet in use
such as the metering agent, sriov agent, and so on.  We can build upon these
new configuration files by adding conditional containers to run these services
in the future.
Also, it is well known even with configuration management the chart is very much
geared toward openvswitch right now.  This PR is not an attempt to solve that
but eventually loosing that dependency can leverage this effort.
Lastly, the mount append effort missing from this PR.  Can be done separately.
Specific reviewers for pull request:
@v1k0d3n @intlabs @larryrensing @wilkers-steve @wilreichert

@alanmeadows Could you please rebase against master?

po/neutron-dhcp-agent-df5sc             1/1       Running   0          1m
po/neutron-l3-agent-2nhlm               1/1       Running   0          1m
po/neutron-metadata-agent-xmzp5         1/1       Running   0          1m
po/neutron-server-3962975493-npcpm      1/1       Running   0          1m

and
alan@hpdesktop:~/Workbench/att/attcomdev/openstack-helm$ neutron agent-list
+--------------------------------------+----------------+-----------+-------------------+-------+----------------+------------------------+
| id                                   | agent_type     | host      | availability_zone | alive | admin_state_up | binary                 |
+--------------------------------------+----------------+-----------+-------------------+-------+----------------+------------------------+
| 24bb6d14-0120-4fe4-932e-8ce2526811c6 | L3 agent       | hpdesktop | nova              | :-)   | True           | neutron-l3-agent       |
| cde796dc-46d0-44ba-bb19-b0ec76eff452 | DHCP agent     | hpdesktop | nova              | :-)   | True           | neutron-dhcp-agent     |
| e744988d-c8f2-46c5-a206-9440e8b00b92 | Metadata agent | hpdesktop |                   | :-)   | True           | neutron-metadata-agent |
+--------------------------------------+----------------+-----------+-------------------+-------+----------------+------------------------+
alan@hpdesktop:~/Workbench/att/attcomdev/openstack-helm$

It works as far as the control plane is concerned.  I would normally like to do a VM add test, but given the time constraints: merge little hamsters, merge!


a typo in README.md

from
dependencies {
  compile 'com.ykhdzr:flow::1.0.0'
}

to
dependencies {
  compile 'com.ykhdzr:flow:1.0.0'
}

Thanks @nestorm001 for noticing


BugFix(#3658)

rollback to old version

why do you change the code, no problem is fond at the moment

problem on use oracle and view.
see #3658

tanks


Error: file name is invalid / File name too long

I'm getting a number of errors on a few files.
It seems to be saying this, for each file - about the source file:
The item couldn’t be saved because the file name xxxxx is invalid.
and this about the destination:
File name too long
Neither of which seem to make sense given the file names/paths displayed. It does create the files in the destination directory, but all the ones I checked are 0 bytes long. I tried deleting these files and re-running but it seems to error on the same files every time.
The destination directory is an SMB network share to an external drive on another Mac, with an HFS+ filesystem (if that matters).
The command I ran and the command output is given below.
Stephen
$ osxphotos --db /Users/ssummer/Pictures/Photos\ Library.photoslibrary export --directory "{created.year}/{created.month}" /Volumes/Media/Photos --update --person-keyword --album-keyword
Exporting 63439 photos to /Volumes/Media/Photos...
  [------------------------------------]    1%  05:01:55Error exporting photo (51092B12-BE85-4797-B12E-457AE617C1F0: DSCN9184.JPG) as DSCN9184.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “51092B12-BE85-4797-B12E-457AE617C1F0.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/5/51092B12-BE85-4797-B12E-457AE617C1F0.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2015/October/DSCN9184.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/5/51092B12-BE85-4797-B12E-457AE617C1F0.jpeg, NSUnderlyingError=0x7fb19889c3b0 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [#-----------------------------------]    5%  04:10:04Error exporting photo (091EE8E2-0C0E-4549-BBDC-8F7E8713390F: DSCN8158.JPG) as DSCN8158.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “091EE8E2-0C0E-4549-BBDC-8F7E8713390F.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/0/091EE8E2-0C0E-4549-BBDC-8F7E8713390F.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2008/July/DSCN8158.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/0/091EE8E2-0C0E-4549-BBDC-8F7E8713390F.jpeg, NSUnderlyingError=0x7fb149267430 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [##----------------------------------]    8%  04:11:04Error exporting photo (21D35FE7-0AC7-4AEA-8C72-300AA1262BE2: PICT0193.JPG) as PICT0193.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “21D35FE7-0AC7-4AEA-8C72-300AA1262BE2.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/2/21D35FE7-0AC7-4AEA-8C72-300AA1262BE2.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2004/October/PICT0193.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/2/21D35FE7-0AC7-4AEA-8C72-300AA1262BE2.jpeg, NSUnderlyingError=0x7fb196fe6520 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [###---------------------------------]    8%  04:10:50Error exporting photo (18A3DD96-1D56-40A5-BC55-3306DF4BFBFE: DSCN6903.JPG) as DSCN6903.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “18A3DD96-1D56-40A5-BC55-3306DF4BFBFE.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/1/18A3DD96-1D56-40A5-BC55-3306DF4BFBFE.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2010/November/DSCN6903.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/1/18A3DD96-1D56-40A5-BC55-3306DF4BFBFE.jpeg, NSUnderlyingError=0x7fb196fe0dd0 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [#####-------------------------------]   15%  03:44:24Error exporting photo (6A0E56F0-438F-49CE-8D7A-06F3E1470943: DSCN8519.JPG) as DSCN8519.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “6A0E56F0-438F-49CE-8D7A-06F3E1470943.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/6/6A0E56F0-438F-49CE-8D7A-06F3E1470943.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2008/December/DSCN8519.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/6/6A0E56F0-438F-49CE-8D7A-06F3E1470943.jpeg, NSUnderlyingError=0x7fb149923760 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [######------------------------------]   16%  03:42:36Error exporting photo (12CA6533-314F-4D4B-A4A2-1058A2E4958D: IMG_3076.PNG) as IMG_3076.PNG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “12CA6533-314F-4D4B-A4A2-1058A2E4958D.png” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/1/12CA6533-314F-4D4B-A4A2-1058A2E4958D.png, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2020/March/IMG_3076.PNG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/1/12CA6533-314F-4D4B-A4A2-1058A2E4958D.png, NSUnderlyingError=0x7fb14978dc10 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [######------------------------------]   17%  03:46:19Error exporting photo (091A53B6-6ABC-4B82-B9FE-557A3DD81E6F: DSCN8275.JPG) as DSCN8275.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “091A53B6-6ABC-4B82-B9FE-557A3DD81E6F.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/0/091A53B6-6ABC-4B82-B9FE-557A3DD81E6F.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2015/October/DSCN8275.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/0/091A53B6-6ABC-4B82-B9FE-557A3DD81E6F.jpeg, NSUnderlyingError=0x7fb14983b3b0 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [######------------------------------]   19%  03:49:34Error exporting photo (37F1D79E-9DC8-4F03-B4EA-100E666D683E: DSC02182.JPG) as DSC02182.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “37F1D79E-9DC8-4F03-B4EA-100E666D683E.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/3/37F1D79E-9DC8-4F03-B4EA-100E666D683E.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2012/March/DSC02182.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/3/37F1D79E-9DC8-4F03-B4EA-100E666D683E.jpeg, NSUnderlyingError=0x7fb14996fdd0 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [########----------------------------]   23%  04:05:39Error exporting photo (D8CAEE3A-87A3-4BB1-8066-EAB5501A9A80: DSCN5731.JPG) as DSCN5731.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “D8CAEE3A-87A3-4BB1-8066-EAB5501A9A80.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/D/D8CAEE3A-87A3-4BB1-8066-EAB5501A9A80.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2010/March/DSCN5731.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/D/D8CAEE3A-87A3-4BB1-8066-EAB5501A9A80.jpeg, NSUnderlyingError=0x7fb14986c780 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [##########--------------------------]   30%  05:08:12Error exporting photo (FD4D979F-D5DB-4B6A-9441-7B1A0D14411E: P7210040_2.JPG) as P7210040_2.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “FD4D979F-D5DB-4B6A-9441-7B1A0D14411E.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/F/FD4D979F-D5DB-4B6A-9441-7B1A0D14411E.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2005/July/P7210040_2 (1).JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/F/FD4D979F-D5DB-4B6A-9441-7B1A0D14411E.jpeg, NSUnderlyingError=0x7fb149ce9070 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}
  [############------------------------]   34%  06:35:47Error exporting photo (2D0A83CC-C646-4D4A-902C-37AD67B566B1: DSCN3865.JPG) as DSCN3865.JPG: Error Domain=NSCocoaErrorDomain Code=514 "The item couldn’t be saved because the file name “2D0A83CC-C646-4D4A-902C-37AD67B566B1.jpeg” is invalid." UserInfo={NSSourceFilePathErrorKey=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/2/2D0A83CC-C646-4D4A-902C-37AD67B566B1.jpeg, NSUserStringVariant=(
    Copy
), NSDestinationFilePath=/Volumes/Media/Photos/2007/June/DSCN3865.JPG, NSFilePath=/Users/ssummer/Pictures/Photos Library.photoslibrary/originals/2/2D0A83CC-C646-4D4A-902C-37AD67B566B1.jpeg, NSUnderlyingError=0x7fb149e04ab0 {Error Domain=NSPOSIXErrorDomain Code=63 "File name too long"}}

Weird.  This may be in SMB issue or may be something weird in the source library.  Please run these command and send me the output.  Either attach here or email to me at<EMAIL_ADDRESS>Note that the debug dump will contain metadata about the photos including keywords, person names, locations, etc.  So if there's anything private in there, you might not want to post here.  The output will be text that's fairly self-explanatory so you can review it before sending it and delete any data that's sensitive.  (for example, the 'reverse_geolocation', 'longitude', 'latitude' fields which aren't relevant to this problem).
osxphotos debug-dump --dump photos --uuid 51092B12-BE85-4797-B12E-457AE617C1F0 > invalid.txt
osxphotos debug-dump --dump photos --uuid 2D0A83CC-C646-4D4A-902C-37AD67B566B1 > toolong.txt

I think this has to do with flakiness of the SMB drive and not something in the actual Photos library.  I've added a --retry X option to osxphotos export to retry the export X times (where X is int >= 0) if there's an error during the export.  Give that a try and let me know if you still errors.  --retry is in release v0.41.6


Summary::getBestWindow() sometimes scans past the begging of the array

Summary::getBestWindow() sometimes scans before the 'ms' array (off-by-one-error).
Solved in our fork, commit https://github.com/privacore/open-source-search-engine/commit/42706b5a41856d5fa3a22cf51169247411153986

thanks, incorporating into master.


When I toggle on picker the page jumps up

I notice that when I toggle on the emoji picker it cause the window to scroll up a bit to fit the picker to screen, how can I cancel it?

It could be a Popper.js issue. Try experimenting with the position options?
Can you fork this CodeSandbox (https://codesandbox.io/s/picmo-popup-demo-6fyoew) and see if you can provide a reproducible case with it?
Thanks!

@reflexLabs Any update on this?

I can't see what's going on if you don't provide a working code example. Please create a CodePen/CodeSandbox or the like with your code that exhibits the issue. Thanks!

Closing due to no response. Feel free to reopen in the future if you can provide some sample code.


Backport:New Monitoring Prometheus PVC doesn't work with OpenEBS

https://github.com/rancher/rancher/issues/19410

can be tested after https://github.com/rancher/rancher/issues/19610 is ready for test

in rc7

I am closing this issue, it has effectively already been tested here: https://github.com/rancher/rancher/issues/19610#issuecomment-490294425
This is because when I validate the UI I ensure the API appears to be working correctly as well.
Using the release name makes the names stay under 63 chars and as such should resolve the OpenEBS issue originally reported.


llm model command does not offer option to delete

While download of a model, I had an internet issue and the download was broken.  After the partial download, the LLM does not make an attempt to restart download.
There seems to no options to remove the partially downloaded model.  I could also not figure out the location of the downloaded model so that I can make a manual delete.
Can we add an option to remove a model? Or, document the location of the models?  I tried to look myself ... but without success.

Ok, I could find the locally stored models -> ~/.cache/gpt4all,
After deletion of partially downloaded file, I could proceed again.

I had the same problem and could not find the folder myself (using locate and find).
Just to make sure this info can be found easily by others:
Also on Mac, all models downloaded by llm-gpt4all are in the folder ~/.cache/gpt4all.


illustrator stored as self(0) in file

I'm using 20170524-beta3. I downloaded works by pixiv member 3723954 (https://www.pixiv.net/member.php?id=3723954) today. The download is fine, but the file is stored in directories like "L:\pic\self(0)\self(0)-❤あや❤(38349407)東方,東方project,落書き,射命丸文.jpg".

Weird, I tried and it stores properly to PixivUtil\testapp\あまの@お仕事募集中 (3723954)\38349407_p0 - ❤あや❤.jpg. Tested with filename format %artist% (%member_id%)\%urlFilename% - %title% in config.ini

also with same issue, seems like it randomly happens?  I have a folder self (0) with tons of images, I just never noticed it was placing files there.  I saw this in my window:
Already downloaded: 31817836
Last Page
Done.
done.
Processing Member Id: 1197484
Reading C:\pixivutils\pixivutil(4b)\config.ini ...
done.
 Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313
Remote filesize = 250.00 KiB (256627 Bytes)
Start downloading... Using Referer: http://www.pixiv.net/member_illust.php?mode=
medium&illust_id=63375313
256627 of    256627 Bytes  Completed in 0.761s (329.32 KiB/s)
Image verified.
Image URL : https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p1
.jpg
Filename  : C:\PixivDownloaded\self (0)\63375313_p1.jpg
Checking local filename... Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313
Remote filesize = 616.00 KiB (631777 Bytes)
Start downloading... Using Referer: http://www.pixiv.net/member_illust.php?mode=
medium&illust_id=63375313
631777 of    631777 Bytes  Completed in 0.863s (714.91 KiB/s)
Image verified.
Image URL : https://i.pximg.net/img-original/img/2017/06/14/03/15/51/63375313_p2
.jpg
Filename  : C:\PixivDownloaded\self (0)\63375313_p2.jpg
Checking local filename... Getting remote filesize...
Using Referer: http://www.pixiv.net/member_illust.php?mode=medium&illust_id=6337
5313

need sample input

Can you try https://github.com/Nandaka/PixivUtil2/releases/tag/20170619-beta4
It will generate an html page if the member id is not able to be parsed. Please upload it here if its happen.

using the beta having writeimageinfo = True in the config doesnt produce the .txt with the comments/tags

I think these are the ones that generated just now
http://s000.tinyupload.com/index.php?file_id=02440308656133072048
http://s000.tinyupload.com/index.php?file_id=01291899624919583339

Hi, I can confirm this bug happens at random, I just close the program, relaunch, and the next time it downloads correctly. Happened with both 20170411 and 20170524-beta3. I'll try the newest one and observe what happens.

one of it is due to deleted image.
other one is the correct file, looks like they update the layout
==> will update the parser.

try https://github.com/Nandaka/PixivUtil2/releases/tag/v201070626-beta5

Just had the exact same issue. I'll check if the new version then.

You can use d. Manage database then 10. Delete image by image_id.
Or delete the wrong images and temporarily change the config.ini values for:
overwrite = True
alwayscheckfilesize = True

Alternatively, purge folder, d. Manage database then c. Cleanup database then redownload.

Delete image by image_id won't work, there is like a thousand images there, haven't launched an update in a while.

overwrite = True
alwayscheckfilesize = True

I assume this will also re-download all the images I deleted prior to that? Which I definitely do not want to happen. Also it will overwrite every image which I modified, by cropping or whatever.
Is there any way to delete the entries from DB by date?
Because all of them were made within a known period of time, which can be specified.

Well, you can find external SQLite DB browser then, and use it to remove all images with directory self(0), but I haven't tried that and got no experience here.

Yeah, that is the last resort and will probably what I'll have to do then.
Re-reading the readme, I don't see any commands for deleting entries by dates.

Delete member by member_id
Delete image by image_id
Delete member and image (cascade deletion)
This is all of what is listed.

@WhiteSp1rit  It will not do the actual download if the local filesize is the same as the server filesize, but it still issue a HEAD request to the server to get the file size.
The other option is using DB editor to delete by dates. You can use SQLiteBrowser (http://sqlitebrowser.org/)

Yeah just opened the DB with this, and here they are, entries with a path like:

Nandaka's PixivUtil2\pixiv_images[0] self\64553359_p0.png

Thankfully since the issue is fairly simple I won't have to re-learn SQL ;)
Filtering by "[0] self" showed all invalid entries.
Looking at the dates earlist one was "2017-07-06 02:46:44", so this problem first appeared more than a month ago.

So the procedure is to use SQLiteBrowser, find all '[0] self' entries, delete them all, and hit Cleanup database and redownload?

"[0] self" is how my folder was named under the naming scheme I used. Yours will probably be different.
For example, if you use just member id for folder names, your folder might be just "0" or something.
I don't even know where you got the "Cleanup database" command or what does it do, you use "Write changes"


fix(scripts): single node celestia app home fix

Overview
The default Celestia app home directory contains a typo
Checklist

[ ] New and updated code has appropriate documentation
[ ] New and updated code has new and/or updated testing
[ ] Required CI checks are passing
[ ] Visual proof for any user facing features like CLI or documentation updates
[ ] Linked issues closed with keywords

Codecov Report

Merging #2467 (2228646) into main (958cc7e) will not change coverage.
Report is 3 commits behind head on main.
The diff coverage is n/a.

@@           Coverage Diff           @@
##             main    #2467   +/-   ##
=======================================
  Coverage   20.61%   20.61%
=======================================
  Files         131      131
  Lines       15270    15270
=======================================
  Hits         3148     3148
  Misses      11820    11820
  Partials      302      302


Rendering @throws Exception Description

Hello people,
when i rendering the documentation the throws exception only renders the description.
Example:
/**
 * Getting DB Values
 *
 * The method connect to localhost DB XYZ
 *
 * @throws Exception Connect to Database failed
 */

What is rendered in PHP Doc:
Getting DB Values
The method connect to localhost DB XYZ
throws                                               Connect to Database failed
I want to know how i can render the exception type. It gives so many exception types (InvalidArgumentException and so on).
If can tell me a way to render this, i would thank you very much
I'm using Version phpDocumentor2-2.8.2 with Template responsive-twig
Greetings
Robert

thanks for your report. This issue will be fixed in the next release.


The AbstractFormWriter.isTypeOrCollection() cannot handle byte[] arrays

Assumptions
There is an endpoint with byte[] parameter. E.g. photo parameter in following snippet.
// byte[] parameter
@RequestLine("POST /send_photo")
@Headers("Content-Type: multipart/form-data")
void sendPhoto (@Param("is_public") Boolean isPublic, @Param("photo") byte[] photo);

The sendPhoto() method is called.
Issue
The AbstractFormWriter.isTypeOrCollection() tries to cast the byte[] value into Object[] which leads to the "java.lang.ClassCastException: class [B cannot be cast to class [Ljava.lang.Object; ([B and [Ljava.lang.Object; are in module java.base of loader 'bootstrap')" error.
Problem is on the line 65: https://github.com/spring-cloud/spring-cloud-openfeign/blob/10ad8a0c6ea0a5b641eb31318c93055bdaf4c4f8/spring-cloud-openfeign-core/src/main/java/org/springframework/cloud/openfeign/support/AbstractFormWriter.java#L63-L65
Expected behaviour
The method should be able to handle arrays of primitive types without casting into Object[].
FYI: @darrenfoong

Hello @vkuzel , looks like a bug. Thanks for reporting it.


PHP security updates

Things done

[ ] Tested using sandboxing
(nix.useSandbox on NixOS,
or option build-use-sandbox in nix.conf
on non-NixOS)
Built on platform(s)

[ ] NixOS
[ ] OS X
[x] Linux

[x] Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[x] Fits CONTRIBUTING.md.

Doesn't introduce any new build failures for me on nox-review

Say what, travis?

@risicle unrelated breakage


Re-enable lineplot and map in pass.tsx

Oops - merged before I saw this and I didn't do a version update ...

Oh, I see you already merged. No worries!

I will do a version update :)

Sorry and thanks.

No worries.


RFC: MustAlias edge and pass

This introduces a new MUST_ALIAS edge, and a simple pass, to handle soot-generated aliases.
Before merging, consider the relevant changes in codescience and the resultant changes in findings.
But you can comment on the general approach already, and whether this justifies the additional edge.
A further change is to treat <operator>.cast like member-access. I think that would be a very sane change -- in JVM languages, we have a lot of cases like
$r1 = (java.util.Map) $r0 // $r0 has static type java.util.HashMap

Now we already have two mechanisms for dealing with aliases: First, all member access operators are treated as aliases-with-path by the entire machinery, deep down; second, we allow policies to declare the -alias flag, which triggers a slightly sketchy alias mechanism. We currently use that slightly sketchy mechanism for <operator>.cast. However, this mechanism does not interact well with the new must_alias pass.
I think we should always have treated <operator>.cast like a member access, for the sake of aliasing. That does not introduce any possible incorrect flows.
The incorrect flows caused by this new must_alias pass are all of the same kind: If a memory write between creation and use of the alias breaks it, then we compute wrong flows. I think this is preferable to the current state (we always assume that a memory write in the meantime destroys the alias).
A more advanced pass would figure out whether intermediate (in CFG) memory writes can break the alias. We already do that in the llvm inliner pass, using the built-in alias analysis and callgraph-based function attributor (esp. readonly, argmemonly).
A similar approach could be attempted for CPG, but for now I'd like to see comments on the AST-based variant.
A more brutal approach would also alias SSA values (single definition, many uses), but I want to start small.

As discussed via Zoom: We cannot merge it like this because this would eliminate the cast operations from the data flow results which is important to have in languages like C since problematic casts are a hole group of security problems in this language.

@ml86 Can you take a look again?
The insight behind the latest change is that we don't need to know user preferences about <operator>.cast in the general code -- we can simply look at presence of mustAlias edges.
re naming: What about SAFE_SSA? The edge after all signifies that something is single-static-assignment (which allows us to elide tracking the LOCAL), and that it is permissible to relocate between DEF and USE.
Furthermore, I think we should move this from enhancements.json into base.json and allow frontends to use that: LLVM has its own version of this computation, and could set these edges (llvm2cpg actually builds an entire forest, not just chains like the simple pass for JVM, and uses actual pointer analysis to determine the SAFE part).
Currently llvm2cpg just recursively inlines the DEF into all USES -- which means that we have potentially exponential blow-up of the size of the CPG (DEF0 is inlined into USE00 and USE01, which are themselves define DEF1 which has USE10 and USE11, et cetera. In other words, int x0 = foo(); int def0 = x0+x0; int def1 = def0 + def0; int def2 = def1 + def1; ... leads to exponential number of copies of def0).

@ml86 @fabsx00 can you take a look again? This work is not high priority, but it has been stuck in review for ~ 3 weeks now.

@bbrehm I am still not convinced that sometimes treating the cast operator as part of the access path is a good idea.

@ml86 it is the correct thing to do in java, where casts are pointer-reinterpret, and added by the compiler due to the way generics / type parameters are implemented.
It would have been better to model NOP-casts (i.e. reinterpret / bitcast of pointers) with a different operator in the first place, but that's where we are.

Closing, because I've been getting daily reminders about it for the last 6 months.


Added link to valid parameter values

@stevestein,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

azureps-cmdlets-docs/ResourceManager/AzureRM.Sql/v3.2.1/New-AzureRmSqlElasticPool.md
:white_check_mark:Succeeded

azureps-cmdlets-docs/ResourceManager/AzureRM.Sql/v3.2.1/Set-AzureRmSqlElasticPool.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

Validation seems hung up. Closing and reopening to give it a nudge...

@stevestein,
Thanks for your contribution as a Microsoft full-time employee or intern. You do not need to sign a CLA.
Thanks,
Microsoft Pull Request Bot

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

azureps-cmdlets-docs/ResourceManager/AzureRM.Sql/v3.2.1/New-AzureRmSqlElasticPool.md
:white_check_mark:Succeeded

azureps-cmdlets-docs/ResourceManager/AzureRM.Sql/v3.2.1/Set-AzureRmSqlElasticPool.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.

#sign-off

:white_check_mark: Validation status: passed

File
Status
Preview URL
Details

azureps-cmdlets-docs/ResourceManager/AzureRM.Sql/v3.2.1/New-AzureRmSqlElasticPool.md
:white_check_mark:Succeeded

azureps-cmdlets-docs/ResourceManager/AzureRM.Sql/v3.2.1/Set-AzureRmSqlElasticPool.md
:white_check_mark:Succeeded

For more details, please refer to the build report.
Note: If you changed an existing file name or deleted a file, broken links in other files to the deleted or renamed file are listed only in the full build report.


Feature/451 add test for cache

Changes

big refactoring of Robot resource imports (better support by VSCode Robot Framework Language Server plugin - IDE features)

tests for detection of issue with caching

extra test suite for tests with cache disabled

[x] add and activate test to CI pipeline

Related issue
resolves https://github.com/ehrbase/project_management/issues/451
Additional information and checks

[ ] Pull request linked in changelog

@stefanspiska please have a first review - although the test is not fully finished, there are some open questions (see linked issue above).
Also note the test is not on CI pipeline yet. To run it locally use this command
robot -d results -L TRACE -i cache robot/CACHE_TESTS

this is how far the test works for me

seems that the template is not available after restart of ehrbase

@stefanspiska the PR is ready for review


CSVW: CSV on the Web

Read/write CSVW

[ ] Read CSVW
[ ] Write CSVW

CSVW
Homepage: https://w3c.github.io/csvw/
Standard: https://www.w3.org/TR/tabular-data-primer/
Standard: https://www.w3.org/TR/tabular-data-model/
Standard: https://www.w3.org/TR/tabular-metadata/
Standard: https://www.w3.org/TR/csv2json/
Standard: https://www.w3.org/TR/csv2rdf/
Namespace: http://www.w3.org/ns/csvw#
xmlns: @prefix csvw: <http://www.w3.org/ns/csvw#> .
@context: http://www.w3.org/ns/csvw.jsonld
CSVW (CSV on the Web) is a set of relatively new standards for representing CSV rows and columns as RDF (and JSON / JSON-LD) along with metadata.

URIs for datatypes (XSD, ...)
URIs for columns (RDF)
Document Metadata
CSV -> JSON ( -> JSON-LD -> RDF )
CSV -> RDF

.

https://wrdrd.github.io/docs/consulting/knowledge-engineering#csvw

From https://twitter.com/westurner/status/901988850330996736 :

@kennethreitz
SQL Query (from any DB) ⟿ Pandas DataFrame, in one line of code:
https://github.com/kennethreitz/records#records-sql-for-humans
@westurner
What about #CSVW #JSONLD? (#nbmeta #linkedreproducibility)
https://w3c.github.io/csvw/primer/
@kennethreitz
want to send a pull request?
@westurner
I'd like to. It may be good to get to 1.0 w/ #nbmeta (maybe retool w/ #attrs) result objects w/ repr_html (repr_rdfa_and_jsonld) be4
@kennethreitz
that'd be good! i just want to make sure curl/requests and friends get back data, not html
@westurner
"Model for Tabular Data and Metadata on the Web" (#JSONLD, #RDFa HTML) is for Data on the Web #dwbp #linkeddata
http://www.w3.org/TR/tabular-data-model/
@kennethreitz
sounds like something to add to tablib! http://docs.python-tablib.org/en/master/
@westurner
The challenge/opportunity is where to add the additional columnar metadata: {@sqlalchemy, #tablib,}
@westurner
"7 metadata header rows (column label, property URI path, DataType, unit, accuracy, precision, significant figures)"
https://wrdrd.github.io/docs/consulting/linkedreproducibility#csv-csvw-and-metadata-rows …
@kennethreitz
hmm, tablib is quite dumb when it comes to that stuff.... could autodetect if all columns match
@kennethreitz
or we just support strings out the gate and call it v0
@westurner
#CSVW defaults to xsd:string if unspecified.
"How do you support units of measure?" #qudt
https://www.w3.org/TR/tabular-data-primer/#units-of-measure
@kennethreitz
i say we stick to strings and call it v0 :)

@westurner
I'd like to. It may be good to get to 1.0 w/ #nbmeta (maybe retool w/ #attrs) result objects w/ repr_html (repr_rdfa_and_jsonld) be4

nbmeta: https://github.com/westurner/nbmeta

@westurner
#CSVW defaults to xsd:string if unspecified.

From https://www.w3.org/TR/tabular-data-primer/#datatypes :+1:

NOTE
You don't have to include "datatype": "string" for columns that are strings — columns are assumed to hold strings if no datatype is explicitly specified.

this sounds awesome!

Unfortunately I don't know when I'll be able to get to this while job
searching.
On Thursday, August 31, 2017, Kenneth Reitz<EMAIL_ADDRESS>wrote:

this sounds awesome!
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/kennethreitz/tablib/issues/305#issuecomment-326480489,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AADGy9bcXuifI956S5qCu2vlKN2f0TmRks5sd388gaJpZM4PJoIM
.

@westurner
I'd like to. It may be good to get to 1.0 w/ #nbmeta (maybe retool w/ #attrs) result objects w/ repr_html (repr_rdfa_and_jsonld) be4

attrs

Src: https://github.com/python-attrs/attrs
Docs: https://attrs.readthedocs.io/en/stable/
Docs: https://attrs.readthedocs.io/en/stable/examples.html#metadata
Docs: https://attrs.readthedocs.io/en/stable/extending.html#extending-metadata

hey @westurner and @hugovk, I would like to collaborate on this feature and get it up and going. Can you guys guide me or may be point me in the right direction to start this. Thanks in advance. :smiley:

Have a look at the dev docs:

https://tablib.readthedocs.io/en/stable/development/

@hugovk thanks for pointing this to me will try to do a WIP and then we can take this ahead.

Autho comment

Comment

@56789KD Please stop spamming.


Write, single bit example

Hello i read bit like this, bud i can't change it and write back.
i got 100.1
100.2
they are false i need change it to true and write it back.
s7client.MBRead(100, 1, function(err, buffer) {
if(err) return console.log(' >> MBRead failed. Code #' + err + ' - ' + s7client.ErrorText(err));
var bit=(buffer[0] & 0x01)!=0;
		console.log(bit); // say's false.

});
Some one please help me to write function that will change bit and write it back

Hi,
if you want to write only a single bit you have to use
S7Client.WriteArea(area, dbNumber, start, amount, wordLen, buffer, [callback])
with wordLen=S7WLBit.
For example if you want to change M100.1 to true:
var snap7 = require('node-snap7');

var s7client = new snap7.S7Client();
s7client.ConnectTo('<IP_ADDRESS>', 0, 1, function(err) {
    if(err)
        return console.log(' >> Connection failed. Code #' + err + ' - ' + s7client.ErrorText(err));

    // Change M100.1 to true
    s7client.WriteArea(s7client.S7AreaMK, 0, (100*8+1), 1, s7client.S7WLBit, new Buffer([0x01]), function(err) {
        if(err)
            return console.log(' >> WriteArea failed. Code #' + err + ' - ' + s7client.ErrorText(err));
    });
});

thank you!!! it's work's !
bud now when i read i any way get false in my function maybe i do somting wrong when read bit?

In your given example you read the full merker byte M100 but you test for the first bit with 0x01 but M100.1 is the second bit and M100.2 the third. So you have to do var bit = !!(buffer[0] & 0x02) for M100.1 and var bit = !!(buffer[0] & 0x04) for M100.2.

And again thank you!


execute does not show responses

When I run the execute in the Swagger UI generated by flasgger, the server does run the route and returns JSON, but flassger UI doesn't show the response.
What do I need to do to get the UI to show the response

Can we get more information about your application.

I got it to work, but I don't remember what I did to fix it.


Not working on Url with redirect

I have a url that returns a code 301 (redirect) with the url of the song. Using that link im not able to play the song. If the url is direct to the song it works fine. Im using a network inspector an the second url is never called.
this is my link:       "https://v23.musuy.com/servlet/com.musuy.musweb.agetstream?/jHtnqj4WR2b7xcC5jE4gwGJsPt5jCZAoxctYaMI0CBBIk55Lg/T9BwZeACkwvsgaTqDBURqSrmIhwN3Jrc5QP06/zdovRcOeP68eWFFoE3hJd8%2BjkvAw3Sr%2B7YPfwl/";
If i use this link in the AVPlayer of AVFoundation in IOS it works perfectly.
I saw the other issue with the code 302 but it was with an http request not https, i allowed arbitrary loads anyways but it didnt change nothing.
Do i have to do something in special?

the second url itself doesn't serve any media file

Yes it does, the generated url (the second one) can only be used one time. If you try that url in the browser it wont work because it make the request 2 times. If you try it out in any player it works.

To add more info im getting this in the console:
player[37893:3340017] Task <3DD9395D-9C9F-475F-B0C9-1AF454EC8E4B>.<1> load failed with error Error Domain=NSURLErrorDomain Code=-999 "cancelled" UserInfo={NSErrorFailingURLStringKey=https://v20.musuy.com/servlet/com.musuy.musweb.agetstream?Cs9PuwKuD8aMbQ67Hg9UZ1Z+sWbQ3xVr+hZSffqhG8kEzXhTRx1kZ7n81pjp+apGZB4BPWEoe8Y4RuabK8P/ydCJMi0Uh13KBjJGOuEDOds91jvScIsWG3LhWwVDF11o, NSErrorFailingURLKey=https://v20.musuy.com/servlet/com.musuy.musweb.agetstream?Cs9PuwKuD8aMbQ67Hg9UZ1Z+sWbQ3xVr+hZSffqhG8kEzXhTRx1kZ7n81pjp+apGZB4BPWEoe8Y4RuabK8P/ydCJMi0Uh13KBjJGOuEDOds91jvScIsWG3LhWwVDF11o, _NSURLErrorRelatedURLSessionTaskErrorKey=(
"LocalDataTask <3DD9395D-9C9F-475F-B0C9-1AF454EC8E4B>.<1>"
), _NSURLErrorFailingURLSessionTaskErrorKey=LocalDataTask <3DD9395D-9C9F-475F-B0C9-1AF454EC8E4B>.<1>, NSLocalizedDescription=cancelled} [-999]

Im trying with a direct url of 7Digital that works in the browser and im getting the same error and no audio

I've tried in Android and it works perfectly with that link. And added an AVPlayer implementation into this project and it works well too (So there is no problem with the plist configuration).
Sorry for spamming i just want to help adding information because i really want to use this library in my project. :)

It was a problem with encoding the url if you look is different the url i provided with the url of the log output. I created another issue related to this. Thanks  #564

Hi,
I have the same error with a supernormal URL:
https://www.myserver.com/audio/7
this URL is a PHP page that checks the auth token and then respond with the file or with an error.
Calling the URL with Postman or with Axios inside the app, all works fine, but with this library I've -999 code error...
I've added all headers required but the request doesn't even reach the server, it's cancelled before doing the request.
Any advise?

That url says me this:
This site can’t be reachedwww.myserver.com’s server IP address could not be found.
DNS_PROBE_FINISHED_NXDOMAIN

Yes, the host is obviously a fake URL to not show you my real server address :) but the /audio/7 is correct.
It's my client's site so I can't write it in clear right now.

In my case all the %2  were encoded by the lib so it was reaching a diferent url. I dont know your url though


"[!] No video results!" with ":a#"

When searching videos with straw-viewer and want to see the latest videos of a certain channel with ":a" and a number it doesn't give any results but shows "[!] No video results!" with the previous list of videos.

I cannot confirm this issue with invidious.snopyta.org. What instance are you using?
Additionally, use the --debug option to see if there are any errors.

The --debug gives;
=>> Select one or more videos to play (:h for help)

:a1
** GET https://invidious.snopyta.org/api/v1/channels/UCPDis9pjXuqyI7RYLJ-TTSA/videos ==> 500 read timeout (1s)
** GET https://invidious.snopyta.org/api/v1/channels/UCPDis9pjXuqyI7RYLJ-TTSA/videos ==> 200 OK (2s)

[!] No video results!

Hmm... It seems to work only for some channels, but not for others. Although, it's not an issue of straw-viewer. It's an issue of invidious.
Changing the instance may help:
straw-viewer --instance=invidious.13ad.de

For example, this works: https://invidious.13ad.de/api/v1/channels/UCXuqSBlHAE6Xw-yeJA0Tunw/videos


yet another Uncaught Error: plugin "record" does not exist

Description
I have been trying to setup videojs-record on a rails setup with rails webpacker and always run into the error Uncaught Error: plugin "record" does not exist
I went trough the related closed issues and that did not solved the problem, hence i am here requesting some input on how to solve it
Steps to reproduce
So without being very familiar with codesandbox i managed to setup one with the bare minimum as per the videojs webpack wiki entry https://github.com/collab-project/videojs-record/wiki/Webpack
The sandbox can be found here https://codesandbox.io/s/unruffled-dream-lkcvp
Results
Expected
Should create one instance of videojs with the videojs-record plugin
Actual
Always bump into the error Uncaught Error: plugin "record" does not exist
Please open the browser console to see the error
Error output
Uncaught Error: plugin "record" does not exist
Additional Information
The versions of the plugins can be found on the codesandbox link

After some attempts i think the problem might be the latest version of videojs v7.7.6 because it does seem to work with v.7.7.5

Interesting. I'll give it a try.

npm install videojs-record installs video.js 7.7.5 so this is basically working. If it breaks with 7.7.6 (pre-release for now) and that is actually released, I'll reopen this.

Hello,
I'm commenting this issue because I got the exact same error but with video.js v7.9.2.
I had to downgrade to 7.8.4 to make it work again. I didn't test 7.9.0 nor 7.9.1, so I don't know if the error comes with these 2 versions too.
I'll gladly help investigating the issue if needed cc @thijstriemstra :-)

I'm not sure if I should open a new issue though

@Kulgar please do.


Any way to track requests in progress? For acceptance tests

I was having intermittent failures in my acceptance tests that I was able to fix using this technique of setting a promise on the controller that the test waits for the resolution of. It seems unfortunate to me to add test-specific code to the application, so I’ve been looking into how the andThen acceptance test helper determines when asynchronous operations are complete so it can check the expectations.
I think the key code is here. This line would be helpful if the application were backed by a REST/AJAX database:
// 2. If there are pending Ajax requests, keep polling
if (Test.pendingAjaxRequests) { return; }

Elsewhere, they register to listen for AJAX events and operate on the counter that way:
jQuery(document).on('ajaxSend', incrementAjaxPendingRequests);
jQuery(document).on('ajaxComplete', decrementAjaxPendingRequests);

So, my question, is there a way I can detect an ongoing PouchDB operation and delay the progression of the test until the operation is complete? I looked at PouchDB.on, but it’s apparently only for created and destroyed events.
Maybe there’s a way to include this kind of functionality in ember-pouch to facilitate acceptance tests. Like setting a flag before a call to this.db.rel.find and the like, and unsetting it when the operation completes. Then I could override `setupForTesting' and pretend that a PouchDB operation is an AJAX operation, for the purposes of testing.
Ideally, ember-testing would have a facility where ongoing asynchronous operations can be registered to delay test progression, but I’m not optimistic on that happening.
This may not be something you want ember-pouch to be concerned with, though, understandably.

I've been using a custom test waiter that is similar to your wait-for-model code, @rsutphin , but just automatically waits for all models to settle (so there's no adapter-specific code needed in your tests).
You can install it with ember install ember-cli-test-model-waiter (note, though, that the model-finding code only works with Ember 2.0 + at the moment)

Neat. Thanks for sharing that.

Inspired by your code, @rsutphin, so thank you :)

Closing unless you still think this is an issue. Thanks for the discussion! :)


crate-admin blank after upgrading to 0.52.3

Hello.
After I upgraded to Crate 0.52.3 admin page is not loading anymore. Page is completely blank. I tried this on multiple nodes and even a new instance on Google Cloud Platform had the same issue.
Browser:
Google Chrome 46.0.2490.86 (32-bits)
This is the error log from browser console:
Uncaught Error: [$injector:modulerr] Failed to instantiate module crate due to:
Error: [$injector:modulerr] Failed to instantiate module truncate due to:
Error: [$injector:nomod] Module 'truncate' is not available! You either misspelled the module name or forgot to load it. If registering a module ensure that you specify the dependencies as the second argument.
http://errors.angularjs.org/1.3.15/$injector/nomod?p0=truncate
at http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:412
at http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:8758
at b (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:8330)
at http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:8642
at http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:17076
at f (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:797)
at n (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:16924)
at http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:17093
at f (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:797)
at n (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:16924)
http://errors.angularjs.org/1.3.15/$injector/modulerr?p0=truncate&p1=Error%…3A4200%2F_plugin%2Fcrate-admin%2Fscripts%2F21417c0c.angular.js%3A1%3A16924)
at http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:412
at http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:17352
at f (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:797)
at n (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:16924)
at http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:17093
at f (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:797)
at n (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:16924)
at Sa (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:18614)
at h (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:6443)
at Object._ [as bootstrap] (http://node1:4200/_plugin/crate-admin/scripts/21417c0c.angular.js:1:6753)
http://errors.angularjs.org/1.3.15/$injector/modulerr?p0=crate&p1=Error%3A%…%3A4200%2F_plugin%2Fcrate-admin%2Fscripts%2F21417c0c.angular.js%3A1%3A6753)

---
Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/28821217-crate-admin-blank-after-upgrading-to-0-52-3?utm_campaign=plugin&utm_content=tracker%2F505659&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F505659&utm_medium=issues&utm_source=github).

hi @Samuman
thanks for reporting the issue. we'll take a look and let you know when it is resolved.
best,
christian

Hi @Samuman ,
fix is already in crate-admin 0.15. (see https://github.com/crate/crate-admin/issues/173)

Fix is in new version of Crate 0.52.4 stable
https://github.com/crate/crate/releases/tag/0.52.4


Issue filtering IPs

Hi,
Just to give you some context on my setup, I have an Apache server sitting in the DMZ as a SSL termination point to proxy the requests to the internal caddy server.
So I'm running into an issue with the ipfilter middleware which does not seem to respect the "allow" rule that is defined in my Caddyfile:
:80
gzip
log /var/log/caddy/access.log {
  rotate {
    size 100
    age  14
    keep 10
  }
}
ipfilter / {
  rule allow
  ip   <IP_ADDRESS> <IP_ADDRESS>-255
}
root /apps/caddy/www/
tls off

Just to give you some context on my setup, I have an Apache server sitting in the DMZ as a SSL termination point to proxy the requests to the internal caddy server.
Basically any requests coming from the proxy are identified as <IP_ADDRESS> in the access.log and the <IP_ADDRESS>-255 is to allow internal IPs accessing the server directly.

Thanks!

I forgot to mention what the log basically shows is that anything coming from an internal IP is allowed but requests coming from the proxy are 403'ed.

Additional information, if I remove the "ipfilter" from my Caddyfile:
:80
gzip
log /apps/caddy/logs/access.log {
  rotate {
    size 100
    age  14
    keep 10
  }
}
root /apps/caddy/www/
tls off

Everything works:
<IP_ADDRESS> - [10/Dec/2015:11:58:27 -0500] "GET /buildroot/ HTTP/1.1" 304 0
<IP_ADDRESS> - [10/Dec/2015:11:58:31 -0500] "GET /buildroot/ HTTP/1.1" 200 37

So I think I've figured out the cause of this issue following a bit more investigation but I can't say I've resolved it.
First of all, even though the logs report that the request is coming from <IP_ADDRESS>, it's actually coming from the proxy server which has the internal IP <IP_ADDRESS>. <IP_ADDRESS> is actually the vlan gateway.
Now, the main issue is that the IPs being logged in the caddy access.log file are not the actual IPs being filtered by the ipfilter middleware. I found out that if I put anything but the proxy server IP (<IP_ADDRESS>), then nothing will work but if I put the proxy server IP (<IP_ADDRESS>) then everything will work.
If you are following my reasoning then you should now understand that the ipfilter middleware is not able to filter out anything.
Caddyfile:
:80
gzip
log /apps/caddy/logs/access.log {
  rotate {
    size 100 # Rotate after 100 MB
    age  14  # Keep log files for 14 days
    keep 10  # Keep at most 10 log files
  }
}
ipfilter / {
  rule allow
  ip   <IP_ADDRESS>
}
root /apps/caddy/www/
tls off

access.log:
<snip>
<IP_ADDRESS> - [10/Dec/2015:14:05:54 -0500] "GET /buildroot/ HTTP/1.1" 200 37
<IP_ADDRESS> - [10/Dec/2015:14:06:06 -0500] "GET /buildroot/ HTTP/1.1" 200 37

Confirmed that caddy logs are using "X-Forwarded-For" otherwise "remote ip" which is why there was a difference between what was logged and what was actually happening.
The ipfilter middleware probably only filters out on "remote ip" which would explain everything and why it is not working when behind a proxy.
Suggestion:
It would be probably be interesting to add a config option to enable different way to filter IPs, such as "strictly X-Forwarded-For", "X-Forwarded-For, otherwise remote ip" and "strictly remote ip". It could perhaps default to the "X-Forwarded-For, otherwise remote ip" which would be in line with what can be seen in the logs but that could be seen as a "breaking change" to a certain extent so it could also default to "strictly remote ip" if you want to remain on the safe side.
(Or, instead of all of this, it could simply be enabling a "proxy mode" which would apply the "X-Forwarded-For, otherwise remote ip" mode)

your findings are right, ipfilter was originally designed to filter only by country codes, which means only remote ip, your suggestion should be added eventually, I would be glad to get a PR though ;)


distill: equations in footnote or a pop-up window

I like distill popup window provided in footnote and cite. I would like to use popup window to annotate part of text (explaining more). This should be done in these places:
(1) in the page left or right sides.
(2) in the footnote.
For (1): Currently I cannot do sides of the page.
For (2): Currently I get ugly format if I add equations (they won't be inline with text), see this image for example:

Can you please provide solutions for both (1) and (2)?

any suggestions for this pop-up window request?


P2P (both EV and batteries), P2H and P2G should only store/convert excess

This is not really a question but more a request for verification. :smile:
What I mean by excess electricity is the production of solar PV (or other volatile producers) that cannot be used by consuming technologies and would otherwise be sent to the network. So, say at time 't' we have a connection of the network with

P2P battery
solar PV producing 10 kW
Television consuming 10 kW
In this case, the P2P battery should not be storing anything.

If the situation is as follows:

P2P battery
solar PV producing 15 kW
Television consuming 10 kW
the battery should store at a rate of 5 kW (if its capacity permits). For realistic batteries, the capacity is about 2 kW however so it would only store at that rate and 3 kW load would flow to the network.

@antw is this indeed how it works?

@antw is this indeed how it works?

TL;DR: Yes!
There are two types of consumption: "mandatory" and "conditional".

Mandatory consumption is that which must be satisfied; if there is insufficient production in the testing ground, the grid will be used to balance the deficit. If you have a television consuming 10 kW and a solar PV producing 5 kW, 5kW will come from the PV and 5 kW from the grid.

Conditional consumption is that which is satisfied only if there is an excess of production within the testing ground; never using the grid. Batteries, P2H, and P2G are all classed as "conditional". If there is no excess, they will receive nothing.

Electric vehicles have both mandatory and conditional consumption. The load required to satisfy the minimum charge of the vehicle (defined by the availability profile) is mandatory. If there is not enough production in the testing ground for the car to charge to its minimum stored volume, it will use the grid. The car will only charge beyond its minimum volume if there is an excess of production.
To put it another way: the area below the availability curve is mandatory and must be fulfilled. The area above the curve is conditional.

Let me know if this is unclear. :smiley:

Oh... that red line should read "Volume", not "Capacity". /faceplam

Let me know if this is unclear. :smiley:

Very clear thanks!

Closing.


Qute textmate grammar

Fixes #182
To test this PR open a .qute file.
This PR adds a test.qute file for you to try out.
What this PR does
This PR adds a new Qute language with its own textmate grammar:

Things to improve:

Within an if section, if we have a comparison with an integer or float, the integer should be a different colour:

{#if item.age > 10}

The same applies for strings:

{#if item.name is 'sword'}

If a method is being called, the method should be a different colour:

{#for item in item.getDerivedItems()}

If brackets are being used inside of a string like in the following image, {person.name} should be a different colour.

First name: <input type="text" name="fname" value="{person.name}">

Currently, the colour scheme is what it is because the textmate grammar provided in this PR is based on FreeMarker's grammar: https://github.com/bburbach/textmate-freemarker-bundle/blob/master/FreeMarker.tmbundle/Syntaxes/FreeMarker.tmLanguage

The colours for keywords like "for" and "if" are given a root group of "entity", therefore its colour is yellow. I think it would be nice to be consistent with programming languages' syntaxes, I think it would be better if these keywords were given "keyword.control.(something)" instead.
Doing so, will give these keywords a more familiar colour (at least for VS Code):

In VS Code, keywords like "for" and "if" from programming languages like Java, Python, Typescript are purple.
I'm not sure if this behaviour will be the same in IDEs like Eclipse or IntelliJ, so for now, this is just a suggestion.
Things I am not sure about

Should parameter declarations<EMAIL_ADDRESS>foo} in the following image) be highlighted in a different colour?

The textmate grammar inherits rules from this HTML grammar: https://github.com/textmate/html.tmbundle/blob/master/Syntaxes/HTML.plist

So, therefore, HTML comments are highlighted in green:

From what I understand, Qute templates have their own comment style:

Are HTML comments valid in Qute templates?

@xorye it looks like very promising! Congrats!

To test this PR open a .qute file.

It's a thing that we must discuss with @mkouba because Qute documentation uses .html file extension and not .qute.
We have 2 choices, @mkouba, @maxandersen  (and Qute community) :

are OK to use .qute file extension. Your PR is perfect. But perhaps we should use .html-qute file extension because Qute can generate anything. So we could imagine to generate json content (.json-qute). In other words we could have a convention name for Qute template. What do you think about that?

wishes to keep .html file extension, is it possible to manage that with vscode, Eclipse IDE, etc? Perhaps Textmate injection could help us https://code.visualstudio.com/api/language-extensions/syntax-highlight-guide#creating-a-basic-injection-grammar but my fear is that it will update all html file from all project of vscode (not only project which have Qute dependency.

Things to improve:

Indeed it should be very nice, but if it's too complex, perhaps it can be enough for the moment. @mkouba, @maxandersen, @fbricon what do you think about that?

Should parameter declarations<EMAIL_ADDRESS>foo} in the following image) be highlighted in a different colour?

I think it can be enough but I let @mkouba, @maxandersen, @fbricon give us feedback.

Are HTML comments valid in Qute templates?

Yes

So I think that we should avoid using .qute if possible because:

Qute itself is not a file format
In Quarkus we use the file extension to detect the media type when working with variant templates

Maybe just use .qute.html? Otherwise we would have to parse the content of the file and look for the Qute constructs {@ ...}, {! ... !}, {#...}, etc.

So I think that we should avoid using .qute

It was my fear :)
@xorye do you know if we could use  .qute.html as file type in vscode ?

.qute.html is fine imo.

.qute.html is fine imo.

I agree but I don't know if vscode, Eclipse IDE, etc considers that .qute.html is a file extension. I fear that .html is a file extension. We need to do some tests about that.

I added a new commit that uses .qute.html and it seems to be working:

I added a new commit that uses .qute.html and it seems to be working for VS Code:

Great! I will check that in Eclipse IDE.

mmm qute.html works as it's recognized as a qute file, but then we lose all the HTML support

mmm qute.html works as it's recognized as a qute file, but then we lose all the HTML support (completion, emmet ...)

how was that different when the file is called .html-qute or something else ?

@maxandersen, @xorye  is studying how to manage this html file extension, we will give you feedback when we will find a clean solution, because the challenge is to have a file extension (if we could keep .html it should be really cool) is to

keep HTML support managed by the HTML LS (completion, hover, etc)
keep the HTML syntax coloration and add Qute syntax (perhaps an injection of TextMate could work ?)
augment the HTML language configuration (matching bracket,etc) with Qute syntax (ex: {! !}).

@maxandersen after several experimentation (correct me if I'm wrong @xorye ) it seems the better solution to support Qute syntax coloration with TextMate WITHOUT loosing HTML support (completion, etc) is NOT to define a Qute language. In other words the only solution is to inject Qute syntax with TextMate to the HTML TextMate grammar.
It works great except that all HTML files have Qute syntax coloration even if a project is not a Qute project. So the idea is to give the capability to use to enable/disable the Qute syntax for a workspace. To do that an idea, @fbricon suggest to create a new vscode extension (ex : vscode-qute-syntax-highlighting) which injects only the Qute TextMate grammar in the given HTML TextMate grammar.
The user can disable the Qute syntax coloration by uninstalling the vscode-qute-syntax-highlighting and by keeping Quarkus (and Qute support managed by language server).
For Qute language configuration (to manage matching bracket for Qute comment ({! !}) I don't know how to do that?
If @fbricon is OK, @xorye please create this new vscode extension, thanks!

that’s a bummer and funky limitation of vscode ;/
I reckon we'll still be able to have a generic ".qute" file support which will just syntax highlight the qute part ?
how about content assist ? is that limited in the same way  or is this truly just for syntax highlighting things gets a bit convoluted ?

that’s a bummer and funky limitation of vscode ;/

Yes I know, you can read an interesting discussion at https://github.com/microsoft/vscode/issues/55088#issuecomment-408031810

I reckon we'll still be able to have a generic ".qute" file support which will just syntax highlight the qute part ?

We could do that, but it will require that we will create a vcode Qute language type and we will loose HTML support completion, etc) because HTML language server works only with vscode HTML language

@angelozerr @maxandersen
It looks like VS Code will soon provide a contribution point for other html-like languages to register to the html language server, as mentioned in point b. in this comment: https://github.com/microsoft/vscode/issues/88377#issuecomment-573568651

I pushed some improvements to the grammar:

There are now two grammar files: qute-html.tmLanguage.json and qute.tmLanguage.json.
qute-html.tmLanguage.json
This is the grammar that the Qute HTML language uses.
It brings in the Qute grammar from qute.tmLanguage.json, brings in HTML grammar, and injects some parts of the Qute grammar into the HTML grammar.
qute.tmLanguage.json
This is the grammar file containing Qute specific grammar. This file does not contain anything specific to HTML.
Improvements:

That looks pretty good @xorye, nice job!
However, I found a case where the highlighting is missing, when the elvis operator (?:)  is involved:
<!DOCTYPE html>

<meta charset="UTF-8">
<title>Qute Hello World</title>

Hello {name ?: "world"}!

Awesome! Well done @xorye !


Override C# binary operators for syntactic sugar

Hello.
I was curious, is there a specific reason Sprache doesn't overload C# binary operators to make the syntax more fluid?
E.g. Parse.Char('a') | Parse.Char('b') could evaluate to Parse.Char('a').Or(Parse.Char('b')).

Unfortunately, it's not possible in Sprache because Parser is a delegate. Operator overloads need to be in a non-static class, and one of the operator arguments needs to be its containing type. IIRC there might be an additional problem using generic types with operator overloads, but I might be wrong about that.

Ah, that's a shame. Do you perhaps see this as a potential improvement for the next major version? Instead of being a delegate, Parser<T> can be a class that has methods Parse()/TryParse(), which would allow for operator overloading.
Personally, I think this could provide tremendous benefit for readability:

Parsers can appear on the same "level"; not having to write an extra set of parentheses
Already familiar semantics of the operators can make it easier to understand the code

Were Parser a class, I think overridden operators may be achievable, but I'm not sure - I know that operator overrides can't use generic types, and I don't know how that impacts operators in classes with generic types. If you're able to get a working proof of concept, I would be interested in that personally - I can think of plenty of parsers I've written that could look a lot prettier if you only gave me | for Or(). I don't know whether or not I'd be in favor of that in Sprache; turning Parser into a class probably breaks a healthy amount of code written against older versions of Sprache. Alas, I'm not one of the owners of Sprache so I'm not the one who needs to be convinced in the end, haha!

Done

Please add more overloads