Mariadb vs MySQL

My current live site is running on mysql 5.5 and it appears this set of containers uses mariadb 10.1.
To your knowledge should I have any issue importing into mariadb from mysql production exports and then re-exporting from mariadb to import back into mysql in production?
If this is unknown is there any way to swap out a mysql container for your mariadb to ensure parity of imports/exports from my live environment to dev?
By the way, I just want to say I really appreciate the work that's been put into this. It makes setting up a wordpress stack on windows so much cleaner.

See this article about MariaDB and MySQL compatibility. And you can always replace MariaDB image with your own or with official MySQL image by modifying compose file though we can't guarantee everything would work fine.


Setting Memory Limit/Request values through spark operator

I would like not to set memory Limit/Request values for spark executor PODs
based on https://github.com/kubernetes/kubernetes/issues/86171 .
I would like not to set container_memory_working_set_bytes to fix OOM issue. How can I achieve it though spark operator?

@liyinan926 @mrow4a Any suggestions on this


OverlayManager class

@kurtzmarc @neilboyd i was looking into OverlayManager class and found these lines
if (mTilesOverlay != null && mTilesOverlay.isEnabled()) {
mTilesOverlay.draw(c, pMapView, true);
}
    if (mTilesOverlay != null && mTilesOverlay.isEnabled()) {
        mTilesOverlay.draw(c, pMapView, false);
    }

    for (final BaseOverlay overlay : mOverlayList) {
        if (overlay.isEnabled()) {
            overlay.draw(c, pMapView, true);
        }
    }

    for (final BaseOverlay overlay : mOverlayList) {
        if (overlay.isEnabled()) {
            overlay.draw(c, pMapView, false);
        }
    }

the boolean argument when passed true it return right away doing nothing.

I don't show any difference when i remove draw(c, pMapView, true);
Is Osmdroid drawing any shadow anywhere as i dont see can we just remove shadow thing??

As part of the Google Maps v1 API, Overlays get two passes to draw. Once to draw a shadow layer, and once to draw regular. A typical behavior during the shadow layer is to draw an icon in grayscale with a skew to give the illusion of 3D depth to the maps. Although we don't take advantage of this (maybe we do in the location overlay), the API specifies that we give Overlays the opportunity to do this. If you don't want to use the shadow layer, add a
 if (shadow)
     return;

to your draw() method.


Update sectors.erb

Updated text of retail, hospitality or leisure care sector to specify gyms
Added personal care sector
Reason: there is a new grant that includes those specifically
⚠ This repo is Continuously Deployed: make sure you follow the guidance ⚠

This was merged in as part of https://github.com/alphagov/smart-answers/pull/5271


Scroll to first available tile on map load

Hello, thank you for the effort you're putting on this.
It will be very nice, especially for long vertical maps that starts at the bottom, to scroll to the first visible tile on page load.
Thank you

Hi @sickDevelopers
Thanks for the thought :-) Definitely worth doing I think.. I'll keep you posted on when it might be available :-)


Unable to decode image: InvalidStateError

Attach (recommended) or Link to PDF file here:
org_AVA89V01U0$ (Black).pdf
Configuration:

Web browser and its version: Chrome Version 118.0.5993.89 (Official Build) (64-bit)
Firefox Version 119.0 (64-bit)
Edge Version 118.0.2088.69 (Official build) (64-bit)
Operating system and its version: Windows 11 Enterprise Version 10.0.22621 Build 22621
PDF.js version: 4.0.132 [34781121c] (test viewer)
Is a browser extension: No

Steps to reproduce the problem:

Open the file in Acrobat. Main area is blacked out but you can see the edges of a CAD drawing along the borders that includes various characters.
Open the file in PDF.js. Only the black rectangle in the center and the blue box in the upper right corner are rendered.

What is the expected behavior? (add screenshot)
This is the upper left corner at 200% zoom in Acrobat:

What went wrong? (add screenshot)
The console on Chrome:
PDF a8c52134c310cca513061a5a2ce6ca91 [1.3 - / -] (PDF.js: 4.0.132 [34781121c])
util.js:367 Warning: Unable to decode image "img_p0_1": "InvalidStateError: The source image could not be decoded.".
util.js:367 Warning: Dependent image isn't ready yet
The console on Firefox:
PDF a8c52134c310cca513061a5a2ce6ca91 [1.3 - / -] (PDF.js: 4.0.132 [34781121c]) app.js:1561:12
Warning: Unable to decode image "img_p0_1": "InvalidStateError: An attempt was made to use an object that is not, or is no longer, usable". pdf.worker.mjs:339:13
Warning: Dependent image isn't ready yet 3 util.js:367:12
The console on Edge:
PDF a8c52134c310cca513061a5a2ce6ca91 [1.3 - / -] (PDF.js: 4.0.132 [34781121c])
util.js:367 Warning: Unable to decode image "img_p0_1": "InvalidStateError: The source image could not be decoded.".
2util.js:367 Warning: Dependent image isn't ready yet
util.js:367 Warning: Dependent image isn't ready yet
This is the upper left corner at 200% zoom in Chrome:

Link to a viewer (if hosted on a site other than mozilla.github.io/pdf.js or as Firefox/Chrome extension):
https://mozilla.github.io/pdf.js/web/viewer.html

Most likely the huge image dimensions are the problem, given the following excerpt (taken from http://brendandahl.github.io/pdf.js.utils/browser/):
Resources (dict)
    ProcSet (array) [id: 5, gen: 0]
    XObject (dict)
        Im0 (stream) [id: 6, gen: 0]
            BitsPerComponent = 1
            ColorSpace = /DeviceGray [id: 7, gen: 0]
            DecodeParms (dict)
            Filter = /CCITTFaxDecode
            Height = 19866
            Length = 1110922
            Name = /Im0
            Subtype = /Image
            Type = /XObject
            Width = 28087
            <view contents> download
        wspe_X1 (stream) [id: 11, gen: 0]
            BitsPerComponent = 1
            ColorSpace = /DeviceGray
            Decode (array)
            Filter = /JBIG2Decode
            Height = 19866
            Length = 76004
            Subtype = /Image
            Type = /XObject
            Width = 28087
        wspe_X2 (stream) [id: 10, gen: 0]
        wspe_X3 (stream) [id: 13, gen: 0]
        wspe_X4 (stream) [id: 17, gen: 0]

Most likely the huge image dimensions are the problem, given the following excerpt (taken from http://brendandahl.github.io/pdf.js.utils/browser/):

So what (roughly) is the limiting factor that keeps the large image from being downloaded and processed? It looks like the promise tasked with fetching the image just quits at some point.

some problem，when run to the await bitmapPromise and createImageBitmap ,  show err msg:
1、Unable to decode image
2、The source image could not be decoded

I also get the same error but only for PDFs that are very large (130+ x 50+ in) and may or may not contain multiple page sizes (each page has different dimensions). My PDFs are the default outputs of using the utility TIFF2PDF.
What should we do with files that are too large? Is there a setting we can configure in PDF.js? Thanks!

I also get the same error but only for PDFs that are very large (130+ x 50+ in) and may or may not contain multiple page sizes (each page has different dimensions). My PDFs are the default outputs of using the utility TIFF2PDF.
What should we do with files that are too large? Is there a setting we can configure in PDF.js? Thanks!
Specifically, I'm using PDF.js in https://github.com/wojtekmaj/react-pdf v7.7.0.

Yes, this is our issue as well. Some further investigation has shown that both the image and the PDF are not very large in terms of memory, only in physical dimensions. Is it a matter of allocating memory for the bitmap within PDF.js?

As mentioned in https://github.com/mozilla/pdf.js/issues/17190#issuecomment-1781872862 the image size is the issue.
After having been decoded it'll require 19866 x 28087 x 4 (RGBA) bytes which is greater than 2**31 -  1 and in Firefox for example the max size of an image is gfx.max-alloc-size (see about:config or https://searchfox.org/mozilla-central/source/modules/libpref/init/StaticPrefList.yaml#6174).
We could probably split the image into several ones, resize them and draw the smaller versions on the same resized canvas.

Has there been any progress on this issue?

As mentioned in #17190 (comment) the image size is the issue. After having been decoded it'll require 19866 x 28087 x 4 (RGBA) bytes which is greater than 2**31 - 1 and in Firefox for example the max size of an image is gfx.max-alloc-size (see about:config or https://searchfox.org/mozilla-central/source/modules/libpref/init/StaticPrefList.yaml#6174). We could probably split the image into several ones, resize them and draw the smaller versions on the same resized canvas.

Chrome's built-in PDF viewer is able to handle the sample document.


Memory Leak??

(node:2440) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 11 resize listeners added to [WriteStream]. Use emitter.setMaxListeners() to increase limit
Steam Guard Code:
I put the auth as '', this is what happens.

Manually inputting steam guard codes in the config file works, suggest not having an input but consider this fixed.


[DRAFT] Minimal QA check on repository

Just to get started, I looked at CONTRIBUTING.md and saw that the linting is not passing currently, so I went ahead and incorporated the minimal steps to lint this project
What is possibly missing from this PR

Knowing what versions of NodeJS this should support, I set it to 20.x and 22.x
Making sure it fails if package.json and package-lock.json disagree
Incorporating the release.sh command, or part of it, into the global npm run build
Marking the web as a module with npm workspaces. that would prevent having two different package-lock.json and could simplify running everything, npm install globally would work and then a task to go along the launch.json for development

Also, the linting is not passing currently. For example
src/globals.js
  13:66  error  '_' is defined but never used      no-unused-vars
  17:11  error  'error' is defined but never used  no-unused-vars

Looks great, thanks! Am I understanding this correctly that this will always run two jobs now, one for node 20 and one for node 22? If so, you can drop the 20 one IMO. Personally I'm always using the latest stable version, so 22 right now, and any more than that would be an overkill.
The other points you have mentioned make sense too, but none of them really critical. We can at least make an issue out of them though.
Next to the linter, a TypeScript type check would be important, so npx tsc -p jsconfig.json --noEmit in the root dir. However, that seems to fail to ignore node_module contents and fails to type check *.vue file contents, so probably https://github.com/vuejs/language-tools/tree/master/packages/tsc vue-tsc is required for the latter.
Can't really investigate much right now. Apart from the Node20 thingy, this PR is good to merge for now.
And yeah, currently there's a lot of type and lint errors in the code. Guess I'll have to start taking the rules more serious myself :) and fix them soon after merge.

Ok.
In that case, I will make that change to only target 22.x.
I can look at making the compilation do type checking on another PR. I have very little experience with vue however.

ok!


Problem following introduction

I followed this guide:
https://sematext.com/blog/2017/03/20/sematext-solr-autocomplete-introduction-and-howto/
but I had problems at the core creation step. The guide left out which files to grab, so I copied solr-autocomplete/example/solr/collection1. This is the error I received:
solr-6.3.0$ bin/solr create -c autocomplete -d ./server/solr/collection1/conf/

Connecting to ZooKeeper at localhost:9983 ...
Uploading /home/aw/Downloads/solr-6.3.0/server/solr/collection1/conf for config autocomplete to ZooKeeper at localhost:9983

Creating new collection 'autocomplete' using command:
http://localhost:8983/solr/admin/collections?action=CREATE&name=autocomplete&numShards=1&replicationFactor=1&maxShardsPerNode=1&collection.configName=autocomplete

ERROR: Failed to create collection 'autocomplete' due to: {<IP_ADDRESS>:8983_solr=org.apache.solr.client.solrj.impl.HttpSolrClient$RemoteSolrException:Error from server at http://<IP_ADDRESS>:8983/solr: Error CREATEing SolrCore 'autocomplete_shard1_replica1': Unable to create core [autocomplete_shard1_replica1] Caused by: org.apache.solr.handler.dataimport.DataImportHandler}

It's also unclear also where to get the schema.xml file from, since one doesn't appear to exist in this project. I tried renaming schema-function-query-ordering.xml to schema.xml and there was no change to the error above. I also obtained this error with solr-6.4.0. before I downgraded to solr-6.3.0 to see if that would make a difference. It didn't. Please advise.

You should be good with using the solr/collection1/conf/ directory from the solr-autocomplete clone. I've updated the guide to make that clear, sorry for being vague there.

Your guide doesn't work. Have you tried it yourself from a fresh install?
I also tried adding in a data-config.xml file (it's not in collection1/conf) that mirrors the readme but that didn't help either.

I did try it from a fresh install to write down the commands for writing the blog post. I will check it again, maybe I've been missing something.

It looks like I did a major oopsie on this one, sorry for that. I forgot that I made some changes and was under the impression that I had a clean install.
The error you saw is likely because it misses the DataImportHandler class (you'd see that in the log), which in turn is because the sample solrconfig.xml doesn't load the jar that contains it.
Also, the example configuration that's meant to be used is in example/solr/collection1/conf (note the addition of example there) which has a sample data-config.xml, too (though we're not using it in this tutorial - it would need a few tweaks to be fully functional, depending on how you'd have your source core/collection).
Anyway, if you clone the repo again or update it, then follow the tutorial, it should work. Please let me know if you hit any more issues, I'm closing this one for now.


ERR_SPDY_COMPRESSION_ERROR Caddy 0.9.3

1. What version of Caddy are you running (caddy -version)?
0.9.3
2. What are you trying to do?
Open a Symfony application served via caddy.
3. What is your entire Caddyfile?
https://hum.localhost:8000
root /Users/MYUSER/projects/MYAPP/web
gzip

tls self_signed

rewrite {
  to {path} {path}/ /app_dev.php?{query}
}

fastcgi / <IP_ADDRESS>:9000 php {
  index app_dev.php
}

4.
Just ran caddy in my app directory via fish shell 2.3.1 on macOS 10.12.1.
5. What did you expect to see?
The running web app.
6. What did you see instead (give full error messages and/or log)?
Chrome's (Version 54.0.2840.87 (64-bit)) error page ERR_SPDY_COMPRESSION_ERROR:

Does this error occur without the fastcgi directive?

No, without fastcgi the php source code is shown in the browser (as the application isn't interpreted anymore).

Okay, so the error seems to have something to do with the fastcgi middleware and/or your backend. Can you provide some details about your backend and even the source code of the page maybe so we can reproduce it?

We have this on every Symfony 2 project on macOS machines since Sierra (doesn't occur on linux, didn't happen on El Capitan). I can't provide source code for now since all are corporate projects.

That's fascinating, that it only happens on Sierra. Symfony bug (incompatibility with Sierra maybe)?

It works when running Apache via https. It just happens with the combination of Sierra and Caddy - very strange.

Are you running Apache with HTTP/2 enabled?

Closing pending answering of question ;)

Sorry, missed it. Yes, I was running it with mod_http2.

Okay. So I don't know what the problem would be. :-/


Relation embedder lambda infra build

What does this change?
Part of https://github.com/wellcomecollection/catalogue-pipeline/issues/2770
Adds relation_emb lambda to the build script, terraform modules and the shell of a relation_emb lambda handler
How to test
Build the image, eg. docker build --target lambda_rie -t lambda_relation .
Run docker run -p 9000:8080 lambda_relation
Run cat scripts/batches.txt | python scripts/post_to_rie.py and see the lambda getting hit
TF plan Plan: 13 to add, 2 to change, 0 to destroy.
13 to add are all relation_embedder_lambda resources
2 to update:
Terraform will perform the following actions:

  # module.pipeline.module.inference_capacity_provider.aws_autoscaling_group.asg will be updated in-place
  ~ resource "aws_autoscaling_group" "asg" {
        id                               = "catalogue-2024-11-05_inferrer_asg"
        name                             = "catalogue-2024-11-05_inferrer_asg"
        # (31 unchanged attributes hidden)

      ~ launch_template {
            id      = "lt-0ae23e659ef16ca3a"
            name    = "catalogue-2024-11-05_inferrer_launch_template"
          ~ version = "3" -> (known after apply)
        }

        # (2 unchanged blocks hidden)
    }

  # module.pipeline.module.inference_capacity_provider.aws_launch_template.launch_template will be updated in-place
  ~ resource "aws_launch_template" "launch_template" {
      ~ default_version                      = 3 -> (known after apply)
        id                                   = "lt-0ae23e659ef16ca3a"
      ~ image_id                             = "ami-082bbd3675c6d658c" -> "ami-09e2d602becc5f1fa"
      ~ latest_version                       = 3 -> (known after apply)
        name                                 = "catalogue-2024-11-05_inferrer_launch_template"
        tags                                 = {}
        # (16 unchanged attributes hidden)

        # (3 unchanged blocks hidden)
    }

How can we measure success?
We can deploy a relation_embedder_lambda alongside the service
Have we considered potential risks?
Very low to inexistent

Adding the ES index, tf plan
Plan: 20 to add, 69 to change, 6 to destroy.
I'm a bit confused as to why all the module.output_topic.aws_iam_role_policy.task_sns for the other services need updating though

Added a commit here to remove some unused variables, and change a type so this plans cleanly. Shall we try and apply it to the new pipeline next week and see if it behaves as expected?
You may want to separate the Dockerfile and terraform entrypoint changes into another PR as they are a change that can be made safely with the expectation that the existing ECS service will continue to work.


MIREOT 'Eastern equine encephalitis virus' from NCBITaxon

Should have closed with commit d64212c, but didn't because of typo.


macast-debug: Add version 0.7

Debug version for macast, with command-line output.

Closes #XXXX

Relates to #7862 and  #7758

[ -] I have read the Contributing Guide.

/verify

I think this needs to be in Versions bucket.

I'm fine with adding this here, since it's stable after all.

I'm fine with adding this here, since it's stable after all.
Ok. Let's keep it here.


Readme says Blackboard instead of Brightspace :)

Also, the Mitchell (1997) link is broken, but you can find it here or here


Cheatsheet InitAudioStream not up to date

In version 4.0.0 InitAudioStream was renamed to LoadAudioStream but the cheatsheet was not updated.

@wbenke Feel free to send a PR with the update, just need to modify https://github.com/raysan5/raylib.com/blob/master/cheatsheet/raylib_audio.c.


Rename DockerRunCommand

based on conversation in https://github.com/fedora-modularity/conu/pull/61#discussion_r152566588

since this would change the API, adding to 0.1.0


no possibility to specify the baseline.xml

For running from gradle I have a detekt-baseline.xml file. The plugin does not know about this, so blacklisting files in the baseline has no effekt for the plugin.
Is it possible to add a configuration for the baseline as well?

I started to look at it but it implies modification to detekt repository as well, since ProcessingSettings does not expose baseline yet.

Take a look at https://github.com/arturbosch/detekt/blob/3543c2fe1aa46b579aac61371a13b9faf23c6e95/detekt-cli/src/main/kotlin/io/gitlab/arturbosch/detekt/cli/baseline/BaselineFacade.kt
The core module does not know about a baseline and though does not ProcessingSettings.
The cli module post process the findings like via the BaselineFacade. Hope that helps.
Thanks for your work!


Tatin user command does not work after running ]tatin.updatetatin

Tatin user command does not work after running ]tatin.updatetatin

I install Dyalog 19.0.48959 Unicode 64-bit.
]activate all
)off and restart dyalog
]tatin.updatetatin

Your version of Tatin is 0.104.1
Latest published version is 0.112.0
Would you like to download and install 0.112.0 ? (Y/n) Y
Tatin successfully updated to version 0.112.0 in C:\Users[...redacted...]
)off and restart dyalog

⍝  now Tatin does not work
]tatin.version

Invalid user command; to see a list of all user commands type
] -?
]tools.version

Dyalog  19.0.48959 64-bit Unicode, BuildID abfdd505
OS      Windows 11 (10.0.22631) 64-bit
Link    4.0.17
SALT    2.915
UCMD    2.6
.NET    .NET Framework 4.8.9241.0
WS      19.0
Tatin   0.112.0
Cider   0.39.0

Please try ]Ureset, that should fix it.
It appears to be a SALT problem: why does SALT not recognize that something has changed, as it should?


Make it possible to retrieve external task error Details

Good day,
Is there any way to obtain errorDetails for a failed external task? The ExternalTaskInfo class does not have errorDetails property and I do not see any action available for External Tasks service in this client library that can be able to obtain that information.
Using postman I have checked that I have to call the following endpoint to obtain the error details for failed external task which works fine:
GET /external-task/{id}/errorDetails.
Here I find the other issue - according to the Camunda REST API Reference documentation there is no such endpoint available at all. Does that mean that retrieving errorDetails for a failed external task is not officially supported?

Resolved by adding ErrorDetails to ExternalTaskInfo https://github.com/jlucansky/Camunda.Api.Client/commit/6aafcd1fc3c1fdcd739a5df33791fb076e62d8f0


vechicle.connect() timeout

I am trying to connect to solo with below code;
vehicle = connect("udpin:<IP_ADDRESS>:14550", wait_ready=True, baud=57600, heartbeat_timeout=30) or
vehicle = connect("udpin:<IP_ADDRESS>:14550", wait_ready=True)
vehicle = connect("udpin:<IP_ADDRESS>:14550", wait_ready=False)

My environment is
pymavlink==1.1.73
dronekit==2.5.0
monotonic==1.0

I get the following errors when running on the solo
ERROR LOADING MAVNATIVE - falling back to python implementation
Connecting to udpin:<IP_ADDRESS>:14550...
>>> Link timeout, no heartbeat in last 5 seconds
>>> No heartbeat in 30 seconds, aborting.
Traceback (most recent call last):
  File "helloworld.py", line 10, in <module>
    vehicle = connect(target)
  File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/__init__.py", line 2813, in connect
    vehicle.initialize(rate=rate, heartbeat_timeout=heartbeat_timeout)
  File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/__init__.py", line 2086, in initialize
    raise APIException('Timeout in initializing connection.')
dronekit.APIException: Timeout in initializing connection.

@janthony, the error you are seeing means that whatever is invoking the script cannot connect to solo on udpin:<IP_ADDRESS>:14550. where are you running the script from?
my scripts running on my laptop connect to solo just fine.

My script runs fine from my laptop as well. But when I run from the solo
computer, then it fails everytime.
On Sun, 10 Jul 2016, 4:06 AM Ahmed Agbabiaka<EMAIL_ADDRESS>wrote:

@janthony https://github.com/janthony, the error you are seeing means
that whatever is invoking the script cannot connect to solo on udpin:
<IP_ADDRESS>:14550. where are you running the script from?
my scripts running on my laptop connect to solo just fine.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/dronekit/dronekit-python/issues/645#issuecomment-231547472,
or mute the thread
https://github.com/notifications/unsubscribe/ABxdF4pGi9l5U1L_WKAEZorRaZjFIL6gks5qT-MNgaJpZM4JInGF
.

--
Jerome Anthony

@janthony i can confirm. the same script that runs on the computer connects to solo and runs just fine. if i move that script to solo and run it, the script can't connect and times out. the error occurs if i run the script directly via python script.py or via solo script pack.
via solo script run
Exception in thread Thread-2 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 504, in run
File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/mavlink.py", line 166, in mavlink_thread_out
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'error'
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 504, in run
File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/mavlink.py", line 201, in mavlink_thread_in
File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/init.py", line 1311, in listener
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'monotonic'

On Sun, 10 Jul 2016, Ahmed Agbabiaka wrote:

@janthony i can confirm. the same script that runs on the computer connects
to solo and runs just fine. if i move that script to solo and run it, the
script can't connect and times out. the error occurs if i run the script
directly via python script.py or via solo script pack.
via solo script run
Exception in thread Thread-2 (most likely raised during interpreter
shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 504, in run
File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/mavlink.py",
line 166, in mavlink_thread_out
: 'NoneType' object has no attribute 'error'
Exception in thread Thread-1 (most likely raised during interpreter
shutdown):
Traceback (most recent call last):
File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
File "/usr/lib/python2.7/threading.py", line 504, in run
File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/mavlink.py",
line 201, in mavlink_thread_in
File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/__init.py",
line 1311, in listener
: 'NoneType' object has no attribute 'monotonic'

OK, so sadly these exceptions don't indicate the source of your issues.
Note the message "during interpreter shutdown", and "'Nonetype'
object..."; the "None" objects there are python modules.  These errors
occur because the main thread is busy deconstructing itself and the
interpreter, but the mavlink threads are still running.
These threads should really be shut down nicely before we quit, but
aren't.
Nobody has gone and fixed that as by the time you get them your program
has finished and it is just messy.  OTOH, it does lead to distractions
from the real issue - like the one you're facing.
Could you give me the exact incantation of solocli you're using, please?
Peter

@peterbarker Thanks for taking your time to reply. The command I run from my computer is
solo script run helloworld.py
To short circuit the debug times, I tried ssh into solo and cd into the script dir and run the python helloworld.py from there.
Both fail for me. I am trying to figure out a way to debug the issue on the solo itself. If you have any pointers as to how I can go about debugging, I could dig into this deeper.

On Sun, 10 Jul 2016, Jerome Anthony wrote:

@peterbarker Thanks for taking your time to reply. The command I run from my
computer is
solo script run helloworld.py

OK, so that one runs fine for me.  Old version of Solo's software,
'though.  I'll update my solo and see if something broken in the recent
Solo releases.

Both fail for me. I am trying to figure out a way to debug the issue on the
solo itself. If you have any pointers as to how I can go about debugging, I
could dig into this deeper.

What does "python helloworld.py" do for you?

On Sun, 10 Jul 2016, Jerome Anthony wrote:

@peterbarker Thanks for taking your time to reply. The command I run from my
computer is
solo script run helloworld.py
To short circuit the debug times, I tried ssh into solo and cd into the
script dir and run the python helloworld.py from there.
Both fail for me. I am trying to figure out a way to debug the issue on the
solo itself. If you have any pointers as to how I can go about debugging, I
could dig into this deeper.

After updating my Solo, I now get a timeout with helloworld.py
3DR may have broken something with telem_forwarder - I no longer have
access to that code.
What does work is this:
diff --git a/examples/helloworld/helloworld.py
b/examples/helloworld/helloworld.py
index b51d02c..89b403d 100644
--- a/examples/helloworld/helloworld.py
+++ b/examples/helloworld/helloworld.py
@@ -2,7 +2,8 @@ from dronekit import connect
import sys
Connect to UDP endpoint (and wait for default attributes to accumulate)
-target = sys.argv[1] if len(sys.argv) >= 2 else 'udpin:<IP_ADDRESS>:14550'
+#target = sys.argv[1] if len(sys.argv) >= 2 else 'udpin:<IP_ADDRESS>:14550'
+target = sys.argv[1] if len(sys.argv) >= 2 else 'udpout:<IP_ADDRESS>:14560'
print 'Connecting to ' + target + '...'
vehicle = connect(target, wait_ready=True)
I'm wondering if 3DR might have nixed those broadcasts on Solo's localhost
interface or something.....
See if that patch works for you?

@peterbarker Thanks. Will test this and get back to you. I thought this connection string you specified,
udpout:<IP_ADDRESS>:14560
is what is used when you are proxying through MapProxy

@peterbarker i am running
solo-cli 1.2.0
dronekit 2.7.0
solo fw 2.4.0
if i run helloworld from solodevguide/examples via solo script run, the script runs (udpin:<IP_ADDRESS>:14550) and i can see all of the vehicle attributes. but on vehicle.close, the script doesn't exit graciously.

@franknitty69 I installed solo-cli 1.2.0. But when I try solo --version it says version solo-cli 1.1.2. Does yours print solo-cli 1.2.0 ?

if i use pip list, solo-cli is 1.2.0. if i use solo --version it says 1.1.2. i'm not sure where the discrepancy is.

On Mon, 11 Jul 2016, Jerome Anthony wrote:

@peterbarker Thanks. Will test this and get back to you. I thought this
connection string you specified,
udpout:<IP_ADDRESS>:14560
is what is used when you are proxying through MapProxy

That's a "pymavlink-style" connection string.  It is passed through to
pymavlink pretty much unmolested by both dronekit-python and mavproxy.
The thing listening on Solo's :14560 is not mavproxy IIRC - it's
telem_forward.

@peterbarker I reset my solo/controller and then the connection timeout went away. I believe the problem was after the software update I had not paired the solo to the controller (strangely I could control the gimbal using the controller, which contradicts this theory).
But I get the following error of script not ending successfully;
ERROR LOADING MAVNATIVE - falling back to python implementation
Using MAVLink 1.0
Connecting to udpout:<IP_ADDRESS>:14560...
>>> PreArm: Waiting for Nav Checks
>>> APM:Copter solo-2.0.20 (609645b5)
>>> PX4: bf80a499 NuttX: a5146c68
>>> Frame: QUAD
>>> PX4v2 0034001C 31355104 36333436
>>> PreArm: Waiting for Nav Checks
>>> PreArm: Waiting for Nav Checks
Vehicle state:
 Global Location: LocationGlobal:lat=xxx,lon=xxx,alt=96.86
 Global Location (relative altitude): LocationGlobalRelative:lat=-xxx,lon=xxx,alt=1.66
 Local Location: LocationLocal:north=None,east=None,down=None
 Attitude: Attitude:pitch=-0.0601995512843,yaw=-0.326566100121,roll=-0.010718235746
 Velocity: [-0.01, 0.01, -0.01]
 Battery: Battery:voltage=15.796,current=0.7,level=74
 Last Heartbeat: 0.856388333
 Heading: 341
 Groundspeed: 0.0399999991059
 Airspeed: 0.0399999991059
 Mode: LOITER
 Is Armable?: True
 Armed: False
>>> PreArm: Waiting for Nav Checks
>>> PreArm: Waiting for Nav Checks
>>> PreArm: Waiting for Nav Checks
>>> PreArm: Waiting for Nav Checks
Done.
Exception in thread Thread-2 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 504, in run
  File "/log/solo-script/env/lib/python2.7/site-packages/dronekit/mavlink.py", line 166, in mavlink_thread_out
<type 'exceptions.AttributeError'>: 'NoneType' object has no attribute 'error'
Exception in thread Thread-1 (most likely raised during interpreter shutdown):
Traceback (most recent call last):
  File "/usr/lib/python2.7/threading.py", line 551, in __bootstrap_inner
  File "/usr/lib/python2.7/threading.py", line 504, in run

This happens both with udpout:<IP_ADDRESS>:14560 and udpin:<IP_ADDRESS>:14550

On Tue, 12 Jul 2016, Jerome Anthony wrote:

But I get the following error of script not ending successfully;

Yes, those threads aren't terminated correctly.  I think I have to do
something about that  - somehow.  Last time I looked it was complicated
due to being daemon threads.

I can contribute to debugging and fixing if you can give a few pointers as to how to proceed.

Do you think you could test that one for me, as it seems you had the interpreter-shutdown error happening quite reliably.

@peterbarker Sure. I will test it and let you know

I have tested this and it works. Thanks for the fix.

i tested the fix as well and can confirm that dronekit-python scripts now exit gracefully.

This was merged in - thanks @franknitty69!

Note that the original issue here - timeout in connect() on Solo - was solved by @janthony with a reset of both Solo and Artoo.


.NET Core 1.0?

Any possibility of getting this ported to .NET Core 1.0? I'd like to use it with some lambda functions.

Version <IP_ADDRESS> will have a .NET 8 compilation target


Clear drag preview material on NOTIFICATION_DRAG_END

Related to: #87125
Currently when dragging a material onto a target it will still apply even after the action 'ui_cancel'. This PR will prevent that.

Cherry-picked for 4.3.1.


Pools fetching and fees calculation for not null withdrawal param takes 3-10mins

Context

Information
-

Version
v2020-08-03

Platform
Linux

Installation
From binaries

Steps to Reproduce

Try to calculate fee with active "withdrawal" param or try to fetch pools list  in the mainnet.
...
PROFIT!

Expected behavior
Waiting for couple seconds for fee calculation and 10-20 seconds for pools (around 800 for now).
Actual behavior
You can have some coffee and watch something funny on the YouTube before response.
Comment
Pools/fees fetching takes up to 10 mins, but, actually in pool's case it's not a problem - I do it as background task and then just  put it to a caching layer. So, at application level pools fetching takes no time for users. But fees are killing the application. I can't prefetch them by obvious reason.
So It's impossible to implement rewards withdrawal for the mainnet because of this issue.
But I've submitted both problems because looks like they a related.
Resolution

QA

@Fell-x27 would you mind opening two separate bug tickets for these two separate issues please :pray:
Regarding pool fetching, are you doing this while syncing? (cf: https://github.com/input-output-hk/cardano-wallet/issues/1846). One of the problem regarding pools listing is that, in order to reply to such request, cardano-wallet needs a few information from the node it's connected to and, the node won't reply to any request regarding the Ledger state until it is within k blocks from the network tip (where k stands for the security parameter here, 108 in the Shelley era). Which makes a lot of the queries look slow.
As for fees, this looks indeed quit abnormal. May you open a separate ticket for this one and if possible, provide your UTxO summary / statistics? If not, perhaps simply an order of magnitude of your wallet's size?

Regarding pool fetching, are you doing this while syncing?
No. The node and wallets are perfectly synced. But node fetches pools list up to 10 mins.

May you open a separate ticket for this one
Sure

and if possible, provide your UTxO summary / statistics? If not, perhaps simply an order of magnitude of your wallet's size?
I can't. But  tested it by myself. It doesn't depend on wallet size\UTxO amount\Reward amount. The same result for funded wallet without reward, rewarded without UTxO, and even for zero-balanced one.

Probably, both problems are related with the count of wallets because there are dozens. But only pools and fees with withdrawal included are so slow.

@KtorZ , ^^^


fix: design updates

Align with latest designs

:tada: This PR is included in version 1.23.2 :tada:
The release is available on GitHub release
Your semantic-release bot :package::rocket:


Sidebar, Playground, Navbar, and other Doc website elements

Feature request 🚀

[ ] I will create Pull Request
[x] It's just a suggestion

Expected

Could the elements from the documentation website be added to the library? The sidebar for example would be really useful. Same with the sticky navbar.
The playground component with the expandable code editor looks like an extension of <Fieldset>, just with a copy button and collapsible footer. Perhaps that functionality could be rolled into it with a collapse prop?

Add borderless prop to <Button> to enable recreating the copy button from the playground.
Could <Textarea> please get a code prop to style the text like in <Code> or <Snippet>? I've been doing this with CSS classes, but a simple prop would be really convenient.

Examples
<Fieldset>
    ...
  <Fieldset.Footer collapse>
    <Fieldset.Footer.Status>
      <Text small>Code Editor</Text>
    </Fieldset.Footer.Status>
    <Fieldset.Footer.Actions>
      <Button iconRight={<Copy />} auto borderless size="small" />
    </Fieldset.Footer.Actions>
    <Fieldset.Footer.Content>
      <Textarea width="100%" code />
    </Fieldset.Footer.Content>
  </Fieldset.Footer>
</Fieldset>

Thanks and much ❤, geist-ui is the best

These functions are a kind of customization, not the smallest unit that can be abstracted in the layout, and we don't want to add these elements to the component library for now.
Thank you for the kind words, sorry this feature is not available.


Multilingual prepare.sh throws an error after downloading

While running prepare.sh the followign errors are thrown for all the languages in multilingual directory:
FileNotFoundError: [Errno 2] No such file or directory: '/home/crocoder/Desktop/transformers/PLBART/multilingual/data/processed/valid.php-en_XX.php'
Traceback (most recent call last):
  File "encode.py", line 92, in <module>
    main()
  File "encode.py", line 88, in main
    process(args)
  File "encode.py", line 49, in process
    with open(args.input_source, 'r', encoding='utf-8') as f1, \
FileNotFoundError: [Errno 2] No such file or directory: '/home/crocoder/Desktop/transformers/PLBART/multilingual/data/processed/test.php-en_XX.php'

Any help on this?

@wasiahmad Thanks!
I will check again the order in which things are run and get back. My processed directory is empty after download.sh. Will see what is causing the issue.

The actual issue was that libclang-6.0 is needed. Probably a dependency for some package involved.
Doing the following fixed the issue:
sudo apt-get install -y libclang-6.0-dev

Ah I see, the tokenizer supports the C++ language too. So, libclang is required. Perhaps, I should remove it since none of the downstream tasks is involved with the C++ language.


README.md: Fix coveralls badge link

Coverage remained the same at 72.951% when pulling 4662ccb80f0eaeb557998e7eaff94248175cba9d on fix-coveralls-link into abaea936fff642898078224c3df0405275ec148d on master.


Fix #332: Remove .settings files

Pull request type

Bugfix (:fire: hotfix!)

What is the current behavior?

Issue Number: #332, .settings files may impede clean import into Eclipse on new machines.
What is the new behavior?

Remove all .settings folders and *.pref files.
Ignore .settings/ folders in .gitignore

Does this introduce a breaking change?

No.
Pull request checklist
Please check if your PR fulfills the following requirements:

[ ] Tests for the changes have been added or the PR is neither a bug fix nor a feature
[ ] Docs have been reviewed and added / updated if needed or the PR is neither a bug fix nor a feature
[ ] The Unreleased section in CHANGELOG.md has been amended to reflect the changes in this PR if needed
[ ] Build (mvn verify) was run locally and any changes were pushed
[ ] The pull request is against the correct branch (master for bug fixes, develop for new functionality)
[ ] Dependencies and citation templates have been updated where necessary or there are no new dependencies

Checklist for maintainers
Releases

Do NOT push the green merge button on GitHub.
Follow the latest version of the developer/maintainer docs for making releases.
Check that license and citation information for dependencies are complete (each dependency has a folder in /THIRD-PARTY/).

Closing due to branching issues.


Gmail virus detection

Gmail virus detection now blocks SeleniumVBA.xlam when trying to attach it into a new message:

I found that this line of WebDriverManager is the only thing responsible for this false positive:

I also found that executing the same file with other methods, like Shell or wsh.Run, is sufficient to not trigger this false positive.
However, these methods don't offer a method like StdOut.ReadAll to read the output string.
Does someone know a better way to read the output with alternative methods than by writing and reading the output to a file or the clipboard?

Good question Diego.
I just tried sending both the .xlsm and .xlam versions between my two gmail accounts and neither was detected as virus.
When Defender was detecting a virus in the add-in quite a while ago, I tried removing many possible triggers but what made it so difficult to narrow down the problem line(s) was repeatability. I would try removing a code line, scan with Defender, then upload to a private repo on Github, then download - ahh - success! Then an hour later Defender (same version!) detects a virus in the downloaded file. I finally gave up because it was taking so much time, and I seemed to be getting nowhere - trying to hit a moving target. Anyway, I hope you've found the problem line and then we can find a solution, but unfortunately, I'm not able to replicate the issue on my end right now.
I'll keep trying.
Cheers.

Ah, wait, then it's important that I add the information that my SeleniumVBA.xlam file doesn't just contain the SeleniumVBA modules but also contains many other modules of mine.
The fact that removing only that wsh.Exec line from WebDriverManager made Gmail no longer detect the virus led me to believe that my additional modules weren't relevant. But apparently they are :(
I just tried again with the original SeleniumVBA.xlsm file downloaded from GitHub, and it doesn't give "Virus detected". I also tried saving it as xlam, and it doesn't give "Virus detected" either. Same as you.
At this point, I'm inclined to think that Gmail's antivirus might assign a "suspicion score" to multiple commands/lines in my file, where the wsh.Exec line just adds points that make it exceed a critical threshold. In other words, this line alone wouldn't be enough to exceed the critical threshold, but it does so in combination with other commands/lines in my xlam file.
(This doesn't explain your case when just the timing seemed to be relevant, anyway)


The middleground of these two priorities is the current sync negotiation. This issue proposes a more aggressive form of this sync negotiation, possible with no protocol changes, but would benefit the syncing node somewhat disproportionally.
Syncing node:

Node A sends block locator hashes to Peer B. (and others in parallel, omitted for brevity onward)
Peer B sends it's 64 sync block proposed hashes to Node A.
Node A simultaneously requests the 64 blocks from Peer B, and immediately reforwards the sync hashes as block locator hashes to Peer B.
Peer B sends the first batch of 64 blocks, and the next 64 sync hashes simultaneously.

We can continue our parallel node approach to load balance block transfers. This change should improve sync time significantly by eliminating a lot of network overhead, and basically being continuous.

An implementation was filed in https://github.com/AleoHQ/snarkOS/pull/1120.


Should be able to pass additional FDs

Right now there is not way to get additional FDs to be passed from parent to daemon.
My goal is to open TCP socket, then get FD (using TCPListener.File()), then pass that FD to the daemon and convert it back to Listener (using net.FileListener())
I think this should be totally doable and not hard to implement, so I don't see why it was not there from the start. There are more uses to this functionality than just this.

@akamensky Hey man, did you get a implement now? I want the feature you mentioned too

@sherlockblaze Nope, to get that working I would need to rewrite large part of this package.

@akamensky I think I got it already.
When you use the lib to start a daemon thread, you should set Args for it.
cntxt := &daemon.Context{
		PidFileName: "pid",
		PidFilePerm: 0644,
		LogFileName: "log",
		LogFilePerm: 0640,
		WorkDir:     "./",
		Umask:       027,
		Args:        []string{"[go-daemon sample]", "test"},
	}

And the Args just like the command line you use. Like nginx start, arg[0] is ngxin and arg[1] is start and like. So, you just need to set the args with your command by using a golang slice, you can achieve your goal.

@sherlockblaze command line arguments have nothing to do with file descriptors...

@akamensky sorry man, I misunderstood your purpose


How to find a translated RFC

As a reader,  all RFCs look the same in the sidebar. How to identify that a RFC has a Chinese version? Perhaps add a prefix emoji like following would help:

2341-const-locals
✅ 2342-const-control-flow
2344-const-looping
2345-const-panic

cc @CYBAI @wusyong

I will propose to have a separated folder which might be clearer and it can be toggled to open in mdBook directly 👀
(btw, I'm wondering if it's good to have some triage labels so that we can find some issues quicker? this is off-topic to this issue though)

People who visit this book are looking for translated pages. Perhaps just remove all untranslated pages and leave some notes on the front page to tell them feel free to open a PR or issue if the RFCs they looking for aren't translated yet.

Fixed by 31eae2d2c81135febbd940ea37c076d1590a7ae6


Updates README with info on using in a node environment

Hey! Thanks for the great library! I'm using this for a project and thought this info might be helpful to others. Let me know if it works.

Thanks!


Submodules with port in the url result in wrong link

We DO NOT take questions or config/deploy problems on GitHub, please use our forum: https://discuss.gogs.io
Please take a moment to search that an issue doesn't already exist.
For bug reports, please give the relevant info:

Gogs version (or commit ref): <IP_ADDRESS>3
Git version: 2.1.4
Operating system: Debian Jessie amd64
Database:

[ ] PostgreSQL
[ ] MySQL
[ x] SQLite

Can you reproduce the bug at http://try.gogs.io:

[x ] Yes (provide example URL)
[ ] No
[ ] Not relevant

Log gist:

Submodules with port in the url result in wrong link
The example shows this: https://try.gogs.io/noppl/testsubmodule
The commands for adding the submodules were:
git submodule add ssh://git@try.gogs.io:22/c29316/myyy.git _submodulewithport1
git submodule add<EMAIL_ADDRESS>_submodule
...
Neither of them show the repository as intended. The submodule with ssh port shows the port as path in the url, the other is just garbled.

Can confirm this.
Same issue here.
This also happens when the submodule is also hosted on the same gogs installation.

It should be easy enough to fix for submodules hosted on the same installation, just a plain string replace instance SSH path with instance HTTP path for all submodules. Outside installations would be far more problematic, but I believe simply stripping the port entirely would at least take care of the vast majority of gogs installations - after all, we have things like SNI and Host headers for HTTP, but not SSH so in most instances, the HTTP(S) port will be the standard 80/443 far more often than the default port 22 for SSH.

@issuehuntfest has funded $20.00 to this issue. See it on IssueHunt

I worked around this issue with a redirect in my reverseproxy (nginx):
# edit the server section in /etc/nginx/sites-enabled/git.your.domain
# add this lines to
# fix the custom port issue for submodules in gogs (port: 2222)
location ~* ^/2222/(.*)$ {
        return 301 /$1;
}


Content Script cannot use React

Hi, I have the content script that will create a div element and use React render to that div then append it into the document.body, but when content script loaded it show this error Cannot use import statement outside a module:

My content script code:

Hey @dim0147,
thanks for opening the issue.
The script is not defined as a module, so ESM imports won't work in this case.
Afaik the simplest workaround would be to use require instead of import. Could you give that a try and see if it works?

Thanks for reply, I will try

Hi @JohnBra, @dim0147, I managed to get around the import part, but the manifest doesn't recognise the assets and tries to load them from the page source.

Do they have to be included in the web_accessible_resources block or the content_scripts block?

Here's what I tried so far:
add all js and css to the manifest (either manually or via a helper on closeBundle() in the make-manifest plugin):
"content_scripts": [
    {
      "matches": [
        "http://*/*",
        "https://*/*",
        "<all_urls>"
      ],
      "js": [
        "src/pages/content/index.js",
        "assets/logo.01164c68.js",
        "assets/react.90f97a3d.js"
      ],
      "css": [
        "contentStyle.css",
        "assets/newtab.0ee40f7f.css",
        "assets/options.b792369d.css",
        "assets/panel.b9bea8bd.css",
        "assets/popup.b8e876c5.css",
        "assets/tailwind.70279efb.css"
      ]
    }
  ],
  "web_accessible_resources": [
    {
      "resources": [
        "contentStyle.css",
        "icon-128.png",
        "icon-34.png",
        "assets/jsx-runtime.80d7c9a6.js",
        "assets/react.1c025b20.js",
        "assets/newtab.0ee40f7f.css",
        "assets/options.b792369d.css",
        "assets/panel.b9bea8bd.css",
        "assets/popup.b8e876c5.css",
        "assets/tailwind.70279efb.css"
      ],
      "matches": [
        "http://*/*",
        "https://*/*",
        "<all_urls>"
      ]
    }
  ]

Same error still: preload-helper:30 Refused to load the script 'https://github.com/assets/react.90f97a3d.js' because it violates the following Content Security Policy directive: "script-src github.githubassets.com". Note that 'script-src-elem' was not explicitly set, so 'script-src' is used as a fallback
Compare to the working repo from the docs (without tailwind): https://github.com/Jonghakseo/chrome-extension-boilerplate-react-vite
I see differences, can't yet understand to begin to merge the two....


Update readme

Picks up README changes and updated troubleshooting section from @predragnikolic 's earlier work in #441.

Coverage increased (+0.09%) to 26.693% when pulling 4638f1b8db45f9fd3b61dc589ce5de39cc6d3570 on update_readme into 7d249e75a9a17f75ec3bf7cb4d837c8e6c25fc63 on master.


Please add 1.20.1 support

Many mods are on 1.20.1 and i am ITCHING to use this mod on 1.20.1.

https://github.com/Project-Alphaa/Improved-Animations/issues/12#issuecomment-1907517036

Nevermind i just did it and it worked, thank you bro you're a lifesaver

How did you exactly do that?

Only works for fabric mods.
Go to the jar file and convert it into a zip file. Then, double click the file and look for "fabric.mob.json". Open it with notepad and you should see the version of minecraft it requires whether it's 1.19.3 or 1.20.4. Make sure to replace it with what version you want, and then save. Convert the zip file back into a jar file, and you're good to go.
If it still says it's incompatible, try doing the process again but when you save it, save it to your desktop. Afterwards, drag the file into the zip file (it may ask to replace the file in the destination, do that).

Cool, ill use a fabric & forge connector - thanks for that!


Too many extensions to Array and Enumerable isn't used elsewhere

Interface of array feels quite big, with a couple of very similar functions. We even have some aliases...
If you count the function definitions in lively.lang.Array you'll get ridiculously close to 50...
Also, the Enumerable object is created in lively.lang.Array and immediately mixed into the Array.prototype. I think that it's not used elsewhere - so we might skip this step as well in an effort to get a smaller Array interface...

with lively.lang a cleanup was done. we could still filter some extensions but the interface is mostly sane. https://github.com/LivelyKernel/lively.lang


Enable hiding graders and instructors

Currently we support hiding student roles, this does not delete them from the database but hides them from the UI and sets various policies so that the student can't interact with the course.
We should also support the ability to hide graders and instructors users:

no database changes needed because the hidden column already exists on the roles table
change grader and instructor views to enable setting the hidden value through the UI
update policies so that hidden roles cannot interact with the given course

Will update as I test more but upon analyzing the code, it looks like the only real difference between a hidden student and a normal student are that hidden students:

Cannot see the course automatically through dashboard (need to manually enter course link)
Cannot be added to a new group (cloned or fresh)
They are not included in the grade entry form CSV when graders download it
Their data is not included in the grade entry form summary stats (they are included for the assignment summary stats)

In all other aspects hidden and normal students are the same

Thanks @anoverga17 for that summary. Here is what I'm thinking for all users:

keep all the things you listed above in place (except don't allow them to access the course even with a direct link)
hidden users cannot view the course at all (this can be implemented by adding a check for hidden: false in the current_role function)
hidden users do not have permission to access repositories (update the check_repo_permissions function to check if a user is hidden or not)
on the graders table (left-hand side table on the Graders tab), add a checkbox to toggle whether hidden graders are shown on that table.
allow instructors to set whether TAs and students are hidden
allow admins to set whether instructors are hidden

@david-yz-liu can you confirm whether that makes sense and if it's consistent with what quercus is doing as well.

Thanks @mishaschwartz that makes a lot of sense. I will get to it.


Use cosmiconfig to resolve configuration files

What is the problem you're trying to solve?
In #1, I implemented a rough heuristic to check if the user already has Stylelint configured: by checking the current working directory for a set of files. However, this ignores several areas that Stylelint users could configure their configurations in, including the stylelint property in package.json.
What solution would you like to see?
In https://github.com/stylelint/create-stylelint/pull/1#discussion_r1088535793, @ntwb said:

Potentially additional files and paths based on cosmiconfig:
https://github.com/davidtheclark/cosmiconfig/#cosmiconfig

"any of the above two inside a .config subdirectory"
"a .config.js or .config.cjs CommonJS module"

i.e., we could use cosmiconfig to detect a valid Stylelint configuration in the CWD.
There may also be a different approach that I'm missing; though, keeping exact parity with Stylelint seems like a good idea.
Regardless of whatever approach we pick, I'm happy to take this on, especially as it's a follow-up to #1.

we could use cosmiconfig to detect a valid Stylelint configuration in the CWD.
keeping exact parity with Stylelint seems like a good idea.

SGTM.

I'm happy to take this on, especially as it's a follow-up to https://github.com/stylelint/create-stylelint/pull/1.

That's fantastic, thank you!


Update Nodejitsu information in README

I think it would be useful to notify people that they no longer can create a new account on nodejitsu platform due to Godaddy acquisition.

Thanks, I will update it. The README was written long before Godaddy's acquisition of Nodejitsu.


temporarily removed cafemaker API due to compatibility issue with dawntrail

Pull request type

[x] This is a bugfix to existing functionality

Pull request details
Due to the temporary unavailability of Chinese services cafemaker.wakingsands.com, The cafemaker.wakingsands.com is not support dawntrail version, So English API will be used instead.
When the cafemaker API adaptation is complete, it needs to be modified back.

Testing / Validation

[x] I used the log(s) listed below to develop and test this bugfix:
https://xivanalysis.com/fflogs/a:wk2x41rLtZcpqBmT/3/199

duplicated #2129

ah, my bad - didn't check prs beforehand; just got sentry pings 😓


Is it possible to set low latency configuration for live streams after manifest is parsed?

Have you read the Tutorials?
yes
Have you read the FAQ and checked for duplicate open issues?
yes
What version of Shaka Player are you using?
latest
Please ask your question
We have live streams where some may be low latency DASH streams, and some might not be.
Is there a way to ask shaka if what we're playing is detected as a low latency stream, and based on that information configure the player with low latency parameters?
So far we've tried:

listening to the manifestParsed-event, can't find information if the stream is low latency, updating the config with lowLatencyMode: true doesn't seem to work (shaka still warns that we're playing a low latency stream without the correct configuration)
adding a manifestFilter and looking for the availabilityTimeOffset-parameter manually (as this seems to be what shaka is using to know if the stream is low latency) setting configuration here doesn't seem to work either, shaka still warns that we're playing low latency without the correct configuration

Have you tried the streaming.autoLowLatency config?
https://shaka-player-demo.appspot.com/docs/api/shaka.extern.html#.StreamingConfiguration
This is off by default because it conflicts with an older Shaka Player feature: response filters.  If you're not using response filters for segments, you can safely set autoLowLatency to true for all content.
Does this help?

I think in the future, we should consider doing the inverse: always using LL features, except when response filters are configured for segments.  If someone is using segment response filters, we could issue a warning and turn off LL features.  We can't do that trivially today because response filters don't declare what type they care about (segment, license, manifest, etc).  Instead, the filter has to check the response type in its callback.  But with a v5 release and a breaking API change for filters, we could make LL require no configuration at all, which is arguably much more common and important than response filters.

Using autoLowLatency works, but we would like more control over other settings (defaultPresentationDelay and rebufferingGoal) and toggle these settings based on if the manifest is low latency or not
As I understand it, using autoLowLatency, rebufferingGoal will be set to 0.01 unless otherwise specified.
I'll toggle it on and run some tests though, thank you!

response filters for segments

we're using synchronous response filters for segments to read the headers, autoLowLatency still seems to give us an improvement though. What's the potential issue that can happen here?

We're running some custom filters to prevent stream stalls (detecting manifests that continuously update without causing shaka to fetch any segments) and detecting CDN switching on segments. Is there no way to get access to the segment headers for LL streams without patching fetch/xhr?

LL DASH uses the ReadableStream API to pipeline data from the network directly to MediaSource.  There is no point in that flow when Shaka Player has a complete segment in memory to filter.
This is partly because the filter API allows both reading and writing.  You could imagine a read-only version of filters that still worked for LL, but would receive the data after it was consumed by MediaSource instead of before.  This, however, doesn't exist in Shaka Player today.

Ok, there's probably something I don't understand here. The ReadableStream you use skips Fetch completely? As in, it's not using the Response.body ReadableStream part of the Fetch API?

Does the ReadableStream you use somehow skip Fetch completely, or are you referring to the Response.body ReadableStream? I'm wondering if patching fetch to read the headers could be an option for us.
A readonly API would be nice, or any kind of solution that would make it possible to access the Response.headers.

Perhaps we could offer a flag for response filters that indicates "read-only is okay" or one that indicates "headers only is okay".  Then those filters could still run in LL-DASH.
I think "headers only is okay" will be much cheaper to implement, FWIW.

That would be a good feature for us to have access to. With some pointers it's possible I could have a go at implementing the change, I'm not sure where I should start looking.

@martinstark, please look at the definitions for registerResponseFilter and related types (some in lib/net/ and some in externs/shaka/).  If you want to add a flag or alternate registration method for read-only, headers-only response filters, that would be the place to start.


build elementary-files with vala 0.48

Motivation for this change
fixes #94707
There is probably a more elegant way to do it but this was the one with the least impact I could find.
ping @vcunat
Things done

[x] Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
Built on platform(s)

[x] NixOS
[ ] macOS
[ ] other Linux distributions

[ ] Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
[ ] Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
[ ] Tested execution of all binary files (usually in ./result/bin/)
[ ] Determined the impact on package closure size (by running nix path-info -S before and after)
[ ] Ensured that relevant documentation is up to date
[ ] Fits CONTRIBUTING.md.

I don't know why @GrahamcOfBorg fails here. It builds for me locally just fine.

Most likely because of --arg config { allowAliases = false; }

We actually need to update vala on stable in the 0.46 series which is completely fine for a stable release.
Once I've verified that works I'm going to revert the update on release-20.03 and merge the vala update to staging-20.03.
And then "revert the revert" on that branch as well.

fixed in https://github.com/NixOS/nixpkgs/commit/caf8427f6ff5cf76e5a00b22dedd9a8fdd329249.

Here's the PR that updates vala and reverts that revert https://github.com/NixOS/nixpkgs/pull/94749


filter_users function limits use of custom usernames in obcca

The following filter is being applied inside the ibc.register() function in order to filter out the users that are of the 'client' usertype. However, this function seems to be specifically build for the standard demo in which the usernames as given in the Marbles demo - app.js file start with user_type1_ .
`//==================================================================
//filter_users() - only get client level usernames - [1=client, 2=nvp, 4=vp, 8=auditor accurate as of 2/18]
//==================================================================
function filter_users(users){
var valid_users = [];
for(var i = 0; i < users.length; i++) {

	if(users[i].username.indexOf('user_type1') == 0){		//type should be 1 for client
		valid_users.push(users[i]);
	}
}
return valid_users;

}`
Suggestion would be to give the user object an additional attribute to store their user type which the SDK filter_users function can filter on.

Thanks for the suggestion, but I want to try to keep the options as clean and user friendly as possible.  I feel this is an unneeded change since you can still achieve what you want with the existing code (see my comment below). I would agree that I did not detail this action at all in my documentation, which I have corrected in https://github.com/IBM-Blockchain/ibm-blockchain-js/commit/165a4d59849e7f721c0dba32a2e048527b3704cd.
I've added a FAQ section to the readme with this response:
Correct behavior of ibc.load() is to remove any usernames that do not contain 'type_1' in their name. This is to conform to the OBC Peer spec of what usernames a dev's app should use. If this is not applicable for your network (ie you have a custom IBM Blockchain network) you can easily create your own version of ibc.load() for your needs. I would copy the code found in ibc.load() then modifiy it to fit your own needs. Everything important that ibc.load() calls is exposed in this module.


Getting the config when the plugin is initialized

For my airtable plugin I need the connection configuration to dynamically create the tables. This doesn't seems to be possible right now.
I haven't digged in the code to see if it's hard to implement or not.
If you have any hints on how I could do that, it would be nice.

Thanks @francois2metz for opening this issue from our prior discussions in Slack. Definitely excited to see what's possible once we can be more dynamic in the table definitions phase!
@kaidaguerre has started thinking about this and mocking some approaches, so I'll let him expand below.

Hi @francois2metz, thanks for raising this issue.
There is a bootstrap ordering issue which makes it hard to provide the parsed connection config at plugin startup time.
So instead what I am proposing to do is provide a callback function which can optionally be used to create the tables. The connection config would be available within this function so dynamic table
So an example usage may look like this:

func Plugin(ctx context.Context) *plugin.Plugin {
	return &plugin.Plugin{
		Name: pluginName,
		ConnectionConfigSchema: &plugin.ConnectionConfigSchema{
			NewInstance: ConfigInstance,
			Schema:      ConfigSchema,
		},

		// set create tables callback function
		TableMapFunc: createTables,
	}
}

// table creation factory function
func createTables(p *plugin.Plugin) (map[string]*plugin.Table, error) {
	// get connection config from plugin
	connectionConfig := GetConfig(p.Connection)

	var tableMap = make(map[string]*plugin.Table)

	// TODO dynamically build tables using connectionConfig

	return tableMap, nil
}

That looks good!

@kaidaguerre Awesome work! I used the new version on https://github.com/francois2metz/steampipe-plugin-airtable/blob/main/airtable/plugin.go.
For some reasons, when using the callback function, the descriptions of the tables and columns is not working.
> .inspect airtable
+----------------------+-------------+
| table                | description |
+----------------------+-------------+
| airtable_acquis      |             |
| airtable_competences |             |
| airtable_domaines    |             |
| airtable_epreuves    |             |
| airtable_referentiel |             |
| airtable_tubes       |             |
+----------------------+-------------+

To get information about the columns in a table, run .inspect {c
onnection}.{table}

> .inspect airtable_acquis
+--------+-------+-------------+
| column | type  | description |
+--------+-------+-------------+
| fields | jsonb |             |
| id     | text  |             |
+--------+-------+-------------+

@francois2metz thanks for spotting this - it is actually a small bug in the Steampipe CLI - when retrieving the plugin schema in order to populate the inspect data, the connection config was not being passed to the plugin. This resulted in the descriptions being omitted for dynamic tables. https://github.com/turbot/steampipe/issues/932
I have fixed this and it will be included in the forthcoming 0.8.3 patch, which will be released this week


Updated roslyn to consume the new builtin support for dotnet pack.

cc @jaredpar. This gets us almost all of the way off of the patch files, with the only one remaining being the global.json patch.

I don't have a linux box to test locally, so I was going to let CI test this. It may be blocked by https://github.com/NuGet/Home/issues/6829.

@dseefeld looks like this should be good to go.


chore(main): release 1.1.0

:robot: I have created a release beep boop
1.1.0 (2024-05-16)
Features

track cash transfers as asset transaction type (#152) (ec3949c)

Bug Fixes

eth indexer fails to attach LP DynamicSource (#154) (5cab359)
remove celo (#141) (2b55255)
transfer debt 2024 handler (#149) (0fc024a)

This PR was generated with Release Please. See documentation.

:robot: Release is at https://github.com/centrifuge/pools-subql/releases/tag/v1.1.0 :sunflower:


Bulk Load Config?

I'm looking for something similar to RocksDB's PrepareForBulkLoad ( https://github.com/facebook/rocksdb/blob/19ec44fd392beff927cacbd3a976415edb3d27e9/options/options.cc#L367 ) which sets all the options to write without doing much organization or compaction, and then you are expects to call CompactRange yourself afterwards. What would be the best set of values to load into Options to cause that kind of behavior?

Transaction may be what you looking for. It skips journaling and writes directly into sstable, and those table will not committed (hence no organization or compaction being done on it) until you call Commit.


SVG tags compatibility with Safari and mobile

Hi!
We've found an issue with the SVG html tags used to render icons (close icon on badges, search icon). It doesn't work properly on Safari, some mobile phones, and Iphones.
This tag could easily be replaced with special fonts (glyphicons), or a simple image.
Thanks

+1

Works as expected.  Tested on safari mobile in iphone. If still the error persists, check your html tag with the html5 standards


Reorganize Utils functions

Description
Move util functions to concrete types
const nftAddress: Address = new NftAddress(Utils.nftIdToBech32(nftId, networkName));

Consider making the nftIdToBech32 function internal to NftAddress, e.g.:
const nftAddress: Address = NftAddress.fromNftId(nftId, networkName));

The same applies to

Utils.aliasIdToBech32.
Utils.computeTokenId. It already returns a TokenId so is just a constructor for TokenId without living on the type itself.
Utils.computeOutputId

Conversion function from Output ID to UTXOInput
Add a constructor for UTXOInput that has the functionality of TransactionHelper.inputFromOutputId from iota.js.
Move top-level functions to Utils namespace
The utf8ToBytes, utf8ToHex and hexToUtf8 functions are directly in the top-level namespace of the sdk. Consider moving thos "util" functions to the Utils namespace.
Nits

Utils.computeNftId returns a string instead of the type alias NftId.

Motivation
The address util functions take a string and return one and it's not immediately obvious what that string represents, especially for new users, I imagine. So having them directly on the stronger-typed classes like AliasAddress seems both clearer and more convenient.
Constructors should live on the classes that they construct rather than in Utils. A user checking the API docs for how to construct a certain type would check that type's docs and rarely consider that a constructor is in Utils.
The constructor new UTXOInput requires a transaction ID and output index, but it's not easily possible to extract those from some outputID, because that is just a string. There should be a convenient way to convert.
Are you planning to do it yourself in a pull request?
No.

const nftAddress: Address = new NftAddress(Utils.nftIdToBech32(nftId, networkName));

NFTAddress  takes an NFTId which is hex encoded string, there's no need to convert it to Bech32. Same for other address classes.

Utils.computeTokenId. It already returns a TokenId so is just a constructor for TokenId without living on the type itself

TokenId is a type alias for string, it doesn't have a constructor.

Add a constructor for UTXOInput that has the functionality of TransactionHelper.inputFromOutputId from iota.js.

Will be added.

The utf8ToBytes, utf8ToHex and hexToUtf8 functions are directly in the top-level namespace of the sdk. Consider moving thos "util" functions to the Utils namespace

Everything is top level right now, since there isn't many util functions, keeping everything top level can be beneficial to avoid breaking changes if directory structure changes.

Utils.computeNftId returns a string instead of the type alias NftId

Will be corrected.

NFTAddress takes an NFTId which is hex encoded string, there's no need to convert it to Bech32. Same for other address classes.

True. Not high priorty, and can be changed non-breakingly, but the parameter name of the constructor of AliasAddress and NftAddress should take aliasId: AliasId and nftId: NftId respectively, not address as it currently does. That's where my confusion that the constructor needed a bech32 address initially came from.


[BugFix] Fixed the problem of inaccurate memory statistics with auto spill turned on

We may trigger bad_alloc when the amount of memory we reserve is less than the amount of memory we actually consume, which will cause us to get an error in our memory statistics
Fixes #issue
What type of PR is this:

[x] BugFix
[ ] Feature
[ ] Enhancement
[ ] Refactor
[ ] UT
[ ] Doc
[ ] Tool

Checklist:

[ ] I have added test cases for my bug fix or my new feature
[ ] This pr will affect users' behaviors
[ ] This pr needs user documentation (for new or modified features or behaviors)

[ ] I have added documentation for my new feature or new function

Bugfix cherry-pick branch check:

[ ] I have checked the version labels which the pr will be auto-backported to the target branch

[x] 3.1
[x] 3.0
[ ] 2.5
[ ] 2.4

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)

@Mergifyio backport branch-3.1

@Mergifyio backport branch-3.0

[FE PR Coverage Check]
:heart_eyes: pass : 0 / 0 (0%)


remove the use of implicits that are not warp-memory

I'd suggest you to add warp_memory implicit rather than completely getting rid of it
#[implicit(warp_memory)]
fn f(...) {
  ...
  warp_memory.read_u128(index);
  ...
}

It would be difficult later to track places where warp_memory needs to be added if we remove it from all of the places now.

I agree 100% with you, actually that's was what I think, that's why on CairoFunctionDefinition implementation I appended a comment of how we plan to use implicit and sort of what's happening with them.
https://github.com/NethermindEth/warp/blob/5cc3efe06dfff437b11b68929a0385b557bbbfba/src/ast/cairoNodes/cairoFunctionDefinition.ts#L18
Right now we still have a set of Implicits foreach function definition. Even considering that the only available implicit is warplib_memory the logic we had to infer the implicits of the current function worth it.
Nevertheless I do remove them from cairoUtilFuncGen scripts that were not updated to Cairo1 syntax, mainly because we had them hardcode in there (was just a string that we insert in the raw string definition of those functions). They didn't have any role at ast level and we do have to review each of of those script to make them Cairo1 compatible.
This is an example:
https://github.com/NethermindEth/warp/blob/db1b5987c7fe19c0f7b26dc80239f819be7f644e/src/cairoUtilFuncGen/memory/implicitConversion.ts#L51
https://github.com/NethermindEth/warp/blob/db1b5987c7fe19c0f7b26dc80239f819be7f644e/src/cairoUtilFuncGen/memory/implicitConversion.ts#L210

I'd suggest you to add warp_memory implicit rather than completely getting rid of it
#[implicit(warp_memory)]
fn f(...) {
  ...
  warp_memory.read_u128(index);
  ...
}

It would be difficult later to track places where warp_memory needs to be added if we remove it from all of the places now.

I agree. Let's just put #[implicit(warp_memory)] in front of those functions, but without updating the rest of their implementation and syntax

As part of this PR you should update cairoParsing.ts and make it parse the custom implicit attribute in order to detect when generated functions requires the warp_memory implicit

Actually that's already done, we just have the remains of the old parseImplicit, but using getRawCairoFunctionInfo is already known the implicit used (if used any). I'm gonna remove that old function right now

Does parseMultipleRawCairoFunctions work properly?

There is also getImplicitsAt in ast.ts. Is it needed?


Moved rules into their own files

Everything under lib/rules/* is a rules file (with the name of the rule in camelCase), re-exported into an array in lib/rules.js.
Moved the regular expressions from lib/rules.js to lib/expressions.js, and the rest of the variables into lib/shared.js.
Fixes #83.

Coverage decreased (-79.2%) to 20.8% when pulling b63647f6b159eddb0897b381508719118af4e8ee on JoshuaKGoldberg:rule-files-split into 77e27cbe09b3a2b8a1b71c11e0f29939dbf7e4c3 on DavidAnson:master.

Coverage decreased (-79.2%) to 20.8%

It's 100% locally :(


[Request] Site Support: Oprah.com

Please follow the guide below

You will be asked some questions and requested to provide some information, please read them carefully and answer honestly
Put an x into all the boxes [ ] relevant to your issue (like this: [x])
Use the Preview tab to see what your issue will actually look like

Make sure you are using the latest version: run youtube-dl --version and ensure your version is 2017.11.06. If it's not, read this FAQ entry and update. Issues with outdated version will be rejected.

[x] I've verified and I assure that I'm running youtube-dl 2017.11.06

Before submitting an issue make sure you have:

[x] At least skimmed through the README, most notably the FAQ and BUGS sections
[x] Searched the bugtracker for similar issues including closed ones

What is the purpose of your issue?

[ ] Bug report (encountered problems with youtube-dl)
[x] Site support request (request for adding support for a new site)
[ ] Feature request (request for a new functionality)
[ ] Question
[ ] Other

The following sections concretize particular purposed issues, you can erase any section (the contents between triple ---) not applicable to your issue

If the purpose of this issue is a bug report, site support request or you are not completely sure provide the full verbose output as follows:
Add the -v flag to your command line you run youtube-dl with (youtube-dl -v <your command line>), copy the whole output and insert it here. It should look similar to one below (replace it with your log inserted between triple ```):
[debug] System config: []
[debug] User config: []
[debug] Custom config: []
[debug] Command-line args: [u'--write-srt', u'--sub-lang', u'en', u'--ap-mso', u'nor105', u'--ap-username', u'PRIVATE', u'--ap-password', u'PRIVATE', u'http://www.oprah.com/own-queensugar/copper-sun', u'-v']
[debug] Encodings: locale UTF-8, fs UTF-8, out UTF-8, pref UTF-8
[debug] youtube-dl version 2017.11.06
[debug] Python version 2.7.5 - Linux-3.10.0-693.2.2.el7.x86_64-x86_64-with-centos-7.4.1708-Core
[debug] exe versions: avconv 10.1, avprobe 10.1, ffmpeg 3.4, ffprobe 3.4
[debug] Proxy map: {}
[generic] copper-sun: Requesting header
WARNING: Falling back on generic information extractor.
[generic] copper-sun: Downloading webpage
[generic] copper-sun: Extracting information
ERROR: Unsupported URL: http://www.oprah.com/own-queensugar/copper-sun
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/extractor/generic.py", line 2159, in _real_extract
    doc = compat_etree_fromstring(webpage.encode('utf-8'))
  File "/usr/lib/python2.7/site-packages/youtube_dl/compat.py", line 2539, in compat_etree_fromstring
    doc = _XML(text, parser=etree.XMLParser(target=_TreeBuilder(element_factory=_element_factory)))
  File "/usr/lib/python2.7/site-packages/youtube_dl/compat.py", line 2528, in _XML
    parser.feed(text)
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1642, in feed
    self._raiseerror(v)
  File "/usr/lib64/python2.7/xml/etree/ElementTree.py", line 1506, in _raiseerror
    raise err
ParseError: not well-formed (invalid token): line 72, column 72
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/youtube_dl/YoutubeDL.py", line 784, in extract_info
    ie_result = ie.extract(url)
  File "/usr/lib/python2.7/site-packages/youtube_dl/extractor/common.py", line 437, in extract
    ie_result = self._real_extract(url)
  File "/usr/lib/python2.7/site-packages/youtube_dl/extractor/generic.py", line 3059, in _real_extract
    raise UnsupportedError(url)
UnsupportedError: Unsupported URL: http://www.oprah.com/own-queensugar/copper-sun

If the purpose of this issue is a site support request please provide all kinds of example URLs support for which should be included (replace following example URLs by yours):

Single video: http://www.oprah.com/own-queensugar/copper-sun

Note that youtube-dl does not support sites dedicated to copyright infringement. In order for site support request to be accepted all provided example URLs should not violate any copyrights.

I wrote a oprah.py script and I got it to extract the m3u8 but when youtube-dl goes to download it, it gets a 403 Forbidden.  I tried via browser and If i am logged in then I can get the m3u8 with no problem.  if I try to do --ap-mso  --netrc it seems to get ignored.  What am I missing?

Correct code obviously.

well that was useful (NOT).  and here I thought I would help the community

Yours either. There are no telepathist here.

I asked for guidance on an issue, I didn't just say 'please help, someone write the code'.  Whatever at this point,  I'll review the docs again and look at the other .py for assistance.

You are not mimicking browser behavior properly - that's all guidance possible without any code. Again, there are no telepathists here, nobody knows what code have you dashed off.

Fair enough re the code.  I'll preface by saying i'm not a python developer so this is new to me.
This is the code that I wrote so far
#coding : utf-8
from __future__ import unicode_literals

import re
import base64
import json

from .common import InfoExtractor
from ..utils import int_or_none

class OprahIE(InfoExtractor):
    _VALID_URL = r'https?:\/\/(?:\w+\.)?oprah\.com\/.*\/(?P<id>.*)'
    _TEST = {
        'url': 'http://oprah.com/own-queensugar/copper-sun',
        'info_dict': {
            'id': 'copper-sun',
            'ext': 'mp4',
            'title': 'Copper Sun'
        }
    }

def _real_extract(self, url):
    video_id = self._match_id(url)
    webpage = self._download_webpage(url, video_id)

    # Create an array of the html
    webpage_array = webpage.split('\n');

    # Find the line that says api_data_hash
    json_data = None
    for htmlLine in webpage_array:
        if "api_data_hash" in htmlLine and 'JSON' not in htmlLine:
            htmlLine = htmlLine.strip()
            api_data = htmlLine.split('"')[1]
            json_data = json.loads(base64.b64decode(api_data))

    # Proceed if json_data has ... data :)
    if json_data is not None:
        clip_data = json_data['clip_data']
        url =  clip_data['contentPath']
        title = clip_data['title']
        description = clip_data['description']

        print("[Oprah] Video ID: %s" % video_id)
        print("[Oprah] Video Title: %s" % title)

        return {
            'id': video_id,
            'title': title,
            'url': url,
            'description': description,
            'ext': 'mp4'
        }
    else:
        print("[Oprah] Error extracting the information from the page")

Any update on this? I'd be interesting in it being added as well.


destroying objects

@astagi
Is there any solution for destroying objects? (problem was found in #29)
I propose to remove Python __del__() and C++ SlimArray::remove() from the code.
Do you have any use case where it is needed?
We can add a new command clean which removes all existing objects. This can be called at start of the script and the end explicitly.

No solution atm, users must explicitly call "del". I'll try to think about it, I was thinking about a global register for all the instances and call "del" on them at the end of the script, or whenever the user wants.

I accidentally commented out __del__()
del x doesn't call x.__del__() so the current solution is wrong. (read this http://stackoverflow.com/questions/6104535/i-dont-understand-this-python-del-behaviour) For destroying remote objects an explicit method should be called like x.destroy()

ok let's go for a custom method :+1:

To keep things simple I'd avoid to maintain a global register of instances and similar solutions, let the user clean his objects explicitly.


53 infer concept relatons

Coverage increased (+0.05%) to 98.839% when pulling 7a315a9b88ea53382b839414557acf3fc3628a63 on 53_infer_concept_relatons into 036928e21d910f075e1d1d69a7f8d1e66b1a9e24 on DEV_0.6.0.


idea: entire projects as plugins

I can't keep up with adding new features to the plugin loader. I added middleware and routes, but then realized I forgot islands. This is under review in #1472. Marvin is looking to introduce the concept of 'actions' and his branch doesn't touch anything with plugins. I'm sure it won't be long after release until someone wants to add an action via a plugin. Now @cbinzer is looking for static files.
What if instead of adding these features individually to the fromManifest method we were to support entire projects as plugins? I'm not entirely sure what this would look like, but I don't think what we're doing now is scalable.
Pinging people who have expressed interest in plugins before: @cbinzer @bjesuiter @Octo8080X @jabolol @iuioiua

This sounds very interesting!
As you say, maybe there should not be a difference between loading a plugin and loading a partial project.
I'm on vacation to the 15th of August, so cannot look into it right now, but I'll happily join the conversation after!

Thanks for opening this thread. I wholeheartedly agree that it's valuable to get our heads together and think of how we want the plugin API to look like more thoroughly.
FWIW: Disregard the actions thingy for now. I meanly went that route because it was the path of least resistance to prototype that idea.
To me islands are special in the sense that they need to be wired up with the bundler internally. That makes them a bit of an edge case. When it comes to the other use cases like routes, static files, or middlewares, I think we have the foundation covered but lack the necessary helpers to make it as nice as it should be.
If we think about it they all are an abstraction on top of a request handler. That's the lowest shared abstraction that Deno provides through the serve API:
Deno.serve(req => new Response("hey"));

A route handler is basically the same thing, just that it checks first if the URL matches an expected format. Same for a middleware. The current plugin approach of adding yet another property doesn't scale too well and I'd love for us to move more in a direction where Fresh itself uses the same methods of adding functionality as plugins. Basically making Fresh itself a plugin which is what I think you @deer had in mind.
That's what I like about https://github.com/denoland/fresh/issues/1487 specifically as it gives us a composable way of nesting plugins and by extension Fresh apps. Expanding on the idea presented in that issue:
const sub = createApp()
  .get("/foo/bar", req => ...)
  .post("/foo", req => ...)

const app = createApp()
  .get("/", req => ...)
  .get("/about", req => ...)
  .use("/sub", sub); // <-- plant subApp at "/sub"

To solve the static file use case we should simply expose the way we do it in Fresh internally:
const myPlugin = createApp()
  .get("/static/:file", req => serveStatic(req, "/path/to/dir"))
  .get("/memory/:file", req => serveFromMemory(req, files))

const app = createApp()
  .use("/*", myPlugin)

I'm admittedly unsure about how to wire islands into that thing. In essence all we need to do is "register" islands so that the bundler knows where to look them up. Maybe something like:
const app = createApp()
  .islands("path/to/island?", { options })

Not sure. I kinda like the idea in general because it allows us to compose apps and by extensions plugins.

I agree that islands can be applied with a plugin. I am excited.
I don't have a concrete implementation image, but the important thing is how can we call islands applied by a plugin? I think it is.
For example
// routes/hoge.ts

// Currently, it is like this.
import SameIslands from "../islands/SameIslands.ts"

// Islands applied by plugins
import {SameIslands} from "@fresh/islandsStore"

// Apply to template

I think that routes applied by plug-ins and directly under routes/ need to be able to be read in the same way.
I hope my opinion is helpful.
Thanks.

I think the current plugin system makes sense. Technically, you can do anything in a plugin given you have access to making routes. "static" files are just an abstraction over routes, we technically don't need to """officially""" support this (thought it'd be nice to). Islands are a bit of a different thing, given they can be used in user code, but I think it makes sense the way it's being implemented.
TLDR; I think the plugin system we have now makes sense, unless someone has a proposal that solves concrete problems we have with the current one.

Hi, thanks for the lively discussion about a Fresh Plugin system. I think a solid built out plugin/module system is needed if Fresh should be a serious framework in the market. Since many developers are comfortable and want to create applications with as little effort as possible, they will end up choosing the framework based on already existing plugins/modules.
I could imagine to drill out the plugin system a bit more. Maybe we should no longer provide single fields like routes or middlewares, but use hooks for registration instead. At the moment there is for example only the render hook. Now we could create another hook, e.g. the setupServer hook. Here you would register a function in which the actual registration of routes, middlewares, islands, static files etc. would be done over helper functions or an application object.
This could look like as the following:
import { Plugin, injectRoute, injectIslands } from "$fresh/server.ts";
import { MyIsland1, MyIsland2 } from "./islands.tsx";

const plugin: Plugin = {
  name: "my_plugin",
  hooks: {
    setupServer: () => {
      injectRoute('/api/entries');
      injectIslands(MyIsland1, MyIsland2);
    }
  }
};

Or like @marvinhagemeister suggested in #1487:
import { Plugin } from "$fresh/server.ts";

const plugin: Plugin = {
  name: "my_plugin",
  hooks: {
    setupServer: (app) => {
      app.use(req => ....)

      app.use({
        route: "/foo/:id",
        async POST(req, ctx) => {},

        Component({ children }) {
        return
Hello I'm a route

      },
    });
    }
  }
};

I've some thoughts on this proposal of @cbinzer.
I generally like the idea of these hooks, maybe we can have a similar syntax /feeling like using rxjs pipe operators (in this case app.inject):
setupServer: (app) => {
    app.inject(
        route('/foo', routeHandlers),
        island(),
        ...
    )
 }

Note: It might be more difficult to type this app.inject() function compared to rxjs, since the output of these functions might differ in more ways, but I think this is solveable.
The great thing of this option would be the ability for fresh to provide nice typings for these handler functions and make them typesafe for the users of fresh.
I'm a bit hesitant to using the "app.use()" naming, because it reminds me too much of express.js and normal route handlers.
And if we allow more than routes in this context, like Islands, I find it inappropriate to use the same syntax as for route and middleware register functions.
Even though we would see the typing in most IDEs, I found it off-putting when many different frameworks use this app.use() pattern but with wildly different typings / usage patterns.
An alternative to this would clearly be that the base extension mechanism of app is app.use for attaching routes,
and all other things are like "factory functions".
This would fit well with the mental model of routes being the basis of the app router and all other things, like staticFiles, page handlers, etc. being an extension to route. We would only need to fit a way for islands into this model.
At last, only for clarification:
For islands, plugin authors would register them in their plugins and users would simply import the Island classes in their code from the Plugin package?
In this case we should add some kind of error, when the user tries to use an Island from a plugin, which is not registered with the fresh app (for example. if the plugin author missed the registration in its plugin definition)
Otherwise the user would have no error from typescript imo, but it would still not work.

@bjesuiter That thought crossed my mind too that app.use() is a bit overloaded. Maybe something like app.plugin() could work?

Yes, app.plugin() sounds good as the root hook for plugins. It's also more descriptive than app.inject()

What about app.setup()? setup is sometimes used in other frameworks to signify that a hook only runs once (on setup/init). I ilke app.plugin() as well though, and in general, I feel like aligning this to the new server API proposal  in https://github.com/denoland/fresh/issues/1487 is the way to go 🚀

@miguelrk The structure for registering a plugin is proposed like this:
import { Plugin } from "$fresh/server.ts";

const plugin: Plugin = {
  name: "my_plugin",
  hooks: {
    setupServer: (app) => {
      app.use(req => ....)

      app.use({
        route: "/foo/:id",
        async POST(req, ctx) => {},

        Component({ children }) {
        return
Hello I'm a route

      },
    });
    }
  }
};

When you look at the "hooks" object, it has a 'setupServer' property. This property contains a function which will be run when the server is started.
This is similar to to the "render" hook, where you can add a function to run when a page is rendered.
So, the app object doesn't need another notion of "when does this function run" but only a "add this thing (route, island, etc.) to the state of the app.

@bjesuiter I like the idea of making plugins more esbuild-like. If we go that route we can drop the indirection of calling app.plugin().
const app = freshApp({
  plugins: [
    twindPlugin,
    {
      name: "my-inline-plugin",
      setup(app) {
        app.use(req => ...)
      }
    }
});

I would go with @marvinhagemeister's approach, since it enables new functionality with a simple API that can be inlined easily with the existing plugins.

@Jabolol exactly which of all the options do you like? 😄

I also like marvin's approach here best 👍🏼

@Jabolol exactly which of all the options do you like? 😄

The option here is the one I prefer. I appreciate simplicity, and the esbuild-like approach seems the one that fits the most this philosophy.

I just want to voice support for this feature as even with a medium size project, I'm having worries about the long term maintainability of the single route hierarchy and it trending towards a big ball (tower) of mud.
The "feature based folders" will at least get it down to one Jenga tower to manage instead of 2 (routes + components) and is a big step forward, but projects as plugins is a path* towards true isolation.
*pun/intended

https://github.com/denoland/fresh/issues/1602#issuecomment-1671740190 seems to be the winning approach so far. I wanted to comment about a particular functional use case of this: SaaSKit. I'm currently a bit confused about what it's doing. It seems like there are two things here:

generic code for SaaS -- the "kit" part, so to speak
a specific "Deno Hunt" app which makes use of the kit

I would love to see SaaSKit move in the "project as plugin" direction. That way I (or anyone else) could just focus on building the business logic of the app. In the Deno Hunt case, all of the logic particular to commenting and voting and such would stay in the Deno Hunt project, which would use SaaSKit as a plugin.
Of course @iuioiua is free to decide as he wants, but this is just how I'm imagining a real world use case of this stuff.

What @deer said would also be a major USP for me for the fresh ecosystem.
The Astro Templates all seem to be a template repo where you have to dable in the already existing code. This may be fine and gives you as a developer the control to change anything you want, but it creates a steep learning curve for the code setup in your template.
With this plugin approach we can simply plug & play different fresh apps to greater ecosystems without cluttering the "userland namespace", so to say.
I can Imagine having fresh plugins providing an opinionated basic blogging infrastructure for example.
This plugin approach even allows for updating this "template" stuff, which is not done so easily in something like Astro, afaik.

I would really love to be able to either extend / override the Esbuild config, so I could add plugins I want for esbuild etc.

Hello everyone! I've been following this issue and https://github.com/denoland/fresh/issues/1487 and wanted to know if we are still on track for this for the 1.5 release in some days @marvinhagemeister and if I could be of any help.
In general, I feel both this issues/proposals should make fresh a direct alternative to hono (which is lately doing some great stuff BTW). Currently, I find myself turning to hono for pure backends (APIs), while I turn to fresh for either pure frontends (UIs), or full-stack (UI+API). I hope these efforts (especially https://github.com/denoland/fresh/issues/1487) make fresh a default also fore pure backends! Or am I stretching this?

Closing as this has been achieved with Fresh 2 which will release soon.