Wrong reference in Debug.xcconfig and Release.xcconfig

I wanted to test announced Flutter 3.13 possibility to change "Runner" workspace name.
I followed the steps exactly as reported here @LouiseHsu in https://github.com/flutter/flutter/issues/9767#issuecomment-1533771721_ in an existing project, but I had some errors in the build process related to some pod misconfiguration.
Finally I found the problem: in the Debug.xcconfig and Release.xcconfig files there are the reference to the old Target Support xcconfig files.
In ios/Flutter/Debug.xcconfig you need to replace
#include? "Pods/Target Support Files/Pods-Runner./Pods-Runner.debug.xcconfig"`
with
#include? "Pods/Target Support Files/Pods-MyNewName/Pods-MyNewName.debug.xcconfig"`
The same in Release.xcconfig

Thanks for filing the issue. I can reproduce this after adding a plugin such as webview_flutter to the project. It seems the pod can not be installed because it can't find the correct target:
[!] Unable to find the Xcode project `/Users/huynq/Desktop/test_rename_project/ios/Runner.xcodeproj` for the target `Pods-Runner`.

flutter doctor -v (stable and master)
[✓] Flutter (Channel stable, 3.13.0, on macOS 13.5 22G74 darwin-x64, locale en-VN)
    • Flutter version 3.13.0 on channel stable at /Users/huynq/Documents/GitHub/flutter
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision efbf63d9c6 (25 hours ago), 2023-08-15 21:05:06 -0500
    • Engine revision 1ac611c64e
    • Dart version 3.1.0
    • DevTools version 2.25.0

[✓] Android toolchain - develop for Android devices (Android SDK version 32.0.0)
    • Android SDK at /Users/huynq/Library/Android/sdk
    • Platform android-33, build-tools 32.0.0
    • ANDROID_HOME = /Users/huynq/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E222b
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.81.0)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.70.0

[✓] Connected device (3 available)
    • iPhone (mobile) • d9a94afe2b649fef56ba0bfeb052f0f2a7dae95e • ios            • iOS 15.7.2 19H218
    • macOS (desktop) • macos                                    • darwin-x64     • macOS 13.5 22G74 darwin-x64
    • Chrome (web)    • chrome                                   • web-javascript • Google Chrome 116.0.5845.96

[✓] Network resources
    • All expected network resources are available.

• No issues found!

[!] Flutter (Channel master, 3.14.0-9.0.pre.49, on macOS 13.5 22G74 darwin-x64, locale en-VN)
    • Flutter version 3.14.0-9.0.pre.49 on channel master at /Users/huynq/Documents/GitHub/flutter_master
    ! Warning: `flutter` on your path resolves to /Users/huynq/Documents/GitHub/flutter/bin/flutter, which is not inside your current Flutter SDK checkout at /Users/huynq/Documents/GitHub/flutter_master. Consider adding /Users/huynq/Documents/GitHub/flutter_master/bin to the front of your path.
    ! Warning: `dart` on your path resolves to /Users/huynq/Documents/GitHub/flutter/bin/dart, which is not inside your current Flutter SDK checkout at /Users/huynq/Documents/GitHub/flutter_master. Consider adding /Users/huynq/Documents/GitHub/flutter_master/bin to the front of your path.
    • Upstream repository https://github.com/flutter/flutter.git
    • Framework revision d8470f713c (62 minutes ago), 2023-08-22 19:18:00 -0700
    • Engine revision b190f90150
    • Dart version 3.2.0 (build 3.2.0-96.0.dev)
    • DevTools version 2.26.2
    • If those were intentional, you can disregard the above warnings; however it is recommended to use "git" directly to perform update checks and upgrades.

[✓] Android toolchain - develop for Android devices (Android SDK version 32.0.0)
    • Android SDK at /Users/huynq/Library/Android/sdk
    • Platform android-33, build-tools 32.0.0
    • ANDROID_HOME = /Users/huynq/Library/Android/sdk
    • Java binary at: /Applications/Android Studio.app/Contents/jbr/Contents/Home/bin/java
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)
    • All Android licenses accepted.

[✓] Xcode - develop for iOS and macOS (Xcode 14.3)
    • Xcode at /Applications/Xcode.app/Contents/Developer
    • Build 14E222b
    • CocoaPods version 1.11.3

[✓] Chrome - develop for the web
    • Chrome at /Applications/Google Chrome.app/Contents/MacOS/Google Chrome

[✓] Android Studio (version 2022.2)
    • Android Studio at /Applications/Android Studio.app/Contents
    • Flutter plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/9212-flutter
    • Dart plugin can be installed from:
      🔨 https://plugins.jetbrains.com/plugin/6351-dart
    • Java version OpenJDK Runtime Environment (build 17.0.6+0-17.0.6b802.4-9586694)

[✓] VS Code (version 1.81.1)
    • VS Code at /Applications/Visual Studio Code.app/Contents
    • Flutter extension version 3.70.0

[✓] Connected device (4 available)
    • RMX2001 (mobile) • EUYTFEUSQSRGDA6D                         • android-arm64  • Android 11 (API 30)
    • iPhone (mobile)  • d9a94afe2b649fef56ba0bfeb052f0f2a7dae95e • ios            • iOS 15.7.2 19H218
    • macOS (desktop)  • macos                                    • darwin-x64     • macOS 13.5 22G74 darwin-x64
    • Chrome (web)     • chrome                                   • web-javascript • Google Chrome 116.0.5845.96

[✓] Network resources
    • All expected network resources are available.

! Doctor found issues in 1 category.

Thanks for filing this issue, I can repro it too, so let me take a look :)

Since the xcconfig files aren't generated, the only viable option is to just change them if you want to rename a project. I've updated my comment with some updated instructions, as well as instructions on what to change in the Podfile.
Thanks @giaesp !


Support Gaps in Row and Column

I’d like a symmetric way to put space between my elements. Today I’m hacking it with margins on N-1 components and it looks bad.
          Row {
            Button(
              modifier = Modifier.margin(Margin(end = 8.dp)),
              text = "Left",
            )
            Button(
              text = "Right",
            )
          }

Spec: https://www.w3.org/TR/css-align-3/#gaps


Chat sample running in Colab fails. - Google Compute Engine metadata service. Status: 404 Response  - Works locally.

Description of the bug:
I have a sample which runs perfect in PyCharm.   However when I put it over on Colab its failing.   With some vague issues with a service account.    Which i do not understand as gemini just uses an API key.

google.auth.exceptions.TransportError: ("Failed to retrieve http://metadata.google.internal/computeMetadata/v1/instance/service-accounts/default/?recursive=true from the Google Compute Engine metadata service. Status: 404 Response:\nb''", <google.auth.transport.requests._Response object at 0x7d1ba77c4f40>)

I'm not sure how to share a colab here but have a link  Gemini_chat_.ipynb
Actual vs expected behavior:
This is the same code just from PyCharm
from dataclasses import asdict, dataclass
from dotenv import load_dotenv
import google.generativeai as genai
import os

load_dotenv()

# The api key for accessing the api. stored in .env
genai.configure(api_key=os.getenv("API_KEY"))

# name of the ai model used in this call.
CHAT_MODEL_NAME = os.getenv("CHAT_MODEL_NAME")

# Init the chatbot with two prompts, there must be two.

# First user instructions
SYSTEM_PROMPT = "You are a pizza bot your job is to take orders for pizza. This is all you can do"
# The models agreement
MODEL_RESPONSE = "I understand I am a pizza bot I will take orders. For pizza"

@dataclass
class Message:
    role: str
    parts: list[str]
    Model = "model"
    User = "user"

    def __init__(self, role: str, content: str):
        """
        Initializes a new Message object with the specified role and content.

        Args:
            role (str): The role of the sender (e.g., "user", "model").
            content (str): The content of the message.
        """

        self.role = role
        self.parts = [content]

class GeminiService:
    generation_config: str = {
        'temperature': 0.9,
        'top_p': 1,
        'top_k': 40,
        'max_output_tokens': 2048,
        'stop_sequences': [],
    }

    safety_settings: list[str] = [{"category": "HARM_CATEGORY_HARASSMENT", "threshold": "BLOCK_NONE"},
                                  {"category": "HARM_CATEGORY_HATE_SPEECH", "threshold": "BLOCK_NONE"},
                                  {"category": "HARM_CATEGORY_SEXUALLY_EXPLICIT", "threshold": "BLOCK_NONE"},
                                  {"category": "HARM_CATEGORY_DANGEROUS_CONTENT", "threshold": "BLOCK_NONE"}]

    def __init__(self):
        self.model = genai.GenerativeModel(model_name=CHAT_MODEL_NAME,
                                           generation_config=self.generation_config,
                                           safety_settings=self.safety_settings
                                           )

    def get_completion(self,
                       messages: list[Message],
                       message: str) -> str:
        # get the conversation history as a dict
        messages = [asdict(m) for m in messages]

        # Start the chat with the conversation history
        convo = self.model.start_chat(history=messages)

        # send message with the new request from the user.
        convo.send_message(message)

        # return the response
        return convo.last.text

class ChattyUI:
    def __init__(
            self,
            service: GeminiService,
            system_prompt: str = SYSTEM_PROMPT,
            system_prompt_response: str = MODEL_RESPONSE
    ) -> None:
        self.service = service
        self.system_prompt = system_prompt
        self.system_prompt_response = system_prompt_response

    def close(self):
        self.interface.close()

    def answer(self, message: str, history: list[str]) -> str:
        # Init the chatbot with the starting prompts.
        # Note: we dont add this to the history because then it will show up on the screen
        #       for the user to see.
        messages = [Message(Message.User, self.system_prompt),
                    Message(Message.Model, self.system_prompt_response)]

        # add user and assistant message from history
        for user_content, assistant_content in history:
            messages.append(Message(Message.User, user_content))
            messages.append(Message(Message.Model, assistant_content))

        return service.get_completion(messages=messages, message=message)

service = GeminiService()
chatty = ChattyUI(service.model)
print(chatty.answer("test", []))

Any other information you'd like to share?
Im working on a YouTube tutorial for this and would like to have a working version in Colab.  I just don't understand why it doesn't work in Colab.
I suppose i could try and add a service account with GOOGLE_APPLICATION_CREDENTIALS but Gemini shouldn't need one.  Unless this is an underlying issue with Colab accessing gemini that i haven't heard of.

sorry this was my bad found the error after sending this.


Key binding is not working

Is command set-syntax:language correct?
I'm trying to bind set-syntax:ruby or set-syntax:json but nothing happens.
As you can see in image below, command was executed but language still Plain Text

Same problem here :(

I found the solution, it's case sensitive. Try JSON instead of json.


Promote more lifecycle to ZiplineCodeSession

Previously TreehouseApp was tracking exceptions on behalf of a class that could do it better.
This introduces listeners at the CodeSession level in addition to the existing listeners at the CodeHost level.

Part of https://github.com/cashapp/redwood/issues/1458

⚠ stacked PR!


Develop

Pull Request Test Coverage Report for Build<PHONE_NUMBER>
Details

39 of 56 (69.64%)  changed or added relevant lines in 5 files are covered.
3 unchanged lines in 2 files lost coverage.
Overall coverage decreased (-0.5%) to 70.293%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

sis_provisioner/routers.py
12
17
70.59%

sis_provisioner/dao/student.py
6
12
50.0%

sis_provisioner/models.py
0
6
0.0%

Files with Coverage Reduction
New Missed Lines
%

sis_provisioner/dao/student.py
1
53.33%

sis_provisioner/models.py
2
52.02%

Totals

Change from base Build<PHONE_NUMBER>:
-0.5%

Covered Lines:
672

Relevant Lines:
956

💛  - Coveralls


Get ASGs to manage based on tag

Describe the feature request
The current tag lookup works if you want to manage all ASGs uniformly, however you cannot exclude ASGs that are still managed by the OSS Cluster Autoscaler.
Support adding a specific tag to enable/disable the update handler on ASGs for the update handler to lookup at runtime.
Why do you personally want this feature to be implemented?
No response
How long have you been using this project?
No response
Additional information
No response

Once https://github.com/TwiN/aws-eks-asg-rolling-update-handler/pull/21 is merged you can come up with your own tags to select which ASGs should be selected.


Added some better and more accurate on how to return errors

Some improved docs on how to generate errors.
To reference #207

Thank you for this documentation - really helpful. Would be interesting if there is any easy "default" way to delete a file on error (shouldn't this be done automatically on exceptions?).

@JHGitty I think its OK, to leave the file, if you dont want the file should be touched, then just use the validation method.
The problem with the validation method, its only for Dropzone and Blueimp, and I dont think that unlink($event->getFile()) is a problem to run

Oh.. any idea how I may do it for FineUploader?

First make a ErrorHandler for Fineuploader so you use the validation method, then add it to the config and it seems like the fineuploader already can handle errors in the controller.

@lsv Thanks for your contribution!


Create Dockerfile

👋 Hey everybody! I'm using LocalStack as part of a Docker Compose setup. As part of this effort I've whipped up a simple Dockerfile so that I can use terraform-local in the same fashion as Hashicorp's official Terraform image to apply Terraform after LocalStack spins up via depends_on and healthchecks. I pushed this to my own Dockerhub for the time being but it'd be great to have an upstream version. Happy to coordinate with you all on this and help in any way that I can; this probably needs a GH Action workflow and an official Dockerhub repo on top of what I have here.

Hi @alexjplant! This is pretty interesting! Could you please elaborate on how you are using this Docker image in your setup / workflow?

Sure! I have a Docker Compose file that spins up localstack as a service with a healthcheck that tests for its endpoints being available. I then have a service that depends on it (i.e. starts once it's available) that runs the terraform-local image that I built. The terraform-local service mounts a volume with the Terraform to be applied to localstack services. This arrangement makes it so that a fellow developer can git pull my project and simply do a docker compose up to get rolling; IMO this is much easier than jumping through pip and brew hoops to get things installed. It also has the added benefit of being as ephemeral as possible so that you don't have to worry about state in between runs.
localstack's value proposition is making AWS development tighter/easier and I think my use case takes this one step further :-). Let me know how I can help or if you have any other questions.


Multiple swagger JSON's in swagger-ui.html

Hello, I have a question. Looks like springfox doesn't automatically document spring-ws endpoints (websocket). I didn't find any solution how to document it manually, so I decided to write Swagger JSON doc manually, serve it as a static content and add it to Springfox's swagger-ui.html page. I was able to do this by adding another  tag directly in a browser
Now I want to add it in the application the code. Could you tell me how to do this? Or how to add endpoints to existing generated documentation manually?
Thanks :)

You need to create a bean that implements the SwaggerResourcesProvider interface. You'd have to make a composite bean that uses the InMemorySwaggerResourcesProvider.java and adds your own json file to it as well. Keep in mind you need to make this new bean a @Primary bean.

This works! Thank you!
@Configuration
public class SwaggerWsEndpointsConfig {

    @Primary
    @Bean
    public SwaggerResourcesProvider swaggerResourcesProvider(InMemorySwaggerResourcesProvider defaultResourcesProvider) {
        return () -> {
            SwaggerResource wsResource = new SwaggerResource();
            wsResource.setName("ws endpoints");
            wsResource.setSwaggerVersion("2.0");
            wsResource.setLocation("/v2/websockets.json");

            List<SwaggerResource> resources = new ArrayList<>(defaultResourcesProvider.get());
            resources.add(wsResource);
            return resources;
        };
    }
}


revise old contract unit tests

We've now got fns for generating different types of account. These are managed to ensure no collisions occur. The newer unit tests use them but the old ones do not, and instead make accounts ad-hoc which could lead to collisions.
We need to transfer the old tests over to using the new account creation methods.

This appears to be low priority but I'm not sure, so moving to needs refinement


TDRD 217 Remove PropertyProtected and use editable instead

With Parliamentary Archives LegalStatus and RightsCopyright will be editable so we can use the existing editable property in the FileProperty table to get protected properties instead.

Will merge this when https://github.com/nationalarchives/tdr-draft-metadata-validator/pull/80 is merged so I can test together.


Missing icon(s?)

This icon is not available through the integration. This is the first icon I've run into that doesn't seem to be available. Am I missing something?

Maybe it is a new icon, then I'll have to release a new version.
Can you please post a screenshot of what you tried? And please try some other icons and see if they work (like si:google).
Also, did you restart Home Assistant after installing the integration?

Oh sorry, overread that this is the first icon that's NOT available. Gonna make an update with the new icons later today

It should work in the newest version!


Add logout via api/logout

@fzaninotto
Francois, this is PR for logout process:

User click on logout button;
Run api request to api/logout
Server should return request with redirectURL
Clean token from COOKIE and remove userId from localStorage
Redirect to URL from request

This issue has been done. PR was approved by Francois and merged to develop.


fix: do not add trailing commit the last positional parameter already…

… has one

Thanks for your contribution! Please replace this text with a description of what this PR is changing or adding and why, list any relevant issues, and review the contribution guidelines below.

[x] I’ve reviewed the contributor guide and applied the relevant portions to this PR.

Contribution guidelines:

See our contributor guide for general expectations for PRs.
Larger or significant changes should be discussed in an issue before creating a PR.
Contributions to our repos should follow the Dart style guide and use dart format.

Note that this repository uses Gerrit for code reviews. Your pull request will be automatically converted into a Gerrit CL and a link to the CL written into this PR. The review will happen on Gerrit but you can also push additional commits to this PR to update the code review.

https://critique.corp.google.com/cl/695463353

In case it wasn't clear to you, the review is happening at https://dart-review.googlesource.com/c/sdk/+/394660. You'll need to use that link to open Garret so that you can read the review comments. You can reply to comments there, but you can also make changes and push new commits to the PR the way you would for any GitHub review and they will be copied to Garret.


python icalendar, downloadable calendar events for classes and meetings,...

-install python icalendar
-downloadable calendar events for classes and meetings
fixes #172 and #173

@karbassi eventually we can open this up as a public classes calendar feed for all classes but I figure since we still only have the one HTML101 for the moment, individual event downloads made more sense.  we can easily expand this to an organization wide calendar if we need it.


feat: support legacy enterprise users

Description

Related Issue(s)

#{issue number}

Verification

[ ] Your code builds clean without any errors or warnings
[ ] Manual testing done (required)
[ ] Relevant automated test added (if you find this hard, leave it and we'll help out)

Documentation

[ ] Documentation is updated (either in docs-directory, Altinnpedia or a separate linked PR in altinn-studio-docs., if applicable)

Main merged to new branch #728


Cannot add captions/labels to multiple charts respectively

问题简述 (One-line summary)
When multiple radar charts (and other kinds of chart, like pie chart) are drawn in the same frame, a separated caption/label for each chart is usually needed to tell things apart.
For example, this is what we have:

Expected:

版本及环境 (Version & Environment)

ECharts 版本 (ECharts version): 2.1.10
浏览器类型和版本 (Browser version): Chrome Version 51.0.2704.103 (64-bit)
操作系统类型和版本 (OS Version): OS X Yosemite 10.10.5

可能哪里有问题 (What went wrong)
Suggestions

add caption object to polar, including properties like caption text, show, formatter, position, etc.
consider the same functionality for other kinds of chart, like funnel chart and etc.

ECharts配置项 (ECharts option)

option = {
...
 polar : [
        {
            indicator : [
                {text : 'Bronx', max : 100},
                {text : 'Manhattan', max  : 100},
                {text : 'Brooklyn', max  : 100},
                {text : 'Staten Island', max  : 100},
                {text : 'Queens', max  : 100}
            ],
            center : ['20%','33%'],
            radius : '38%'
        },
        {
           indicator : [
                {text : 'Bronx', max : 100},
                {text : 'Manhattan', max  : 100},
                {text : 'Brooklyn', max  : 100},
                {text : 'Staten Island', max  : 100},
                {text : 'Queens', max  : 100}
            ],
            center : ['40%','33%'],
            radius : '38%'
        },
        ...
...
}

其他信息 (Other comments)

You can have multiple title. Like this example

But grid and multiple title are supported by Echarts 3 rather than Echarts 2 right?

I'm afraid there are no way to do it in ECharts 2 unless you use dom to show the captions

Got you, thank you @pissang


The "No Styles Available" page link to the blog post doesn't work

The link in the middle of the screen "this blog post" isn't clickable.
http://esri.github.io/arcgis-vectortile-style-editor/

It's not clickable for me either, but FYI - the URL it's pointing to is https://blogs.esri.com/esri/arcgis/2015/11/19/how-to-customize-esri-vector-basemaps/


Context record id is nil when creating a new resource with an attachment

I have a custom storage set up for Vimeo and I'm passing in some record information so that I can query a few variables like video title, privacy settings, update status, etc. to store in Vimeo.
Here's a quick gist of what I'm doing and where my issue lies,
class Shrine
  module Storage
    class Vimeo
      # …

      def upload(io, id, shrine_metadata: {}, **upload_options)
        # This line always fails on record_type#create because record_id is nil
        resource = shrine_metadata["record_type"].constantize
                     .find shrine_metadata["record_id"]

        # …
      end

      # …
    end
  end
end

require "shrine"
require "shrine/storage/file_system"
require_dependency "lib/shrine/storage/vimeo"

Shrine.storages = {
  store: Shrine::Storage::FileSystem.new("public", prefix: "uploads/store"),
  vimeo: Shrine::Storage::Vimeo.new(access_token: ENV['VIMEO_ACCESS_TOKEN'])
}

Shrine.plugin :activerecord
Shrine.plugin :cached_attachment_data
Shrine.plugin :backgrounding
Shrine::Attacher.promote { |data| PromoteAttachmentWorker.perform_async data }
Shrine::Attacher.delete { |data| DeleteAttachmentWorker.perform_async data }

class VideoUploader < Shrine
  plugin :default_storage, store: :vimeo
  plugin :add_metadata

  add_metadata do |io, context|
    { record_id: context[:record].id, record_type: context[:record].class.name }
  end
end

class Record < ApplicationRecord
  include VideoUploader[:video]
end

Whenever the attachment's meta is stored, the record's ID is actually nil (I'm assuming it hasn't been persisted to the database yet). What do I need to do in order to get the correct context within my custom storage class? Is there a better way to get the attachment's context? I couldn't find anything documented that suggests there is.
It's also worth noting that I'm running the uploader in the background using Sidekiq + Shrine's backgrounding plugin. And I know sometimes Sidekiq is a bit too fast, so this might be a race condition that needs to be solved another way.
Everything works correctly when updating an existing resource. It's only an issue whenever a new resource is created with a video attachment.

This actually was a race condition. I resolved it by adding a delay to the worker,
Shrine::Attacher.promote { |data| PromoteAttachmentWorker.perform_in 30.seconds, data }
Shrine::Attacher.delete { |data| DeleteAttachmentWorker.perform_in 30.seconds, data }

Might be worth noting in the docs?

Yes, when extracting metadata the record will be nil on creation, because metadata is extracted as soon as the file is attached, which happens before the record is actually saved.
However, you shouldn't need to retrieve the record from inside the storage class, since when cached file is promoted to permanent storage the record is already retrieved. Also, the storage class should always just know about managing files (SRP 😉), and receive any upload options from the outside. You can pass options from the outside using the upload_options plugin:
class VideoUploader < Shrine
  plugin :upload_options, store: -> (io, context) do
    video = context[:record]
    {
      title: video.title,
      privacy: video.privacy,
      ...
    }
  end
end

The returned options will then be forwarded to Shrine::Storage::Vimeo#upload:
class Shrine
  module Storage
    class Vimeo
      # ...

      def upload(io, id, shrine_metadata: {}, **options)
        options #=> {title: "...", privacy: "...", ...}
        # ...
      end

      # ...
    end
  end
end

It's strange that you have to add a delay for spawning background workers (though great that you've figured it out!), because Shrine's activerecord plugin use the after_commit callback to call the block that spawns the background jobs, which means the jobs are spawned after the database transaction for creating/updating the record was committed, so the record should always be retrievable no matter how fast the worker starts processing the job 🤔

I didn't know about the upload_options plugin, that's perfect. I didn't like the hack I put together and figured there was a better way, so thanks for pointing me in the right direction. I'm not really sure why the job processes before the record is persisted.

@ezekg Happy to help 😃. Can you confirm whether you're still experiencing the race condition where you need to delay spawning the job? Were you receiving it only on creation because of the blank record ID, or did you receive it on update as well?

It was just on creation, updates worked fine. Not why how Sidekiq would run before the record is persisted if Shrine is using the after_commit hook. I can dig in more Monday if need be.

@ezekg If you're manually wrapping record creation/update around a transaction, that could be one reason (I'm pretty sure ActiveRecord doesn't nest transactions, though not 100%).
Ok, I'll close this issue for now, and if you discover that it persisted, feel free to open another issue with any details around the race condition that you found.


Update sbt-scalafix to 0.10.1

Updates ch.epfl.scala:sbt-scalafix from 0.10.0 to 0.10.1.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "ch.epfl.scala", artifactId = "sbt-scalafix" } ]

labels: sbt-plugin-update, early-semver-minor, semver-spec-patch, commit-count:1

Codecov Report

Merging #478 (0e47e68) into main (8125d39) will not change coverage.
The diff coverage is n/a.

@@            Coverage Diff             @@
##              main      #478    +/-   ##
==========================================
  Coverage   100.00%   100.00%
==========================================
  Files           66        56    -10
  Lines         2753      1862   -891
  Branches       234        81   -153
==========================================
- Hits          2753      1862   -891

Impacted Files
Coverage Δ

packages/recheck/src/lib/exe.ts

...ges/eslint-plugin-redos/src/rules/no-vulnerable.ts

packages/recheck/src/lib/java.ts

packages/recheck/src/lib/agent.ts

packages/recheck/src/lib/env.ts

packages/recheck/src/lib/pure.ts

packages/recheck/src/lib/native.ts

packages/recheck/src/browser.ts

packages/recheck/src/main.ts

packages/recheck/src/lib/worker-pool.ts

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 8125d39...0e47e68. Read the comment docs.


Fix local debuginfo inline functions and line info

Closes #832.
Relies on #834.

This generally LGTM but in the JSON output the function_offset fields appear to be wrong. Specifically, it seems that they're filled with the function address, not the function offset (i.e. the function address minus the module base address).

That's odd, that shouldn't be from symbolic since that is opening the modules directly. I'll check and see how that's occurring.

Ah, it looks like it needs the module base address added to the function offset in the trait interface.

Looking good, but can you rebase this so I can merge it?

Done!


Add check_config API

Description:
Expose our check_config script as an API endpoint for integration into the frontend.
To do:

[ ] Write tests.
[ ] Fix tests.

Frontend PR: https://github.com/home-assistant/home-assistant-polymer/pull/206
Related issue (if applicable): fixes #
Pull request in home-assistant.github.io with documentation (if applicable): home-assistant/home-assistant.github.io#<home-assistant.github.io PR number goes here>
Example entry for configuration.yaml (if applicable):
config:

Checklist:
If the code does not interact with devices:

[ ] Local tests with tox run successfully. Your PR cannot be merged unless tests pass
[ ] Tests have been added to verify that the new code works.

Oh nice. As next we have a editor :smile:


DateTimeField and auto_now_add=True

Would be nice, for now I store date as isoformat() in string and do the logic on my side.
I may give it a try but I wonder how the sorting will behave.

Hi,
Typed fields would be great, is there any progress on this?   If they exist, I have some I'd like to add..
S

I totally forgotten this, and, sadly, I have absolutely no idea where my draft is :-/
So no, no news on this, sorry.

I realise you're busy. .. just one thing though; I'm trying to look at the feasibility of typed fields:
I copied StringField to IntegerField, but am having a bit of trouble working out where to put the code along the lines of return int(result)  - any hints ?

Doh, worked something out, sorry for the noise.

:)
If you have something that works, is generic enough to ease the creation of other typed field, and is tested, I would love to take a look and maybe integrate it (in the contrib part, or maybe it could be in a separate repository as we did for limyd-extensions)

Yup - nothing generic right now, just a POC where I overrode the get method.
Are these assumptions correct:

get isn't the only method that can return a value
some methods might return more than one value

Assuming the above, I need to add a list of which fields are getters + need results to be transformed and any that return sequences of results (and need to transform each one).
Ungeneric version:
class IntegerField(StringField):

    to_python = int

    def get(self):
        result = super(IntegerField, self).get()
        return self.to_python(result)```

Given infinite time, Sider looks like a great option for handling types (it seems to have no ORM, it just does redis <-> python type conversion).

Your assumptions are correct.
But you're "lucky": each field has two attributes available_getters and available_modifiers. They hold the names of the command that are allowed to get/set data.
But it will be cumbersome to work on each of these getters and setters. In a "long term thinking", the first thing to do I think is to add a way in the internals of limpyd, to have a hook that intervenes before a setter, and after a getter.

Do you think Sider (I don't know this lib) could be plugged with Lipmyd?

But you're "lucky": each field has two attributes available_getters and available_modifiers. They hold the names of the command that are allowed to get/set data.

I had a look at available_getters, but not every one of those would need transforming, e.g. in StringField:  available_getters = ('get', 'getbit', 'getrange', 'strlen', 'bitcount', )

Do you think Sider (I don't know this lib) could be plugged with Lipmyd?

Maybe .... it would involve some rewriting, OTH both projects are basically missing what the other provides.
https://sider.readthedocs.io/en/0.3.1

About the getters, let's imagine you create your field based on a StringField, you can set yoursel the available getters and setters of your field. Because calling getbit on a DateTimeField doesn't make any sense.
So, first, with these attributes you can learn what are the default getters and setters, redefine these attributes in your field class without the ones that don't make any sense, and override the setters and getters methods to do what you want.
PS: I checked on my current and previous computer for my old work on this subject, without success :(

I guess it's a similar story for modifiers ?

yes of course

I've been experimenting in this branch   https://github.com/stuaxo/redis-limpyd/tree/sider-types
It's not uber clean, but there is an ipython notebook I've been trying stuff in.
It creates typed.IntegerField and others by wrapping the Sider types, though I've only really tested IntegerField and BooleanField for get and set so far.
to_storage is more tricky than to_python, as it's not obvious which field is going to be a value, there is the start of some code for it, but not hooked up.
The only code in lympid itself is a flag to let you override _make_command_method - kind of clunky, but it works:
https://github.com/stuaxo/redis-limpyd/blob/sider-types/limpyd/fields.py#L95

While I got quite a bit working, as I'm on a tight deadline, I switched tack and got redisco working on python3 (as I'm getting existing features working there, not adding new ones).
I can definitely help to polish this, up, but it's still a bridge.  A proper fix would be more tightly integrated - or go the other way and add the missing features to Sider.


fix: deploy_configs was removed when invoke SetCreateParams

这个 PR 实现什么功能/修复什么问题:
修复: 创建 server 时 deploy_configs 被 SetCreateParams 删掉了，导致部署文件都失效
是否需要 backport 到之前的 release 分支:

release/2.8.0
release/2.9.0

/area region
/cc @swordqiu @wanyaoqi

/cherrypick release/2.9.0

/cherrypick release/2.8.0

Codecov Report

Merging #696 into master will increase coverage by 0.01%.
The diff coverage is n/a.

@@            Coverage Diff            @@
##           master    #696      +/-   ##
=========================================
+ Coverage    7.47%   7.48%   +0.01%
=========================================
  Files         347     347
  Lines       54900   54898       -2
=========================================
+ Hits         4103    4110       +7
+ Misses      50417   50408       -9
  Partials      380     380

Flag
Coverage Δ

#aFlag
7.48% <ø> (+0.01%)
:arrow_up:

Impacted Files
Coverage Δ

pkg/compute/models/guests.go
0.67% <ø> (ø)
:arrow_up:

pkg/hostman/isolated_device/isolated_device.go
3.35% <0%> (+0.01%)
:arrow_up:

pkg/util/rbacutils/rabc.go
72.34% <0%> (+2.65%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update f56be35...e242f40. Read the comment docs.

/lgtm
/approve


SSLHandler does not detect or act on incoming close_notify messages

It appears that Netty does not detect the SSLEngine processing a close_notify, so you end up with a situation where the SSLEngine is in state CLOSED, but no upstream notifications are generated and there is no way for the application to know it has happened.
This can happen if the client closes the SSLSocket but not the underlying Socket. The Netty server receives  the close_notify, unwraps it, and sends back the reply of close_notify, however the SSLHandler code itself does not detect the CLOSED state of the SSLEngine. This can be tested with a client closing an SSLSocket which has been created by wrapping a Socket, or by a Netty client calling SSLHandler.close().
From the SSLEngine JDK docs...

"A peer will signal its intent to close by sending its own closure handshake message. After this message has been received > and processed by the local SSLEngine's unwrap() call, the application can detect the close by calling unwrap() and looking > for a SSLEngineResult with status "CLOSED", or if isInboundDone() returns true"

The SSLHandler does process the handshake, but does not detect the status "CLOSED".
I suggest that an optional Future be allowed on the SSLHandler that can be notified of the SSLEngine closing, as this is not a common situation, as usually the underlying socket will also be closed which will be detected normally.
I have a situation where I initially communicate via SSL, then close the SSLEngines and continue using TCP over the same socket, this works fine using SSLSocket directly as it returns -1 on a get when the SSLSocket closes but the underlying Socket remains open.

I will have a look

ok sounds like a bug..


fix: don't raise an error for 'shell'

A "shell" is a software application (for example, /bin/bash or /bin/sh) that provides an interface to a computer. Do not use this term to describe the prompt where you type commands.

fixes https://github.com/Vale-at-Red-Hat/vale-at-red-hat/issues/88


issue about VelaUX RBAC，an account has a role about project list，can't see project list

Describe the bug

To Reproduce

@haima95 Please provide more info. Such as the role config.

@haima95 Please provide more info. Such as the role config.

@haima95 Please provide more info. Such as the role config.

I has upload my video about my action to baiducloud，you can check。 because the video is 30M，so I have to upload cloud
链接：https://pan.baidu.com/s/1XTWHWj1udQLjIyWCW4C7Xg?pwd=g9q8
提取码：g9q8


[LANG-1612] testGetAllFields and testGetFieldsWithAnnotation sometimes fail

Issue link: https://issues.apache.org/jira/browse/LANG-1612
Don't use assertArrayEquals() because it would fail when the array returned by FieldUtils.getAllFields(...) is not ordered in the way the test assumes in annotatedFields and ArrayUtils.addAll(fieldsInteger, fieldsNumber).
Instead, do order-independent assertions on: 1) the size of the returned array, 2) the element it contains.

Hi @tongxin97
Thank you for your PR.
There is a simpler solution: sorting, see git master

Ah I see. I didn't do it this way since Field is not comparable. I saw your workaround with the sort() method. Thanks for keeping me updated.


forecast not updating because of api limit reached

Describe the bug
forecast not updating because of api limit reached. It has worked for a few days I think and new I see this error.
To Reproduce
Steps to reproduce the behavior:
No Idea. I was in hospital after I updated to the latest version of solcast and didn't really monitor after I did the update.
Expected behavior
That it updates. The api limit can't be reached yet.
Screenshots

Logs
`Deze fout is ontstaan door een aangepaste integratie.

Logger: custom_components.solcast_solar.solcastapi
Bron: custom_components/solcast_solar/solcastapi.py:2313
integratie: Solcast PV Forecast (documentatie, problemen)
Eerst voorgekomen: 08:06:12 (9 gebeurtenissen)
Laatst gelogd: 08:16:17

The API is busy, pausing 77 seconds before retry
The API is busy, pausing 92 seconds before retry
The API is busy, pausing 117 seconds before retry
The API is busy, pausing 132 seconds before retry
The API is busy, pausing 149 seconds before retry`
`Deze fout is ontstaan door een aangepaste integratie.

Logger: custom_components.solcast_solar.solcastapi
Bron: custom_components/solcast_solar/solcastapi.py:2337
integratie: Solcast PV Forecast (documentatie, problemen)
Eerst voorgekomen: 08:18:46 (1 gebeurtenissen)
Laatst gelogd: 08:18:46

API was tried 10 times, but all attempts failed`
`Deze fout is ontstaan door een aangepaste integratie.

Logger: custom_components.solcast_solar.solcastapi
Bron: custom_components/solcast_solar/solcastapi.py:1981
integratie: Solcast PV Forecast (documentatie, problemen)
Eerst voorgekomen: 08:18:46 (1 gebeurtenissen)
Laatst gelogd: 08:18:46

Forecast update for site x failed so not getting remaining sites`
`Deze fout is ontstaan door een aangepaste integratie.

Logger: custom_components.solcast_solar.solcastapi
Bron: custom_components/solcast_solar/solcastapi.py:2005
integratie: Solcast PV Forecast (documentatie, problemen)
Eerst voorgekomen: 08:18:46 (1 gebeurtenissen)
Laatst gelogd: 08:18:46

At least one site forecast failed to fetch, so forecast has not been built`

This does not look like the API limit is exceeded.
The Solcast API was unavailable many times and the integration gave up. There are five attempts logged, but it gives up at ten, so has the log that you shared been altered?
Dutch:
_Het lijkt er niet op dat de API-limiet wordt overschreden.
De Solcast API was vaak niet beschikbaar en de integratie stopte. Er zijn vijf pogingen geregistreerd, maar hij stopt bij tien. Is de log die je hebt gedeeld gewijzigd?_

No log timestamps to give a sense of timing makes things very hard to interpret.
It does interest me that there were only five retries reported, and you have two rooftops. Ten divided by two is... I will take a look at that, but I'm fairly certain that it should do up to ten tries per site. I wrote the code.
TL>DR: "API was tried 10 times, but all attempts failed." It failed. The API limit was not exceeded.
Did it fail because the integration has an issue? I doubt it. 4,500+ user experiences would agree with that. Did it fail because the Solcast API was issuing "429 error, go away we're busy" responses? Most likely.
But I could be wrong. I'm holding this issue open for now, but I strongly suggest that you try a 'developer tools' "Solcast PV Forecast: Update" (or run your fetch automation), or (if you're using the wonderful auto-update then) "Solcast PV Forecast: Force Update" and report back.
At times Solcast gets really busy, and the 429 responses seem to last ages. The other night I had a thirty minute period where I could get nothing from them.

@bert1111, thanks heaps for being a user by the way! Our interaction makes me want to book a trip back to the Netherlands. What an ace place.
(From a HAOS shell for me, via SSH... [Spoiler alert for a debug logging improvement coming]. The 'Troubleshooting' discussion has the details about how to get these logs.)
less /config/home-assistant.log, then type &solcast, then shift F to follow. Magic.
2024-10-17 16:23:06.432 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Checking for stale usage cache
2024-10-17 16:23:06.432 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site b68d-c05a-c2b3-2cf9
2024-10-17 16:23:06.433 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site b68d-c05a-c2b3-2cf9 lastday 2024-10-24 numhours 176
2024-10-17 16:23:06.442 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/b68d-c05a-c2b3-2cf9/forecasts
2024-10-17 16:23:06.442 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2024-10-17 16:23:07.539 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch successful
2024-10-17 16:23:07.539 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API returned data, API counter incremented from 2 to 3
2024-10-17 16:23:07.539 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing API usage cache file: /config/solcast-usage.json
2024-10-17 16:23:07.557 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned data type <class 'dict'>
2024-10-17 16:23:07.557 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session status 200/Success
2024-10-17 16:23:07.557 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] 353 records returned
2024-10-17 16:23:07.795 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecasts dictionary length 9069
2024-10-17 16:23:07.795 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Undampened forecasts dictionary length 1056
2024-10-17 16:23:07.796 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP data call processing took 0.238 seconds
2024-10-17 16:23:07.800 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site 83d5-ab72-2a9a-2397
2024-10-17 16:23:07.800 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site 83d5-ab72-2a9a-2397 lastday 2024-10-24 numhours 176
2024-10-17 16:23:07.800 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/83d5-ab72-2a9a-2397/forecasts
2024-10-17 16:23:07.801 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2024-10-17 16:23:08.321 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch successful
2024-10-17 16:23:08.322 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API returned data, API counter incremented from 3 to 4
2024-10-17 16:23:08.322 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing API usage cache file: /config/solcast-usage.json
2024-10-17 16:23:08.346 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned data type <class 'dict'>
2024-10-17 16:23:08.348 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session status 200/Success
2024-10-17 16:23:08.349 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] 353 records returned
2024-10-17 16:23:08.586 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecasts dictionary length 9069
2024-10-17 16:23:08.587 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Undampened forecasts dictionary length 1056
2024-10-17 16:23:08.588 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP data call processing took 0.238 seconds
2024-10-17 16:23:08.946 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecast data from 2024-10-17 to 2024-10-24 contains all 48 intervals
2024-10-17 16:23:08.946 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Calculating splines
2024-10-17 16:23:08.979 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Build forecast processing took 0.387 seconds
2024-10-17 16:23:09.376 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Saved dampened forecast cache
2024-10-17 16:23:09.399 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Saved undampened forecast cache
2024-10-17 16:23:09.400 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update completed successfully

I'm from Belgium by the way but you're close :).
I'm going to set for debug in a bit but after rebooting HA and trying the force update I now see following in my HA logs.
Setting for the debug now.
`Deze fout is ontstaan door een aangepaste integratie.
Logger: custom_components.solcast_solar.solcastapi
Bron: custom_components/solcast_solar/solcastapi.py:2237
integratie: Solcast PV Forecast (documentatie, problemen)
Eerst voorgekomen: 10:46:40 (3 gebeurtenissen)
Laatst gelogd: 10:47:10
Exception in __http_data_call(): 'fetch': Traceback (most recent call last): File "/config/custom_components/solcast_solar/solcastapi.py", line 2158, in __http_data_call resp_dict = self.tasks['fetch'].result() ~~~~~~~~~~^^^^^^^^^ KeyError: 'fetch'`

Damn. Soooo close. 🤣
The only way you could have hit that error is if the 'past' forecast data was trying to be fetched, indicating that your solcast.json in the config folder has been somehow killed. The integration is attempting to get the "estimated actuals" for the past, which makes no sense given your last forecast update was about five hours ago according to your screen shot.
What is the content of /config/solcast.json?

You haven't messed with your Solcast API key at all have you? There is an issue that I have fixed, yet not released yet on that front...

I just see someone else make a bug report with almost the same settings I am using. He uses two accounts and I only use one but he also uses the automatic updates. No I haven't changed anything. The data in the solcast.json is ranging from forecasts": [{"period_start": "2024-06-24T22:30:00+00:00 to estimate90": 0.0}, {"period_start": "2024-10-24T21:30:00+00:00", "pv_estimate": 0.0, "pv_estimate10": 0.0, "pv_estimate90": 0.0}], "tally": 4.6573}}, "last_updated": "2024-10-17T08:46:39.226400+00:00", "version": 4}

Very different settings. Very different issues i believe.
Debug logs for the win. I can't do much by guessing.

2024-10-17 10:54:58.529 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration solcast_solar which has not been tested by Home Assist
ant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant
2024-10-17 10:55:13.234 DEBUG (MainThread) [custom_components.solcast_solar] Auto-update options: {'auto_update': 1}
2024-10-17 10:55:13.235 DEBUG (MainThread) [custom_components.solcast_solar] Estimate to use options: {'key_estimate': 'estimate'}
2024-10-17 10:55:13.235 DEBUG (MainThread) [custom_components.solcast_solar] Attribute options: {'attr_brk_estimate': True, 'attr_brk_estimate10': False,
'attr_brk_estimate90': False, 'attr_brk_site': False, 'attr_brk_halfhourly': True, 'attr_brk_hourly': False, 'attr_brk_detailed': False}
2024-10-17 10:55:13.235 DEBUG (MainThread) [custom_components.solcast_solar] Custom sensor options: {'customhoursensor': 1}
2024-10-17 10:55:13.235 DEBUG (MainThread) [custom_components.solcast_solar] Hard limit: {'hard_limit': 100000}
2024-10-17 10:55:13.236 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Configuration directory is /config
2024-10-17 10:55:13.249 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Sites cache exists
2024-10-17 10:55:13.249 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Connecting to https://api.solcast.com.au/rooftop_sites?format=json
&api_key=PqOvKO
2024-10-17 10:55:14.128 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned status 200/Success in __sites_data()
2024-10-17 10:55:14.129 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing sites cache
2024-10-17 10:55:14.164 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Sites data: {'sites': [{'name': 'Thuis oost', 'resource_id': '4220
-0dc5-b46c-5b6e', 'capacity': 5, 'capacity_dc': 3.04, 'longitude': ., 'latitude': ., 'azimuth': -65, 'tilt': 30, 'install_date': '2022-02-
08T23:00:00.0000000Z', 'loss_factor': 0.94, 'tags': ['Tessenderlo']}, {'name': 'Thuis west', 'resource_id': '', 'capacity': 5, 'capacit
y_dc': 3.04, 'longitude': ., 'latitude': ., 'azimuth': 112, 'tilt': 30, 'install_date': '2022-02-21T23:00:00.0000000Z', 'loss_factor': 0.9
4, 'tags': ['****']}], 'page_count': 1, 'current_page': 1, 'total_records': 2}
2024-10-17 10:55:14.165 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Sites loaded for ******PqOvKO
2024-10-17 10:55:14.167 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Usage cache exists for ******PqOvKO
2024-10-17 10:55:14.219 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Usage cache for ******PqOvKO last reset 2024-10-17 02:00:00
2024-10-17 10:55:14.219 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Usage loaded for ******PqOvKO
2024-10-17 10:55:14.220 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API counter for ******PqOvKO is 1/8
2024-10-17 10:55:14.220 DEBUG (MainThread) [custom_components.solcast_solar] Successful init
2024-10-17 10:55:14.220 INFO (MainThread) [custom_components.solcast_solar]
If all else fails, then open an issue and our community will try to
help: https://github.com/BJReplay/ha-solcast-solar/issues
2024-10-17 10:55:14.416 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data cache /config/solcast.json exists, file type is <class 'dict'

2024-10-17 10:55:14.493 INFO (MainThread) [custom_components.solcast_solar.solcastapi] /config/solcast.json data loaded
2024-10-17 10:55:14.529 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data cache /config/solcast-undampened.json exists, file type is <c
lass 'dict'>
2024-10-17 10:55:14.532 INFO (MainThread) [custom_components.solcast_solar.solcastapi] /config/solcast-undampened.json data loaded
2024-10-17 10:55:15.026 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-17 contains all 48 records
2024-10-17 10:55:15.027 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-18 contains all 48 records
2024-10-17 10:55:15.028 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-19 contains all 48 records
2024-10-17 10:55:15.029 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-20 contains all 48 records
2024-10-17 10:55:15.032 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-21 contains all 48 records
2024-10-17 10:55:15.034 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-22 contains all 48 records
2024-10-17 10:55:15.036 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-23 contains all 48 records
2024-10-17 10:55:15.040 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-24 contains all 48 records
2024-10-17 10:55:15.040 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Calculating splines
2024-10-17 10:55:15.092 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Build forecast processing took 0.558 seconds
2024-10-17 10:55:15.101 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Sun rise / set today: 08:06:12 / 18:43:03
2024-10-17 10:55:15.103 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Auto update: Total seconds 38211, divisions: 4 updates, interval:
9552 seconds
2024-10-17 10:55:15.103 INFO (MainThread) [custom_components.solcast_solar.coordinator] Auto-update will update forecasts 4 times between sunrise and suns
et
2024-10-17 10:55:15.103 INFO (MainThread) [custom_components.solcast_solar.coordinator] Auto-scheduled forecast update for today at 13:24:36, 16:03:48
2024-10-17 10:55:15.104 INFO (MainThread) [custom_components.solcast_solar.coordinator] Auto-scheduled forecast update for tomorrow at 08:07:52, 10:46:09,
13:24:26, 16:02:43
2024-10-17 10:55:15.105 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Started task listeners
2024-10-17 10:55:15.106 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Started task check_fetch
2024-10-17 10:55:15.106 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Started task midnight_update
2024-10-17 10:55:15.106 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Finished fetching solcast_solar data in 0.000 seconds (success: T
rue)
2024-10-17 10:55:17.688 DEBUG (MainThread) [custom_components.solcast_solar] UTC times are converted to Europe/Brussels

in the integration I now see this.

Thuis oost and west look like they survived a perfectly normal startup sequence, with auto-update enabled. Good on you!
Perfect logs for the job.
API used is showing as '1'. This is weird for a two-site set up, but may be explained by a previous failure. The integration tracks this in a solcast-usage.json cache, so the content of that file in the `/config' folder will confirm that.
You've got an update scheduled for today at 13:24:36. Let's see if it turns to custard or not. Post logs, but I'm fairly certain it will succeed. (Your earlier error about the 'fetch' task I cannot work out. I have tried this code path many times without incident.

I think it is the api is busy, this is the second morning in a row that the 09:xx has failed owing to 429 error, here are my logs from today.2024-10-17 08:07:52.740 WARNING (SyncWorker_0) [homeassistant.loader] We found a custom integration solcast_solar which has not been tested by Home Assistant. This component might cause stability problems, be sure to disable it if you experience issues with Home Assistant2024-10-17 09:36:00.415 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 16 seconds before retry2024-10-17 09:36:16.440 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 43 seconds before retry2024-10-17 09:36:59.466 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 48 seconds before retry2024-10-17 09:37:47.572 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 62 seconds before retry2024-10-17 09:38:49.609 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 81 seconds before retry2024-10-17 09:40:10.640 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 103 seconds before retry2024-10-17 09:41:53.679 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 106 seconds before retry2024-10-17 09:43:39.743 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 127 seconds before retry2024-10-17 09:45:46.781 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] The API is busy, pausing 147 seconds before retry2024-10-17 09:48:13.900 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] API was tried 10 times, but all attempts failed2024-10-17 09:48:13.900 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update for site 02d4-edbc-faba-cb5b failed2024-10-17 09:48:13.900 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] At least one site forecast failed to fetch, so forecast has not been builtCheersHenrySent from my iPadOn 17 Oct 2024, at 09:37, Steve Saunders @.***> wrote:
No log timestamps to give a sense of timing makes things very hard to interpret.
It does interest me that there were only five retries reported, and you have two rooftops. Ten divided by two is... I will take a look at that, but I'm fairly certain that it should do up to ten tries per site. I wrote the code.
TL>DR: "API was tried 10 times, but all attempts failed." It failed. The API limit was not exceeded.
Did it fail because the integration has an issue? I doubt it. 4,500+ user experiences would agree with that. Did it fail because the Solcast API was issuing "429 error, go away we're busy" responses? Most likely.
But I could be wrong. I'm holding this issue open for now, but I strongly suggest that you try a 'developer tools' "Solcast PV Forecast: Update" (or run your fetch automation), or (if you're using the wonderful auto-update then) "Solcast PV Forecast: Force Update" and report back.
At times Solcast gets really busy, and the 429 responses seem to last ages. The other night I had a thirty minute period where I could get nothing from them.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: @.***>

2024-10-17 11:27:17.736 INFO (MainThread) [custom_components.solcast_solar] Service call: update_forecasts
2024-10-17 11:27:17.737 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Checking for stale usage cache
2024-10-17 11:27:17.739 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site 4220-0dc5-b46c-5b6e
2024-10-17 11:27:17.740 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site xxxxxxxxxx lastday 2024-10-24 numhou
rs 181
2024-10-17 11:27:17.741 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/xxxxxxxxxxxxxxx/forecasts
2024-10-17 11:27:17.741 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2024-10-17 11:27:18.857 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch successful
2024-10-17 11:27:18.857 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API returned data, using force fetch so not incrementing API counter
2024-10-17 11:27:19.414 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned data type <class 'dict'>
2024-10-17 11:27:19.415 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session status 200/Success
2024-10-17 11:27:19.415 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] 363 records returned
2024-10-17 11:27:19.480 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecasts dictionary length 5855
2024-10-17 11:27:19.482 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Undampened forecasts dictionary length 1056
2024-10-17 11:27:19.892 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP data call processing took 0.477 seconds
2024-10-17 11:27:19.898 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site cee7-9c91-f834-32ad
2024-10-17 11:27:19.899 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site xxxxxxxxxxxxxxx lastday 2024-10-24 numhours 181
2024-10-17 11:27:19.902 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/cxxxxxxxxxxxxxxxxxxxxxxx/forecasts
2024-10-17 11:27:19.902 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2024-10-17 11:27:21.066 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch successful
2024-10-17 11:27:21.067 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API returned data, using force fetch so not incrementing API counter
2024-10-17 11:27:21.614 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned data type <class 'dict'>
2024-10-17 11:27:21.615 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session status 200/Success
2024-10-17 11:27:21.615 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] 363 records returned
2024-10-17 11:27:21.664 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecasts dictionary length 5855
2024-10-17 11:27:21.666 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Undampened forecasts dictionary length 1056
2024-10-17 11:27:22.059 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP data call processing took 0.444 seconds
2024-10-17 11:27:22.547 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-17 contains all 48 records
2024-10-17 11:27:22.548 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-18 contains all 48 records
2024-10-17 11:27:22.549 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-19 contains all 48 records
2024-10-17 11:27:22.550 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-20 contains all 48 records
2024-10-17 11:27:22.551 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-21 contains all 48 records
2024-10-17 11:27:22.554 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-22 contains all 48 records
2024-10-17 11:27:22.556 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-23 contains all 48 records
2024-10-17 11:27:22.559 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Data for 2024-10-24 contains all 48 records
2024-10-17 11:27:22.560 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Calculating splines
2024-10-17 11:27:22.612 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Build forecast processing took 0.546 seconds
2024-10-17 11:27:22.947 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Saved forecast cache /config/solcast.json
2024-10-17 11:27:23.003 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Saved forecast cache /config/solcast-undampened.json
2024-10-17 11:27:23.003 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update completed successfully

And normal service returns, @bert1111. Looks perfect.

But why does it throw an error that the api limit was exceeded? Or isn't that what the error was? Anyway I'll keep an eye on it.

I think it is the api is busy

@hwb45, they do get feisty at times. I have built in retries to try tenaciously, unlike Oziee versions that just tossed their toes in the air at the first smell of a 429. But sometimes... bad things happen.
Unexplained things have happened in this issue report, including a raised exception, which should not happen, nor be possible, but that may be because @bert1111 (to misquote @BJReplay, and no offence intended) may have tried sacrificing a chicken or some other unnatural act to try and revive something that was simply a temporary error, as noted in the readme.
(You didn't go the voodoo route @bert1111, did you, by changing API keys and expecting a different result? 🤣 I'm still interested in the exception that you got that should not be, though. I cannot explain it, and that code was tested many times just days ago. Weird. If you did use voodoo it would be useful to know...)

But why does it throw an error that the api limit was exceeded? Or isn't that what the error was? Anyway I'll keep an eye on it.

I didn't see anywhere that the limit was exceeded. I just saw that it gave up after ten attempts at getting one of your sites. Ten tries. API limit ten. Coincidence.

No voodoo here 🤣. I wish I understood HA and programming better so I could be of more assistance. I tried but my memory and concentration issues aren't helping being a programmer.

This is the log from the last automatic update.
2024-10-17 13:20:00.330 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Forecast will update in 276 seconds 2024-10-17 13:24:36.332 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Checking for stale usage cache 2024-10-17 13:24:36.334 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site 4220-0dc5-b46c-5b6e 2024-10-17 13:24:36.335 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site 4220-0dc5-b46c-5b6e lastday 2024-10-24 numhours 179 2024-10-17 13:24:36.335 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/4220-0dc5-b46c-5b6e/forecasts 2024-10-17 13:24:36.336 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast 2024-10-17 13:24:37.522 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch successful 2024-10-17 13:24:37.523 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API returned data, API counter incremented from 1 to 2 2024-10-17 13:24:37.523 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing API usage cache file: /config/solcast-usage.json 2024-10-17 13:24:38.091 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned data type <class 'dict'> 2024-10-17 13:24:38.091 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session status 200/Success 2024-10-17 13:24:38.092 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] 359 records returned 2024-10-17 13:24:38.142 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecasts dictionary length 5855 2024-10-17 13:24:38.143 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Undampened forecasts dictionary length 1056 2024-10-17 13:24:38.548 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP data call processing took 0.457 seconds 2024-10-17 13:24:38.553 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site cee7-9c91-f834-32ad 2024-10-17 13:24:38.555 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site cee7-9c91-f834-32ad lastday 2024-10-24 numhours 179 2024-10-17 13:24:38.556 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/cee7-9c91-f834-32ad/forecasts 2024-10-17 13:24:38.556 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast 2024-10-17 13:24:39.190 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing API usage cache file: /config/solcast-usage.json 2024-10-17 13:24:39.195 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] API allowed polling limit has been exceeded, API counter set to 8/8 2024-10-17 13:24:39.196 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update for the last site queued failed (cee7-9c91-f834-32ad) so not getting remaining sites - API use count may be odd 2024-10-17 13:24:39.196 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] At least one site forecast failed to fetch, so forecast has not been built 2024-10-17 13:24:39.210 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Next forecast update scheduled for 2024-10-17 16:03:48

This is the log from the last automatic update.
2024-10-17 13:20:00.330 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Forecast will update in 276 seconds 2024-10-17 13:24:36.332 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Checking for stale usage cache 2024-10-17 13:24:36.334 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site 4220-0dc5-b46c-5b6e 2024-10-17 13:24:36.335 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site 4220-0dc5-b46c-5b6e lastday 2024-10-24 numhours 179 2024-10-17 13:24:36.335 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/4220-0dc5-b46c-5b6e/forecasts 2024-10-17 13:24:36.336 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast 2024-10-17 13:24:37.522 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch successful 2024-10-17 13:24:37.523 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API returned data, API counter incremented from 1 to 2 2024-10-17 13:24:37.523 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing API usage cache file: /config/solcast-usage.json 2024-10-17 13:24:38.091 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned data type <class 'dict'> 2024-10-17 13:24:38.091 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session status 200/Success 2024-10-17 13:24:38.092 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] 359 records returned 2024-10-17 13:24:38.142 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecasts dictionary length 5855 2024-10-17 13:24:38.143 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Undampened forecasts dictionary length 1056 2024-10-17 13:24:38.548 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP data call processing took 0.457 seconds 2024-10-17 13:24:38.553 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site cee7-9c91-f834-32ad 2024-10-17 13:24:38.555 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site cee7-9c91-f834-32ad lastday 2024-10-24 numhours 179 2024-10-17 13:24:38.556 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/cee7-9c91-f834-32ad/forecasts 2024-10-17 13:24:38.556 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast 2024-10-17 13:24:39.190 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing API usage cache file: /config/solcast-usage.json 2024-10-17 13:24:39.195 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] API allowed polling limit has been exceeded, API counter set to 8/8 2024-10-17 13:24:39.196 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update for the last site queued failed (cee7-9c91-f834-32ad) so not getting remaining sites - API use count may be odd 2024-10-17 13:24:39.196 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] At least one site forecast failed to fetch, so forecast has not been built 2024-10-17 13:24:39.210 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Next forecast update scheduled for 2024-10-17 16:03:48

Please format logs enclosed in tripple back-ticks. ```

'''2024-10-17 13:20:00.330 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Forecast will update in 276 seconds
2024-10-17 13:24:36.332 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Checking for stale usage cache
2024-10-17 13:24:36.334 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site 4220-0dc5-b46c-5b6e
2024-10-17 13:24:36.335 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site 4220-0dc5-b46c-5b6e lastday 2024-10-24 numhours 179
2024-10-17 13:24:36.335 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/4220-0dc5-b46c-5b6e/forecasts
2024-10-17 13:24:36.336 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2024-10-17 13:24:37.522 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch successful
2024-10-17 13:24:37.523 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] API returned data, API counter incremented from 1 to 2
2024-10-17 13:24:37.523 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing API usage cache file: /config/solcast-usage.json
2024-10-17 13:24:38.091 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session returned data type <class 'dict'>
2024-10-17 13:24:38.091 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP session status 200/Success
2024-10-17 13:24:38.092 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] 359 records returned
2024-10-17 13:24:38.142 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Forecasts dictionary length 5855
2024-10-17 13:24:38.143 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Undampened forecasts dictionary length 1056
2024-10-17 13:24:38.548 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] HTTP data call processing took 0.457 seconds
2024-10-17 13:24:38.553 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site cee7-9c91-f834-32ad
2024-10-17 13:24:38.555 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Polling API for site cee7-9c91-f834-32ad lastday 2024-10-24 numhours 179
2024-10-17 13:24:38.556 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetch data url: https://api.solcast.com.au/rooftop_sites/cee7-9c91-f834-32ad/forecasts
2024-10-17 13:24:38.556 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Fetching forecast
2024-10-17 13:24:39.190 DEBUG (MainThread) [custom_components.solcast_solar.solcastapi] Writing API usage cache file: /config/solcast-usage.json
2024-10-17 13:24:39.195 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] API allowed polling limit has been exceeded, API counter set to 8/8
2024-10-17 13:24:39.196 WARNING (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update for the last site queued failed (cee7-9c91-f834-32ad) so not getting remaining sites - API use count may be odd
2024-10-17 13:24:39.196 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] At least one site forecast failed to fetch, so forecast has not been built
2024-10-17 13:24:39.210 DEBUG (MainThread) [custom_components.solcast_solar.coordinator] Next forecast update scheduled for 2024-10-17 16:03:48'''

Back ticks and then GitHub will get you. ```
Not single quotes '''. If you want to quote a single thing, lile entity.my_thing, then enclose it inline with single back-ticks around the words using `.
What I think has happened here is that the internal counter in the integration of API used is zero, while Solcast says different. Then the integration adjusts based on the specific API response of usage counter exceeded. This will reset at midnight UTC.

Furher supported by

API returned data, using force fetch so not incrementing API counter

You've been force-fetching, and the integration specifically does not keep count of this with auto-update enabled. It is by design.
 Maybe they will, maybe they won't...
A 'force fetch' defeats the counter on purpose. It has to, and so bad things can happen if you cause them to be bad.

(If you're listening, @billy-solcast, how about into the suggestions box: Including api used in the response for a forecast fetch? That's low cost, single call and could be used to re-adjust an internal counter...)

Should force-fetches decrement available API count?
If only to avoid issues like this one being raised 🤣?
At least until @billy-solcast comes to the party with a remaining API count in the API response (though, TBF, I expect that's a significant change for them to consider, as it would either require an optional parameter for the request to say that you're asking for it, or a added component in the returned json, that might be breaking for people not expecting it).

Should force-fetches decrement available API count?
If only to avoid issues like this one being raised  🤣?

Well I just want to say, that wasn't the issue I raised.
And it seems although the call failed it used up api calls somehow.
I did use force fetch but not that many times that all my calls where used for yesterday. I believe that I used six when it failed due to calls being used up. The other calls returned api is busy so my total of ten shouldn't have been used up and it should have failed because of that at a later time.

Well I just want to say, that wasn't the issue I raised.

I know, but...

I believe that I used six that returned data

The thing is that every call uses one of your calls.
@autoSteve made a judgement call that it was reasonable for a user might want to force the integration to refresh the forecast - even if the integration thought that all calls were used - so changed the force fetch call to both ignore the number of calls that had been made - so that it would run even if the count was already at the limit - and not increase the count of calls made.
So, if you made say six forced calls yesterday, even if you weren't complaining about this, this is what might have happened:
You might have thought that the force fetch call neither used an API count nor increased the used count - but as far as Solcast is concerned, each one very much did use an API count.
The integration didn't know you'd made six calls - because it wasn't counting.
Later on - after only 2 calls that it was counting - it got the error from Solcast indicating that you had exceeded your limit.
So, as Steve pointed out - it now knows that the limit has been exceeded, so it sets its internal counter to the limit (which is 8 - as you've configured).
That's why I suggested we change the way force-fetch works (slightly) - it ignores the fact that the integration thinks the limit might have been exceeded (so if I user thinks or knows that it hasn't), it at least tries), but it still attempts to fetch.  But it increments the API used count otherwise.

Well I just want to say, that wasn't the issue I raised.

And, in regards to the issue you did originally raise - we're not ignoring it - but after the fact, without timestamped logs, it's hard to say what did happen.
What we do know now, is that things appear to be working as expected:

Solcast does occasionally get very busy, and despite up to 10 retries, a forecast will fail to be fetched.
This can sometimes happen after one out of two arrays (sites) are fetched, leading to an odd number of API calls being used up.
The integration will use a call to initialise if the cached configuration files require this, for various reasons, again ending up using an odd number of API calls for a two array (site) system.

Regarding not incrementing the API use counter for a force update, this may be a silly thing to be doing @BJReplay, because it does leave a bit of an information gap.
Rumbing about in the back of my head is the notion of a triad of API limit, API used, and auto-update API used. All three could be displayed on the dignostics page giving far greater situational awareness.
What would result however, is the need to add yet another configuration option (involving Urdu) that would be something like "API calls to use for auto-update" or a name far more thought through.

The present implementation is simple. I like simple.  (Until it does...)

a triad of API limit, API used, and auto-update API used

Seems overly complex

is the need to add yet another configuration option

Seems overly complex

referring to see how many folks step on the API use rake head and get smacked in the face with the handle

Seems fair enough

The present implementation is simple. I like simple.

Agreed.
Perhaps the only variation would be to ignore the fact that you might appear to be at the limit, and attempt an update if already at the limit if force is called.

Perhaps the only variation would be to ignore the fact that you might appear to be at the limit, and attempt an update if already at the limit if force is called.

Just revert the change here?
https://github.com/BJReplay/ha-solcast-solar/blob/e9a792ba4cfd1f16e33141913d494d6ed279269f/custom_components/solcast_solar/solcastapi.py#L2321
But not the change higher up that checks the current limit

Just revert the change here?

Yeah, nah. It's a force fetch. Can't update the API used, as that's against the current simple implementation "rules" of "auto-update should only mess with that counter or it will get lost".
Force is force. It's up to the user to be mindful of actual total used count above what auto-update will consume. Which brings my brain back to instrumentation of this, and added complexity, which brings my brain back to Urdu can't deal.

Fair enough.

P.S. Have been re-netflixing Three Body Problem.  Is still good second time around.  Less blurry, too.

Hi,
The total API limit was not used for me today.
I have set Automatic update of forecasts from sunrise to sunset
It was OK with automation in the old version.
Don't know what the problem could be?

Thanks

Hi, The total API limit was not used for me today. I have set Automatic update of forecasts from sunrise to sunset It was OK with automation in the old version.

You are going to need to provide a tonne more information for anyone to say why @matomatusov.

How does your issue remotely relate to hitting the API limit?
When you say "automation" was that an HA automation used with "the old version", or the auto-update feature of the integration?
What version are you running?
What version was "the old version"?
What happened in the logs?
Turn info level logging on for the integration and restart (if info is not already your default). The proposed update times throughout the day will be logged on startup. Are these wrong?
When is UTC midnight for you (i.e. what constitutes "a day", i.e. when is usage reset)?
At what time of day is six calls used up as shown?
When was the first forecast fetch of the day, and what was the API count at the time? Did this align with the proposed updates logged at startup?

I could keep asking questions...

Despite providing very little information @matomatusov, I think I have an answer for you.

Note in my diagnostic usage the value says 8. This is wrong. I have used two API calls in this UTC day period, and the logged forecast fetches reflect this. Now open the diagnostic sensor. I expect you will see something like this. The history shows correctly, yet the API used does not.

So this is an instrumentation bug, and thanks for pointing it out. It will be fixed in a future version.

Is it a bug with multi-array systems?
I have a tell-tale on one of my background dashboards where I have tweaks and controls, and it doesn't seem to miss a beat.
History doesn't go back that far because I'm in the habit of uninstalling and re-installing for testing 🤣

I don't know if any one else has problems with the api calls but yesterday every single one failed with API busy except the one at 5:32 AM.
Today up until now the same result, API busy try again. Does everyone have the same problem then or not? I tried logging in to my solcast account and downloading the JSON from there and that works, probably until my API calls run out? Did solcast maybe change something you are not aware of? Just trying to help.

I don't know if any one else has problems with the api calls but yesterday every single one failed with API busy except the one at 5:32 AM.

Similar story for me.

Perhaps more load in your hemisphere?

Did solcast maybe change something you are not aware of?

Not that we're aware of.

Just trying to help.

Speculation is not actually helping.
I am going to close this issue.
I don't believe that it represents a problem that can be traced back to anything other the Solcast servers being busy.
https://status.solcast.com/ doesn't tell us there is anything wrong and the history doesn't suggest that there have been problems since June


Prevent local backend file fragmentation by file preallocation.

What does this PR change? What problem does it solve?
File preallocation has already been added to the restorer, but not to the local backend.
This PR moves preallocation code from restorer to the fs package and adds preallocation to the local backend.
Before this PR:
restic init --copy-chunker-params
created restic repository f0e0af3f2f at repo2

restic backup "C:\Program Files"
no parent snapshot found, will read all files

Files:       94897 new,     0 changed,     0 unmodified
Dirs:         9490 new,     0 changed,     0 unmodified
Added to the repo: 9.117 GiB

processed 94897 files, 10.011 GiB in 5:04
snapshot 5b287af7 saved

contig /a /s repo2

...
repo2\data\0d\0da13caef4a1746608248a8d1286dbddd20ef89dc524c6c2adb8da5c9e4a8a41 is in 16 fragments
repo2\data\0e\0eaa27e755e25a65f45327e400d058df6b426cdefa0b083c54a743d10fb90db9 is in 14 fragments
repo2\data\2c\2cb9a6bc2645656f8f288d131f251edcf5a76314f46cb67f2300febea6b61aaf is in 13 fragments
repo2\data\34\34a9dfd06cc971ef6d5768471245ae8ad26572942133d54313baa6484d116bb6 is in 12 fragments
repo2\data\74\74cfcbf057fc7af25b4fe930b766d95e2a96f1c0a166d66dff9812da85f8df30 is in 12 fragments
repo2\data\85\854e876ad694fcff94b5927520ac1aeb85dd555cf47f5d2525212047de4f0a7c is in 17 fragments
repo2\data\be\be958f56d448fe766f6fd7289d015c831f730678529d7355bb5b2ed790741a0b is in 16 fragments
repo2\data\bf\bfd119b20a3a4cdc7831d126f551f0211aaa9c56ae06603365a31365c6e4d3e9 is in 12 fragments
...

Summary:
     Number of files processed:      2322
     Number unsuccessfully procesed: 0
     Average fragmentation       : 2.9186 frags/file

Many files are split into 10-20 fragments.
After this PR:
restic init --copy-chunker-params
created restic repository 665523814e at repo2

restic backup "C:\Program Files"
no parent snapshot found, will read all files

Files:       94897 new,     0 changed,     0 unmodified
Dirs:         9490 new,     0 changed,     0 unmodified
Added to the repo: 9.117 GiB

processed 94897 files, 10.011 GiB in 4:55
snapshot f97d7203 saved

contig /a /s repo2

...
repo2\data is in 8 fragments
...

Summary:
     Number of files processed:      2317
     Number unsuccessfully procesed: 0
     Average fragmentation       : 1.00388 frags/file

Only the data dir (containing 256 subdirs) is fragmented.
Was the change discussed in an issue or in the forum before?
https://github.com/restic/restic/issues/2679
https://github.com/restic/restic/pull/2893
Checklist

[x] I have read the Contribution Guidelines
[x] I have enabled maintainer edits for this PR
[ ] I have added tests for all changes in this PR
[ ] I have added documentation for the changes (in the manual)
[x] There's a new file in changelog/unreleased/ that describes the changes for our users (template here)
[x] I have run gofmt on the code in all commits
[x] All commit messages are formatted in the same style as the other commits in the repo
[x] I'm done, this Pull Request is ready for review

Additionally I tried various Windows-specific prealocation methods instead of the generic os.File.Truncate().
For backup, all methods are the same within margin of error but results for external SMR drive are very strange.
Preallocmethod
CMR SATA HDD
CMR SATA HDD
CMR USB HDD
SMR USB HDD

None
96.93 +0.00%2.16
256.76 -0.00%3.19
173.06 +0.00%2.34
410.82 -0.00%2.52

Truncate
94.25 -2.77%1.00
236.74 -7.80%1.00
166.09 -4.02%1.00
557.40 +35.68%1.01

Method5
94.33 -2.68%1.00
234.44 -8.69%1.00
166.21 -3.96%1.00
557.66 +35.74%1.01

Method65
94.75 -2.25%1.00
236.49 -7.90%1.00
165.86 -4.16%1.00
558.55 +35.96%1.01

None - Master branch
Truncate - This PR
Method5 - SetFileInformationByHandle(FileAllocationInfo)
Method65 - SetFileInformationByHandle(FileEndOfFileInfo) + SetFileInformationByHandle(FileAllocationInfo)
The data in the columns is not comparable because they correspond to different computers and backup sources.
The first sub column is the average time of 10 runs, the second is the average fragmentation.

Thanks a lot for the review, I'll fix it tomorrow.

As far as I know, Truncate internally calls SetFileInformationByHandle(FileEndOfFileInfo).

No, it's FileEndOfFileInfo+FileAllocationInfo (Method65):
benchstat Restore_Truncate.txt Restore_Method5.txt
name           old time/op  new time/op  delta
CMR_HDD1_SATA    202s ВЄ 2%    209s ВЄ 1%  +3.40%  (p=0.000 n=10+10)
CMR_HDD2_SATA    658s ВЄ 3%    714s ВЄ 2%  +8.45%  (p=0.000 n=9+9)
CMR_HDD_USB      315s ВЄ 1%    317s ВЄ 1%  +0.83%  (p=0.002 n=9+9)
SMR_HDD_USB      370s ВЄ 8%    371s ВЄ 2%    ~     (p=0.549 n=9+10)

benchstat Restore_Truncate.txt Restore_Method65.txt
name           old time/op  new time/op  delta
CMR_HDD1_SATA    202s ВЄ 2%    202s ВЄ 2%   ~     (p=0.796 n=10+10)
CMR_HDD2_SATA    658s ВЄ 3%    663s ВЄ 2%   ~     (p=0.278 n=9+10)
CMR_HDD_USB      315s ВЄ 1%    318s ВЄ 5%   ~     (p=1.000 n=9+9)
SMR_HDD_USB      370s ВЄ 8%    374s ВЄ 6%   ~     (p=0.400 n=9+10)

Could you try what happens when there's only a single filestorer or when the local backend does not fsync?

Yes, of course, but every test run takes many hours, so give me a few days.

@MichaelEischer TL;DR By disabling fsync, you will get a speedup of two or three times. Single filestorer is a bad thing on SMR.

CMR SATA HDD
SMR USB HDD

None
276.49 +0.00%3.19
419.75 +0.00%2.57

Truncate
254.81 -7.84%1.00
561.25 +33.71%1.01

Single
244.71 -11.50%1.00
708.39 +68.76%1.01

NoSync
130.04 -52.97%1.00
170.36 -59.41%1.01

NoSync+Single
130.77 -52.71%1.00
165.94 -60.47%1.01

None - Master branch
Truncate - This PR
Single - This PR and a single filestorer
NoSync - This PR and no fsync
benchstat Backup_None.txt Backup_Truncate.txt
name           old time/op  new time/op  delta
CMR_HDD2_SATA    276s ± 2%    255s ± 1%   -7.73%  (p=0.000 n=9+10)
SMR_HDD_USB      421s ± 4%    563s ± 3%  +33.69%  (p=0.000 n=10+10)

benchstat Backup_Truncate.txt Backup_NoSync.txt
name           old time/op  new time/op  delta
CMR_HDD2_SATA    255s ± 1%    130s ± 1%  -48.96%  (p=0.000 n=10+10)
SMR_HDD_USB      563s ± 3%    173s ±14%  -69.25%  (p=0.000 n=10+10)

benchstat Backup_Truncate.txt Backup_Single.txt
name           old time/op  new time/op  delta
CMR_HDD2_SATA    255s ± 1%    245s ± 1%   -3.98%  (p=0.000 n=10+10)
SMR_HDD_USB      563s ± 3%    713s ± 1%  +26.67%  (p=0.000 n=10+9)

benchstat Backup_Truncate.txt Backup_NoSync_Single.txt
name           old time/op  new time/op  delta
CMR_HDD2_SATA    255s ± 1%    131s ± 1%  -48.69%  (p=0.000 n=10+10)
SMR_HDD_USB      563s ± 3%    166s ± 8%  -70.52%  (p=0.000 n=10+9)

benchstat Backup_NoSync.txt Backup_NoSync_Single.txt
name           old time/op  new time/op  delta
CMR_HDD2_SATA    130s ± 1%    131s ± 1%   ~     (p=0.165 n=10+10)
SMR_HDD_USB      173s ±14%    166s ± 8%   ~     (p=0.278 n=10+9)

Raw benchstat data.

It's not the same SMR drive (now i have only two ST6000DM003), but results are very promising:
             │ Backup_None.txt │        Backup_Truncate.txt         │
             │     sec/op      │   sec/op    vs base                │
CMR_HDD_SATA       56.16 ± 10%   53.50 ± 8%        ~ (p=0.101 n=10)
CMR_HDD_USB        181.4 ±  4%   165.5 ± 5%   -8.77% (p=0.000 n=10)
SMR_HDD_USB        351.0 ±  1%   247.8 ± 2%  -29.40% (p=0.000 n=10)


[Machine Learning] Time Series Analysis and Forecasting using Auto Time Series

Proposal Submission
Proposed title of article
[Machine Learning] Time Series Analysis and Forecasting using Auto Time-Series
Proposed article introduction
Time series is a type of machine learning model that is used to forecast future events based on previous events that have been observed at regular time intervals. Times series models can be used to forecast stock market prices, forex prices, pandemic spread, and weather forecasts.
Time series models are used to extract meaningful statistics and other characteristics of the data. This makes the model understand patterns from data and gain useful insights. The model uses the knowledge gained to make forecasts of future values such as a stock. The time series model uses curve fitting techniques to plot the forecast values and come up with a line of best fit. The plot diagrams are used to represent the predicted values.
In this tutorial, we will download our dataset and perform dataset preprocessing. We will then use the auto series library to build our model. The auto series library will use a machine learning pipeline to automate the time series modeling process. Auto time series will use multiple models such as Facebook Prophet, ARIMA, SARIMA, and VAR to train the model. It will train the model and produce an optimized model. Auto time series will search through all the time series libraries and select the one with the best results.
Key takeaways

Types of Time series models. (Univariate, bivariate, and multivariate)
Time series libraries such as Facebook Prophet, ARIMA, SARIMA, and VAR
Time series decomposition and analysis.
Training the model using multiple models.
Select the best model and make forecasts.

Article quality
This tutorial is unique because it will discuss different types of time series models and libraries. We will compare libraries such as Facebook Prophet, ARIMA, SARIMA, and VAR. We will then combine all these libraries using Auto time series and train the same model. The final model will be fully optimized and give correct forecasts/predictions. It will have detailed and guided steps that are easy for a reader to follow.
References
Please list links to any published content/research that you intend to use to support/guide this article.
Conclusion
Finally, remove the Pre-Submission advice section and all our blockquoted notes as you fill in the form before you submit.
We look forwarding to reviewing your topic suggestion.
Templates to use as guides

How To Guide Template
Software Review Template
Tutorial Template

Great topic 🚀 , make sure it matches the following:

Adds value beyond what is in the official docs and what is covered in other blog sites.
To avoid potential article closure, it should not overlap with existing EngEd articles or incoming topic suggestions.

Please reference any relevant EngEd articles in yours and build a unique project - Approved.