Memory Leak in bufferPool

Summary:
After upgrading from v0.30.0 to v0.35.1, I have noticed a memory leak in my server application. The memory consumption keeps increasing and is not released unless I restart the service.
Expected Behavior:
Under normal circumstances, the server should manage memory appropriately, without any memory leaks.
Actual Behavior:
After upgrading to v0.35.1, I observed a continuous increase in memory usage, which is not released over time. As a result, the server becomes memory-intensive and requires a restart to free up the memory.
Environment:
Previous Version: v0.30.0
Current Version: v0.35.1
Operating System:[Linux debian 6.1.0-7-amd64
go: 1.19

pprof
File: webService.run
Build ID: 046cd5ef5de1f12c5d4b70dbb96ebafb3be85a33
Type: inuse_space
Time: Jun 3, 2023 at 4:22pm (+0330)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) web
(pprof) top
Showing nodes accounting for 388.27MB, 86.67% of 447.98MB total
Dropped 194 nodes (cum <= 2.24MB)
Showing top 10 nodes out of 116
      flat  flat%   sum%        cum   cum%
  201.79MB 45.04% 45.04%   201.79MB 45.04%  github.com/quic-go/quic-go.init.0.func1
   58.38MB 13.03% 58.08%    58.38MB 13.03%  github.com/webservice/utility.ReadHTTPDataBodyResponse
   32.08MB  7.16% 65.24%    32.08MB  7.16%  github.com/smallnest/goframe.(*lengthFieldBasedFrameConn).ReadFrame
   29.01MB  6.48% 71.71%    30.01MB  6.70%  github.com/quic-go/quic-go.newConn
   15.77MB  3.52% 75.23%    15.77MB  3.52%  regexp.(*Regexp).get
   13.22MB  2.95% 78.18%    15.36MB  3.43%  compress/flate.NewWriter
   11.50MB  2.57% 80.75%    11.50MB  2.57%  runtime.malg
   11.50MB  2.57% 83.32%    11.50MB  2.57%  golang.org/x/net/internal/socket.(*mmsghdrsPacker).prepare
    8.01MB  1.79% 85.11%     8.01MB  1.79%  runtime.allocm
       7MB  1.56% 86.67%    41.27MB  9.21%  github.com/quic-go/quic-go.(*Transport).init.func1

this is how i used quic:
func listenQuic(httpServer *http.Server) {
	var SupportedVersions = []quic.VersionNumber{quic.VersionDraft29, quic.Version1, quic.Version2}
	quicConf := &quic.Config{EnableDatagrams: true, Versions: SupportedVersions}

	serverQuic = http3.Server{
		Handler:         httpServer.Handler,
		Addr:            httpServer.Addr,
		QuicConfig:      quicConf,
		EnableDatagrams: true,
		MaxHeaderBytes:  httpServer.MaxHeaderBytes,
		TLSConfig:       httpServer.TLSConfig,
	}

	if err := serverQuic.ListenAndServe(); err != nil {
		log.Printf("Quic Server error: %v", err)
	}

}

you may have overlooked invoking b.Release() in some critical paths.
i will conduct further investigation to analyze the memory leak issue. I will provide additional information once I have completed the investigation.

this is a qlog connection tracer :
	quicConf.Tracer = func(ctx context.Context, p logging.Perspective, connID quic.ConnectionID) logging.ConnectionTracer {
		filename := fmt.Sprintf("/tmp/server_%x.qlog", connID)
		f, err := os.Create(filename)
		if err != nil {
			log.Fatal(err)
		}
		log.Printf("Creating qlog file %s.\n", filename)
		return qlog.NewConnectionTracer(NewBufferedWriteCloser(bufio.NewWriter(f), f), p, connID)
	}

cat /tmp/server_656231376164393166313537623036323462633636396134.qlog
{"qlog_format":"NDJSON","qlog_version":"draft-02","title":"quic-go qlog","configuration":{"code_version":"v0.35.1"},"trace":{"vantage_point":{"type":"server"},"common_fields":{"ODCID":"eb17ad91f157b0624bc669a4","group_id":"eb17ad91f157b0624bc669a4","reference_time":1685820897102.842,"time_format":"relative"}}}
{"time":0.029979,"name":"recovery:congestion_state_updated","data":{"new":"slow_start"}}
{"time":0.038887,"name":"transport:parameters_set","data":{"owner":"local","original_destination_connection_id":"eb17ad91f157b0624bc669a4","stateless_reset_token":"c6dd7f3687e7299e15e1e19a8a05b2ce","initial_source_connection_id":"c3efb732","disable_active_migration":true,"max_idle_timeout":30000,"max_udp_payload_size":null,"ack_delay_exponent":3,"max_ack_delay":26,"active_connection_id_limit":4,"initial_max_data":786432,"initial_max_stream_data_bidi_local":524288,"initial_max_stream_data_bidi_remote":524288,"initial_max_stream_data_uni":524288,"initial_max_streams_bidi":100,"initial_max_streams_uni":100,"max_datagram_frame_size":1200}}
{"time":0.067908,"name":"security:key_updated","data":{"trigger":"tls","key_type":"client_initial_secret"}}
{"time":0.06813,"name":"security:key_updated","data":{"trigger":"tls","key_type":"server_initial_secret"}}
{"time":0.130437,"name":"transport:version_information","data":{"server_versions":["ff00001d","1","6b3343cf"],"chosen_version":"1"}}
{"time":0.132042,"name":"transport:connection_started","data":{"ip_version":"ipv4","src_ip":"<IP_ADDRESS>","src_port":443,"dst_ip":"<IP_ADDRESS>","dst_port":30820,"src_cid":"(empty)","dst_cid":"eb17ad91f157b0624bc669a4"}}
{"time":0.14231,"name":"transport:packet_received","data":{"header":{"packet_type":"initial","packet_number":0,"version":"1","scil":0,"dcil":12,"dcid":"eb17ad91f157b0624bc669a4"},"raw":{"length":1252,"payload_length":1230},"frames":[{"frame_type":"crypto","offset":0,"length":295}]}}
{"time":0.193827,"name":"transport:packet_sent","data":{"header":{"packet_type":"initial","packet_number":0,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":1252,"payload_length":1238},"frames":[{"frame_type":"ack","acked_ranges":[[0]]},{"frame_type":"crypto","offset":0,"length":56}]}}
{"time":0.196167,"name":"recovery:metrics_updated","data":{"min_rtt":0,"smoothed_rtt":0,"latest_rtt":0,"rtt_variance":0,"congestion_window":40064,"bytes_in_flight":1252,"packets_in_flight":1}}
{"time":0.196913,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"initial","delta":199.987628}}
{"time":1.637331,"name":"transport:packet_received","data":{"header":{"packet_type":"initial","packet_number":1,"version":"1","scil":0,"dcil":4,"dcid":"c3efb732"},"raw":{"length":1252,"payload_length":1238},"frames":[{"frame_type":"ack","acked_ranges":[[0]]},{"frame_type":"crypto","offset":295,"length":396}]}}
{"time":1.640369,"name":"recovery:congestion_state_updated","data":{"new":"application_limited"}}
{"time":1.641004,"name":"recovery:metrics_updated","data":{"min_rtt":1.404801,"smoothed_rtt":1.404801,"latest_rtt":1.404801,"rtt_variance":0.7024,"bytes_in_flight":0}}
{"time":1.641461,"name":"recovery:loss_timer_updated","data":{"event_type":"cancelled"}}
{"time":6.555634,"name":"security:key_updated","data":{"trigger":"tls","key_type":"client_handshake_secret"}}
{"time":6.572389,"name":"security:key_updated","data":{"trigger":"tls","key_type":"server_handshake_secret"}}
{"time":8.409237,"name":"security:key_updated","data":{"trigger":"tls","key_type":"server_1rtt_secret","generation":0}}
{"time":8.507491,"name":"transport:parameters_set","data":{"owner":"remote","initial_source_connection_id":"(empty)","disable_active_migration":true,"max_idle_timeout":30000,"max_udp_payload_size":1452,"ack_delay_exponent":3,"max_ack_delay":26,"active_connection_id_limit":4,"initial_max_data":786432,"initial_max_stream_data_bidi_local":524288,"initial_max_stream_data_bidi_remote":524288,"initial_max_stream_data_uni":524288,"initial_max_streams_uni":100,"max_datagram_frame_size":1220}}
{"time":8.530086,"name":"transport:packet_sent","data":{"header":{"packet_type":"initial","packet_number":1,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":232,"payload_length":218},"frames":[{"frame_type":"ack","acked_ranges":[[0,1]]},{"frame_type":"crypto","offset":56,"length":191}]}}
{"time":8.530686,"name":"transport:packet_sent","data":{"header":{"packet_type":"handshake","packet_number":0,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":1020,"payload_length":1007},"frames":[{"frame_type":"crypto","offset":0,"length":985}]}}
{"time":8.532936,"name":"recovery:metrics_updated","data":{"bytes_in_flight":232,"packets_in_flight":1}}
{"time":8.533428,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"initial","delta":4.203356}}
{"time":8.536384,"name":"recovery:metrics_updated","data":{"bytes_in_flight":1252,"packets_in_flight":2}}
{"time":8.53969,"name":"transport:packet_sent","data":{"header":{"packet_type":"handshake","packet_number":1,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":1252,"payload_length":1239},"frames":[{"frame_type":"crypto","offset":985,"length":1216}]}}
{"time":8.541179,"name":"recovery:metrics_updated","data":{"bytes_in_flight":2504,"packets_in_flight":3}}
{"time":8.543572,"name":"transport:packet_sent","data":{"header":{"packet_type":"handshake","packet_number":2,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":1252,"payload_length":1239},"frames":[{"frame_type":"crypto","offset":2201,"length":1216}]}}
{"time":8.544965,"name":"recovery:metrics_updated","data":{"bytes_in_flight":3756,"packets_in_flight":4}}
{"time":8.54727,"name":"transport:packet_sent","data":{"header":{"packet_type":"handshake","packet_number":3,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":1252,"payload_length":1239},"frames":[{"frame_type":"crypto","offset":3417,"length":1216}]}}
{"time":8.548598,"name":"recovery:metrics_updated","data":{"bytes_in_flight":5008,"packets_in_flight":5}}
{"time":8.555632,"name":"transport:packet_sent","data":{"header":{"packet_type":"handshake","packet_number":4,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":859,"payload_length":846},"frames":[{"frame_type":"crypto","offset":4633,"length":823}]}}
{"time":8.556285,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":0,"key_phase_bit":"0"},"raw":{"length":91},"frames":[{"frame_type":"new_connection_id","sequence_number":3,"retire_prior_to":0,"length":4,"connection_id":"2a27fe2f","stateless_reset_token":"910bfaa1ae04aeeca04128e4be78fe13"},{"frame_type":"new_connection_id","sequence_number":2,"retire_prior_to":0,"length":4,"connection_id":"9b41a388","stateless_reset_token":"c4914c45e32d360ab5a865fdb21fb727"},{"frame_type":"new_connection_id","sequence_number":1,"retire_prior_to":0,"length":4,"connection_id":"3bfef526","stateless_reset_token":"a12884541c7636a75545ef8b0a060ed9"}]}}
{"time":8.557681,"name":"recovery:metrics_updated","data":{"bytes_in_flight":5867,"packets_in_flight":6}}
{"time":8.558646,"name":"recovery:metrics_updated","data":{"bytes_in_flight":5958,"packets_in_flight":7}}
{"time":8.686783,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":1,"key_phase_bit":"0"},"raw":{"length":29},"frames":[{"frame_type":"stream","stream_id":3,"offset":0,"length":8}]}}
{"time":8.688204,"name":"recovery:metrics_updated","data":{"bytes_in_flight":5987,"packets_in_flight":8}}
{"time":8.94724,"name":"transport:packet_received","data":{"header":{"packet_type":"initial","packet_number":2,"version":"1","scil":0,"dcil":4,"dcid":"c3efb732"},"raw":{"length":1252,"payload_length":1238},"frames":[{"frame_type":"crypto","offset":295,"length":396}]}}
{"time":8.96204,"name":"transport:packet_received","data":{"header":{"packet_type":"initial","packet_number":3,"version":"1","scil":0,"dcil":4,"dcid":"c3efb732"},"raw":{"length":1252,"payload_length":1238},"frames":[{"frame_type":"crypto","offset":295,"length":396}]}}
{"time":8.969164,"name":"transport:packet_sent","data":{"header":{"packet_type":"initial","packet_number":2,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":37,"payload_length":23},"frames":[{"frame_type":"ack","acked_ranges":[[0,3]]}]}}
{"time":8.980301,"name":"transport:packet_received","data":{"header":{"packet_type":"initial","packet_number":4,"version":"1","scil":0,"dcil":4,"dcid":"c3efb732"},"raw":{"length":1252,"payload_length":1238},"frames":[{"frame_type":"crypto","offset":295,"length":396}]}}
{"time":8.988827,"name":"transport:packet_received","data":{"header":{"packet_type":"initial","packet_number":5,"version":"1","scil":0,"dcil":4,"dcid":"c3efb732"},"raw":{"length":1252,"payload_length":1238},"frames":[{"frame_type":"crypto","offset":295,"length":396}]}}
{"time":8.991204,"name":"transport:packet_sent","data":{"header":{"packet_type":"initial","packet_number":3,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":37,"payload_length":23},"frames":[{"frame_type":"ack","acked_ranges":[[0,5]]}]}}
{"time":12.627103,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"handshake","delta":0.136226}}
{"time":12.627787,"name":"security:key_discarded","data":{"trigger":"tls","key_type":"server_initial_secret"}}
{"time":12.627787,"name":"security:key_discarded","data":{"trigger":"tls","key_type":"client_initial_secret"}}
{"time":12.630244,"name":"transport:packet_received","data":{"header":{"packet_type":"handshake","packet_number":0,"version":"1","scil":0,"dcil":4,"dcid":"c3efb732"},"raw":{"length":36,"payload_length":23},"frames":[{"frame_type":"ack","acked_ranges":[[0]]}]}}
{"time":12.633364,"name":"recovery:metrics_updated","data":{"smoothed_rtt":1.74,"latest_rtt":4.094877,"rtt_variance":1.199,"bytes_in_flight":4735,"packets_in_flight":6}}
{"time":12.633675,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"handshake","delta":2.451253}}
{"time":12.671107,"name":"transport:packet_received","data":{"header":{"packet_type":"handshake","packet_number":1,"version":"1","scil":0,"dcil":4,"dcid":"c3efb732"},"raw":{"length":36,"payload_length":23},"frames":[{"frame_type":"ack","acked_ranges":[[0,2]]}]}}
{"time":12.676353,"name":"recovery:metrics_updated","data":{"smoothed_rtt":2.037,"latest_rtt":4.117516,"rtt_variance":1.493,"bytes_in_flight":2231,"packets_in_flight":4}}
{"time":12.677176,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"handshake","delta":3.880752}}
{"time":12.89069,"name":"transport:packet_received","data":{"header":{"packet_type":"handshake","packet_number":2,"version":"1","scil":0,"dcil":4,"dcid":"c3efb732"},"raw":{"length":83,"payload_length":70},"frames":[{"frame_type":"ack","acked_ranges":[[0,4]]},{"frame_type":"crypto","offset":0,"length":44}]}}
{"time":12.894638,"name":"recovery:metrics_updated","data":{"smoothed_rtt":2.323,"latest_rtt":4.325338,"rtt_variance":1.691,"bytes_in_flight":120,"packets_in_flight":2}}
{"time":12.895083,"name":"recovery:loss_timer_updated","data":{"event_type":"cancelled"}}
{"time":12.931574,"name":"security:key_updated","data":{"trigger":"tls","key_type":"client_1rtt_secret","generation":0}}
{"time":12.947782,"name":"transport:packet_sent","data":{"header":{"packet_type":"handshake","packet_number":5,"version":"1","scil":4,"scid":"c3efb732","dcil":0},"raw":{"length":36,"payload_length":23},"frames":[{"frame_type":"ack","acked_ranges":[[0,2]]}]}}
{"time":12.95347,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":0,"key_phase_bit":"0"},"raw":{"length":29,"payload_length":26},"frames":[{"frame_type":"ack","ack_delay":4.12,"acked_ranges":[[0,1]]}]}}
{"time":12.955569,"name":"recovery:metrics_updated","data":{"smoothed_rtt":2.556,"latest_rtt":4.191387,"rtt_variance":1.735,"bytes_in_flight":0}}
{"time":12.984483,"name":"security:key_discarded","data":{"trigger":"tls","key_type":"server_handshake_secret"}}
{"time":12.984483,"name":"security:key_discarded","data":{"trigger":"tls","key_type":"client_handshake_secret"}}
{"time":13.030842,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":2,"key_phase_bit":"0"},"raw":{"length":208},"frames":[{"frame_type":"handshake_done"},{"frame_type":"new_token","token":{"data":"8a043452ac469b3b7af132e7045a9a2a6b33ca8950197c1467a5a3075a299f49f46376bb8d57fa39aee54aeea5ab288691086a9a6192f40b2421d83bea729a8fd5bab0934d853a8b3366"}},{"frame_type":"crypto","offset":0,"length":107}]}}
{"time":13.031932,"name":"recovery:metrics_updated","data":{"bytes_in_flight":208,"packets_in_flight":1}}
{"time":13.03235,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"application_data","delta":35.493397}}
{"time":13.041111,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":1,"key_phase_bit":"0"},"raw":{"length":25,"payload_length":22},"frames":[{"frame_type":"retire_connection_id","sequence_number":0}]}}
{"time":13.046389,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":2,"key_phase_bit":"0"},"raw":{"length":33,"payload_length":30},"frames":[{"frame_type":"stream","stream_id":2,"offset":0,"length":8}]}}
{"time":13.060378,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":3,"key_phase_bit":"0"},"raw":{"length":251,"payload_length":248},"frames":[{"frame_type":"stream","stream_id":0,"offset":0,"length":226,"fin":true}]}}
{"time":13.070417,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":3,"key_phase_bit":"0"},"raw":{"length":24},"frames":[{"frame_type":"ack","ack_delay":0.034513,"acked_ranges":[[0,3]]}]}}
{"time":39.128965,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":4,"key_phase_bit":"0"},"raw":{"length":1352,"payload_length":1349},"frames":[{"frame_type":"ping"}]}}
{"time":39.13917,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":5,"key_phase_bit":"0"},"raw":{"length":29,"payload_length":26},"frames":[{"frame_type":"ack","ack_delay":25.896,"acked_ranges":[[0,3]]}]}}
{"time":39.144531,"name":"recovery:metrics_updated","data":{"bytes_in_flight":0}}
{"time":39.145138,"name":"recovery:loss_timer_updated","data":{"event_type":"cancelled"}}
{"time":39.152806,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":4,"key_phase_bit":"0"},"raw":{"length":1352},"frames":[{"frame_type":"ping"}]}}
{"time":39.153844,"name":"recovery:metrics_updated","data":{"bytes_in_flight":1352,"packets_in_flight":1}}
{"time":64.68538,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":6,"key_phase_bit":"0"},"raw":{"length":29,"payload_length":26},"frames":[{"frame_type":"ack","ack_delay":25.36,"acked_ranges":[[0,4]]}]}}
{"time":64.689262,"name":"recovery:metrics_updated","data":{"smoothed_rtt":5.425,"latest_rtt":25.514021,"rtt_variance":7.04,"bytes_in_flight":0}}
{"time":64.696427,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":5,"key_phase_bit":"0"},"raw":{"length":24},"frames":[{"frame_type":"ack","ack_delay":0.027531,"acked_ranges":[[0,6]]}]}}
{"time":64.873069,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":7,"key_phase_bit":"0"},"raw":{"length":1402,"payload_length":1399},"frames":[{"frame_type":"ping"}]}}
{"time":90.407958,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":6,"key_phase_bit":"0"},"raw":{"length":1402},"frames":[{"frame_type":"ping"}]}}
{"time":90.410981,"name":"recovery:metrics_updated","data":{"bytes_in_flight":1402,"packets_in_flight":1}}
{"time":90.416884,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":7,"key_phase_bit":"0"},"raw":{"length":25},"frames":[{"frame_type":"ack","ack_delay":25.563849,"acked_ranges":[[0,7]]}]}}
{"time":90.642717,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":8,"key_phase_bit":"0"},"raw":{"length":1427,"payload_length":1424},"frames":[{"frame_type":"ping"}]}}
{"time":116.080169,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":8,"key_phase_bit":"0"},"raw":{"length":25},"frames":[{"frame_type":"ack","ack_delay":25.455878,"acked_ranges":[[0,8]]}]}}
{"time":116.101789,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":9,"key_phase_bit":"0"},"raw":{"length":29,"payload_length":26},"frames":[{"frame_type":"ack","ack_delay":25.496,"acked_ranges":[[0,7]]}]}}
{"time":116.105683,"name":"recovery:metrics_updated","data":{"bytes_in_flight":0}}
{"time":116.27423,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":10,"key_phase_bit":"0"},"raw":{"length":1439,"payload_length":1436},"frames":[{"frame_type":"ping"}]}}
{"time":142.199776,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":9,"key_phase_bit":"0"},"raw":{"length":1427},"frames":[{"frame_type":"ping"}]}}
{"time":142.207846,"name":"recovery:metrics_updated","data":{"bytes_in_flight":1427,"packets_in_flight":1}}
{"time":142.214345,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":10,"key_phase_bit":"0"},"raw":{"length":25},"frames":[{"frame_type":"ack","ack_delay":25.958275,"acked_ranges":[[0,10]]}]}}
{"time":167.861574,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":11,"key_phase_bit":"0"},"raw":{"length":29,"payload_length":26},"frames":[{"frame_type":"ack","ack_delay":25.384,"acked_ranges":[[0,10]]}]}}
{"time":167.880031,"name":"recovery:metrics_updated","data":{"bytes_in_flight":0}}
{"time":607.189979,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":11,"key_phase_bit":"0"},"raw":{"length":1439},"frames":[{"frame_type":"ping"}]}}
{"time":607.199008,"name":"recovery:metrics_updated","data":{"bytes_in_flight":1439,"packets_in_flight":1}}
{"time":607.214136,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":12,"key_phase_bit":"0"},"raw":{"length":1427},"frames":[{"frame_type":"stream","stream_id":0,"offset":0,"length":1406}]}}
{"time":607.215804,"name":"recovery:metrics_updated","data":{"bytes_in_flight":2866,"packets_in_flight":2}}
{"time":607.216717,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"application_data","delta":59.581768}}
{"time":607.225507,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":13,"key_phase_bit":"0"},"raw":{"length":1427},"frames":[{"frame_type":"stream","stream_id":0,"offset":1406,"length":1404}]}}
{"time":607.226399,"name":"recovery:metrics_updated","data":{"bytes_in_flight":4293,"packets_in_flight":3}}
{"time":607.226711,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"application_data","delta":59.583225}}
{"time":607.233933,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":14,"key_phase_bit":"0"},"raw":{"length":1309},"frames":[{"frame_type":"stream","stream_id":0,"offset":2810,"length":1286}]}}
{"time":607.23545,"name":"recovery:metrics_updated","data":{"bytes_in_flight":5602,"packets_in_flight":4}}
{"time":607.235758,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"application_data","delta":59.582637}}
{"time":607.256309,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":15,"key_phase_bit":"0"},"raw":{"length":1427},"frames":[{"frame_type":"stream","stream_id":0,"offset":4096,"length":1404}]}}
{"time":607.257363,"name":"recovery:metrics_updated","data":{"bytes_in_flight":7029,"packets_in_flight":5}}
{"time":607.257743,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"application_data","delta":59.583037}}
{"time":607.26414,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":16,"key_phase_bit":"0"},"raw":{"length":166},"frames":[{"frame_type":"stream","stream_id":0,"offset":5500,"length":143}]}}
{"time":607.26688,"name":"recovery:metrics_updated","data":{"bytes_in_flight":7195,"packets_in_flight":6}}
{"time":607.267172,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"application_data","delta":59.581549}}
{"time":607.307097,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":17,"key_phase_bit":"0"},"raw":{"length":23},"frames":[{"frame_type":"stream","stream_id":0,"offset":5643,"length":0,"fin":true}]}}
{"time":607.309547,"name":"recovery:metrics_updated","data":{"bytes_in_flight":7218,"packets_in_flight":7}}
{"time":607.310566,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"application_data","delta":59.579806}}
{"time":607.674653,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":12,"key_phase_bit":"0"},"raw":{"length":28,"payload_length":25},"frames":[{"frame_type":"ack","ack_delay":0.08,"acked_ranges":[[0,17]]}]}}
{"time":607.689084,"name":"recovery:metrics_updated","data":{"min_rtt":0.346082,"smoothed_rtt":4.79,"latest_rtt":0.346082,"rtt_variance":6.549,"bytes_in_flight":0}}
{"time":607.689617,"name":"recovery:loss_timer_updated","data":{"event_type":"cancelled"}}
{"time":607.696568,"name":"transport:packet_sent","data":{"header":{"packet_type":"1RTT","packet_number":18,"key_phase_bit":"0"},"raw":{"length":22},"frames":[{"frame_type":"max_streams","stream_type":"bidirectional","maximum":101}]}}
{"time":607.69744,"name":"recovery:metrics_updated","data":{"bytes_in_flight":22,"packets_in_flight":1}}
{"time":607.697934,"name":"recovery:loss_timer_updated","data":{"event_type":"set","timer_type":"pto","packet_number_space":"application_data","delta":56.984099}}
{"time":633.144044,"name":"transport:packet_received","data":{"header":{"packet_type":"1RTT","packet_number":13,"key_phase_bit":"0"},"raw":{"length":29,"payload_length":26},"frames":[{"frame_type":"ack","ack_delay":25.208,"acked_ranges":[[0,18]]}]}}
{"time":633.162029,"name":"recovery:metrics_updated","data":{"smoothed_rtt":7.368,"latest_rtt":25.421199,"rtt_variance":10.069,"bytes_in_flight":0}}
{"time":633.162853,"name":"recovery:loss_timer_updated","data":{"event_type":"cancelled"}}
{"time":30633.500952,"name":"transport:connection_closed","data":{"owner":"local","trigger":"idle_timeout"}}

This doesn't seem unusual. If you're handling connections, the pool might hold a few objects. What makes you think that there's a leak here?

Thank you for your response. After upgrading to v0.35.1, I have observed that the memory usage of my application keeps increasing over time and does not get released, eventually consuming all available OS memory. Here are the details of my observation:
Before the upgrade: With the previous version (v0.30.0), I did not encounter this memory leak issue. The memory usage remained stable, and there was no excessive memory consumption.

After the upgrade: Once I upgraded to v0.35.1, I noticed a significant increase in memory usage. The memory keeps growing as the application continues to handle connections and perform its regular operations. Even during periods of low traffic, the memory is not released, leading to continuous memory growth.

Restarting the service: To temporarily mitigate the memory consumption, I have resorted to regularly restarting the service. This restart releases the accumulated memory, but the issue reoccurs over time.

Given these observations, I suspect that there might be an underlying memory leak introduced in the v0.35.1 version. The upgrade seems to have triggered the abnormal memory behavior, resulting in unreleased memory consumption until a service restart is performed.

Before the upgrade: With the previous version (v0.30.0) and (go 1.17) I did not encounter this memory leak issue. The memory usage remained stable, and there was no excessive memory consumption.

That's a quite big jump, which makes it hard to pinpoint which release is the culprit.

I understand that identifying the specific release causing the memory leak might be challenging due to the significant jump in versions. To provide more helpful information, I will conduct further investigation and gather additional details to assist in narrowing down the possible causes. Could you please let me know what specific information would be most useful for you in this case? This can include any particular logs, profiling data, or code snippets that you believe would be beneficial in analyzing the issue. Your guidance will greatly aid in our collaborative efforts to identify and address the memory leak effectively. Thank you for your assistance.

One thing you could do is see if this also happens with the latest v0.34 release. v0.35 introduced a lot of changes, so it would be great to know if we just recently introduced the bug.
Other than that, can you check if the leak also occurs if you don't have any tracer set (qlog or otherwise)?

Thank you for your suggestions. I will test the latest v0.34 release tomorrow to see if the memory leak issue persists.
Regarding the tracer, I have also tested without any tracer set, including qlog, and unfortunately, the memory leak still occurs. Therefore, it appears that the issue is not specific to the tracer configuration.
I will provide an update tomorrow after testing v0.34 and share the results with you. If there are any specific logs, profiling data, or code snippets you would like me to gather during the testing, please let me know.
Thank you for your guidance.

Great! Thanks for your help debugging this!

That's good and bad news at the same time :)
Would you mind narrowing down the versions a bit further? Lots of things happened between v0.30 and v0.34.

@marten-seemann I have exciting news to share regarding the memory leak issue. After switching to v0.33.0 of github.com/quic-go/quic-go, I am pleased to inform you that the memory stability has significantly improved. Over a continuous period of 17 hours, the memory usage remained stable, and there was no trace of the previously observed github.com/quic-go/quic-go.init.0.func1 process in the top list.
Even with a higher number of users compared to previous days, the memory consumption stayed under control, indicating that v0.33.0 has effectively addressed the memory leak problem. This positive outcome brings us closer to a stable and reliable solution.
I will continue monitoring the application and gather more data to solidify these positive results. Please let me know if there are any specific tests, logs, or profiling information you would like me to provide to further investigate and validate the improvements.
Thank you for your continued guidance and collaboration in this issue.
 That narrows it down to these commits: https://github.com/quic-go/quic-go/compare/v0.33.0...v0.34.0
There's nothing in there that immediately looks like the culprit. Maybe it could be https://github.com/quic-go/quic-go/pull/3767. Could you run the https://github.com/quic-go/quic-go/tree/copy-stateless-reset-token branch (it's based off v0.35.1) and see if that resolves the memory leak?

Apologies for the delay in my response. I was occupied with upgrading my Go version to 1.20, which turned out to be quite a challenging process. Nevertheless, I have made the necessary changes in my environment and continued with the suggested branch.
I have updated my go.mod file to include github.com/quic-go/quic-go v0.35.2-0.20230606103216-c48350a7d2ea and conducted further testing. Unfortunately, I observed that the memory leak issue resurfaced once again after this update.
This indicates that the specific version v0.35.2-0.20230606103216-c48350a7d2ea did not resolve the memory leak problem. As a result, it seems that the issue persists despite the changes made.
I will continue my investigation and explore other potential factors that could be causing the memory leak. If you have any further suggestions or insights regarding this matter, please let me know. I appreciate your patience and support in resolving this issue.
I will keep you updated on any new developments or findings. Thank you for your understanding.
File: webService.run
Build ID: affe79e86423871f824934a5c79d52c953e4a879
Type: inuse_space
Time: Jun 7, 2023 at 7:11pm (+0330)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) top
Showing nodes accounting for 274.65MB, 85.69% of 320.51MB total
Dropped 145 nodes (cum <= 1.60MB)
Showing top 10 nodes out of 110
      flat  flat%   sum%        cum   cum%
  105.15MB 32.81% 32.81%   105.15MB 32.81%  github.com/quic-go/quic-go.init.0.func1
   52.35MB 16.33% 49.14%    52.35MB 16.33%  github.com/webservice/utility.ReadHTTPDataBodyResponse
   33.67MB 10.51% 59.65%    33.67MB 10.51%  github.com/webservice/utility.ReadCloser
      27MB  8.42% 68.07%       27MB  8.42%  github.com/smallnest/goframe.(*lengthFieldBasedFrameConn).ReadFrame
      15MB  4.68% 72.75%    15.50MB  4.84%  github.com/quic-go/quic-go.newConn
   13.22MB  4.13% 76.88%    16.97MB  5.29%  compress/flate.NewWriter
   10.25MB  3.20% 80.07%    10.25MB  3.20%  regexp.(*Regexp).get
       8MB  2.50% 82.57%        8MB  2.50%  golang.org/x/net/internal/socket.(*mmsghdrsPacker).prepare
       6MB  1.87% 84.44%        6MB  1.87%  runtime.malg
       4MB  1.25% 85.69%    22.34MB  6.97%  github.com/quic-go/quic-go.(*Transport).init.func1

Could you provide the pprof file? It would be really helpful to find out where caused the memory leak.

Thank you for your suggestion. I understand that the pprof file would be valuable in identifying the root cause of the memory leak. However, due to security protocols within our organization, I am unable to directly provide the pprof file at this time.
However, I can still provide you with relevant information from the pprof analysis if it helps in diagnosing the issue. If there are any specific details or insights you would like me to extract from the pprof file, please let me know, and I will do my best to provide that information to you, given the necessary permissions from our security team.
I apologize for any inconvenience caused by this limitation, and I appreciate your understanding in prioritizing security measures. Please feel free to provide any specific instructions or guidance on what aspects of the pprof analysis would be most beneficial for you.
Thank you for your cooperation in resolving this."

@raszia Would you please use list <name of the function> to list which line of the code consumed the memory in the pprof interactive mode?
I am interested in github.com/quic-go/quic-go.init.0.func1, github.com/quic-go/quic-go.newConn and github.com/quic-go/quic-go.(*Transport).init.func1. The result shouldn't contain any code from your own project.
Also, a call stack graph might be useful as well if you can provide one :)

Thank you for offering your help. I will be glad to provide you with the data you requested. Here is the information you asked for:
File: webService.run
Build ID: affe79e86423871f824934a5c79d52c953e4a879
Type: inuse_space
Time: Jun 7, 2023 at 7:11pm (+0330)
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) list github.com/quic-go/quic-go.(\*Transport).init.func1
Total: 320.51MB
(pprof) list github.com/quic-go/quic-go.\(\*Transport\)\.init.func1
Total: 320.51MB
ROUTINE ======================== github.com/quic-go/quic-go.(*Transport).init.func1 in<EMAIL_ADDRESS>       4MB    22.34MB (flat, cum)  6.97% of Total
         .          .    171:	t.initOnce.Do(func() {
         .   858.34kB    172:		getMultiplexer().AddConn(t.Conn)
         .          .    173:
         .    15.50MB    174:		conn, err := wrapConn(t.Conn)
         .          .    175:		if err != nil {
         .          .    176:			t.initErr = err
         .          .    177:			return
         .          .    178:		}
         .          .    179:
         .          .    180:		t.logger = utils.DefaultLogger // TODO: make this configurable
         .          .    181:		t.conn = conn
         .        2MB    182:		t.handlerMap = newPacketHandlerMap(t.StatelessResetKey, t.enqueueClosePacket, t.logger)
  512.05kB   512.05kB    183:		t.listening = make(chan struct{})
         .          .    184:
       3MB        3MB    185:		t.closeQueue = make(chan closePacket, 4)
  512.02kB   512.02kB    186:		t.statelessResetQueue = make(chan *receivedPacket, 4)
         .          .    187:
         .          .    188:		if t.ConnectionIDGenerator != nil {
         .          .    189:			t.connIDGenerator = t.ConnectionIDGenerator
         .          .    190:			t.connIDLen = t.ConnectionIDGenerator.ConnectionIDLen()
         .          .    191:		} else {
(pprof) list github.com/quic-go/quic-go.init.0.func1
Total: 320.51MB
ROUTINE ======================== github.com/quic-go/quic-go.init.0.func1 in<EMAIL_ADDRESS>  105.15MB   105.15MB (flat, cum) 32.81% of Total
         .          .     75:	bufferPool.New = func() interface{} {
    2.50MB     2.50MB     76:		return &packetBuffer{
  102.65MB   102.65MB     77:			Data: make([]byte, 0, protocol.MaxPacketBufferSize),
         .          .     78:		}
         .          .     79:	}
         .          .     80:}
(pprof) list github.com/quic-go/quic-go.newConn
Total: 320.51MB
ROUTINE ======================== github.com/quic-go/quic-go.newConn in<EMAIL_ADDRESS>      15MB    15.50MB (flat, cum)  4.84% of Total
         .          .     68:func newConn(c OOBCapablePacketConn) (*oobConn, error) {
         .          .     69:	rawConn, err := c.SyscallConn()
         .          .     70:	if err != nil {
         .          .     71:		return nil, err
         .          .     72:	}
         .          .     73:	needsPacketInfo := false
         .          .     74:	if udpAddr, ok := c.LocalAddr().(*net.UDPAddr); ok && udpAddr.IP.IsUnspecified() {
         .          .     75:		needsPacketInfo = true
         .          .     76:	}
         .          .     77:	// We don't know if this a IPv4-only, IPv6-only or a IPv4-and-IPv6 connection.
         .          .     78:	// Try enabling receiving of ECN and packet info for both IP versions.
         .          .     79:	// We expect at least one of those syscalls to succeed.
         .          .     80:	var errECNIPv4, errECNIPv6, errPIIPv4, errPIIPv6 error
         .          .     81:	if err := rawConn.Control(func(fd uintptr) {
         .          .     82:		errECNIPv4 = unix.SetsockoptInt(int(fd), unix.IPPROTO_IP, unix.IP_RECVTOS, 1)
         .          .     83:		errECNIPv6 = unix.SetsockoptInt(int(fd), unix.IPPROTO_IPV6, unix.IPV6_RECVTCLASS, 1)
         .          .     84:
         .          .     85:		if needsPacketInfo {
         .          .     86:			errPIIPv4 = unix.SetsockoptInt(int(fd), unix.IPPROTO_IP, ipv4RECVPKTINFO, 1)
         .          .     87:			errPIIPv6 = unix.SetsockoptInt(int(fd), unix.IPPROTO_IPV6, ipv6RECVPKTINFO, 1)
         .          .     88:		}
         .          .     89:	}); err != nil {
         .          .     90:		return nil, err
         .          .     91:	}
         .          .     92:	switch {
         .          .     93:	case errECNIPv4 == nil && errECNIPv6 == nil:
         .          .     94:		utils.DefaultLogger.Debugf("Activating reading of ECN bits for IPv4 and IPv6.")
         .          .     95:	case errECNIPv4 == nil && errECNIPv6 != nil:
         .          .     96:		utils.DefaultLogger.Debugf("Activating reading of ECN bits for IPv4.")
         .          .     97:	case errECNIPv4 != nil && errECNIPv6 == nil:
         .          .     98:		utils.DefaultLogger.Debugf("Activating reading of ECN bits for IPv6.")
         .          .     99:	case errECNIPv4 != nil && errECNIPv6 != nil:
         .          .    100:		return nil, errors.New("activating ECN failed for both IPv4 and IPv6")
         .          .    101:	}
         .          .    102:	if needsPacketInfo {
         .          .    103:		switch {
         .          .    104:		case errPIIPv4 == nil && errPIIPv6 == nil:
         .          .    105:			utils.DefaultLogger.Debugf("Activating reading of packet info for IPv4 and IPv6.")
         .          .    106:		case errPIIPv4 == nil && errPIIPv6 != nil:
         .          .    107:			utils.DefaultLogger.Debugf("Activating reading of packet info bits for IPv4.")
         .          .    108:		case errPIIPv4 != nil && errPIIPv6 == nil:
         .          .    109:			utils.DefaultLogger.Debugf("Activating reading of packet info bits for IPv6.")
         .          .    110:		case errPIIPv4 != nil && errPIIPv6 != nil:
         .          .    111:			return nil, errors.New("activating packet info failed for both IPv4 and IPv6")
         .          .    112:		}
         .          .    113:	}
         .          .    114:
         .          .    115:	// Allows callers to pass in a connection that already satisfies batchConn interface
         .          .    116:	// to make use of the optimisation. Otherwise, ipv4.NewPacketConn would unwrap the file descriptor
         .          .    117:	// via SyscallConn(), and read it that way, which might not be what the caller wants.
         .          .    118:	var bc batchConn
         .          .    119:	if ibc, ok := c.(batchConn); ok {
         .          .    120:		bc = ibc
         .          .    121:	} else {
         .   512.04kB    122:		bc = ipv4.NewPacketConn(c)
         .          .    123:	}
         .          .    124:
    3.50MB     3.50MB    125:	msgs := make([]ipv4.Message, batchSize)
         .          .    126:	for i := range msgs {
         .          .    127:		// preallocate the [][]byte
    1.50MB     1.50MB    128:		msgs[i].Buffers = make([][]byte, 1)
         .          .    129:	}
         .          .    130:	oobConn := &oobConn{
         .          .    131:		OOBCapablePacketConn: c,
         .          .    132:		batchConn:            bc,
         .          .    133:		messages:             msgs,
         .          .    134:		readPos:              batchSize,
         .          .    135:	}
         .          .    136:	for i := 0; i < batchSize; i++ {
      10MB       10MB    137:		oobConn.messages[i].OOB = make([]byte, oobBufferSize)
         .          .    138:	}
         .          .    139:	return oobConn, nil
         .          .    140:}
         .          .    141:
         .          .    142:func (c *oobConn) ReadPacket() (*receivedPacket, error) {

Regarding the call stack graph, did you mean the one in this link: https://github.com/google/pprof/tree/main/doc#graphical-reports ? I wanted to make sure I understood you correctly.

did you mean the one in this link: https://github.com/google/pprof/tree/main/doc#graphical-reports ?

Yes, this is what I mean. Would please provide one? A SVG format would be nice.

The profile is showing us where the memory was originally allocated (it's indeed the buffer pool), but it doesn't tell us why these buffers are not returned to the pool and garbage collected. I'm not aware of any pprof tool that would give us that information.
There are not a lot of commits that could potentially change the way memory is managed. Could you try to build quic-go using the commit just before b03585160fb27abf0d03e51484430cd036e99177 (e.g. by importing quic-go at af517bdef10a483bacd5eb96e67ebca4e4e151b2)?

I have created a test repository to investigate this issue with different versions and commits. The main branch, which is using version 33, is functioning correctly without any memory leaks. However, other branches are experiencing memory leaks.
This demonstration closely resembles the scenario we have in our product.
I hope that this demonstration will assist you in debugging the issue effectively. By replicating a similar environment and observing the memory leak behavior, you can analyze the problem more closely and work towards finding a solution.
In this demonstration, a QUIC socket is opened and subsequently, multiple client connections are established to it. after 20 seconds, the client connections will cease. in my PC, I have observed that the memory is released approximately 10 minutes after running the code using version 33. however, in other versions like version 34, the memory is not released, even after a significantly longer duration, such as 2 hours.
https://github.com/raszia/quicTest
I have tried using the following go.mod requirement in my test repository: github.com/quic-go/quic-go v0.33.1-0.20230408045314-af517bdef10a. However, I have observed that the memory issue still persists.
@Glonee Please review my repository and, if you believe it would be helpful, generate a heap file.
https://github.com/raszia/quicTest/blob/quic33-1-af517bd/profile-af517bdef10a.pb.gz

@raszia Thanks for the test repo! That was really helpful.
I noticed that you created a brand new http.Client at each request and didn't close the undying http3.RoundTripper after the request was complete:
https://github.com/raszia/quicTest/blob/fdcefa25873cdcbd70519c893326ab53eecb89d3/client/qClient.go#L155-L170
Is that the same as what you did in the profuction environment? Since https://github.com/quic-go/quic-go/commit/6d7280b7dc6c3535086d9faa83451704c9842bda the http3.RoundTripper multiplexes all connections on a single UDPConn, you have to close it manually if you don't use anymore. This might be the reason of the memory leak.
You can try to build quic-go at https://github.com/quic-go/quic-go/commit/97dae87bf4ceb11d98ff6c192e93c51ba820e067 (should not encounter memory leak) and https://github.com/quic-go/quic-go/commit/6d7280b7dc6c3535086d9faa83451704c9842bda (should encounter memory leak) in your test. You can also take a look at my test: https://github.com/Glonee/quicTest. It uses the http3.RoundTripper correctly and won't encounter memory leak in any version of quic-go.

@raszia Have you made any progress on debugging this issue?


Decode stdout using preferred encoding not ascii, and combine multibyte as necessary

Original use case was using the rich.inspect within epdb session.
https://pypi.org/project/rich/ is coloring output and uses UTF-8
characters for tables etc.  epdb  client was crashing unable to
decode:
(Epdb) rich.inspect(self, value=False)
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/home/yoh/proj/datalad/datalad-mihextras/venvs/dev3/lib/python3.9/site-packages/epdb/__init__.py", line 1091, in connect
	t.interact()
  File "/home/yoh/proj/datalad/datalad-mihextras/venvs/dev3/lib/python3.9/site-packages/epdb/epdb_client.py", line 146, in interact
	sys.stdout.write(text.decode('ascii'))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 5: ordinal not in range(128)

I think hardcoding to ascii is the wrong thing to do since AFAIK
there is no guarantee that it would not be UTF-8

instead of leaving to default (not providing encoding) decided
to go for prefered encoding

there is a code block which follows for reading/writing stdin,
and I encoded also into preferable encoding which allows now
to pass unicode to be encoded to utf-8 happen someone needs
to enter it.

Signed-off-by: Yaroslav Halchenko<EMAIL_ADDRESS>demo of it working:

This looks okay to me, but I'd like at least one other reviewer. @mibanescu or @mtharp, what do you think?

ping @mibanescu @mtharp , your feedback would be appreciated.

ping on this PR -- I keep coming to the need to use patched version.  Please let me know if I should improve on anything


Intersected Message Channels

This is a simple pull request that adds support for message channels that, when created, combine messages channels in such a way that only members present in every channel are sent messages. This is quite useful for plugins interacting. I couldn't properly import this into my IDE so I had to write this in GitHub, hopefully it works properly.

Please test locally before making a PR -- this doesn't even compile. Also squash your commits.
imo this would be a useful addition if done efficiently -- it adds another set operation to MessageChannels.

I would have done it locally if I could, but I couldn't get it to clone to my IDE on this computer. I am currently in the process of moving so I can't properly do this, so I apologize.

uhhh it's just git clone git@github.com:Wundero/SpongeAPI.git then ./gradlew not that hard

@kashike Is this PR ok to merge or are there changes that are needing to be made?


mesh manifest migrate breaks on values-e2e.yaml

howardjohn commented yesterday:
mesh manifest migrate install/kubernetes/helm/istio/test-values/values-e2e.yaml > file
seems to be broken

fixed in the PR above


add-eks-templates

add templates to create a EKS cluster

this is just very alpha version of EKS cluster, not meant for general use yet


Add possibility to add principal name in log message

It will be very useful to be able to add in config a possibility to add in log message also principal name.

Can you please answer all the required questions described in the issue template? For your convenience let me paste it here

Describe the bug
Please provide details of the problem, including the version of Spring Cloud that you
are using.
Sample
If possible, please provide a test case or sample application that reproduces
the problem. This makes it much easier for us to diagnose the problem and to verify that
we have fixed it.

It is not a bug, it is a feature request.

Your description didn't provide any useful information. Are you referring to spring security? Where should be the prinicpal name be added. Are you talking about a span tag? Do you want to add it to mdc? Please provide thorough information about what you want to achieve and why do you think it's not possible to achieve it now

I want to configure Sleuth to print into logs also principal name from security, besides Application name, TraceId, SpanId, Export, as it is by default.

So you can use the existing solutions such as setting a baggage entry of a principal and configure Sleuth to automatically put it to MDC. You can read more about this here https://docs.spring.io/spring-cloud-sleuth/docs/current/reference/html/project-features.html#features-baggage


Website reference ES5 transpiled version of supercluster dependency

The default path loads ES6 code into the bundle.

Coverage remained the same at 80.211% when pulling 8a347d625d7fe8c16b1ac8b7d69e44659111d86d on fix-supercluster-dep into<PHONE_NUMBER>951d5f8aa06490db2bd4d20611d718 on master.


GET /wayfinding/buildings API fails with internal server error on dev

Currently GET wayfinding/buildings on dev fails with the following reason:
GET https://api-dev.rokwire.illinois.edu/gateway/api/wayfinding/buildings
500 {"message":"Error getting building: invalid character '\u003c' looking for beginning of value","status":"internal-server-error"}

Source: rokwire/illinois-app#3178
Additional resource in Slack.

I had sent the wrong url to use for the campus endpoint, ccf.fs.illinois.edu instead of api.ccf.fs.illinois.edu. Stephen changed the entry in the config and I verified that a postman call to wayfinding/buildings does return data now.


Heroku defaults

This makes creating and configuring Heroku apps the default, as all of our past months projects has been deployed to Heroku, but for some of them we configured Heroku apps manually later on, by adding required gems, configuring static assets serving, updating bin/setup script with Heroku remotes ...
Heroku can be skipped though with --skip_heroku option.
It introduces also --heroku-flags option where we can pass in any Heroku option like the region (was hardcoded before to EU) or some addons.
roll app --heroku-flags "--region eu --addons newrelic,pgbackups,sendgrid"

:+1:

This totally makes sense (as I used to forget the -H flag everytime ;)). :+1:

Good then!
Merging it


#931 - Use Flow for Development

This PR is to propose that we use Flow to help the team be more productive and make the code more expressive for others to understand and start to contribute more.
This is mostly for the benefit to the developers that maintain the SkateJs library and at least at this point, it is NOT for the users of the library.
One alternative could be TypeScript.
The choice is really between ES6+Flow   vs.   TypeScript
TypeScript adds a level of abstraction.
Flow only adds annotations to the existing ES6 code.
Syntactically they both looks quite similar.
I vote for Flow, because I think it is beneficial for SkateJs to stay with ES6 + Babel.
It is best to stay closer to the platform when dealing with buggy Polyfills or buggy browser native implementations.
I let the Team vote…

I work on the TypeScript team, and for what it's worth, TypeScript is also mostly just ECMAScript with types. In fact, ever since the iteration speed of TC39 picked up, our focus has been on the type system and tooling aspects, rather than on creating any new constructs. We heavily strive to align with ES.
I'll also mention that TypeScript tends to be a lot more flexible in the type system in terms of what you can do (e.g. being able to reopen existing types, function overload signatures, type-checked tagged templates, more granularity for nullable types, etc.).
I think regardless of what your team picks, I'd appreciate getting an idea of what affects your decision. I'd also totally be open to answering any questions you have about TS.

I think regardless of what your team picks, I'd appreciate getting an idea of what affects your decision. I'd also totally be open to answering any questions you have about TS.

Thanks @DanielRosenwasser, we definitely might have to take you up on that offer :)

@DanielRosenwasser In September we could not use TypeScript for Web Component development because of this issue:  https://github.com/Microsoft/TypeScript/issues/7574 (now is resolved).
Before ES6 became widely used TypeScript was a life saver for me. I still like it a lot!
But with a big community for Babel plugins and because ES6 will eventually run in future browsers I'm favoring ES6.
Also with Webpack one can set up a project where some files are ES6 and other are TypeScript but working that way is not simple because one can only use "import" within the same language and then one must use "require" when crossing languages.

Thanks for the response @andreawyss!
I actually remember you from your comment on https://github.com/Microsoft/TypeScript/pull/10762 now! I definitely get that the super() call issue was a problem for Custom Elements. I can say that 2.1 will be out very soon.
I also agree that the Babel plugin ecosystem has grown pretty nicely - in the 2.2 timeframe, we're looking into opening our new emit pipeline for custom transforms to enable the same sort of development.

because ES6 will eventually run in future browsers I'm favoring ES6.

But as soon as you pick either TypeScript or Flow, this isn't necessarily true. As soon as you have a type annotation, a type declaration, etc., that needs to get stripped out of the file in some respect. In Babel, this is through a preset, and in TypeScript this is using the ESNext target.
I don't think this is a terrible thing though. With either approach, you can opt out. I know that with TS, we've always worked hard to ensure that people never felt locked in. And with either technology, you will get some benefits. Our teams are working hard to demonstrate the value of typed JS in the ecosystem.

Also with Webpack one can set up a project where some files are ES6 and other are TypeScript but working that way is not simple because one can only use "import" within the same language and then one must use "require" when crossing languages.

If I understand you correctly, what you mean is that TypeScript won't understand an import to a .js file.
You can actually fix this by using the --allowJs flag which allows you to mix TS & JS code. You'll still get the experience you expect in TS files, and if your input JS files use JSDoc annotations, TypeScript can even give you accurate type info, completions, and errors based on that.
If you've used VS Code with JavaScript files, you've already seen a bit of what that's like, since the experience is powered by TypeScript itself.

These are some TypeScript features that are not just type annotations on type of ES6, are not part of the latest accepted ECMAScript standard, and/or are in some way specialized in TypeScript:

Modules Namespaces
Decorators
Class fields (including use of private keyword)
Mixins

This list is possibly incomplete or misinformed (I'll amend it with any offered corrections). I don't know much about Flow so I can't offer comparable list for that language.

That is true, but decorators & class fields are both stage 2 ECMAScript proposals right now, and class fields are required to use classes in either type system. Accessibility modifiers (e.g. public, private) are simply a type-level thing that have no emit impact, and you can feel free to not use them.
While namespaces are mostly subsumed by ES modules, namespaces also act as containers of types and can be used to describe some complex patterns. You might not use it in a new project, but they're incredibly useful for describing existing JS libraries.
I'm not sure what you're referring to with respect to mixins. We have intersection types but those are just type annotations.

👍 for TS. I ❤ TypeScript.

@DanielRosenwasser
Private class fields are currently proposed to use a different syntax.
I'm not 100% sure about the finer details, but I understand that decorators in TypeScript are not implemented as described in the current TC39 draft.

I'm not sure what you're referring to with respect to mixins. We have intersection types but those are just type annotations.

https://www.typescriptlang.org/docs/handbook/mixins.html

Also, I completely understand the rationale for namespaces. But, let's be fair: ECMAScript it ain't.

@cdata The examples on the mixins page are just common JS idioms and how to type them in TypeScript.

Thanks for the input @cdata. I'll try to address your points.
The private fields proposal doesn't impact our implementation of accessibility. One is design-time, the other is run-time. It's not ideal conceptually, but they can coexist, and if you're aiming for readable ES5 emit and care mainly about intent by consumers, TypeScript privates are probably more useful.
If you're referencing the ability to implement an interface, that's entirely a type-level thing that's helpful for enforcing the shape of a class. It doesn't have any emit impact at all, and is actually something Flow doesn't currently support.
You're right that the decorators implementation is based on an older proposal, but this isn't a problem unique to TypeScript anyway. Luckily, decorator authors can write them in a way that can feature-detect and avoid imposing breaking changes on consumers.

@justinfagnani Thanks for the clarification. I've amended my comment.

@DanielRosenwasser
Thanks for the helpful info related to these TypeScript features. It's valuable to understand the areas where TypeScript is not just ECMAScript with types.

I submitted TypeScript edition on #935

I have been using TypeScript for more then 2 years and I really like it a lot.
Before that I spent about 10 years in ActionScript and TypeScript is like a superset of ActionScript.
Now for the last 3 month I took a deep dive into ES6 + Flow and I discovered a new development experience.
The syntax is very similar and one can achieve about the same results with both. (I miss "private" from TypeScript)
The real difference relies in the type of potential issues/warnings that are reported by the 2 code checkers.
Flow is really more than a type checker; It is a code linter on steroids!
E.g. in the sample code on this PR Flow found this problem:
In the IPropConfig interface the optional deserialize method is declared as:
deserialize?: (v:?string) => any;
in the skate props.array strategy it was implemented as:
deserialize: JSON.parse,
Flow gave me an error because: JSON.parse(undefined) throws an error!
Therefore, I had to make the following change to satisfy Flow:
deserialize: val => (val === undefined ? undefined : JSON.parse(String(val))),
TypeScript on the other hand didn’t report this kind of issue.
I’m sure that one can find scenarios where the opposite is true.
During these 3 month with Flow, I have been pleasantly surprised at the kind of potential runtime issue that it warns me about. It is very good at detecting potential problems with undefined and null values.

Yes, I think that you'll also find situations where only TS catches certain errors. A pretty easy example is with something like this, whereas 2.1 will report such an issue.
Have either of you run into situations where Flow is more lax than you'd anticipated? How do you typically ensure that it's not overlooking something?

this is not probably related but maybe @DanielRosenwasser can help brainstorm on this on #936

Yes, I find cases where Flow is more relaxed then TypeScript.
I just submitted PR #938 for a 1-to-1 comparison with this PR #931:
#938 only converts one file (/src/api/prop.js) to prop.ts.
#931(this one) only adds Flow annotations to that same file.
One can see that TypeScript detected a different issue on the same file.
In the IPropConfig interface the optional serialize method is declared as:
TypeScript: serialize?: (v:any) => string|null|undefined;
Flow: serialize?: (v:any) => ?string;
in the skate prop.number strategy it was implemented as:
serialize: alwaysUndefinedIfNotANumberOrNumber
I had to make the following change to satisfy TypeScript:
serialize: alwaysUndefinedIfNotANumberOrString
In this case this issue would not have cause a runtime error since what gets returned from serialize is then used and automatically converted to a string in element.setAttribute( ‘name’, serializedValue );
On the other hand, JSON.parse(undefined) will definitely throw a runtime error.

@DanielRosenwasser
For most libraries I would not have a strong preference between TypeScript vs. ES+Flow+Babel.
As long as a type checker and linter are in place the team will be more productive.
The reasons that for the SkateJs library I’m voting for ES+Flow+Babel are:

SkateJs is already currently using Babel and some ES features.
I don't want to add an abstraction(TS) but just decorate(Flow) the existing code.
SkateJs must handle early implementations of WebComponents from both Polyfills and Browsers(native). To do this sometimes one needs to write them with low level JS Object manipulations.

For SkateJs it is beneficial to stay in JS proper and have the most control on how some ES or TS code gets transpiled.
Currently it is easier to achieve this in the Babel ecosystem than not in TypeScript ecosystem.
With TypeScript 2.2 this will slowly change.
I also agree that it is possible to keep some files in JS and not convert everything to TypeScript. The dev environment gets a bit more complex with separate checker, linters that may not be aware of each other codebase without some extra work (defining some types both in TS and Flow).
When I first submitted issue #918 - Add FlowType and Comments to the code
@Treshugart stated:
I'd prefer to stay in JS proper, but don't oppose definitions for those JS supersets. Easier to maintain in the long run. Imagine if Skate was written in CoffeeScript, for example.
…
I really value @Treshugart and @justinfagnani opinions because they have a lot of experience working with early implementations of WebComponents from both Polyfills and Browsers(native).

@andreawyss what version of TypeScript were you using with that JSON.parse() warning? TypeScript 2.0 with strict null checks would find that error as well. I run TS with strictNullChecks, noImplicitAny, noUnusedLocals, and noUnusedParameters. It catches a lot of things.
I don't really view TypeScript as adding any abstractions, so I wonder what you're referring to. In general, TS doesn't add any runtime semantics, with the possible exception of class properties which have a high degree of alignment with the ECMA proposal, and decorators which are experimental and have to be explicitly enabled. I only use JS modules and imports and don't explicitly use namespaces, except where I come across them in third party type declarations.
TypeScript's type system is by now insanely powerful and suited to describing JS. It's recently added data-flow sensitive type checking, mapped types, the keyof operator, and custom elements APIs to it's DOM lib, which allows us to do things like this:
class MySlider extends HTMLElement {
  slide() {}
}
customElements.define('my-slider', MySlider);

// Add 'my-slider' globally to the list of valid tag names and associate it with the right type
interface HTMLElementTagNameMap {
  'my-slider': MySlider;
}

document.createElement('my-slider').slide() // this typechecks!
document. getElementsByTagName('my-slider')[0].slide() // so does this!

Also, even though decorators are experimental, they add enough value for ergonomics, IMO, that we're going to ship some TS decorators for Polymer 2.0: https://github.com/Polymer/polymer/pull/3954
They're completely optional, and you could implement them so your JS interface doesn't change at all, but users could do this:
import {skate, attribute} from '../skate/decorators.js';

@define('x-hello')
class HelloElement extends skate.Component {

  @attribute name: string;

  renderCallback () {
    return skate.h('div', `Hello, ${this.name}`);
  }
}

(some may not like that, but I do :) )
My experience with @DanielRosenwasser and @mhegazy has been awesome and they've been extremely responsive with adding features we've needed for awesome custom elements support, including https://github.com/Microsoft/TSJS-lib-generator/pull/150 and https://github.com/Microsoft/TypeScript/issues/7574
I really can't recommend it highly enough.

@andreawyss

SkateJs must handle early implementations of WebComponents from both Polyfills and Browsers(native).

v0 support is long gone :) and the original polyfills had issues, so were never supported.
I'm inclined to lean towards TypeScript. I like both, but I do this for a few reasons, unrelated to the type checking details and differences:

It will simplify the build
It mostly aligns to the ECMAScript standard following stage 2 (?) TC39 specs
We don't have to use things that deviate from the specs (but we should definitely flag them)
Custom element type checking; that is awesome
It feels like it will speed up those using both TypeScript (more) or standard JS (less)
This is a big one: TS is far more mature.

The things that worry me are:

Dependencies that don't use TS
We are trusting TS not to deviate to far from ES, seems there's a few gotchas currently
I've seen issues with Skate JSX support, is this solvable? Maybe it is and I missed it.

Whatever happens out of this, there will need to be a blog post :p

Thank you @treshugart, @justinfagnani and @DanielRosenwasser.
It looks like most team members are concurring with moving forward with TypeScript.
Therefore, I close this PR so we can start working on a version of SkateJs written in TypeScript.

@treshugart

I've seen issues with Skate JSX support, is this solvable? Maybe it is and I missed it.

what issues do you mean?
If skate could export just React alias it would be even more easier.
Currently if one wants to use jsx he has to manually do const React = { h: createElement }

@Hotell sounds like TypeScript needs to change here, not Skate. cc @DanielRosenwasser

@treshugart I'm not specifically familiar with what you mean, but in 2.1 we'll support the jsxFactory flag (https://github.com/Microsoft/TypeScript/issues/9582). If this doesn't cover your use-case and there isn't already an issue on us, feel free to file one on us so we can discuss it a little bit more. Our team has tried to make our JSX support flexible enough and sufficiently decoupled from React so that it can be used with multiple libraries.

I made a usecase for SkateJS 4.1.0 https://github.com/vvakame/til/tree/master/skatejs-newdef

I've seen issues with Skate JSX support, is this solvable? Maybe it is and I missed it.

Is this mean https://github.com/Microsoft/TypeScript/issues/7004 ?
There are no signs to solve it at the moment.
We need support from @DanielRosenwasser I think 😉
now, We can detour that issue. not cool.
    renderCallback() {
        const anyProps: any = {};
        return (
            <div>
                <CountUpComponent count={100} {...anyProps}></CountUpComponent>
            </div>
        );
    }

Just want to pitch in and cast my vote for Typescript!


[Improvement]: Remove $moduleExecute function from stack trace in strand dump output

Description
$title
Due to the changes added by the PR https://github.com/ballerina-platform/ballerina-lang/pull/39418, $moduleExecute function is shown in the stack trace in strand dump output.
Describe your problem(s)
import ballerina/lang.runtime;

public function main() {
    runtime:sleep(100);
}

gives
Ballerina Strand Dump [2023/03/27 09:37:47]
===========================================

Total strand group count        :       3
Total strand count              :       1
Active strand group count       :       1
Active strand count             :       1

group 3 [QUEUED]: [1]
        strand 1 "main" [$anon...0:main] [BLOCKED]:
                at      ballerina.lang.runtime.0.0.0:sleep(runtime.bal:86)
                        $anon...0.0.0:main(sd.bal:4)
                        $moduleExecute

===========================================

in the master branch
Describe your solution(s)
No response
Related area
-> Runtime
Related issue(s) (optional)
No response
Suggested label(s) (optional)
No response
Suggested assignee(s) (optional)
No response

Fix in the compile-time


Change the "FetchBuiltInCMap"/"FetchStandardFontData" message-handlers to be asynchronous

This way we can directly throw Errors, rather than having to "manually" return rejected Promises, which is ever so slightly shorter.
Also, since useWorkerFetch is always true in MOZCENTRAL builds these message-handlers should not be invoked there.

/botio test


Move librdkafka to main pool

Coverage remained the same at 89.706% when pulling f1d916e8d3d67d8aa44b5ae50223865193537335 on Pchelolo:librdkafka_pool into 33e0e5b58e82699d8d06ee174667c747c11cfe3b on wikimedia:master.


Enforce branch policies on the repository

To improve the security of the ORAS project we need to enforce the branch policies for this repository. I propose that we enforce the policies as follows:

Use the following rules for main and release/* branches:

Require PR before merging

[ ] Require 3 approvals
[ ] Dismiss stale PR approvals when new commits are pushed
[x] Require review from Code Owners
[ ] Require status checks to pass before merging
[ ] Require conversation resolution before merging
[ ] Require signed commits
[ ] Do not allow bypass the above settings

Please add your comments and proposals for additional changes to this issue.

If a code owner created the PR, is that one approval?

That is correct @TerryHowe - only codeowners count. And no, this is in addition to the person who submitted the PR as far as I know. We can have a relaxed policy and ask for 2 codeowner approvals only.

Few comments:

release/* does not apply to libraries. Besides, we have a special branch named v1.
"Require 3 approvals" is not applicable to this repository since we only have 2 active code owners.


📡 Problem with the Faucet

Faucet Type
Web
Problem
not getting faucet kindly send it on 5Gsw6WA8QaoM4SoALp78XTmTG1bzJNqhPKH6JHFMYifLxRkS
and kindly also fix it on website
Address
No response
Network ID
Relay Chain
Relevant log output
No response

https://github.com/availproject/polkadot-testnet-faucet


emsdk 3.1.55 and later support

Codecov Report
All modified and coverable lines are covered by tests :white_check_mark:

Project coverage is 90.47%. Comparing base (069a637) to head (ff28d55).

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #102   +/-   ##
=======================================
  Coverage   90.47%   90.47%
=======================================
  Files           6        6
  Lines          42       42
  Branches        3        3
=======================================
  Hits           38       38
  Misses          4        4

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.


Add VR fixes

Feel free to cherry pick stuff you may want. This compiles and is working out of the box for VR.
EDIT: In case you wanted to share DP, we've noticed from other mods with multiple versions that 5% of users tend to be VR so I'd be happy with that. But don't feel obligated.

Thank you so much for your fixes and support for VR. Since I don't have VR and can't help users who will have VR related questions, if you want, you can create a mod page on nexus for VR, I will link it in the description.
I will integrate the non-VR fixes you pointed out, but I leave VR to you.

Oh given the dll will be identical since you're using NG, I think it's probably silly for me to host another whole page to host the dll. I'll just monitor your main page's posts. You can also flag me if you see something. That's what I'm doing for doodles, etc.


chore(*): tweak priorities for linear algebra

We make sure that the canonical path from NonAssocSemiring to Ring passes through Semiring,
as this is a path which is followed all the time in linear algebra where the defining semilinear map
σ : R →+* S depends on the NonAssocSemiring structure of R and S while the module
definition depends on the Semiring structure.
Tt is not currently possible to adjust priorities by hand (see lean4#2115). Instead, the last
declared instance is used, so we make sure that Semiring is declared after NonAssocRing, so
that Semiring -> NonAssocSemiring is tried before NonAssocRing -> NonAssocSemiring.

I have tried other options to streamline things, with less success. This change is both the least disruptive and the best I have found.

!bench

bors r+

bors r+


Easier CLI with Clap?

This is just an idea, but given that you're currently rolling your own interface, have you considered using Clap? It makes parsing arguments pretty simple, and even generates help text from meta comments. Like my other proposal, I can probably make a PR for this at some point if you give me the green light on it.

Absolutely! I even mention it as a TODO somewhere. I just didn't get around to it, because I didn't want to develop it into a real™ tool yet.
Are you on Matrix or Telegram? I'd like to chat with you :)

I'm not at the moment sadly, but feel free to email me! If you're on discord, I can email you my tag and we can talk there ^-^

Ah, this got closed by #3, whoops. Should have noted that there!


Added the possibility to automatically jump to first diff ocurrence

The merge addon now can jump to the first difference if configured so. I'm not quite happy with the hard coded 8 in line 64, maybe someone has a tip how to easily get the font size of the editor?

You should add a comment when you add patches -- otherwise, I won't notice them.
Anyway, your implementation still isn't really what I was looking for. See my attached patch for an alternative.


psrqpy doesn't load psrcat.db (the sqlite version) even when it's placed in the cache dir

If I download/extract the psrcat.db from the latest version of the ATNF catalog (v2.0.0) and place it in the CACHEDIR it isn't found by psrqpy when I call QueryATNF().
However, if I use QueryATNF(loadfromdb=psrqpy.CACHEDIR+'psrcat.db') it works just fine.
I suspect the reason is that the change in "db" format from a text file to an SQLite db is confusing the file reader.

Hi @PaulHancock, thanks for reporting this. I had noticed this too. I'm not quite sure what the issue is as the downloadable tarball has the same information in and the psrcat.db file is the same format (apparently George has to produce this file from the SQL database for backwards compatibility). I'll take a look into the issue.

The issue with the latest version of the psrcat_pkg.tar.gz tarball is that the archive stores the directories with a leading "./", which psrqpy does not expect. I've asked George if this discrepancy can be fixed, but if not I'll put a workaround in psrqpy and issue a new release.

@PaulHancock The psrcat_pkg.tar.gz tarball from the catalogue download page has now been fixed, so psrqpy should work for v2.0.0 of the catalogue. Thanks again for reporting this.


Backend : Token should be validated on all APIs

Description
All APIs should have token Validation

I have already implemented ValidateToken logic, you should be able to use it for token validation in all APIs


pkgs/stdenv/linux: add powerpc64 bootstrap files

Description of changes
Adds powerpc64 bootstrap files to linux' stdenv, so that nixpkgs is usable from a powerpc64 machine, or using emulatedSystems = [ "powerpc64-linux" ];.
I'm not sure I got the hashes right the first time, so they might need to be updated the tarballs are uploaded. I'm not sure how to test it while we are missing the uploads.
Things done

Built on platform(s)

[ ] emulated system
[ ] native

[ ] Tested, as applicable:

NixOS test(s) (look inside nixos/tests)
and/or package tests
or, for functions and "core" functionality, tests in lib/tests or pkgs/test

[x] Fits CONTRIBUTING.md.

Until these are uploaded to tarballs.nixos.org, please use working URLs for ease of testing/verification. Then once we get someone to do the upload, we can change the URLs to point to tarballs.nixos.org and verify that the hashes are the same before merging.

so after 12h of compilation, Nix started to compile the "real" gcc, but it failed with this output, apparently because the C preprocessor isn't there/detected? But it succeeded in compiling bison, gettext, perl, and other things, so it could be a bug elsewhere, but I'm not sure where.
On another note, would it be possible to replace the value of system by the config config on in nixpkgs systems attrset? Right now cross-compiling for powerpc64-linux and powerpc64-unknown-linux-gnu leads to two different builds of gcc and the rest, to my knowledge only because the name differs.

Closing as I no longer have a use for it, if someone is interested, feel free to re-open or tag me on another PR.


Allow unsafe raw SQL injection in PostgresQuery

Motivation
Sometimes developers want to have the freedom to inject values into a PostgresQuery that are not bindable. Examples for this can be tableNames or columnNames.
Changes

Allow adopters to interpolate raw sql into the PostgresQuery

Result
More use-cases covered. Make unsafe code explicit unsafe.

Codecov Report

Merging #258 (284ce3b) into main (57fda42) will increase coverage by 4.05%.
The diff coverage is 100.00%.

:exclamation: Current head 284ce3b differs from pull request most recent head c1e28eb. Consider uploading reports for the commit c1e28eb to get more accurate results

@@            Coverage Diff             @@
##             main     #258      +/-   ##
==========================================
+ Coverage   39.78%   43.84%   +4.05%
==========================================
  Files         115      115
  Lines        9513     9523      +10
==========================================
+ Hits         3785     4175     +390
+ Misses       5728     5348     -380

Flag
Coverage Δ

unittests
43.84% <100.00%> (+4.05%)
:arrow_up:

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files
Coverage Δ

Sources/PostgresNIO/New/PostgresQuery.swift
87.17% <100.00%> (+0.51%)
:arrow_up:

...ces/PostgresNIO/New/PostgresRow-multi-decode.swift
4.44% <0.00%> (+0.19%)
:arrow_up:

...es/PostgresNIO/Connection/PostgresConnection.swift
17.48% <0.00%> (+0.93%)
:arrow_up:

Sources/PostgresNIO/Data/PostgresRow.swift
50.00% <0.00%> (+1.02%)
:arrow_up:

...rces/PostgresNIO/New/Extensions/Logging+PSQL.swift
19.60% <0.00%> (+1.42%)
:arrow_up:

...s/PostgresNIO/New/Data/Array+PostgresCodable.swift
93.25% <0.00%> (+2.24%)
:arrow_up:

...ostgresNIO/Deprecated/PostgresMessageDecoder.swift
95.45% <0.00%> (+2.27%)
:arrow_up:

...ostgresNIO/New/PostgresBackendMessageDecoder.swift
93.28% <0.00%> (+2.98%)
:arrow_up:

...urces/PostgresNIO/New/Messages/ErrorResponse.swift
32.30% <0.00%> (+3.07%)
:arrow_up:

Sources/PostgresNIO/New/PostgresCell.swift
96.77% <0.00%> (+3.22%)
:arrow_up:

... and 21 more


why i add triggerListener to the handle failed

I let player and collide,but nothing happen when i press keyboard

Even if I use UserAction to solve this problem,it is useless

I typed so fast that there is something wrong.I made player and test collide.


Changelog catchup - May month to date

Type of Change

[ ] Bug fix (non-breaking change, fixes #)
[ ] New feature (non-breaking change, adds functionality)
[ ] Breaking change (effects multiple commands or functionality)
[ ] Ran manual Pester test and has passed (`.\tests\manual.pester.ps1)
[ ] Adding code coverage to existing functionality
[ ] Pester test is included
[ ] If new file reference added for test, has is been added to github.com/sqlcollaborative/appveyor-lab ?
[ ] Nunit test is included
[X] Documentation
[ ] Build system

Purpose
Catching up the changelog as of the latest release

thanks, bud!


Fix get method on http.Headers

Pull Request Test Coverage Report for Build<PHONE_NUMBER>

0 of 10 (0.0%)  changed or added relevant lines in 1 file are covered.
No unchanged relevant lines lost coverage.
Overall coverage increased (+1.9%) to 82.558%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

authorize/program.go
0
10
0.0%

Totals

Change from base Build<PHONE_NUMBER>:
1.9%

Covered Lines:
355

Relevant Lines:
430

💛  - Coveralls


[AMBARI-24142] Show/Hide config-section not working for Ranger and Ra…

…nger KMS theme.
What changes were proposed in this pull request?
Map ThemeCondition to SubSection and SubSectionTab using name and theme there could be multiple subsections/subsectiontabs with the same 'name' property
How was this patch tested?
21798 passing (22s)
48 pending

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2775/
Test PASSed.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2774/
Test PASSed.

Refer to this link for build results (access rights to CI server needed):
https://builds.apache.org/job/Ambari-Github-PullRequest-Builder/2783/
Test FAILed.
Test FAILured.

Ambari Web succeeds. Failures with Metrics Collector and Server


Extract Literal attribute ?

I do like your idea of Literal Attribute - convert text into string. I did put into the collection of my Roslyn Source Code Generators -  https://ignatandrei.github.io/RSCG_Examples/v2/docs/CodeAnalysis
However, it comes with a lot of other generated files. Could you extract into a new package ?

Actually, I already did it in my code,  using Shared Project. (It's really useful !). I didnt package it bcs it was just me who use it. I'll extract and pack it soon as possibile.

I'v published on nuget : Feast.CodeAnalysis.Literal
And make the latest version of Feast.CodeAnalysis depends on it

Thanks! Modified !


Doom's infernal blade recording bugged

I just played a game, I was looking at the the damage tab and there is two seperate damage inflicters for doom's rightclick. Upon looking at the performances tab, there was one attack called "parsererror: SyntaxError: Unexpected end of JSON input". Presumably both of this were caused by doom manually casting his Infernal Blade.
Upon looking at the skill builds section, Infernal Blade was also using Lvl death's icon.

The first is likely due to creep abilities.  Since Valve's ability data doesn't have entries for these, the API returns a bad response.

The second is a known issue.  https://github.com/odota/core/issues/807


Request

Would it be possible to add a state monitor in the download of the mods? In this last version it is not possible to know if the mod is downloading or if it simply stays in stand by or lost connection.

What exactly do you mean?
How did it look before compared to now?

i mean the percent downloaded in the console. to know the state of the current download.

I was already wondering about that as well, but it appears that steamcmd does not show the progress for workshop items anymore


Rebass with Radium

In an effort to be truly CSS-independent, I'm trying to get Radium to work with Rebass - specifically because of pseudo classes like :hover and media queries. Unfortunately, I have not been able to get it to work because both Radium and Rebass use the style prop.
Any suggestions here?

radium or any JSS framework/library would be superawesome

Same with Aphrodite as well.

Aphroditing the rebass, so awesome!

What appears to be a simple solution (at least for the className usecase) is to just append the className prop that gets passed in to every component.
Instead of className='Breadcrumbs', maybe just
className={`Breadcrumbs ${props.className}`}

Glamor and Aphrodite are probably front runners for being used in the library itself. As for the className prop, v0.4 supports passing custom classnames, so they should work there

Sounds good @jxnblk. My vote is for Aphrodite. 👍
When do you expect 0.4 to be out?

@jxnblk I think https://github.com/rofrischmann/fela is probably the winner. Check it.

@steida Hello, I love the suggestion you gave and read the fela documentation.
I wonder how do you use fela with rebass. Can you maybe offer an example please?
Thanks!

:wave: I'd be interested in helping move this over to Glamor.

And... I ended up using styled-components. Mostly for the community, active development, and extensibility. Thanks all for the discussion! 😎


Unexpectedly high cpu/gpu use

Was playing around with your subplots example the other day to get a feel for the library (am very excited about reducing my matplotlib render times), and I ran into an unexpected issue. While the figure was active, and without any mouse or keyboard input, one of my cpu cores was fully loaded, and gpu use was around ~30%. Did a  little digging, and over a ~25 second profiling run, it seems like fastplotlib/layouts/_plot_area.py:363(render) is calling the pygfx viewport render function almost 40,000 times. Is this expected/desired behavior? Based on my limited understanding of the current codebase, there is a render loop, which I assume is constantly redrawing the figure. This cpu/gpu use seems problematic to me, especially if the figure isn't being interacted with most of the time. I do not observe this behavior if I use snapshot().
My code, run in jupyterlab (using version 0.2.0):
names = [['subplot']*8]*16
figure_grid = fpl.Figure(shape=(16,8), size=(1000,2000), names=names)
for subplot in figure_grid:
    data = np.random.rand(512, 512)
    subplot.add_image(data, name="rand-img")

pr = cProfile.Profile()
pr.enable()
figure_grid.show()
## wait ~25 seconds before running the next block
pr.disable()
pr.create_stats()
pr.print_stats(sort='tottime')

Top 10 calls by tottime
Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
  1029298    4.403    0.000    5.023    0.000 _helpers.py:304(proxy_func)
      409    2.081    0.005    2.081    0.005 {method 'poll' of 'select.epoll' objects}
 20191744    1.979    0.000    2.226    0.000 _weakrefset.py:63(__iter__)
   526029    1.260    0.000    2.323    0.000 _api.py:101(_new_struct_p)
    39168    1.100    0.000    3.220    0.000 environment.py:402(check_inactive)
    39168    0.745    0.000    1.567    0.000 matrix.py:452(mat_orthographic)
   203812    0.734    0.000    0.734    0.000 {built-in method builtins.dir}
   201508    0.634    0.000    1.054    0.000 structs.py:18(<listcomp>)
    78489    0.628    0.000    5.812    0.000 _api.py:2202(begin_render_pass)
    39168    0.521    0.000   23.825    0.001 renderer.py:392(render)

Hi, thanks for posting!
Is there a usecase for 128 subplots? That's a bit more than I anticipated use cases for. Anyways, it does seem like there's something slower about using subplots than putting all those graphics in one subplot, for example this is much more performant:
import fastplotlib as fpl
import numpy as np

fig = fpl.Figure()

for i in range(8):
    for j in range(16):
        fig[0, 0].add_image(np.random.rand(1000, 2000), offset=(i * (2000 + 500), j * (1000 + 500), 0))

While the figure was active, and without any mouse or keyboard input, one of my cpu cores was fully loaded, and gpu use was around ~30%.

I wonder if the cpu usage is related to this: https://github.com/pygfx/pygfx/issues/763, are you on pygfx@main? This isn't in the latest release yet.
Also, GPU usage can be misleading without the current wattage as well, if you're using an nvidia GPU this is quite easy to measure.

Did a little digging, and over a ~25 second profiling run, it seems like fastplotlib/layouts/_plot_area.py:363(render) is calling the pygfx viewport render function almost 40,000 times. Is this expected/desired behavior? Based on my limited understanding of the current codebase, there is a render loop, which I assume is constantly redrawing the figure. This cpu/gpu use seems problematic to me, especially if the figure isn't being interacted with most of the time.

I wonder if there's a way to not re-draw if nothing has changed, @almarklein any thoughts?

I do not observe this behavior if I use snapshot().

Yup because a snapshot is a static png

I'm coming from matplotlib land, and I have a couple of monster figures with that many subplots (definitely niche though). If fastplotlib doesn't require multiple subplots in order to add multiple images to the same figure, then I don't necessarily see a usecase. Just so I understand it better -- what is the point of subplots in fastplotlib? Is it to be able to have different controllers on different subplots?
I am on pygfx@main.  https://github.com/pygfx/pygfx/issues/763 could very well be the culprit. I am on numpy 1.24, and my profiling from this morning points to the same methods called out in the thread over there. camera.update_projection_matrix() came up in the profiling, but was just outside the top 10 I put in my report. I believe it calls mat_orthographic though, which you can see in the top 10.
For completeness, I profiled your example as well. It also uses less cpu on my machine. Shown below are results from a ~37 second profile run. Calls to render are down to only 552 from 40,000.  I may be reading it wrong, but when I looked at the code this morning it seemed like each subplot calls render independently, in addition to the top-level figure. Not sure if it's helpful, but the jupyter busy/idle indicator goes nuts when a figure is showing (flips between busy and idle a couple dozen times a second).
Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     2936   23.875    0.008   23.875    0.008 {method 'poll' of 'select.epoll' objects}
   592364    2.570    0.000    3.248    0.000 _helpers.py:304(proxy_func)
991392/284832    0.767    0.000    1.069    0.000 _base.py:432(iter)
   141312    0.566    0.000    3.336    0.000 pipeline.py:631(draw)
   283176    0.501    0.000    1.818    0.000 _api.py:2040(set_bind_group)
      553    0.375    0.001    0.700    0.001 _jpg.py:54(_encode)
    10425    0.363    0.000    0.363    0.000 {method 'acquire' of '_thread.lock' objects}
      552    0.325    0.001    0.325    0.001 {method 'copy' of 'numpy.ndarray' objects}
   311658    0.321    0.000    0.542    0.000 api.py:242(new)
   592364    0.253    0.000    0.350    0.000 _helpers.py:247(release)
   592364    0.243    0.000    0.328    0.000 _helpers.py:243(capture)
1437857/1436699    0.238    0.000    0.249    0.000 {built-in method builtins.isinstance}
     1104    0.236    0.000    1.992    0.002 _base.py:420(traverse)
      553    0.232    0.000    8.413    0.015 renderer.py:392(render)

I can check the gpu wattage tomorrow.

I'm coming from matplotlib land, and I have a couple of monster figures with that many subplots (definitely niche though).

Yea fastplotlib works very differently from matplotlib (I maybe should've chosen a different name, too late for that :laughing: ). You can add multiple graphics of any type to a single subplot, use offsets if necessary.

If fastplotlib doesn't require multiple subplots in order to add multiple images to the same figure, then I don't necessarily see a usecase. Just so I understand it better -- what is the point of subplots in fastplotlib? Is it to be able to have different controllers on different subplots?

Yes each subplot has its own controller and camera, and you can also add different subplot-level animation functions to each subplot.

each subplot calls render independently, in addition to the top-level figure.

There's no rendering at the figure level. Figure.render() just calls subplot.render() for every subplot, so there will be n_subplot many render calls, and each subplot.render() essentially renders the viewport representing that subplot using the camera for that subplot. I'll see if Almar has any thoughts, but this will change once pygfx refactors the viewport and renderer.

Not sure if it's helpful, but the jupyter busy/idle indicator goes nuts when a figure is showing (flips between busy and idle a couple dozen times a second).

As far as I understand this indicates communication with jupyterlab, it's the remote frame buffer and it will always be active when receiving frames.

I can check the gpu wattage tomorrow.

Thanks! I also get about 30% usage but my GPU wattage under 50% of max so I don't think it's really pushing the GPU, it's something else.

This could be related to https://github.com/pygfx/pygfx/issues/763, but I would not be surprised if this is simply due to there being 128 subplots, i.e. 128 render calls 😅

I wonder if there's a way to not re-draw if nothing has changed

That would be a very obvious solution. Yes, most pygfx examples have code like this:
def animate():
    renderer.render(...)
    renderer.reques_draw()  # <-- this schedules a new draw

Some examples request a new draw on every draw, so the scene is rendered perpetually. This is advantageous in some use-cases, like simulations. (though we plan to provide different support for that). For a plotting library, it makes sense to not do this. Instead, call renderer.reques_draw() when you know that a redraw is needed (and when pygfx not already sdchedules a draw by itself).

call renderer.reques_draw() when you know that a redraw is needed (and when pygfx not already sdchedules a draw by itself).

I think we might be able to get away with this, started a new issue #586
@n-garc I also started #585 , if we do it it won't be anytime soon, maybe in a few months at the earliest, I would just set offsets like my previous example above


Hoe upgrade stuff

Pretty much that.

Please use the new config system, the old config.java is on its way out.
Sent from my iPhone

On 13. May 2018, at 12:38, Yotam Gur<EMAIL_ADDRESS>wrote:
Warning: This message has had one or more attachments removed
Warning: (msg-32472-2.html).
Warning: Please read the "j-e-b-Attachment-Warning.txt" attachment(s) for more information.
Pretty much that.
You can view, comment on, or merge this pull request online at:
https://github.com/SleepyTrousers/EnderIO/pull/4625
-- Commit Summary --

Hoe upgrade stuff

-- File Changes --
M enderio-base/src/main/java/crazypants/enderio/base/config/Config.java (4)
M enderio-base/src/main/java/crazypants/enderio/base/handler/darksteel/DarkSteelRecipeManager.java (2)
M enderio-base/src/main/java/crazypants/enderio/base/item/darksteel/ItemDarkSteelAxe.java (50)
A enderio-base/src/main/java/crazypants/enderio/base/item/darksteel/upgrade/hoe/HoeUpgrade.java (27)
M enderio-base/src/main/resources/assets/enderio/lang/en_us.lang (3)
-- Patch Links --
https://github.com/SleepyTrousers/EnderIO/pull/4625.patch
https://github.com/SleepyTrousers/EnderIO/pull/4625.diff
--
You are receiving this because you are subscribed to this thread.
Reply to this email directly or view it on GitHub:
https://github.com/SleepyTrousers/EnderIO/pull/4625
This is a message from the MailScanner E-Mail Virus Protection Service
The original e-mail message contained potentially dangerous content,
which has been removed for your safety.
The content is dangerous as it is often used to spread viruses or to gain
personal or confidential information from you, such as passwords or credit
card numbers.
Due to limitations placed on us by the Regulation of Investigatory Powers
Act 2000, we were unable to keep a copy of the original attachment.
The content filters found this:
MailScanner: Found a script in HTML message
--
Postmaster
MailScanner thanks transtec Computers for their support