L515: Set max depth(code)

Required Info

Camera Model
{ L515 }

Firmware Version
(<IP_ADDRESS>)

Operating System & Version
{Win (8.1/10)

Platform
PC

Language
{python}

Issue Description
I want to set the maximum depth of point cloud obtained by L515, I know it cloud be set on RealSense Viewer, but I want to operate it in coding.
I set the min depth using this:
dist = sensor_dep.set_option( rs.option.min_distance, 0)
So I try to set max depth similarly, like
dist = sensor_dep.set_option( rs.option.max_distance, 0)
but failed. I also tried the method used for D400 series, as
advnc_mode.set_depth_control(current_std_depth_control_group)
but I could not even run the code, it seems that L515 is not supported in find_device_that_supports_advanced_mode().
So if there is any code to set the max depth of L515? I would like to do that because I have to clipp points far away to keep the cloud efficient. Thank you!

Hi @weisterki  The L515 camera model does not have Advanced Mode.  It also only supports the 'Temporal' depth post-processing filter, so the Threshold filter of the 400 Series cameras cannot be used to set a minimum and maximum depth.  Limits for the rendering of depth can instead be set with the depth visualization settings Min Distance and Max Distance, as you have done.
The Min Distance and Max Distance depth visualization settings are not sensor options but instead are software settings of the depth colorizer.  Here is some Python code for defining Min and Max Distance for the colorizer.
colorizer = rs.colorizer()
colorizer.set_option(rs.option.visual_preset, 1) # 0=Dynamic, 1=Fixed, 2=Near, 3=Far
colorizer.set_option(rs.option.min_distance, value_min)
colorizer.set_option(rs.option.max_distance, value_max)

depth_colormap = np.asanyarray(colorizer.colorize(depth_frame).get_data())

Thanks Marty,  I used the code in this way:

but I can not see any difference. I tried to change several settings but they look same. By the way, could this method wipe off points farther than 1.5m?  I mean change the point cloud but not only the visualization.

I noe that in your script the max_distance code is placed after the depth_image has been stored in the numpy array.  I recommend moving the colorizer code further up the script, such as just after the pipe start instruction.

I move the code to this place, but still not work :<

Please try setting the preset to '0' for Dynamic, as that is the one that the RealSense Viewer uses by default.

Please try setting the preset to '0' for Dynamic, as that is the one that the RealSense Viewer uses by default in its colorization settings.

Sorry to say that it does still not work :(. I don't know which module results in this situation, so I post my code
Show camera stream and export PLY data
from ctypes import sizeof
import math
import time
import pyrealsense2 as rs
import numpy as np
import cv2
import os
class AppState:
def init(self, *args, **kwargs):
self.WIN_NAME = 'My RealSense Controller'
self.pitch, self.yaw = math.radians(-10), math.radians(-15)
self.translation = np.array([0, 0, -1], dtype=np.float32)
self.distance = 2
self.prev_mouse = 0, 0
self.mouse_btns = [False, False, False]
self.paused = False
self.decimate = 0
self.scale = True
self.color = True
def reset(self):
    self.pitch, self.yaw, self.distance = 0, 0, 2
    self.translation[:] = 0, 0, -1

@property
def rotation(self):
    Rx, _ = cv2.Rodrigues((self.pitch, 0, 0))
    Ry, _ = cv2.Rodrigues((0, self.yaw, 0))
    return np.dot(Ry, Rx).astype(np.float32)

@property
def pivot(self):
    return self.translation + np.array((0, 0, self.distance), dtype=np.float32)

def mouse_cb(event, x, y, flags, param):
if event == cv2.EVENT_LBUTTONDOWN:
    state.mouse_btns[0] = True

if event == cv2.EVENT_LBUTTONUP:
    state.mouse_btns[0] = False

if event == cv2.EVENT_RBUTTONDOWN:
    state.mouse_btns[1] = True

if event == cv2.EVENT_RBUTTONUP:
    state.mouse_btns[1] = False

if event == cv2.EVENT_MBUTTONDOWN:
    state.mouse_btns[2] = True

if event == cv2.EVENT_MBUTTONUP:
    state.mouse_btns[2] = False

if event == cv2.EVENT_MOUSEMOVE:

    h, w = out.shape[:2]
    dx, dy = x - state.prev_mouse[0], y - state.prev_mouse[1]

    if state.mouse_btns[0]:
        state.yaw += float(dx) / w * 2
        state.pitch -= float(dy) / h * 2

    elif state.mouse_btns[1]:
        dp = np.array((dx / w, dy / h, 0), dtype=np.float32)
        state.translation -= np.dot(state.rotation, dp)

    elif state.mouse_btns[2]:
        dz = math.sqrt(dx**2 + dy**2) * math.copysign(0.01, -dy)
        state.translation[2] += dz
        state.distance -= dz

if event == cv2.EVENT_MOUSEWHEEL:
    dz = math.copysign(0.1, flags)
    state.translation[2] += dz
    state.distance -= dz

state.prev_mouse = (x, y)

def project(v):
"""project 3d vector array to 2d"""
h, w = out.shape[:2]
view_aspect = float(h)/w
# ignore divide by zero for invalid depth
with np.errstate(divide='ignore', invalid='ignore'):
    proj = v[:, :-1] / v[:, -1, np.newaxis] * \
        (w*view_aspect, h) + (w/2.0, h/2.0)

# near clipping(clipping if the view too close)
znear = 0.03
proj[v[:, 2] < znear] = np.nan#################A
return  proj

def view(v):
"""apply view transformation on vector array"""
return np.dot(v - state.pivot, state.rotation) + state.pivot - state.translation
def line3d(out, pt1, pt2, color=(0x80, 0x80, 0x80), thickness=1):
"""draw a 3d line from pt1 to pt2"""
p0 = project(pt1.reshape(-1, 3))[0]# Shape[0] ouput matrix row numbers
p1 = project(pt2.reshape(-1, 3))[0]
if np.isnan(p0).any() or np.isnan(p1).any():
return
p0 = tuple(p0.astype(int))# Transform array to tuple
p1 = tuple(p1.astype(int))
rect = (0, 0, out.shape[1], out.shape[0])
inside, p0, p1 = cv2.clipLine(rect, p0, p1)# If are in same plane
if inside:
cv2.line(out, p0, p1, color, thickness, cv2.LINE_AA)
def grid(out, pos, rotation=np.eye(3), size=1, n=10, color=(0x80, 0x80, 0x80)):
"""draw a grid on xz plane"""
pos = np.array(pos)
s = size / float(n)
s2 = 0.5 * size
for i in range(0, n+1):
x = -s2 + is
line3d(out, view(pos + np.dot((x, 0, -s2), rotation)),
view(pos + np.dot((x, 0, s2), rotation)), color)
for i in range(0, n+1):
z = -s2 + is
line3d(out, view(pos + np.dot((-s2, 0, z), rotation)),
view(pos + np.dot((s2, 0, z), rotation)), color)
def axes(out, pos, rotation=np.eye(3), size=0.075, thickness=2):
"""draw 3d axes"""
line3d(out, pos, pos +
np.dot((0, 0, size), rotation), (0xff, 0, 0), thickness)
line3d(out, pos, pos +
np.dot((0, size, 0), rotation), (0, 0xff, 0), thickness)
line3d(out, pos, pos +
np.dot((size, 0, 0), rotation), (0, 0, 0xff), thickness)
def frustum(out, intrinsics, color=(0x40, 0x40, 0x40)):
"""draw camera's frustum"""
orig = view([0, 0, 0])
w, h = intrinsics.width, intrinsics.height
for d in range(1, 6, 2):
    def get_point(x, y):
        p = rs.rs2_deproject_pixel_to_point(intrinsics, [x, y], d)# d = depth
        line3d(out, orig, view(p), color)
        return p

    top_left = get_point(0, 0)
    top_right = get_point(w, 0)
    bottom_right = get_point(w, h)
    bottom_left = get_point(0, h)

    line3d(out, view(top_left), view(top_right), color)
    line3d(out, view(top_right), view(bottom_right), color)
    line3d(out, view(bottom_right), view(bottom_left), color)
    line3d(out, view(bottom_left), view(top_left), color)

def pointcloud(out, verts, texcoords, color, painter=True):
"""draw point cloud with optional painter's algorithm"""
if painter:
# Painter's algo, sort points from back to front
    # get reverse sorted indices by z(in view-space)
    # https://gist.github.com/stevenvo/e3dad127598842459b68
    v = view(verts)
    s = v[:, 2].argsort()[::-1]
    proj = project(v[s])
else:
    proj = project(view(verts))

if state.scale:
    proj *= 0.5**state.decimate

h, w = out.shape[:2]

# proj now contains 2d image coordinates
j, i = proj.astype(np.uint32).T# Transposition

# create a mask to ignore out-of-bound indices
im = (i >= 0) & (i < h)
jm = (j >= 0) & (j < w)
m = im & jm

cw, ch = color.shape[:2][::-1]# [::-1] reverse
if painter:
    # sort texcoord with same indices as above
    # texcoords are [0..1] and relative to top-left pixel corner,
    # multiply by size and add 0.5 to center(texture coordinate)
    v, u = (texcoords[s] * (cw, ch) + 0.5).astype(np.uint32).T
else:
    v, u = (texcoords * (cw, ch) + 0.5).astype(np.uint32).T
# clip texcoords to image(limit to 0~ch-1)
np.clip(u, 0, ch-1, out=u)
np.clip(v, 0, cw-1, out=v)

# perform uv-mapping
out[i[m], j[m]] = color[u[m], v[m]]

state = AppState()
Configure depth and color streams
pipeline = rs.pipeline()
config = rs.config()
Get device product line for setting a supporting resolution
pipeline_wrapper = rs.pipeline_wrapper(pipeline)
pipeline_profile = config.resolve(pipeline_wrapper)
device = pipeline_profile.get_device()
found_rgb = False
for s in device.sensors:
if s.get_info(rs.camera_info.name) == 'RGB Camera':
found_rgb = True
break
if not found_rgb:
print("The color sensor not found.")
exit(0)
Set L515
config.enable_stream(rs.stream.depth, 1024, 768, rs.format.z16, 30)
config.enable_stream(rs.stream.color, 1920, 1080, rs.format.rgb8, 30)#############B
Start streaming
profile = pipeline.start(config)###################C
Set Minimum Depth
sensor_dep = profile.get_device().first_depth_sensor()
dist = sensor_dep.set_option( rs.option.min_distance, 0)
dist = sensor_dep.get_option(rs.option.min_distance)
print ("min_distance =  ", dist)
value_max = 1
colorizer = rs.colorizer()
colorizer.set_option(rs.option.visual_preset, 0) # 0=Dynamic, 1=Fixed, 2=Near, 3=Far
#colorizer.set_option(rs.option.min_distance, value_min)
colorizer.set_option(rs.option.max_distance, value_max)
Get stream profile and camera intrinsics
depth_profile = rs.video_stream_profile(profile.get_stream(rs.stream.depth))# profile
depth_intrinsics = depth_profile.get_intrinsics()
w, h = depth_intrinsics.width, depth_intrinsics.height
Processing blocks
pc = rs.pointcloud()
decimate = rs.decimation_filter()
decimate.set_option(rs.option.filter_magnitude, 2 ** state.decimate)
colorizer = rs.colorizer()
cv2.namedWindow(state.WIN_NAME, cv2.WINDOW_AUTOSIZE)
cv2.resizeWindow(state.WIN_NAME, w, h)
cv2.setMouseCallback(state.WIN_NAME, mouse_cb)
out = np.empty((h, w, 3), dtype=np.uint8)
print("Input  to reset\nIpnut  to save")
Streaming loop
while True:
# Grab camera data
if not state.paused:
# Wait for a coherent pair of frames: depth and color
frames = pipeline.wait_for_frames()
    depth_frame = frames.get_depth_frame()
    color_frame = frames.get_color_frame()

    depth_frame = decimate.process(depth_frame)# Process the depth data

    # Grab new intrinsics(may be changed by decimation)
    depth_intrinsics = rs.video_stream_profile(
        depth_frame.profile).get_intrinsics()
    w, h = depth_intrinsics.width, depth_intrinsics.height

    depth_image = np.asanyarray(depth_frame.get_data())# Transform list to array
    color_image = np.asanyarray(color_frame.get_data())

    depth_colormap = np.asanyarray(
        colorizer.colorize(depth_frame).get_data())

    if state.color:
        BGR_color_image = cv2.cvtColor(color_image, cv2.COLOR_RGB2BGR)
        mapped_frame, color_source = color_frame, BGR_color_image
    else:
        mapped_frame, color_source = depth_frame, depth_colormap

    points = pc.calculate(depth_frame)
    pc.map_to(mapped_frame)

    # Pointcloud data to arrays
    v, t = points.get_vertices(), points.get_texture_coordinates()
    verts = np.asanyarray(v).view(np.float32).reshape(-1, 3)# xyz(3 cols)
    texcoords = np.asanyarray(t).view(np.float32).reshape(-1, 2)# uv(2 cols)

# Render
now = time.time()

out.fill(0)# fill with 0

grid(out, (0, 0.5, 1), size=1, n=10)
frustum(out, depth_intrinsics)
axes(out, view([0, 0, 0]), state.rotation, size=0.1, thickness=1)

if not state.scale or out.shape[:2] == (h, w):
    pointcloud(out, verts, texcoords, color_source)
else:
    tmp = np.zeros((h, w, 3), dtype=np.uint8)
    pointcloud(tmp, verts, texcoords, color_source)
    tmp = cv2.resize(
        tmp, out.shape[:2][::-1], interpolation=cv2.INTER_NEAREST)
    np.putmask(out, tmp > 0, tmp)

if any(state.mouse_btns):
    axes(out, view(state.pivot), state.rotation, thickness=4)

dt = time.time() - now

cv2.setWindowTitle(
    state.WIN_NAME, "My RealSense Controller: (%dx%d) %dFPS (%.2fms) %s " %
    (w, h, 1.0/dt, dt*1000, "PAUSED" if state.paused else ""))

cv2.imshow(state.WIN_NAME, out)
key = cv2.waitKey(1)

if key == ord("r"):
    state.reset()

if key == ord("p"):
    state.paused ^= True

if key == ord("e"):
    file = "./output"
    if not os.path.exists(file):
        os.mkdir(file)
    points.export_to_ply("./output/out.ply", mapped_frame)
    print("Output Successfully!")

if key in (27, ord("q")) or cv2.getWindowProperty(state.WIN_NAME, cv2.WND_PROP_AUTOSIZE) < 0:
    break

Stop streaming
pipeline.stop()

Whilst min_distance and max_distance will affect how the depth data is colored, it will not remove depth data that is outside of the defined range like the post-processing threshold filter on the 400 Series cameras, which the L515 camera model does not support.
An approach that has been used in the past to exclude certain depth values is to check the depth of each pixel and set its value to '0' (no depth) if the depth value is larger than a certain number.  A C++ example of doing so is at https://github.com/IntelRealSense/librealsense/issues/1027 (there is not a Python example of the technique available).
Alternatively, it may be worth trying the L515 instruction rs2_option_enable_max_usable_range which "turns on / off the maximum usable depth sensor range, given the amount of ambient light in the scene".  A Python example of turning off the maximum usable range with this instruction (setting it to '0') in combination with min_distance is at https://github.com/IntelRealSense/librealsense/issues/8244#issuecomment-779127501

Whilst min_distance and max_distance will affect how the depth data is colored, it will not remove depth data that is outside of the defined range like the post-processing threshold filter on the 400 Series cameras, which the L515 camera model does not support.
An approach that has been used in the past to exclude certain depth values is to check the depth of each pixel and set its value to '0' (no depth) if the depth value is larger than a certain number. A C++ example of doing so is at #1027 (there is not a Python example of the technique available).
Alternatively, it may be worth trying the L515 instruction rs2_option_enable_max_usable_range which "turns on / off the maximum usable depth sensor range, given the amount of ambient light in the scene". A Python example of turning off the maximum usable range with this instruction (setting it to '0') in combination with min_distance is at #8244 (comment)

Thanks, Marty! Your answer is useful. I tried the method of https://github.com/IntelRealSense/librealsense/issues/8244#issuecomment-779127501, I set laser power to 100 and receiver gain at 18, but the result was acceptable only when I set the confidence at 3. Here is my code, hope to be useful for any guy has same problem.
sensor_dep.set_option(rs.option.enable_max_usable_range, 0)
power = sensor_dep.set_option(rs.option.laser_power, 100)
gain = sensor_dep.set_option(rs.option.receiver_gain, 18)
confidence = sensor_dep.set_option(rs.option.confidence_threshold, 3)

It's great to hear that you achieved a solution!  Thanks so much for the update and for sharing your solution with the RealSense community  :)

Whilst min_distance and max_distance will affect how the depth data is colored, it will not remove depth data that is outside of the defined range like the post-processing threshold filter on the 400 Series cameras, which the L515 camera model does not support.
An approach that has been used in the past to exclude certain depth values is to check the depth of each pixel and set its value to '0' (no depth) if the depth value is larger than a certain number. A C++ example of doing so is at #1027 (there is not a Python example of the technique available).
Alternatively, it may be worth trying the L515 instruction rs2_option_enable_max_usable_range which "turns on / off the maximum usable depth sensor range, given the amount of ambient light in the scene". A Python example of turning off the maximum usable range with this instruction (setting it to '0') in combination with min_distance is at #8244 (comment)

Thanks, Marty! Your answer is useful. I tried the method of #8244 (comment), I set laser power to 100 and receiver gain at 18, but the result was acceptable only when I set the confidence at 3. Here is my code, hope to be useful for any guy has same problem. sensor_dep.set_option(rs.option.enable_max_usable_range, 0) power = sensor_dep.set_option(rs.option.laser_power, 100) gain = sensor_dep.set_option(rs.option.receiver_gain, 18) confidence = sensor_dep.set_option(rs.option.confidence_threshold, 3)

Whilst min_distance and max_distance will affect how the depth data is colored, it will not remove depth data that is outside of the defined range like the post-processing threshold filter on the 400 Series cameras, which the L515 camera model does not support.
An approach that has been used in the past to exclude certain depth values is to check the depth of each pixel and set its value to '0' (no depth) if the depth value is larger than a certain number. A C++ example of doing so is at #1027 (there is not a Python example of the technique available).
Alternatively, it may be worth trying the L515 instruction rs2_option_enable_max_usable_range which "turns on / off the maximum usable depth sensor range, given the amount of ambient light in the scene". A Python example of turning off the maximum usable range with this instruction (setting it to '0') in combination with min_distance is at #8244 (comment)

Whilst min_distance and max_distance will affect how the depth data is colored, it will not remove depth data that is outside of the defined range like the post-processing threshold filter on the 400 Series cameras, which the L515 camera model does not support.
An approach that has been used in the past to exclude certain depth values is to check the depth of each pixel and set its value to '0' (no depth) if the depth value is larger than a certain number. A C++ example of doing so is at #1027 (there is not a Python example of the technique available).
Alternatively, it may be worth trying the L515 instruction rs2_option_enable_max_usable_range which "turns on / off the maximum usable depth sensor range, given the amount of ambient light in the scene". A Python example of turning off the maximum usable range with this instruction (setting it to '0') in combination with min_distance is at #8244 (comment)

Thanks, Marty! Your answer is useful. I tried the method of #8244 (comment), I set laser power to 100 and receiver gain at 18, but the result was acceptable only when I set the confidence at 3. Here is my code, hope to be useful for any guy has same problem. sensor_dep.set_option(rs.option.enable_max_usable_range, 0) power = sensor_dep.set_option(rs.option.laser_power, 100) gain = sensor_dep.set_option(rs.option.receiver_gain, 18) confidence = sensor_dep.set_option(rs.option.confidence_threshold, 3)

However, I found the depth filter used in this issue https://github.com/IntelRealSense/librealsense/issues/10344#issue-1184715482 could bring better performance. I tried the depth filter in this way:
frames = pipeline.wait_for_frames()
depth_frame = frames.get_depth_frame(),
 threshold_filter = rs.threshold_filter(min_dist=0.1, max_dist=1)
depth_frame = threshold_filter.process(depth_frame)
color_frame = frames.get_color_frame()
depth_frame = decimate.process(depth_frame)
and yes could bring point cloud in required range, but the edge of the cloud always flickering and drifting, as shown in the figure.

I thought that may caused by the colour frame, thus I change to this:
frames = pipeline.wait_for_frames()
depth_frame = frames.get_depth_frame()
threshold_filter = rs.threshold_filter(min_dist=0.1, max_dist=1)
frame = threshold_filter.process(depth_frame)
color_frame = frames.get_color_frame()
depth_frame = decimate.process(depth_frame)
I don't know why but it worked. And if I  change the
depth_frame = decimate.process(depth_frame) to
depth_frame = decimate.process(frame) it flickers again. I want to ask am I right to use the depth filter, and if yes, how to use it corrrectly?

In https://github.com/IntelRealSense/librealsense/issues/10344 I highlight later in the discussion at https://github.com/IntelRealSense/librealsense/issues/10344#issuecomment-1081885052 that the L515 only supports the Temporal filter for depth and the Decimation filter for color.
If you are experiencing depth flickering then changing the Temporal filter's Smooth Alpha setting to a value of '0.1' instead of its default of '0.4' stabilizes fluctuations and helps to keep the depth values consistent.

Temporal filter

Thanx. Could you please explain the temporal filter in L515?

Information about the temporal filter can be found in Intel's RealSense post-processing filter guide at the link below.
https://dev.intelrealsense.com/docs/post-processing-filters#temporal-filter

Information about the temporal filter can be found in Intel's RealSense post-processing filter guide at the link below.
https://dev.intelrealsense.com/docs/post-processing-filters#temporal-filter

Whilst min_distance and max_distance will affect how the depth data is colored, it will not remove depth data that is outside of the defined range like the post-processing threshold filter on the 400 Series cameras, which the L515 camera model does not support.
An approach that has been used in the past to exclude certain depth values is to check the depth of each pixel and set its value to '0' (no depth) if the depth value is larger than a certain number. A C++ example of doing so is at #1027 (there is not a Python example of the technique available).
Alternatively, it may be worth trying the L515 instruction rs2_option_enable_max_usable_range which "turns on / off the maximum usable depth sensor range, given the amount of ambient light in the scene". A Python example of turning off the maximum usable range with this instruction (setting it to '0') in combination with min_distance is at #8244 (comment)

Thanks, Marty! Your answer is useful. I tried the method of #8244 (comment), I set laser power to 100 and receiver gain at 18, but the result was acceptable only when I set the confidence at 3. Here is my code, hope to be useful for any guy has same problem. sensor_dep.set_option(rs.option.enable_max_usable_range, 0) power = sensor_dep.set_option(rs.option.laser_power, 100) gain = sensor_dep.set_option(rs.option.receiver_gain, 18) confidence = sensor_dep.set_option(rs.option.confidence_threshold, 3)

Whilst min_distance and max_distance will affect how the depth data is colored, it will not remove depth data that is outside of the defined range like the post-processing threshold filter on the 400 Series cameras, which the L515 camera model does not support.
An approach that has been used in the past to exclude certain depth values is to check the depth of each pixel and set its value to '0' (no depth) if the depth value is larger than a certain number. A C++ example of doing so is at #1027 (there is not a Python example of the technique available).
Alternatively, it may be worth trying the L515 instruction rs2_option_enable_max_usable_range which "turns on / off the maximum usable depth sensor range, given the amount of ambient light in the scene". A Python example of turning off the maximum usable range with this instruction (setting it to '0') in combination with min_distance is at #8244 (comment)

I tried to turn on the maximum range of L515 using the code:
sensor_dep.set_option(rs.option.enable_max_usable_range, 1),
but I got this ERROR：‘Please set 'VGA' resolution and 'Max Range' preset before enabling Max Usable Range‘. The same ERROR occurred in the RealSense Viewer as well. How to deal with that?

VGA resolution on the L515 is 640x480.  I confirmed that this error occurs unless the Visual Preset is Max Range and the resolution is 640x480.
The 'Max Range' preset is not the same as the Colorizer Visual Preset, which is in the 'Depth Visualization' section of the Viewer's side-panel.  You can set the main Visual Preset to 'Max Range' and the resolution to 640x480 but change the colorizer preset and the Enable Max Usable Range option will still work.

n

Problem solved! Thank you：)!

That's excellent news.  You are very welcome  :)

Case closed due to solution achieved and no further comments received.


Make visibility of listed dataset appropriate for version curation statuses and user

I had a much bigger ticket but broke it into multiple.  It is this one  https://github.com/CDL-Dryad/dryad-product-roadmap/issues/300
This PR also goes with a corresponding one in the other repo.
Added tests that show it working.
If this turns out to be too slow later with a large number of records, we may need to find a more efficient way to join rather than doing a subquery.

Just need to rebase off master. Feel free to merge yourself once the conflicts have been resolved

Thanks Brian.  I changed the string to a heredoc.
Merging now after merging the latest changes back into this branch from master.


Inventory parsing nonexistent NBT incorrectly for items sharing the same itemscript

This bug shows up in two ways, using<EMAIL_ADDRESS>and this script:
- foreach<EMAIL_ADDRESS>{
    - narrate "<def[value].nbt[key]>"
}

All items without an item script return the expected values for the NBT key.

You can replicate using the following steps:

Create an itemscript using any NBT, and give yourself one. Example:

samplestone:
    type: item
    material: i@stone[nbt=key/0]
    display name: "Version 0"

Edit the itemscript such that the NBT key does not exist on the updated version. Give yourself one. It is important that you do not edit any other instances of the item in your inventory. Example:

samplestone:
    type: item
    material: i@stone
    display name: "Version null"

Afternote: You do not need to give yourself another instance of the item. You just need the two previous copies, one with the NBT key key and one without.

Give the itemscript the NBT key again. It can be any value. Example:

samplestone:
    type: item
    material: i@stone[nbt=key/99999]
    display name: "Version 99 999"

Run /ex narrate <player.inventory.list_contents.parse[nbt[key]]> or run this script:

sample_inventory_parser:
    type: task
    script:
    - foreach<EMAIL_ADDRESS>{
        - narrate "<def[value].nbt[key]>"
    }

You will see that any instance of the item with the NBT key key will return the expected value (in this example, "0"), but any instance of the item without the NBT key key will return the value in the file, not "null".

That's expected. An item script is the reference for what any items using it are to be. Any value lost from it can regenerate itself easily. Item scripts generally shouldn't have a complex spec and then it's deleted later... an item script is the item, and if you intend to use it as a base then modify it, you should remove the ID from the item (using no_id).


OTEL Install Collector for Ansible, Chef, Puppet

Purpose of this pull request
Add instructions for installing OTEL collector for Chef, Puppet, Ansible. Do not merge until @ajaiswals approves.

Select the type of change:

[ ] Minor Changes - Typos, formatting, slight revisions
[x] Update Content - Revisions and updating sections
[x] New Content - New features, sections, pages, tutorials
[ ] Site and Tools - Updates, maintenance, and new packages for the site, Gatsby, React, etc

Looks great, @kimsauce. I just made one tiny tweak.

Thanks, John! @ajaiswals , can you please review?


Merge Bitpay's version of Bitcoin Core (additional indexes)

Resolved conflicts:
.travis.yml (ignoring changes, keeping dash version)
src/Makefile.am
src/main.cpp
src/main.h
src/rpcclient.cpp
src/rpcserver.cpp
src/script/script.cpp
src/script/script.h

Fixed tests:
qa/rpc-tests/addressindex.py
qa/rpc-tests/spentindex.py
qa/rpc-tests/txtindex.py
src/test/script_P2PKH_tests.cpp

Maybe we need a compiler switch (like DISABLE_WALLET) to guard these features

Is this still WIP?

Changed title ;)
This merge is kind of huge so I still would like to hear any comments and/or (ut/n)ACKs from @evan82 though.

Yep, i agree - waiting for comments and/or (ut/n)ACKs from @evan82 :)

This is insanely useful and built entirely separate from core... merged :+1:


There is a memory leak defect at line 263 in the file /vim/src/vim9execute.c.

Steps to reproduce
1.At line 216 in vim9execute.c, a pointer variable named 'dict' is defined. This pointer is allocated a block of dynamic memory by the function dict_alloc at line 224. When the if statement at line 225 returns false, it indicates that the memory allocation for the variable 'dict' was successful. However, when the if statement at line 262 returns true, the program returns at line 263. Unlike line 255 where dict_unref(dict); is called before returning, there is no such operation executed before returning at line 263. As a result, this constitutes a memory leak defect, as illustrated in the diagram below:
https://github.com/LuMingYinDetect/vim_defects/blob/main/vim_19.png
Expected behaviour
If this is indeed a memory leak defect, it should be addressed to prevent unnecessary memory consumption.
Version of Vim
9.1.0192
Environment
Ubuntu 22.04
Logs and stack traces
No response

I am a master's student, specializing in static analysis of programs. During the course of today, I have submitted multiple issues to provide feedback on potential memory leak problems in Vim. Of course, some of the memory leak defects I have identified may be false positives, and I apologize for any inconvenience this may have caused. I am proud to be able to contribute to the Vim project! If there are any questions regarding the memory leak defects I have raised, please feel free to contact me at any time!

And thank you for this. How did you trace those memory leaks?

And thank you for this. How did you trace those memory leaks?

I am excited to contribute to Vim! I am using a detection tool developed by our lab to detect memory leaks. The foundation of this detection tool is the clang static analyzer. Currently, only a portion of the tool has been developed, and it is not yet fully completed. After further development, we plan to open-source it.


Add support for dbt-spark adapter

We should add all of the dbt supported platforms (and then move to the community ones). This will require two things:

[ ]  default profile that maps to an Airflow connection

[ ]  new extra in setup.cfg

See Apache Spark setup in dbt docs.

See OSS Airflow Docs for Apache Spark Connection

@chrishronek I think #254 added support to this:
https://github.com/astronomer/astronomer-cosmos/tree/main/cosmos/profiles/spark
Please, reopen if there is something missing :)


Check Project 2 Submission

Hi here is a link to Inseption
Will be waiting for your feedback @kob-e


RestrictedBlock piston bypass

*Detailed description of the issue:
You can basically bypass the restrictedblock with a piston and gain money by auto farming.
https://www.youtube.com/watch?v=hNTfEwp7C_8
  General:
    PlaceAndBreak:
      # Enable blocks protection, like ore, from exploiting by placing and destroying same block again and again.
      # Modify restrictedBlocks.yml for blocks you want to protect
      Enabled: true
      # Enabling this we will ignore blocks generated in ore generators, liko stone, coublestone and obsidian. You can still use timer on player placed obsidian block
      IgnoreOreGenerators: true
      # For how long in days to keep block protection data in data base
      # This will clean block data which ones have -1 as cooldown value
      # Data base cleanup will be performed on each server startup
      # This cant be more then 14 days
      KeepDataFor: 14
      # All blocks will be protected X sec after player places it on ground.
      GlobalBlockTimer:
        Use: true
        Timer: 3
      # Enable silk touch protection.
      # With this enabled players wont get paid for broken blocks from restrictedblocks list with silk touch tool.
      SilkTouchProtection: false

Jobs version: <IP_ADDRESS>
Server Type (Spigot/Paper/etc): Paper
Server Version (using /ver): paper 788

c7db265f9e675fcbb4840fcd491fa5201c313156


Test script tries to test v2 after adding v3

Context
Earlier today, I opened this PR. You can see from the changed files that I added ember-qunit v3 types.
Problem
When I run npm test, I am told that ember-qunit v2 is a dependent package. However, there is no v2 folder, so the test fails.
I only added types for v3. Why is the script trying to test v2?
Here are the logs from the CI.

Turns out I read the DefinitelyTyped README incorrectly. Old types are supposed to go in a vX folder, and the new types are supposed to go into the root. I did it the other way around. I should have moved existing types into v2 and added new types to the root.


Asynchronously queue requests

Because of the rate limitation it might be nice to be able to queue a request, the request will sit in a queue and wait for its turn before being sent to the Blizzard API.
Some ideas on how this could be done...

Receive an estimated time until request will be made (based on the minute & day limitations and how many requests are queued)
When queueing a request and a request for the same information already exists no new request is added to the queue, instead it will return a reference to the already existing request

Going to do some research and suggest a few solutions

The limitation is 100 requests / seconds towards the Blizzard API.
If a request is made and the 100 request rate limit has been reached, instead of throwing an exception the request my suggestion is to put the request in a queue. So that, once the rate limitation has passed its duration we can immediately send the next 100 requests from the queue and return the response for those.
I haven't gotten around to creating an implementation as it fully depends on Blizzard and how they handle the rate limitation. One example is, as you suggest, having a loop which sends 100 requests every second. This loop can be triggered as soon as a request is being sent, and terminate if there are no requests remaining in the queue.
Will do a little more research to try to understand what the response from Blizzard might look like once the rate limit has been met.


Configuration value from new options structure is overridden by deprecated options

I'm trying to use a new options structure to set up the environment name (sentry.options.environment) and it seems it doesn't work properly. The structure I've got looks like that:
sentry:
  dsn: http://xxxxxx
  options:
    environment: staging

Unfortunately all errors I receive in sentry are tagged with the environment name of "prod" which seems to be the default value for the symfony application (running through web/app.php front controller). I was wondering if sentry bundle still defaults to the deprecated option (sentry.environment). I think I found the issue here:
https://github.com/getsentry/sentry-symfony/blob/ea4cf2356a286319a3fc9a6326bdd852ae75d4ad/src/Sentry/SentryBundle/Resources/config/services.yml#L7
It seems the client object is constructed with options from the sentry.options and then some of options are overridden by deprecated values (for example sentry.environment);

I've just tested it locally with
sentry:
  dsn: "******"
  options:
    environment: "%kernel.environment%"

and it works. Thanks anyway because I've found a different bug, which triggers an error when both are used at the same time but with identical values, and I will fix it ASAP.
But still, it works for me.
Are you sure that you are not watching errors from prod? You have to set debug at false to avoid Symfony catching all the errors and preventing Sentry from sending events.

This was fixed in releases 0.8.4 and .6


Update docs to reflect that clients are tested with Python 3.8

According to the pipeline the client is tested with Python 3.8, but this is not reflected in the docs yet.

Hi @estensen
Thanks for the report! I updated our templating system (see PR attached), so next release will declare 3.8 correctly (template are applied at release time, so it won't change until the next release, but that's matching what is actually on PyPI).
Thanks again!


IGListDisplayDelegate methods are not called when section controller displays only supplementary views without cells.

New issue checklist

[x] I have reviewed the README and documentation
[x] I have searched existing issues and this is not a duplicate

General information

IGListKit version: latest on master
iOS version(s): 10.1
Reproducible in the demo project? : yes with minor modifications

With the current implementation non of the display delegates are called when number of items is equal to 0. It makes sense that these two methods are not called:
- (void)listAdapter:(IGListAdapter *)listAdapter willDisplaySectionController:(IGListSectionController <IGListSectionType> *)sectionController
               cell:(UICollectionViewCell *)cell
            atIndex:(NSInteger)index;
- (void)listAdapter:(IGListAdapter *)listAdapter didEndDisplayingSectionController:(IGListSectionController <IGListSectionType> *)sectionController
               cell:(UICollectionViewCell *)cell
            atIndex:(NSInteger)index;

but I think the other two must be called since the section is displayed and has content (supplementary views):
- (void)listAdapter:(IGListAdapter *)listAdapter willDisplaySectionController:(IGListSectionController <IGListSectionType> *)sectionController;
- (void)listAdapter:(IGListAdapter *)listAdapter didEndDisplayingSectionController:(IGListSectionController <IGListSectionType> *)sectionController;

Very interesting. This could actually improve display event notifications sent when a supplementary view is about to appear but the cells haven't yet.
Since we support iOS 8+ this should be easy. We can tap into:
- (void)collectionView:(UICollectionView *)collectionView willDisplaySupplementaryView:(UICollectionReusableView *)view forElementKind:(NSString *)elementKind atIndexPath:(NSIndexPath *)indexPath;

- (void)collectionView:(UICollectionView *)collectionView didEndDisplayingSupplementaryView:(UICollectionReusableView *)view forElementOfKind:(NSString *)elementKind atIndexPath:(NSIndexPath *)indexPath;

To give us display start/end events and use the display handler. Great find @almas73!


Detect not authorized request

Detect if a User isn't authorized to make a request and automatically sign out from the application.

Hello, Is this active Repo and I would like to contribute to this lib to make it better,Let me know if it is going to be active project  ?

@varun2india89 Hi, it is. We are transforming it to be a tool for building community platforms. We are in the process of shaping an idea. When we'll finish working on the idea and create a roadmap for the project. We'll be very glad for the contribution.


Refactor: Use Environment Variables for URLs and @Autowired for Dependency Injection

Replaced hardcoded URLs "owners/createOrUpdateOwnerForm" and "pets/createOrUpdatePetForm" with environment variables, enhancing flexibility.
Removed the static final type for variables to enable injection, allowing for easier configuration changes.
Replaced constructor injection for OwnerRepository with the @Autowired annotation, reducing lines of code and improving readability.

@gabemagioli Please sign the Contributor License Agreement!
Click here to manually synchronize the status of this Pull Request.
See the FAQ for frequently asked questions.

@gabemagioli Thank you for signing the Contributor License Agreement!

Thanks for the PR.

Replaced hardcoded URLs "owners/createOrUpdateOwnerForm" and "pets/createOrUpdatePetForm" with environment variables, enhancing flexibility.

There's actually no requirement for being able to customize views like this.

Replaced constructor injection for OwnerRepository with the @Autowired annotation, reducing lines of code and improving readability.

Actually, that isn't a best practice. First of all, the field is not final as it should be and it makes testing of the component harder.


Shell: Support for nested hyper-parameters.

Issue #, if available:
Description of changes:
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Niiiiiiice!

Niiiiiiice!


menu in light mode not visable

The letter in the light mode menu are not visible because it's set to white text both in dark and light mode and the background is white.

Temporally fixed by setting the text to grey color #aaaaaa


SPO-List-Site-External-Users: Created script using PnP PowerShell

Created script to get external users from all site collection and export it to CSV using PnP PowerShell.

Hey @chandaniprajapati , thank you for the sample and your continued support 😊👏 This should be live shortly 😁


Transaction value is showing as 0 on checking account.

-Tested on live app version 1.3.2(171)
-Now the transaction value is showing as 0.
-The old transactions in the checking account are also showing 0 when I am sending from a checking account.

Need to replicate this. Depends on #2154

@cakesoft-swati can you please reproduce it and let me know the how-tos? On testnet it doesn't seem to happen, can probably be a mainnet specific issue but ya a confirmation would definiely help.

Would recommending testing this in v1.3.2 # 172 together with the send crash fix.

-
-I sent 15k from Donation account(just did a upgrade from 171-172) to checking account(fresh installed 172),still the transaction value is showing as 0 on checking account.

The mainnet wrapper seems to return an incorrect amount value for the above.

Can we retest this please?

Screen short from Test-Live v1.3.2 # 172
Mainnet Checking Account
Sent and received transactions are showing correct amount.

-Now it's working.Verified on Hexa live app version 1.3.2(172)


Update Python to 3.8.8

This PR

updates the Python fwlinks to the new Python 3.8.8 packages
adds checks for ADS Python 3.6.6 users and prompts to upgrade

Note: We are removing the bundle version (0.0.1) from the default python path. Fixes https://github.com/microsoft/azuredatastudio/issues/14622

Pull Request Test Coverage Report for Build 796462783

26 of 67 (38.81%)  changed or added relevant lines in 5 files are covered.
1 unchanged line in 1 file lost coverage.
Overall coverage decreased (-0.02%) to 43.391%

Changes Missing Coverage
Covered Lines
Changed/Added Lines
%

extensions/notebook/src/dialog/configurePython/configurePythonWizard.ts
0
1
0.0%

extensions/machine-learning/src/common/utils.ts
0
5
0.0%

extensions/notebook/src/jupyter/jupyterServerInstallation.ts
20
55
36.36%

Files with Coverage Reduction
New Missed Lines
%

extensions/notebook/src/jupyter/jupyterServerInstallation.ts
1
50.81%

Totals

Change from base Build 794097825:
-0.02%

Covered Lines:
26321

Relevant Lines:
55197

💛  - Coveralls


Unhandled exception. System.Exception: Expected redirect did not occur

Hi,
Noticed this has been happening the past couple of days, not made any changes. Updated to 2.1.0 and still an issue.
Anyone else getting this?
Thanks

Yeah, getting the same issue. Maybe some authentication changes has been done by tesla.

Yes, something has changed on the Tesla side. I'm trying to figure out what to do, but it's not caused by the MFA this time.
Note that the flow using the embedded browser is still working.

So it's the CAPTCHA again. Tesla seem to be enabling and disabling this randomly. As per the Readme, the headless authentication mode won't work when the CAPTCHA is switched on.

The error code paths do need a little work.  I agree the CAPTCHA is the biggest cause of problems here.  However, we may end up in a similar place in the code due to a locked Tesla account.  The wrong MFA code also needs to be handled.
I'm still experimenting with this, but thought I'd throw this out there for a point of reference.  I'd much rather see this where appropriate:
System.Exception: Can't log in - Tesla CAPTCHA support needed.
This is in the bottom half-ish of GetAuthorizationCodeAsync.
            if (result.StatusCode != HttpStatusCode.Redirect)
            {
                if (result.StatusCode == HttpStatusCode.OK && resultContent.Contains("passcode"))
                {
                    if (string.IsNullOrEmpty(mfaCode))
                    {
                        throw new MultiFactorAuthenticationException(String.Format("Multi-factor code required to authenticate for account {0}", username), username);
                    }

                    return await GetAuthorizationCodeWithMfaAsync(mfaCode, loginInfo, client, cancellationToken);
                }
                else if (result.StatusCode == HttpStatusCode.OK)
                {
                    // CAPTCHA requirement, probably.
                    bool hasCaptcha = resultContent.Contains("captcha");
                    bool hasRedirectLocation = result.Headers.Location != null;
                    if (!hasCaptcha)
                    {
                        // This is unexpected.  What is this?
                        throw new Exception("Can't log in - expected redirect did not occur.  hasRedirect: "+hasRedirectLocation);
                    }
                    throw new Exception("Can't log in - Tesla CAPTCHA support needed.");
                }
                else
                {
                    // Possible causes for ending up here:
                    // 1) Account could require an MFA code and we didn't provide it, maybe.  Captcha check may come first though.
                    // What happens with the wrong MFA code?  Does that come here?
                    // We believe cases like the account being locked fall into a different codepath below.
                    throw new Exception("Expected redirect did not occur - probably need multi-factor authentication code.  Status code: " + result.StatusCode);
                }
            }

            var location = result.Headers.Location;

            if (location == null)
            {
                throw new SecurityException($"Logging in failed for {username}.  The account may be locked.");
            }

Thanks @briangru. I agree the error paths aren't great. Are you able to raise a PR with your suggested improvements?

Not quite yet - I'm going to be occupied for several days.
FYI - I also saw this for an expired refresh token.  As you may know, we seem to have several failure behaviors with refresh tokens:

They don't work from the cloud (Forbidden)
They don't work due to Tesla changing something - that's an internal server error on their side
This relatively new-to-me "login_required" exception.  I'm seeing this due to a refresh token that expired 6 weeks ago.  I of course need to add some better handling from this case, but it seems like another exception type:

Exception: TeslaAuthHelper RefreshTokenAsync failed.  Status: Unauthorized  Reason: Unauthorized  Serialized response: {"error":"login_required","error_description":"Login required","error_uri":"https://auth.tesla.com/error/reference/ae25acf2-2486-43b0-bc95-3c4e9142ad7e-1628758009290"}

FYI - here's the HTML for the Forbidden error case:
`
TeslaLib couldn't refresh a login token while logging in for account<EMAIL_ADDRESS> Will try to log in again.  Token created at: 4/4/2021 6:50:32 AM  Expires: 5/19/2021 6:50:32 AM  Exception: TeslaAuthHelper RefreshTokenAsync failed.  Status: Forbidden  Reason: Forbidden  Serialized response:
Access Denied

Access Denied
You don't have permission to access "http://auth.tesla.com/oauth2/v3/token" on this server.
Reference #18.b7622317.1628736600.962b0d1

`


Data missing after import from TFS to VSTS

Guys,
We has the plugin installed on TFS and have many teams using it. We just imported out entire collection to VSTS and the data for the plugin is missing.
Can you help?
-MrHinsh

That would work...

Hi @willsmythe , is this already in place ? Can we utilize this to move the data ?

The feature to export calendar data has not been added (that I know of). We are happy to take a contribution from the community if someone is interesting in adding this, however!

We're closing old issues so we can get back to actively maintaining the extension. Feel free to reopen if this is still an issue for you.


support arrays and maps as arguments

arrays and maps as arguments are not supported at the time of writing.
If you run following code, it will end up with empty arguments.
// addArr concatenates two slices together
func addArr(a, b []interface{}) []interface{} {
	return append(a, b...)
}

I would like to help out, If you could tell me where to start it would be great


Extention success but with no information about assosited PRs, WI, etc

Azure DevOps Extensions
Generate Release Notes (Node Cross Platform)
Platform
Azure DevOps Services
Azure DevOps Server (TFS) Version
No response
Extension Version
4.19.2
Describe the bug
The extention together with the template seems to work as expected but I see nothing in the file regardless the build number, no work items, no PRs ,etc.
Logs are showing above.
Thanks for helping folks it's really important for us,
Niv
Repo Steps

Set up test release in Azure DevOps
Attach an artifact from some random CIs
Extention is recognize the artifact including build number
Nothing appears regarding PRs, work-items...

Expected Behavior
Showing the complete output with all the relevant assosited workitems, bugs, etc.
Logging Information
Log from the extention itself:
2024-10-06T08:24:52.4376455Z
2024-10-06T08:24:52.5455680Z Removing duplicate Commits from master list
2024-10-06T08:24:52.5461613Z Expanding the truncated commit messages...
2024-10-06T08:24:52.5462353Z Expanded truncated commit messages 0
2024-10-06T08:24:52.5472677Z Leaving CS in default order as returned by API
2024-10-06T08:24:52.5474645Z Find any WorkItems linked from GitHub using the AB#123 format
2024-10-06T08:24:52.5480558Z Adding 0 found using AB#123 links in GitHub comments
2024-10-06T08:24:52.5480988Z Removing duplicate WorkItems from master list
2024-10-06T08:24:52.5482452Z Getting tests linked to WorkItems
2024-10-06T08:24:52.5484635Z Checking for user defined WIQL query
2024-10-06T08:24:52.5488641Z No WIQL query parameters defined
2024-10-06T08:24:52.5488969Z Leaving WI in default order as returned by API
2024-10-06T08:24:52.6791663Z The default artifact for the release was not linked to an Azure DevOps Git Repo Pull Request
2024-10-06T08:24:52.6792519Z Getting all completed Azure DevOps Git Repo PRs in the Team Project X
2024-10-06T08:24:52.6800036Z Getting all completed Azure DevOps Git Repo PRs for all repositories
2024-10-06T08:24:52.6800496Z Getting all completed Azure DevOps Git Repo PRs for all refs
2024-10-06T08:24:52.6800845Z Getting associated PRs
2024-10-06T08:27:23.0943976Z Found 115928 Azure DevOps PRs in the repo
2024-10-06T08:27:23.0961208Z Enriching known Pull Requests
2024-10-06T08:27:23.0961465Z Total Builds: [0]
2024-10-06T08:27:23.0961726Z Total Commits: [0]
2024-10-06T08:27:23.0961967Z Total Workitems: [0]
2024-10-06T08:27:23.0962261Z Total Related Workitems (Parent/Children): [0]
2024-10-06T08:27:23.0969588Z Total Release Tests: [0]
2024-10-06T08:27:23.0977002Z Total Tests: [0]
2024-10-06T08:27:23.0980295Z Total Manual Test Runs: [0]
2024-10-06T08:27:23.0980712Z Total Manual Test Configurations: [0]
2024-10-06T08:27:23.0981061Z Total Pull Requests: [0]
2024-10-06T08:27:23.0981404Z Total Indirect Pull Requests: [0]
2024-10-06T08:27:23.0981749Z Total Associated Test WI: [0]
2024-10-06T08:27:23.0982090Z Total Published Artifacts: [0]
2024-10-06T08:27:23.0982411Z Total Consumed Artifacts: [0]
2024-10-06T08:27:23.0982736Z Total WIQL Workitems: [0]
2024-10-06T08:27:23.0983116Z Generating the release notes, the are 1 template(s) to process
2024-10-06T08:27:23.0984146Z Using in-line template
2024-10-06T08:27:23.2171751Z Loading handlebars-helpers extension
2024-10-06T08:27:24.1824193Z Completed processing template
2024-10-06T08:27:24.1824854Z Writing output file D:\Azure-Agents\Agent2\_work\r760\a\releasenotes.md
2024-10-06T08:27:24.1859081Z Finished writing output file D:\Azure-Agents\Agent2\_work\r760\a\releasenotes.md
2024-10-06T08:27:24.1859614Z Output variable 'undefined' set to value of first generated release notes

Output of the file:
2024-10-06T08:27:30.8256559Z Hello World
2024-10-06T08:27:31.0293951Z # Notes for build
2024-10-06T08:27:31.0295243Z **Build Number**: 4399392
2024-10-06T08:27:31.0296550Z **Build Trigger PR Number**:
2024-10-06T08:27:31.0297557Z
2024-10-06T08:27:31.0298920Z # Associated Pull Requests (0)
2024-10-06T08:27:31.0301090Z
2024-10-06T08:27:31.0304052Z # Global list of WI (0)
2024-10-06T08:27:31.0305053Z
2024-10-06T08:27:31.0307224Z # Global list of CS (0)
2024-10-06T08:27:31.0308663Z
2024-10-06T08:27:31.0310097Z ## Global list of ConsumedArtifacts (0)
2024-10-06T08:27:31.0311378Z | Category | Type | Version Name | Version Id | Commits | Workitems |
2024-10-06T08:27:31.0312521Z |-|-|-|-|-|-|

I'm missing something?
We use the git in Azure DevOps so no external auth is needed.

How so if this is mandantory?
Any master branch is a must assosited with WI + PR.

Can you run the pipeline with the variable set system.debug=true, or use checkbox with diagnostic login on the run dialog. That will show the complete discovery process.
The usual reason for not getting any associate items is that there have been no WI associations since the last successful run of that pipeline definition on the specified branch.

@rfennell Thanks for the quick response!
Let's break it down:

Commits with the WI id is working and i'm able to get those commits only when we mentioned the id there.
Regarding the PR, I've merged a PR with linked WI, following what you've mentioned I'm assuming that only after 1 successful PR merged i'd be able to see the list though?
Do you already have a template which includes only the necessery details like: assosited PRs, bugs for this purpose?

I'm following mostly this article https://medium.com/digikare/create-automatic-release-notes-on-azuredevops-f235376ec533#:~:text=The purpose of this article is to demonstrate and this is the exact formant I'm aiming to have on for my developers.
Let me send you asap the .debug logs once i'm able to test again.

I would expect to see PR associated WI listed. I will look at the detailed logs when provide to see if they explain what is happening.
As to point 3 - this folder contains all the sample I, and other users, have created for documentation or testing issues. There is no definitive  set of templates as everyone needs diff

@rfennell debug mode doesn't show a lot in the CI, i'm running the extention in the release, how can I do debug on release steps?
Here are the outputs above.
Thanks again!
## Associated Pull Requests
{{#forEach pullRequests}}
* **[{{this.pullRequestId}}]({{replace (replace this.url "_apis/git/repositories" "_git") "pullRequests" "pullRequest"}})** {{this.title}}
* Created by {{this.createdBy.displayName}}  {{thos.creationDate}}
* Reviewed by:
{{#forEach this.reviewers}}
  * {{this.displayName}}
{{/forEach}}
* Associated Work Items
{{#each_with_sort_by_field  this.associatedWorkitems "System.Tags"}}
   {{#with (lookup_a_work_item ../../relatedWorkItems this.url)}}
    - [{{this.id}}]({{replace this.url "_apis/wit/workItems" "_workitems/edit"}}) - {{lookup this.fields 'System.Title'}}
   {{/with}}
{{/each_with_sort_by_field}}
* Associated Commits (this includes commits on the PR source branch not associated directly with the build)
{{#forEach this.associatedCommits}}
    - [{{truncate this.commitId 7}}]({{this.remoteUrl}}) - {{get_only_message_firstline this.comment}}
    {{#with (lookup_a_pullrequest_by_merge_commit ../../inDirectlyAssociatedPullRequests  this.commitId)}}
      - Associated PR {{this.pullRequestId}} - {{this.title}}
    {{/with}}
{{/forEach}}
{{/forEach}}

I added system.debug=true for the CI, needs to confirmation for the PR in order to see some additinal output.
Print the output from extention:
2024-10-07T12:14:38.3333623Z Creating the credential handler from the OAUTH token
2024-10-07T12:14:38.5555873Z Getting the current release details
2024-10-07T12:14:38.6765908Z Show associated items for primary artifact only is set to true
2024-10-07T12:14:38.6767365Z Getting the Environment Id
2024-10-07T12:14:38.6767773Z Identified [release notes extension] as having id [33056]
2024-10-07T12:14:38.6773363Z Finding successful deployments
2024-10-07T12:14:38.8497551Z Found 31 releases to consider
2024-10-07T12:14:38.8511457Z Finding the last successful release ReleaseNotes.TaskGroup.Test.20241007.2-1
2024-10-07T12:14:38.8511852Z Getting all artifacts in the current release...
2024-10-07T12:14:38.8515225Z Found 1 artifacts for current release
2024-10-07T12:14:38.9072931Z Getting all artifacts in the most recent successful release [ReleaseNotes.TaskGroup.Test.20241007.2-1]...
2024-10-07T12:14:38.9073448Z Found 1 artifacts for most recent successful release
2024-10-07T12:14:38.9073870Z Looking at artifact [nivgabay-devops-cw-general]
2024-10-07T12:14:38.9074229Z Artifact type [Build]
2024-10-07T12:14:38.9074531Z Build Definition ID [16119]
2024-10-07T12:14:38.9074856Z Build Number: [20241007.3]
2024-10-07T12:14:38.9075165Z Is Primary: [true]
2024-10-07T12:14:38.9075592Z Looking for the [nivgabay-devops-cw-general] in the most recent successful release [ReleaseNotes.TaskGroup.Test.20241007.2-1]
2024-10-07T12:14:38.9076152Z Found artifact [nivgabay-devops-cw-general] with build number [20241007.2] in release [ReleaseNotes.TaskGroup.Test.20241007.2-1]
2024-10-07T12:14:38.9076672Z Checking what commits and workitems have changed from [20241007.2][ID 4413243] => [20241007.3] [ID 4413264]
2024-10-07T12:14:38.9077200Z Defaulting on the workaround for build API limitation (see issue #349 set 'ReleaseNotes.Fix349=false' to disable)
2024-10-07T12:14:38.9077658Z Using workaround for build API limitation (see issue #349)
2024-10-07T12:14:39.2416818Z Processing commits found
2024-10-07T12:14:39.2423854Z Enriched change ae3413bcd11e0815aefb76de27b046859210a6da of type TfsGit
2024-10-07T12:14:39.3573891Z Enriched with details of 1 files
2024-10-07T12:14:39.3578345Z Enriched change e7a687cfba7cb4222ab79d53ed2bf73b3d41692b of type TfsGit
2024-10-07T12:14:39.5765107Z Enriched with details of 1 files
2024-10-07T12:14:39.5768116Z Getting test associated with the latest build [4413264]
2024-10-07T12:14:39.7679966Z Adding the build [4413264] and its associations to the unified results object
2024-10-07T12:14:39.7680406Z Detected 2 commits/changesets between the current build and the last successful one
2024-10-07T12:14:39.7680825Z Detected  1 workitems between the current build and the last successful one
2024-10-07T12:14:39.7681236Z Found 0 tests associated with the build [4413264] adding any not already in the global test list to the list
2024-10-07T12:14:39.8360277Z Found 0 manual tests associated with the build [4413264] adding any not already in the global test list to the list
2024-10-07T12:14:39.8360678Z Get details of [1] WIs
2024-10-07T12:14:39.8368030Z Getting full details of WI batch from index: [0] to [1]
2024-10-07T12:14:39.9892023Z Adding [1] items
2024-10-07T12:14:39.9892348Z Getting all parents of build WorkItems
2024-10-07T12:14:39.9895101Z Looking for parents of WI [933635]
2024-10-07T12:14:39.9898662Z Added 0 parent WI to the list of direct relations
2024-10-07T12:14:39.9898901Z
2024-10-07T12:14:40.0754684Z Removing duplicate Commits from master list
2024-10-07T12:14:40.0762370Z Expanding the truncated commit messages...
2024-10-07T12:14:40.1936367Z Expanded truncated commit messages 1
2024-10-07T12:14:40.1936723Z Leaving CS in default order as returned by API
2024-10-07T12:14:40.1937060Z Find any WorkItems linked from GitHub using the AB#123 format
2024-10-07T12:14:40.1937402Z Adding 0 found using AB#123 links in GitHub comments
2024-10-07T12:14:40.1937710Z Removing duplicate WorkItems from master list
2024-10-07T12:14:40.1937988Z Get details of [1] WIs
2024-10-07T12:14:40.1945002Z Getting full details of WI batch from index: [0] to [1]
2024-10-07T12:14:40.3157892Z Adding [1] items
2024-10-07T12:14:40.3158329Z Getting direct parents and children of WorkItems
2024-10-07T12:14:40.3178644Z Looking for parents and children of WI [933635]
2024-10-07T12:14:40.3180579Z Getting all parents of known WorkItems
2024-10-07T12:14:40.3181001Z Looking for parents of WI [933635]
2024-10-07T12:14:40.3181406Z Added 0 parent WI to the list of direct relations
2024-10-07T12:14:40.3181810Z Checking for user defined WIQL query
2024-10-07T12:14:40.3182210Z No WIQL query parameters defined
2024-10-07T12:14:40.3182667Z Leaving WI in default order as returned by API
2024-10-07T12:14:40.3647106Z The default artifact for the release was not linked to an Azure DevOps Git Repo Pull Request
2024-10-07T12:14:40.3648243Z Getting all completed Azure DevOps Git Repo PRs in the Team Project X
2024-10-07T12:14:40.3648575Z Skipping getting associated PRs
2024-10-07T12:14:40.3648880Z Enriching known Pull Requests
2024-10-07T12:14:40.3653132Z Total Builds: [1]
2024-10-07T12:14:40.3653383Z Total Commits: [2]
2024-10-07T12:14:40.3653627Z Total Workitems: [1]
2024-10-07T12:14:40.3653896Z Total Related Workitems (Parent/Children): [1]
2024-10-07T12:14:40.3654193Z Total Release Tests: [0]
2024-10-07T12:14:40.3654424Z Total Tests: [0]
2024-10-07T12:14:40.3654677Z Total Manual Test Runs: [0]
2024-10-07T12:14:40.3654957Z Total Manual Test Configurations: [0]
2024-10-07T12:14:40.3655213Z Total Pull Requests: [0]
2024-10-07T12:14:40.3655482Z Total Indirect Pull Requests: [0]
2024-10-07T12:14:40.3655738Z Total Associated Test WI: [0]
2024-10-07T12:14:40.3656012Z Total Published Artifacts: [0]
2024-10-07T12:14:40.3656298Z Total Consumed Artifacts: [0]
2024-10-07T12:14:40.3665462Z Total WIQL Workitems: [0]
2024-10-07T12:14:40.3665819Z Generating the release notes, the are 1 template(s) to process
2024-10-07T12:14:40.3667733Z Using in-line template
2024-10-07T12:14:40.5235562Z Loading handlebars-helpers extension
2024-10-07T12:14:41.5479974Z Completed processing template
2024-10-07T12:14:41.5480353Z Prepending output to file D:\Azure-Agents\Agent1\_work\r879\a\releasenotes.md
2024-10-07T12:14:41.5524749Z Finished writing output file D:\Azure-Agents\Agent1\_work\r879\a\releasenotes.md
2024-10-07T12:14:41.5525341Z Output variable 'undefined' set to value of first generated release notes
2024-10-07T12:14:41.5814334Z ##[section]Finishing: Generate Release Notes

Print the output file:
2024-10-07T12:14:45.9381448Z ## Release Name: ReleaseNotes.TaskGroup.Test.20241007.3-1
2024-10-07T12:14:45.9382423Z * **Source Branch**: refs/heads/master
2024-10-07T12:14:45.9383589Z * **Build Number**: 4413264
2024-10-07T12:14:45.9384937Z * **Build Started**: Mon Oct 07 2024 15:13:22 GMT+0300 (Israel Daylight Time)
2024-10-07T12:14:45.9386084Z * **Build Completed**: Mon Oct 07 2024 15:13:32 GMT+0300 (Israel Daylight Time)
2024-10-07T12:14:45.9395286Z * **Tags**:
2024-10-07T12:14:45.9395397Z
2024-10-07T12:14:45.9395625Z # Global list of WI (1)
2024-10-07T12:14:45.9395921Z ## Associated Work Items (only shown if  WI)
2024-10-07T12:14:45.9396183Z *  **933635**  Test WI
2024-10-07T12:14:45.9396460Z    - **WIT** Product Backlog Item
2024-10-07T12:14:45.9396716Z    - **Tags**
2024-10-07T12:14:45.9397058Z    - **Assigned**  Niv Gabay
2024-10-07T12:14:45.9397497Z
2024-10-07T12:14:45.9399389Z # Global list of CS (2)
2024-10-07T12:14:45.9399838Z ### Associated commits  (only shown if CS)
2024-10-07T12:14:45.9402125Z * ** IDae3413bcd11e0815aefb76de27b046859210a6da**
2024-10-07T12:14:45.9402613Z   -  **Message:** Merged PR 412811: 933635 wi-test-2 removing EEE
2024-10-07T12:14:45.9403385Z
2024-10-07T12:14:45.9404558Z 933635 wi-test-2 removing EEE
2024-10-07T12:14:45.9405297Z
2024-10-07T12:14:45.9406519Z Related work items: #933635
2024-10-07T12:14:45.9407532Z   -  **Commited by:** Niv Gabay
2024-10-07T12:14:45.9408338Z
2024-10-07T12:14:45.9409556Z * ** IDe7a687cfba7cb4222ab79d53ed2bf73b3d41692b**
2024-10-07T12:14:45.9411057Z   -  **Message:** 933635 wi-test-2 removing EEE
2024-10-07T12:14:45.9411730Z   -  **Commited by:** Niv Gabay
2024-10-07T12:14:45.9413494Z
2024-10-07T12:14:45.9415166Z ## Associated Pull Requests
2024-10-07T12:14:46.4174959Z ##[section]Finishing: Print output RELEASENOTES.md file
2024-10-07T12:14:46.7289066Z ##[section]Starting: Finalize Job
2024-10-07T12:14:46.7323553Z Cleaning up task key
2024-10-07T12:14:46.7336069Z Start cleaning up orphan processes.
2024-10-07T12:14:46.7373012Z ##[section]Finishing: Finalize Job
2024-10-07T12:14:46.7459346Z ##[section]Finishing: Agent for release notes

@rfennell Thanks again!
PR does work for me after enabling. It reaches 3-4 mins so I guess we can live we that for now.
Also the debug inside the release.
I've also added helper it does works as well but doesn't gives me what I was hoping to get (Related PR like the Related Work Items which gives you a nice look + attached link to the PR itself)
I think i know what budders me now, the template which i'm using in your repository, it makes the wiki looks really heavy.
What I've thought is if there is such template (simple one) similar to this: https://miro.medium.com/v2/resize:fit:1100/format:webp/1*H3P65LuotbnoZGynVwz_iA.png
Following the guy who's published this article isn't relevant due to the changes you've made already regarding the template rendering.
Template I'm using now:
----------------------------------------------------------------
# Release Note - {{releaseDetails.name}}
* **Source Branch**: {{buildDetails.sourceBranch}}
* **Build Number**: {{buildDetails.id}}
* **Build Started**: {{buildDetails.startTime}}
* **Build Completed**: {{buildDetails.finishTime}}
* **Tags**: {{buildDetails.tags}}

## Associated Work Items ({{workItems.length}})
{{#forEach workItems}}
{{#if isFirst}}## Associated Work Items {{/if}}
*  **{{this.id}}**  {{lookup this.fields 'System.Title'}}
   - **WIT** {{lookup this.fields 'System.WorkItemType'}}
   - **Tags** {{lookup this.fields 'System.Tags'}}
   - **Assigned** {{#with (lookup this.fields 'System.AssignedTo')}} {{displayName}} {{/with}}
{{/forEach}}

## Associated Pull Requests
{{#forEach pullRequests}}
{{#if isFirst}}### Associated Pull Requests {{/if}}
*  **{{this.pullRequestId}}** {{this.title}}
{{/forEach}}
----------------------------------------------------------------

Does the part of "Associated Work Items" is the one who's showing the Related Work Items?

Thanks again and again!
Now my focus is to make the template fit as much simple and productive as possible.

@rfennell Following my last comment, do you maybe have some smaller template with the related work items without having the commits assosited to the release notes?
Thanks!

The majority of the sample template I have are here,  the WIKI or in various support issues.
I am unclear what you need given the sample template you included in this thread does not return the commits.
Remember, the basic process to create templates are

Add or remove loops on the Work Item, Commit etc. arrays as noted in the WIKI
In most cases you can filter results with basic handlebar syntax
If you need special filter, that you can't do in handlebar syntax, in the loops you need to write a custom filter, there are sample here

@rfennell I'm not that good with handlebar syntax and how to manipulate the data.
Although,
What I'm looking for is this:

If you see the related work items attach to the commit, it happens only when the "function" behind the loop i'm assuming is execting the related.
I want just the related work items without showing the commits if possible. I've tried searching for some exmaples even of yours but didn't find this sepcific need.
Thanks again!

If you don't want the commits just remove the whole commits block i.e.
{{#forEach commits}}
{{#if isFirst}}### Associated commits {{/if}}
* **{{this.id}}**
  -  **Message:** {{this.message}}
{{/forEach}}

The idea is the template is to give you full flexibility to  choose which parts of the data set to show in your release notes

@rfennell I still see commits showing in the wiki marked in red.
There is no option to avoid this and only showing related work items?
Let me know if i'm missing something here.
Thanks,
Niv

What is handbars template you are currently using for the task

Analysis of your template
The content of the generated release notes is dependant on the handbars template you use. The one you are using contains  three general blocks
The first block provides the document header
# Release Note - {{releaseDetails.name}}
* **Source Branch**: {{buildDetails.sourceBranch}}
* **Build Number**: {{buildDetails.id}}
* **Build Started**: {{buildDetails.startTime}}
* **Build Completed**: {{buildDetails.finishTime}}
* **Tags**: {{buildDetails.tags}}`

The next block loops across the commits associated with the pipeline run, displaying each commit's ID and commit message
{{#forEach commits}}
{{#if isFirst}}### Associated commits {{/if}}
* **{{this.id}}**
  -  **Message:** {{this.message}}
{{/forEach}}

Finally there is a loop across the PRs associated with the pipeline run, this time listing out the PR ID and title
{{#forEach pullRequests}}
{{#if isFirst}}### Associated Pull Requests {{/if}}
*  **{{this.pullRequestId}}** {{this.title}}
{{/forEach}}

How to list work items
As we have seen the current template lists the Commits and PR related to the pipeline run. The only reason your current template lists any Work Items is because your commit message mention WI using the #1234 format in the commit messages, and Azure DevOps is intelligent enough to render the resultant markdown content as the WI link you see.
Normally when using this task, if you wish to list work items associated with a pipeline run you add a handlebars block in the general form, you choose the WI fields you wish to use.
{{#forEach this.workItems}}
{{#if isFirst}}### Associated WorkItems {{/if}}
*  **{{this.id}}**  {{lookup this.fields 'System.Title'}}
   - **WIT** {{lookup this.fields 'System.WorkItemType'}}
   - **Tags** {{lookup this.fields 'System.Tags'}}
   - **Assigned** {{#with (lookup this.fields 'System.AssignedTo')}} {{displayName}} {{/with}}
   - **Description** {{{lookup this.fields 'System.Description'}}}
   - **PRs**
{{/if}}
{{/forEach}}

Summary
So, to answer your question, you need to update your handlebars template to match your release notes needs, adding a block to loop across work items, and maybe removing the blocks that loop across commits and PRs
You can find more details and sample in this projects wiki

2024-10-29T08:47:09.9541239Z Using in-line template
2024-10-29T08:47:09.9541813Z ##[debug]Loading handlebar template
2024-10-29T08:47:09.9542510Z ##[debug]Processing template
2024-10-29T08:47:09.9548492Z ##[debug]  WI: 4
2024-10-29T08:47:09.9548986Z ##[debug]  CS: 3
2024-10-29T08:47:09.9549471Z ##[debug]  PR: 1
2024-10-29T08:47:09.9549971Z ##[debug]  Builds: 1
2024-10-29T08:47:09.9550482Z ##[debug]  Manual Tests: 0
2024-10-29T08:47:09.9551012Z ##[debug]  Manual TestConfigurations: 0
2024-10-29T08:47:09.9551562Z ##[debug]  Release Tests: 0
2024-10-29T08:47:09.9552077Z ##[debug]  Related WI: 4
2024-10-29T08:47:09.9552593Z ##[debug]  Related Tests: 0
2024-10-29T08:47:09.9553107Z ##[debug]  Indirect PR: 0
2024-10-29T08:47:09.9553625Z ##[debug]  Consumed Artifacts: 0
2024-10-29T08:47:09.9554173Z ##[debug]  Published Artifacts: 0
2024-10-29T08:47:09.9554709Z ##[debug]  Query WI: 0
2024-10-29T08:47:09.9555234Z ##[debug]Processing handlebar template
2024-10-29T08:47:10.0841896Z Loading handlebars-helpers extension
2024-10-29T08:47:11.2382855Z ##[debug]Saving custom Handlebars code passed as a string in the parameter 'customHandlebarsExtensionCode' to file D:\Azure-Agents\Agent2\_work\_tasks\XplatGenerateReleaseNotes_7b0a5887-75c4-4ade-a915-f92a93c934cb\4.21.2\customHandlebarsExtension.js
2024-10-29T08:47:11.2384243Z Writing output file D:\Azure-Agents\Agent2\_work\_tasks\XplatGenerateReleaseNotes_7b0a5887-75c4-4ade-a915-f92a93c934cb\4.21.2\customHandlebarsExtension.js
2024-10-29T08:47:11.2420765Z Finished writing output file D:\Azure-Agents\Agent2\_work\_tasks\XplatGenerateReleaseNotes_7b0a5887-75c4-4ade-a915-f92a93c934cb\4.21.2\customHandlebarsExtension.js
2024-10-29T08:47:11.2422290Z ##[debug]Attempting to load custom handlebars extension from D:\Azure-Agents\Agent2\_work\_tasks\XplatGenerateReleaseNotes_7b0a5887-75c4-4ade-a915-f92a93c934cb\4.21.2\customHandlebarsExtension.js}
2024-10-29T08:47:11.2449656Z Loaded handlebars extension file
2024-10-29T08:47:11.2580010Z As the parameter 'stopOnError' is set to false the above Handlebars processing error has been logged but the task not marked as failed. To fail the task when this occurs, change the parameter to true
2024-10-29T08:47:11.2632678Z ##[error]Error Processing handlebars [Error: forEach doesn't match if - 9:3]
2024-10-29T08:47:11.2643356Z ##[debug]Processed: ##vso[task.issue type=error;]Error Processing handlebars [Error: forEach doesn't match if - 9:3]
2024-10-29T08:47:11.2644177Z Prepending output to file D:\Azure-Agents\Agent2\_work\r760\a\releasenotes.md
2024-10-29T08:47:11.2644573Z Finished writing output file D:\Azure-Agents\Agent2\_work\r760\a\releasenotes.md
2024-10-29T08:47:11.2644958Z Output variable 'undefined' set to value of first generated release notes
2024-10-29T08:47:11.2645671Z ##[debug]task result: Succeeded
2024-10-29T08:47:11.2654236Z ##[debug]Processed: ##vso[task.complete result=Succeeded;]
2024-10-29T08:47:11.4465782Z ##[section]Finishing: Generate Release Notes

@rfennell Faild when added this block.
Thanks for helping again.

Sorry a typo in the sample I sent...
The error in the logs is fairly clear

##[debug]Processed: ##vso[task.issue type=error;]Error Processing handlebars [Error: forEach doesn't match if - 9:3]

The foreach and if block opening and closing mismatch in the sample, there are two opening if and only one closing /if
It should have been i.e. remove the extra /if on the last but one line before the foreach
{{#forEach this.workItems}}
{{#if isFirst}}### Associated WorkItems {{/if}}
*  **{{this.id}}**  {{lookup this.fields 'System.Title'}}
   - **WIT** {{lookup this.fields 'System.WorkItemType'}}
   - **Tags** {{lookup this.fields 'System.Tags'}}
   - **Assigned** {{#with (lookup this.fields 'System.AssignedTo')}} {{displayName}} {{/with}}
   - **Description** {{{lookup this.fields 'System.Description'}}}
   - **PRs**
{{/forEach}}

I recommended having a look at the Handlebars site to get a working knowledge of the syntax

@rfennell I'm struggling to understand how to work with handlbars so your help is helping me so much just to let you know!
I feel that i'm might reaching the goal of what's I'm expecting from the release notes to be!
Small notes regarding your last response:

Where can I add description in WIT in order to see in the final release note in Wiki? Nothing is showing there and actually is really good to have from my side.
Example bug WIT: (Where the description supposed to be?)

Eventually, there are numbers of WIT showing but no links/images (much pretier) like it was before.
There is a option to take each WIT showing like the above SS:

And show it instead of number plaintext just with the image + hyperlink like the above per each WIT:

Nothing appears on PRs although those WIT already assosited with it

Thanks again for your huge solution and support!

General
I can't stress enough the importance of having a reasonable working knowledge of Handlebars to use this task.
You must get used to editing your handlebars template, dealing with error, making small changes to focus in the release notes you require.
To aid this I have written numerious of samples of handlebars in this repo and there is the detailed wiki documentation, as well as issues and discussions on this subject to add context.
I have also created the local test console to help with repeated running of the same template during the editing process, without the need to rerun who pipeline.
There is also a conference video on how we use this extension that can provide context.
To your specific questions...
Q1 - Showing specific fields
You control the handlebars templates, you pick what fields to show using the {{lookup this.fields '....'}} syntax
So it is just a case of editing the rows within the Handlebars work items loop to list the fields you want to see. Obviously the ones you pick will be dependant on your reporting needs and what work item types you been discovered associated with the pipeline run
The sample I sent looks for System.Description, but does not exist for the Bug WIT, hence it is empty in the generate release notes.
You have to decide if you wish to

Leave it as is, so for PBI you see a description, but for Bugs this row is empty
Remove the description, so it is never shown
Show another field, such as the Repo Steps, e.g. - **Repo Steps** {{ lookup this.fields 'Microsoft.VSTS.TCM.ReproSteps' }}

Note WIT field can have strange names, the free WIQL editor extension for Azure  DevOps is really useful for finding out the full names of WIT as it lets you edit WIQL in a text format as opposed to the UI based Azure DevOps query editor

Maybe, as well as step 3 use an {{if}} construct to show either the description or repo step based on WIT e.g. something like this check handlebars for syntax

{{#if (eq (lookup this.fields 'System.WorkItemType) 'Bug'))}}
- **Repo Steps** {{ lookup this.fields 'Microsoft.VSTS.TCM.ReproSteps' }}
{{ else }}
   - **Description** {{{lookup this.fields 'System.Description'}}}
{{/if}}

Q2 - Tidy up your related WI for commits
As I said previously, the reason you are seeing these rich UI links to WI associated within the commit blocks is due to how the browser renders markdown files and that you used the #123 syntax in commit comment.
I suspect what you really need to only show the first line of the commit message, most people do this so it keep your list neat (there is no easy way to control formatting otherwise)
I have included a Handlebars helper within the task for just this purpose, so your handlebars template will be something like
{{#forEach commits}}
{{#if isFirst}}### Associated commits {{/if}}
* **{{this.id}}**
  -  **Message:** {{get_only_message_firstline this.message}}
{{/forEach}}

This will remove all the extra lines in the commit messages, and the associated WI links
If you want to know how this works, see the wiki and the code in the samples of this repo
If you want to make hyper links in the generated release notes you just need to format the generated markdown to include a link, injecting the values with Handlebars
[ The text to show in the release notes]( the {{...}} lookup to get the URL)

The actual field lookup you need here depends on if it is loop for a WIT, PR or Commit, the best way to find what fields are available is to use the built in  json handlebars extension to dump out all that is available for the selected object
{{#forEach this.workItems}}
**{{this.id}}**  {{lookup this.fields 'System.Title'}}
{{ json this }}
{{/forEach}}

Q3 - Why is the PR block empty
The block is empty because in the sample I sent there is not lookup, the - PR is not followed by any {{... }} block.
Remember, this sample is one I quickly created from the WIKI, I expected you to edit it to your needs, maybe in this case you could delete the  - **PRs** in the work item loop as you already have a loop to show the associated Pull Requests

@rfennell Thank you for the response!
I thought about the situation which needs to cover PRs which we have and also without PR but the WI in the case attaches to the branch when you created the branch itself.
So with PR and without PR, showing only the primary WI and not any related. Is this make sense and possible?
I've tried following your docs and extension settings but nothing so far could bring me the WI in the release notes without the PR.
Thanks again!!

The WI you are after must be being found by reference in the PR, not direct association with the build.
If this is the case, with the current logic I am not sure there much that can be done
Have you considered using a WIQL query to try to get the WI?

Actually now I'm using WIQL:
SELECT [System.Id], [System.Title], [System.WorkItemType], [System.State], [System.AssignedTo] FROM WorkItems WHERE [System.TeamProject] = '@project' AND [System.Links.LinkType] = 'System.LinkTypes.Related' AND [System.Links.Target] = 'PullRequest' AND [System.Links.TargetId] = @pullRequestId ORDER BY [System.Id]
The number of WI in the PR looks like showing exactly in the generated release notes.
Is it make sense right?


File Manager

I want to use from a file manager like uni sharp in laravel php framework. serene have like this?

serene has its own file-upload but you can develop your custom behaviour and why not share for the others :)

No it doesn't.


Homework 1, 2,3

What changes are you trying to make? (e.g. Adding or removing code, refactoring existing code, adding reports)
What did you learn from the changes you have made?
Was there another approach you were thinking about making? If so, what approach(es) were you thinking of?
Were there any challenges? If so, what issue(s) did you face? How did you overcome it?
How were these changes tested?
A reference to a related issue in your repository (if applicable)
Checklist

[ ] I can confirm that my changes are working as intended

From HW4 Union query is missing.

HW 5 missing.

HW4, 5, 6 Marked, Passed.


Update config.yml with SHA check for codecov

Update config.yml with SHA check for codecov
and fix a typo in the test.

Description

Related Issue

Motivation and Context

How Has This Been Tested?

Screenshots (if appropriate):
Types of changes

[ ] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to change)

Checklist:

[ ] I have signed the Adobe Open Source CLA.
[ ] My code follows the code style of this project.
[ ] My change requires a change to the documentation.
[ ] I have updated the documentation accordingly.
[ ] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes.
[ ] All new and existing tests passed.

closing this PR since we have the fix in another PR.


default quota fpr new users

After running hinkskalle now for two months, we found a nice to have enhancement. It would be great if we can set a default quota for users and groups on their first login or when the group is created.

there is a config option: https://github.com/csf-ngs/hinkskalle/blob/master/docs/configuration.md?plain=1#L41
see also here: https://csf-ngs.github.io/hinkskalle/configuration.html, but I can't link directly there, it seems.
But it only applies as a db default for new entities. It should be

a field on the user/group object
preset to the default config value
applied to new entities for this user/group

@grisuthedragon I did a bit of a rewrite of the whole quota handling. It was based on entities for $reasons, but that does not work so well when groups come into play. So now it's based on users (and images uploaded by users, no matter where) and groups (still tied to the entity). Quotas can be changed in the user and group views (admins only).
Details are here: https://github.com/csf-ngs/hinkskalle/blob/43-default-quota-for-new-users/docs/quotas.md
Maybe you would like to give it a go:
docker pull ghcr.io/csf-ngs/hinkskalle:43-default-quota-for-new-users

and let me know what you think?

I will take a look on this asap. But reading the description it sounds good.


TestNG swallows exceptions silently if @ClassRule is used

Testcase:
pom.xml:
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <modelVersion>4.0.0</modelVersion>
    <groupId>com.mycompany</groupId>
    <artifactId>JUnitTestcase</artifactId>
    <version>1.0-SNAPSHOT</version>
    <packaging>jar</packaging>
    <dependencies>
        <dependency>
            <groupId>junit</groupId>
            <artifactId>junit</artifactId>
            <version>4.12</version>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <version>6.9.8</version>
            <scope>test</scope>
        </dependency>
    </dependencies>
    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <version>2.19.1</version>
                <configuration>
                    <trimStackTrace>false</trimStackTrace>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

MyTest.java
import org.junit.ClassRule;
import org.junit.Test;
import org.junit.rules.ExternalResource;

public class MyTest {

    @ClassRule
    public static ExternalResource resource = new ExternalResource() {
        @Override
        protected void before() throws Throwable {
            throw new IllegalArgumentException("before");
        }

        @Override
        protected void after() {
            throw new IllegalArgumentException("after");
        }
    };

    @Test
    public void myTest() {
        System.out.println("yay!");
    }
}

If I disable TestNG in the pom file I get:
Running MyTest
Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.064 sec <<< FAILURE! - in MyTest
MyTest  Time elapsed: 0.063 sec  <<< ERROR!
java.lang.IllegalArgumentException: before
	at MyTest$1.before(MyTest.java:12)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:46)
	at org.junit.rules.RunRules.evaluate(RunRules.java:20)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:367)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeWithRerun(JUnit4Provider.java:274)
	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:238)
	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:161)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:290)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:242)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:121)

If I enable TestNG I get:
Running MyTest
Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.69 sec - in MyTest

I am expecting to get similar output whether the TestNG plugin is enabled or not.
The popular Dropwizard framework relies on @ClassRule so any minor configuration problem leads to unit tests silently failing.

Hi,
what are you trying to do? Why do you run junit tests with TestNG instead of JUnit itself?
The problem is that TestNG doesn't support @ClassRule feature for the moment.
It is easy to translate @ClassRule to TestNG listener and/or you can use https://gist.github.com/jhalterman/9814439
But I'm aggree, TestNG should, at least, detect the test.

Our team codes exclusively against JUnit but the parent POM (mandated on us by company policy) includes TestNG in the classpath (probably because they use TestNG somewhere).
At this point we'll wait for you to add support for this feature, but thanks for pointing out potential workarounds.
It's worth noting that it isn't strictly true that TestNG doesn't support @ClassRule. It actually runs the annotation just fine on success. The only thing that seems to be broken is error handling.

Ok, I will have a deaper look.
Another workaround is to configure surefire so use junit as runner: check "Running TestNG and JUnit Tests" from https://maven.apache.org/surefire/maven-surefire-plugin/examples/testng.html

For anyone running into this in the future, here is how I worked around the problem (hat tip @juherr):
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-surefire-plugin</artifactId>
<version>2.19.1</version>
<dependencies>
    <!-- WORKAROUND: https://github.com/cbeust/testng/issues/1045#issuecomment-220086979 -->
    <dependency>
	<groupId>org.apache.maven.surefire</groupId>
	<artifactId>surefire-junit4</artifactId>
	<version>2.19.1</version>
    </dependency>
</dependencies>

The issue comes from https://github.com/cbeust/testng/blob/285cf3d9825eb4ed6f05a35c1626c989e5aece96/src/main/java/org/testng/junit/JUnit4TestRunner.java#L137
In case of an exception into a @ClassRule (but into a @BeforeClass too), tr is null because the description is not related to a test.
TestNG has a specific listener for configuration methods but JUnit has not and I have not enough context information in the object. BTW, I need to find a way to manage it.
From TestNG point-of-view, the output is:
Total tests run: 1, Failures: 0, Skips: 1
Configuration Failures: 1, Skips: 1

From JUnit point-of-view, the output is:
Total tests run: 1, Failures: 0, Skips: 1
Configuration Failures: 2, Skips: 0


Consider if to align manager flag names with upstream Kubernetes components

While working on adding support for Prometheus in CAPI TiltFile we noticed that latest CAPK is still using old flags name for metrics and leader election (see
https://cluster-api.sigs.k8s.io/developer/providers/v0.3-to-v0.4.html#align-manager-flag-names-with-upstream-kubernetes-components).
Even this is not mandatory, I suggest the CAPK team consider aligning flag names for sake of consistency across providers; also deploying Prometheus in local dev env using the CAPI Tiltfile won't work without this change as well as "--metrics-bind-addr=<IP_ADDRESS>:8080" in your manager.yaml file suggested in the same doc linked above.

thanks for bringing this up @fabriziopandini !

for anyone curious about fixing this, these lines will need to be changed (as per the advice linked above), https://github.com/kubernetes-sigs/cluster-api-provider-kubemark/blob/main/main.go#L55-L58
also, as Fabrizio mentioned, the manifest for the manager deployment will need to changed as well, see https://github.com/kubernetes-sigs/cluster-api-provider-kubemark/blob/main/config/manager/manager.yaml

hi @elmiko ,
I have also found references of older name "--metrics-addr"  and "--enable-leader-election" in the below files

https://github.com/kubernetes-sigs/cluster-api-provider-kubemark/blob/main/config/manager/manager_auth_proxy_patch.yaml#L24-L25
https://github.com/kubernetes-sigs/cluster-api-provider-kubemark/blob/main/config/default/manager_auth_proxy_patch.yaml#L24-L25

Let me know if the above files also needs to be changed. Ill create a PR once verified..
Changes :

Renaming  --metrics-addr to --metrics-bind-addr
Renaming  --enable-leader-election to --leader-elect

good investigation @deepakmega !
those flags are for the kube-rbac-proxy that is deployed in the cluster, i think we should leave them alone for now and just fix the flag names for the kubemark provider. my reasoning for this is that we have version v0.4.1 in that manifest, and it looks like upstream is at v0.11.0 (https://github.com/brancz/kube-rbac-proxy/releases).
so, i think it would be nice to upgrade that image to the latest version, and also update the flags. but i think we should do it on another pull request. if you would like to investigate and test fixing that as well, i think it would be awesome =)

Sorry for creating multiple PR's , had issue with EasyCla.
Have created a PR #35 to fix the same.
@elmiko  I will create another PR for updating the image to latest version along with updated flags and test the same.


Loses transformation information in assembly

When indexing the assembly, it loses transformations. Is there a way to retain this information when collecting items from the assembly?
assembly2 = vedo.Assembly({"box": vedo.Box()})
assembly2.bounds()  # (-0.5, 0.5, -1.0, 1.0, -1.5, 1.5)
a = assembly2.shift(dx=5)
a.bounds()  # (4.5, 5.5, -1.0, 1.0, -1.5, 1.5)
a["box"].bounds()  # array([-0.5,  0.5, -1. ,  1. , -1.5,  1.5], dtype=float32)

Hi. the assembly2.transform will give you that!
Note that .shift() is an in-place operation.

What I'm after is to have the content of the assembly be manipulated collectively, rather than having to use apply_transform for each individually.

There cannot be an automatic way because that would mess up the assembly:
from vedo import *
asse = Assembly({"sphere1": Sphere(), "sphere2": Sphere(c="blue5").shift(dx=3)})
asse.shift(dx=5)
# note that this will screw up the assembly (unless you clone() the objects):
# for key in [obj.name for obj in asse.objects]:
#     asse[key].apply_transform(asse.transform)
print(asse.xbounds())
print(asse["sphere2"].xbounds())
show(asse, axes=1)

No worries. In that case, I'll close the issue. The information here will be enough for someone wanting to apply the same approach in the future.


int_in_range for 1.1.4 now returning out of range values for 0..1i8

int_in_range(0..=1i8) is generating values from -1 to 1. It generates -1 less often than the others.
I tried reasoning about the latest code change, but couldn't figure it out.
It also generates negative numbers for 0..=i32::MAX and 0..=i64::MAX.

Ought to be fixed by https://github.com/rust-fuzz/arbitrary/pull/118
(we should cut a release)


[GURPS] 2.16.0

Submission Checklist
Required

[x] The pull request title clearly contains the name of the sheet I am editing.
[x] The pull request title clearly states the type of change I am submitting (New Sheet/New Feature/Bugfix/etc.).
[x] The pull request makes changes to files in only one sub-folder.
[x] The pull request does not contain changes to any json files in the translations folder (translation.json is permitted)

Changes / Description

ENHANCEMENT: Update Combat Tab > Tools > Demolition > Fragmentation tool; added skill modifiers for hitting targets.
NEW: Added Breaking Bindings to Tools > Structural tools tab. Based on Breaking Bindings rules from GURPS Low Tech Page 130.

Character Sheet Info Roll20 Internal Use only.


New kernel not used after install

Hi,
I used the command rpi-update to update the kernel of a Banana Pi.
But after 2 reboot, the kernel used by Raspbian is still the original 3.4 (I'm using uname -r command).
When I display the content of the folder /lib/modules/, I see this:
drwxrwxr-x  3 1000 pi   4096 Dec 23  2014 3.4.103
drwxr-xr-x  3 root root 4096 Jun 25 21:48 4.4.13+
drwxr-xr-x  3 root root 4096 Jun 25 21:48 4.4.13-v7+

Do I have to delete the 3.4.103 folder, or change the user/group?
Did I misunderstood something?
Thanks!

Can you report the output of
uname -a
vcgencmd version

Are you using NOOBS?

Wow, you answered me so quickly!
Linux tiger 3.4.103 #1 SMP PREEMPT Thu Dec 18 13:07:12 CST 2014 armv7l GNU/Linux

-bash: vcgencmd: command not found

I'm using the "
Raspbian For BananaPi v1412" image downloaded from here.
It's a Wheezy Raspbian. I think it's kinda unoficial by the way.

If you are using a banana pi then I can't help.
rpi-update only makes sense on a raspberry pi.
Banana Pi will have different kernel and firmware.

I understand... Thank you for your answers btw :+1:


protection tab - calculation error message

Describe the bug
on the production tab, i get this error

this was as a result from faulty api converter address.
cannot reproduce.


fix(Climbing): reset body collision and false fall end triggers

Climbing would mess with the enableBodyColliders parameter in
the body physics object. If a player set that to off, it would
override that value and never set it back. This change fixes
that problem. In addition, falling would sometimes get triggered
as being complete (velocity based) too early based on race cases
between when the FixedUpdate and fall was called. The system now
waits enough time (3 fixed updates worth) before declaring
falling to be complete.

Code changes are in.

Just checked sleeping, it doesn't sleep when the player is moving so I don't think that would be an exact way to test. Since timing isn't precise (even with fixedUpdate timing), you have to give a buffer to even be sure it goes through a physics update. The only other alternative that I can think of is doing a countdown after x FixedUpdates which seems more invasive. Would you rather I go that route @thestonefox?

are we going to run into a situation where sometimes this wait 3 ticks trips us up because in a certain circumstance it needs to wait 4, or 2 is too many?
I know from bitter experience that numbers that work, end up not working :(
I guess in the short term (if everyone is happy) we can go with this approach and solve the other issue later if it arises.

Well before this change, it was completely broke anyway probably 30% of the time. It's an improvement if nothing else. Really I just want to let the physics cycle through a time or two to make sure any forces are being applied before setting the fall as complete.

Added another important state, when grabbing with a second controller, the body physics would go into a falling state (from the first controller ungrab), but never back into the correct grabbing state. ResetFall is now called when grabbing an object to make sure the state goes back to the way it's supposed to be. This might be at least part of the problem with the physics freak out seen when climbing @thestonefox.

@mattboy64  Shall we merge this? Or do you think there will be additional tweaks as you're going through your stuff?

I'd say lets go ahead and merge it. I don't anticipate any other tweaks right now.


Text changes.

Still in process, just putting this PR out there so y'all know what I'm up to.

@jlucido2 and @mbucknell Anything stick out here as wrong? I think I'm done for now.

Looks good to me 👍

Looks ok to me.


[#161411167] Remove health snapshot

What
This reverts 86653a3c811f9169fc4b0bf12fd19d99f12dd695. It was not
reverted using git revert due to conflicts.
This health snapshot was broken when we enabled UAA in Bosh and would
have taken significant development time to fix. It is a maintenance
burden that is not currently delivering value, so it has been removed.
How to review
Code review should be sufficient.
Who can review
@keymon

LGTM


Fix workspace logpath to not use ROOT

follow-up from #335
With this patch, I can successfully build on nixos :)

Thank you


Future request: Fixing mob and block hitboxes

I know that hitboxes of some mobs and blocks really frustrate people. For example hoglins and scaffolding. Maybe oak doors can have those holes get carved too or something. This kind of future could be helpful and make things a bit easier when hitting mobs like hoglins.

No, for a couple reasons.

This isn't a vanilla bug. It's a matter of preference and/or opinion.
It's a vanilla feature change. For a mod like Chocolate who's intention is to keep vanilla parity while fixing obvious  issues, this is just out of place.
It's completely unrelated to world generation and/or custom worlds (the focus of this mod).