My fresh installed Raspberry Pi 4B encountered problems when using Xorg and Xvnc

First of all, I've just come into contact with Linux. Forgive me for my difficulty in accurately finding the key information to solve the problem. If you still need information to locate the problem, I will provide it.
OS: 2021-10-30-raspios-bullseye-arm64.img
I had installed xrdp (0.9.17-2~bpo11+rpt1) ,and can log in normally through Xorg. But The response to my mouse pointer was particularly slow, so was typing. then I tried to use Xvnc, but there is no response after login in.
log is here:
pi@raspberrypi:~ $ sudo systemctl status xrdp
● xrdp.service - xrdp daemon
     Loaded: loaded (/lib/systemd/system/xrdp.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-12-17 23:47:31 CST; 4min 49s ago
       Docs: man:xrdp(8)
             man:xrdp.ini(5)
    Process: 555 ExecStartPre=/bin/sh /usr/share/xrdp/socksetup (code=exited, status=0/SUCCESS)
    Process: 570 ExecStart=/usr/sbin/xrdp $XRDP_OPTIONS (code=exited, status=0/SUCCESS)
   Main PID: 578 (xrdp)
      Tasks: 1 (limit: 8987)
        CPU: 40.607s
     CGroup: /system.slice/xrdp.service
             └─578 /usr/sbin/xrdp

12月 17 23:48:27 raspberrypi xrdp[749]: [INFO ] sending login info to session manager, please wait...
12月 17 23:48:28 raspberrypi xrdp[749]: [INFO ] xrdp_wm_log_msg: login successful for display 10
12月 17 23:48:28 raspberrypi xrdp[749]: [INFO ] login successful for display 10
12月 17 23:48:28 raspberrypi xrdp[749]: [INFO ] loaded module 'libxup.so' ok, interface size 10296, version 4
12月 17 23:48:28 raspberrypi xrdp[749]: [INFO ] started connecting
12月 17 23:48:28 raspberrypi xrdp[749]: [INFO ] lib_mod_connect: connecting via UNIX socket
12月 17 23:48:28 raspberrypi xrdp[749]: [INFO ] lib_mod_log_peer: xrdp_pid=749 connected to X11rdp_pid=782 X11rdp_uid=1000 X11rdp_gid=1000 client_ip=::ffff:<IP_ADDRESS> client_port=60473
12月 17 23:48:28 raspberrypi xrdp[749]: [INFO ] connected ok
12月 17 23:51:48 raspberrypi xrdp[749]: [ERROR] xrdp_iso_send: trans_write_copy_s failed
12月 17 23:51:48 raspberrypi xrdp[749]: [ERROR] Sending [ITU T.125] DisconnectProviderUltimatum failed

=======================================
pi@raspberrypi:~ $ which Xvnc
/usr/bin/Xvnc
pi@raspberrypi:~ $ sudo systemctl status xrdp
● xrdp.service - xrdp daemon
     Loaded: loaded (/lib/systemd/system/xrdp.service; enabled; vendor preset: enabled)
     Active: active (running) since Fri 2021-12-17 23:47:31 CST; 7min ago
       Docs: man:xrdp(8)
             man:xrdp.ini(5)
    Process: 555 ExecStartPre=/bin/sh /usr/share/xrdp/socksetup (code=exited, status=0/SUCCESS)
    Process: 570 ExecStart=/usr/sbin/xrdp $XRDP_OPTIONS (code=exited, status=0/SUCCESS)
   Main PID: 578 (xrdp)
      Tasks: 2 (limit: 8987)
        CPU: 40.749s
     CGroup: /system.slice/xrdp.service
             ├─ 578 /usr/sbin/xrdp
             └─1833 /usr/sbin/xrdp

12月 17 23:54:32 raspberrypi xrdp[1833]: [WARN ] local keymap file for 0x00000409 found and doesn't match built in keymap, using local keymap file
12月 17 23:54:39 raspberrypi xrdp[1833]: [INFO ] connecting to sesman ip <IP_ADDRESS> port 3350
12月 17 23:54:39 raspberrypi xrdp[1833]: [INFO ] xrdp_wm_log_msg: sesman connect ok
12月 17 23:54:39 raspberrypi xrdp[1833]: [INFO ] sesman connect ok
12月 17 23:54:39 raspberrypi xrdp[1833]: [INFO ] sending login info to session manager, please wait...
12月 17 23:54:39 raspberrypi xrdp[1833]: [INFO ] xrdp_wm_log_msg: login successful for display 11
12月 17 23:54:39 raspberrypi xrdp[1833]: [INFO ] login successful for display 11
12月 17 23:54:39 raspberrypi xrdp[1833]: [ERROR] xrdp_wm_log_msg: error loading libvnc.so specified in xrdp.ini, please add a valid entry like lib=libxrdp-vnc.so or similar
12月 17 23:54:39 raspberrypi xrdp[1833]: [INFO ] error loading libvnc.so specified in xrdp.ini, please add a
12月 17 23:54:39 raspberrypi xrdp[1833]: [INFO ] valid entry like lib=libxrdp-vnc.so or similar
pi@raspberrypi:~ $ Xvnc
VNC(R) Server 6.7.2 (r43081) ARMv8-A (Sep 21 2020 00:15:42)
Copyright (C) 2002-2020 RealVNC Ltd.
RealVNC and VNC are trademarks of RealVNC Ltd and are protected by trademark
registrations and/or pending trademark applications in the European Union,
United States of America and other jurisdictions.
Protected by UK patent 2481870; US patent 8760366; EU patent 2652951.
See https://www.realvnc.com for information on VNC.
For third party acknowledgements see:
https://www.realvnc.com/docs/6/foss.html
OS: Debian GNU/Linux 11, Linux 5.10.63, aarch64
<11> 2021-12-17T15:55:15.853Z raspberrypi Xvnc[1845]: VendorConfig: Error in Certificate "CN=GlobalSign,O=GlobalSign,OU=GlobalSign Root CA - R2": X.509 Error: Certificate expired

Fatal server error:
Server is already active for display 0
        If this server is no longer running, remove /tmp/.X0-lock
        and start again.

then I try to upgrade the system, use raspi-config to enable VNC, change the sources.list, remove realvnc-vnc-server, install tigervnc-standalone-server and reinstall xrdp(0.9.12). i can not login in with Xorg :-(, it just crashed(mstsc) after i login in. and Xvnc works
~/.xsession-errors
Xsession: X session started for pi at 2021年 12月 19日 星期日 18:58:35 CST
WARNING: tempfile is deprecated; consider using mktemp instead.
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting DISPLAY=:13.0
X connection to :13.0 broken (explicit kill or server shutdown).
xhost:  unable to open display ":13.0"
dbus-update-activation-environment: setting USER=pi
dbus-update-activation-environment: setting LANGUAGE=zh_CN.UTF-8
dbus-update-activation-environment: setting TEXTDOMAIN=Linux-PAM
dbus-update-activation-environment: setting XDG_SESSION_TYPE=x11
dbus-update-activation-environment: setting HOME=/home/pi
dbus-update-activation-environment: setting XRDP_PULSE_SINK_SOCKET=xrdp_chansrv_audio_out_socket_13
dbus-update-activation-environment: setting XRDP_PULSE_SOURCE_SOCKET=xrdp_chansrv_audio_in_socket_13
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting PULSE_SCRIPT=/etc/xrdp/pulse/default.pa
dbus-update-activation-environment: setting LOGNAME=pi
dbus-update-activation-environment: setting XDG_SESSION_CLASS=user
dbus-update-activation-environment: setting PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
dbus-update-activation-environment: setting XRDP_SOCKET_PATH=/run/xrdp/sockdir
dbus-update-activation-environment: setting XDG_RUNTIME_DIR=/run/user/1000
dbus-update-activation-environment: setting DISPLAY=:13.0
dbus-update-activation-environment: setting LANG=zh_CN.UTF-8
dbus-update-activation-environment: setting UID=1000
dbus-update-activation-environment: setting SHELL=/bin/bash
dbus-update-activation-environment: setting NO_AT_BRIDGE=1
dbus-update-activation-environment: setting XRDP_SESSION=1
dbus-update-activation-environment: setting GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
dbus-update-activation-environment: setting PWD=/home/pi
dbus-update-activation-environment: setting LC_ALL=zh_CN.UTF-8
** Message: 18:58:36.900: main.vala:103: Session is LXDE-pi
** Message: 18:58:36.900: main.vala:104: DE is LXDE

(lxsession:3508): Gtk-WARNING **: 18:58:36.904: cannot open display: :13.0

@Brownlzy - let's concentrate on one thing at a time.
Can you reboot the system, try to log in with Xorg and see what happens? Assuming it doesn't work, can you provide the following:-

A copy of ~/.xsession-errors
In your ~/.xsession-errors, there will be a display number visible. As an example, in the file above, there's a line saying setting DISPLAY=:13.0, so the display number is 13. Can you also provide ~/.xorgxrdp.$DISPLAY.log where $DISPLAY is the display number.

Thanks.

~/.xsession-errors
Xsession: X session started for pi at 2021年 12月 20日 星期一 21:07:58 CST
WARNING: tempfile is deprecated; consider using mktemp instead.
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting DISPLAY=:10.0
xrdb: Resource temporarily unavailable
xrdb: Can't open display ':10.0'
xhost:  unable to open display ":10.0"
dbus-update-activation-environment: setting USER=pi
dbus-update-activation-environment: setting LANGUAGE=zh_CN.UTF-8
dbus-update-activation-environment: setting TEXTDOMAIN=Linux-PAM
dbus-update-activation-environment: setting XDG_SESSION_TYPE=x11
dbus-update-activation-environment: setting HOME=/home/pi
dbus-update-activation-environment: setting JRE_HOME=/usr/lib/jvm/jdk-17.0.1/jre
dbus-update-activation-environment: setting XRDP_PULSE_SINK_SOCKET=xrdp_chansrv_audio_out_socket_10
dbus-update-activation-environment: setting XRDP_PULSE_SOURCE_SOCKET=xrdp_chansrv_audio_in_socket_10
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting PULSE_SCRIPT=/etc/xrdp/pulse/default.pa
dbus-update-activation-environment: setting LOGNAME=pi
dbus-update-activation-environment: setting XDG_SESSION_CLASS=user
dbus-update-activation-environment: setting PATH=/usr/lib/jvm/jdk-17.0.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/games:/usr/games
dbus-update-activation-environment: setting XRDP_SOCKET_PATH=/run/xrdp/sockdir
dbus-update-activation-environment: setting XDG_RUNTIME_DIR=/run/user/1000
dbus-update-activation-environment: setting DISPLAY=:10.0
dbus-update-activation-environment: setting LANG=zh_CN.UTF-8
dbus-update-activation-environment: setting UID=1000
dbus-update-activation-environment: setting SHELL=/bin/bash
dbus-update-activation-environment: setting NO_AT_BRIDGE=1
dbus-update-activation-environment: setting XRDP_SESSION=1
dbus-update-activation-environment: setting CLASSPATH=.:/usr/lib/jvm/jdk-17.0.1/lib:/usr/lib/jvm/jdk-17.0.1/jre/lib
dbus-update-activation-environment: setting GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
dbus-update-activation-environment: setting PWD=/home/pi
dbus-update-activation-environment: setting JAVA_HOME=/usr/lib/jvm/jdk-17.0.1
dbus-update-activation-environment: setting LC_ALL=zh_CN.UTF-8
** Message: 21:08:01.517: main.vala:103: Session is LXDE-pi
** Message: 21:08:01.517: main.vala:104: DE is LXDE

(lxsession:935): Gtk-WARNING **: 21:08:01.573: cannot open display: :10.0

~/.xorgxrdp.$DISPLAY.log
[   316.544]
X.Org X Server 1.20.11
X Protocol Version 11, Revision 0
[   316.544] Build Operating System: linux Debian
[   316.544] Current Operating System: Linux raspberrypi 5.10.63-v8+ #1488 SMP PREEMPT Thu Nov 18 16:16:16 GMT 2021 aarch64
[   316.544] Kernel command line: coherent_pool=1M 8250.nr_uarts=0 snd_bcm2835.enable_compat_alsa=0 snd_bcm2835.enable_hdmi=1  smsc95xx.macaddr=E4:5F:01:6E:01:BC vc_mem.mem_base=0xec00000 vc_mem.mem_size=0x10000000  console=ttyS0,115200 console=tty1 root=PARTUUID=7659a15b-02 rootfstype=ext4 fsck.repair=yes rootwait quiet splash plymouth.ignore-serial-consoles
[   316.544] Build Date: 05 August 2021  11:35:55AM
[   316.544] xorg-server 2:1.20.11-1+rpt1 (https://www.debian.org/support)
[   316.544] Current version of pixman: 0.40.0
[   316.544]    Before reporting problems, check http://wiki.x.org
        to make sure that you have the latest version.
[   316.544] Markers: (--) probed, (**) from config file, (==) default setting,
        (++) from command line, (!!) notice, (II) informational,
        (WW) warning, (EE) error, (NI) not implemented, (??) unknown.
[   316.544] (++) Log file: ".xorgxrdp.10.log", Time: Mon Dec 20 21:07:58 2021
[   316.546] (++) Using config file: "/etc/X11/xrdp/xorg.conf"
[   316.546] (==) Using system config directory "/usr/share/X11/xorg.conf.d"
[   316.549] (**) Option "defaultserverlayout" "X11 Server"
[   316.549] (**) ServerLayout "X11 Server"
[   316.549] (**) |-->Screen "Screen (xrdpdev)" (0)
[   316.549] (**) |   |-->Monitor "Monitor"
[   316.549] (**) |   |-->Device "Video Card (xrdpdev)"
[   316.549] (**) |-->Input Device "xrdpMouse"
[   316.550] (**) |-->Input Device "xrdpKeyboard"
[   316.550] (**) Option "DontVTSwitch" "on"
[   316.550] (**) Option "AutoAddDevices" "off"
[   316.550] (**) Not automatically adding devices
[   316.550] (==) Automatically enabling devices
[   316.550] (==) Automatically adding GPU devices
[   316.550] (==) Max clients allowed: 256, resource mask: 0x1fffff
[   316.559] (WW) The directory "/usr/share/fonts/X11/cyrillic" does not exist.
[   316.559]    Entry deleted from font path.
[   316.565] (==) FontPath set to:
        /usr/share/fonts/X11/misc,
        /usr/share/fonts/X11/100dpi/:unscaled,
        /usr/share/fonts/X11/75dpi/:unscaled,
        /usr/share/fonts/X11/Type1,
        /usr/share/fonts/X11/100dpi,
        /usr/share/fonts/X11/75dpi,
        built-ins
[   316.565] (==) ModulePath set to "/usr/lib/xorg/modules"
[   316.565] (II) Loader magic: 0x55944d0e28
[   316.565] (II) Module ABI versions:
[   316.565]    X.Org ANSI C Emulation: 0.4
[   316.565]    X.Org Video Driver: 24.1
[   316.565]    X.Org XInput driver : 24.1
[   316.565]    X.Org Server Extension : 10.0
[   316.568] (II) systemd-logind: took control of session /org/freedesktop/login1/session/c1
[   316.570] (II) xfree86: Adding drm device (/dev/dri/card1)
[   316.571] (EE) systemd-logind: failed to take device /dev/dri/card1: Operation not permitted
[   316.572] (II) xfree86: Adding drm device (/dev/dri/card0)
[   316.572] (EE) systemd-logind: failed to take device /dev/dri/card0: Operation not permitted
[   316.573] (II) no primary bus or device found
[   316.573]    falling back to /sys/devices/platform/gpu/drm/card1
[   316.573] (II) "glx" will be loaded. This was enabled by default and also specified in the config file.
[   316.573] (II) LoadModule: "dbe"
[   316.573] (II) Module "dbe" already built-in
[   316.573] (II) LoadModule: "ddc"
[   316.573] (II) Module "ddc" already built-in
[   316.573] (II) LoadModule: "extmod"
[   316.573] (II) Module "extmod" already built-in
[   316.573] (II) LoadModule: "glx"
[   316.575] (II) Loading /usr/lib/xorg/modules/extensions/libglx.so
[   316.619] (II) Module glx: vendor="X.Org Foundation"
[   316.619]    compiled for 1.20.11, module version = 1.0.0
[   316.620]    ABI class: X.Org Server Extension, version 10.0
[   316.620] (II) LoadModule: "int10"
[   316.620] (II) Loading /usr/lib/xorg/modules/libint10.so
[   316.625] (II) Module int10: vendor="X.Org Foundation"
[   316.625]    compiled for 1.20.11, module version = 1.0.0
[   316.625]    ABI class: X.Org Video Driver, version 24.1
[   316.625] (II) LoadModule: "record"
[   316.625] (II) Module "record" already built-in
[   316.625] (II) LoadModule: "vbe"
[   316.625] (II) Loading /usr/lib/xorg/modules/libvbe.so
[   316.627] (II) Module vbe: vendor="X.Org Foundation"
[   316.627]    compiled for 1.20.11, module version = 1.1.0
[   316.627]    ABI class: X.Org Video Driver, version 24.1
[   316.627] (II) LoadModule: "glamoregl"
[   316.627] (II) Loading /usr/lib/xorg/modules/libglamoregl.so
[   316.658] (II) Module glamoregl: vendor="X.Org Foundation"
[   316.658]    compiled for 1.20.11, module version = 1.0.1
[   316.658]    ABI class: X.Org ANSI C Emulation, version 0.4
[   316.658] (II) LoadModule: "xorgxrdp"
[   316.658] (II) Loading /usr/lib/xorg/modules/libxorgxrdp.so
[   316.662] (II) Module XORGXRDP: vendor="X.Org Foundation"
[   316.662]    compiled for 1.20.11, module version = 0.2.17
[   316.662]    ABI class: X.Org Video Driver, version 24.1
[   316.662] xorgxrdpSetup:
[   316.662] (II) LoadModule: "fb"
[   316.662] (II) Loading /usr/lib/xorg/modules/libfb.so
[   316.666] (II) Module fb: vendor="X.Org Foundation"
[   316.666]    compiled for 1.20.11, module version = 1.0.0
[   316.666]    ABI class: X.Org ANSI C Emulation, version 0.4
[   316.666] (II) LoadModule: "xrdpdev"
[   316.667] (II) Loading /usr/lib/xorg/modules/drivers/xrdpdev_drv.so
[   316.668] (II) Module XRDPDEV: vendor="X.Org Foundation"
[   316.668]    compiled for 1.20.11, module version = 0.2.17
[   316.668]    ABI class: X.Org Video Driver, version 24.1
[   316.669] xrdpdevSetup:
[   316.669] (II) LoadModule: "xrdpmouse"
[   316.669] (II) Loading /usr/lib/xorg/modules/input/xrdpmouse_drv.so
[   316.670] (II) Module XRDPMOUSE: vendor="X.Org Foundation"
[   316.670]    compiled for 1.20.11, module version = 0.2.17
[   316.670]    Module class: X.Org XInput Driver
[   316.670]    ABI class: X.Org XInput driver, version 24.1
[   316.670] rdpmousePlug:
[   316.670] (II) LoadModule: "xrdpkeyb"
[   316.671] (II) Loading /usr/lib/xorg/modules/input/xrdpkeyb_drv.so
[   316.672] (II) Module XRDPKEYB: vendor="X.Org Foundation"
[   316.672]    compiled for 1.20.11, module version = 0.2.17
[   316.672]    Module class: X.Org XInput Driver
[   316.672]    ABI class: X.Org XInput driver, version 24.1
[   316.672] rdpkeybPlug:
[   316.672] rdpIdentify:
[   316.672] (II) XRDPDEV: driver for xrdp: XRDPDEV
[   316.672] rdpDriverFunc: op 10
[   316.672] (WW) Falling back to old probe method for XRDPDEV
[   316.672] rdpProbe:
[   316.672] (II) Loading sub module "fb"
[   316.672] (II) LoadModule: "fb"
[   316.672] (II) Loading /usr/lib/xorg/modules/libfb.so
[   316.672] (II) Module fb: vendor="X.Org Foundation"
[   316.672]    compiled for 1.20.11, module version = 1.0.0
[   316.672]    ABI class: X.Org ANSI C Emulation, version 0.4
[   316.672] rdpProbe: found DRMDevice xorg.conf value []
[   316.672] rdpProbe: found DRI3 xorg.conf value [1]
[   316.672] (II) XRDPDEV(0): using default device
[   316.672] (WW) VGA arbiter: cannot open kernel arbiter, no multi-card support
[   316.672] rdpPreInit:
[   316.672] rdpPreInit:  open failed
[   316.672] (**) XRDPDEV(0): Depth 24, (--) framebuffer bpp 32
[   316.672] (==) XRDPDEV(0): RGB weight 888
[   316.672] (==) XRDPDEV(0): Using gamma correction (1.0, 1.0, 1.0)
[   316.672] (==) XRDPDEV(0): Default visual is TrueColor
[   316.672] (==) XRDPDEV(0): DPI set to (96, 96)
[   316.672] (II) XRDPDEV(0):   mode "640x480" ok
[   316.672] (II) XRDPDEV(0):   mode "800x600" ok
[   316.672] (II) XRDPDEV(0): Virtual size is 800x600 (pitch 800)
[   316.672] (**) XRDPDEV(0):  Default mode "800x600": 36.0 MHz (scaled from 0.0 MHz), 35.2 kHz, 56.2 Hz
[   316.672] (II) XRDPDEV(0): Modeline "800x600"x0.0   36.00  800<PHONE_NUMBER>  600 601 603 625 +hsync +vsync (35.2 kHz d)
[   316.676] rdpScreenInit: virtualX 800 virtualY 600 rgbBits 8 depth 24
[   316.676] rdpScreenInit: pfbMemory bytes 1920000
[   316.676] rdpScreenInit: pfbMemory 0x7fb7e8c010
[   316.676] rdpSimdInit: assigning yuv functions
[   316.679] (==) XRDPDEV(0): Backing store enabled
[   316.682] rdpClientConInit: disconnect idle session after [0] sec
[   316.682] rdpClientConInit: kill disconnected [0] timeout [0] sec
[   316.682] rdpXvInit: depth 24
[   316.682] rdpScreenInit: out
[   316.683] (II) Initializing extension Generic Event Extension
[   316.683] (II) Initializing extension SHAPE
[   316.683] (II) Initializing extension MIT-SHM
[   316.684] (II) Initializing extension XInputExtension
[   316.687] (II) Initializing extension XTEST
[   316.687] (II) Initializing extension BIG-REQUESTS
[   316.688] (II) Initializing extension SYNC
[   316.688] (II) Initializing extension XKEYBOARD
[   316.688] (II) Initializing extension XC-MISC
[   316.688] (II) Initializing extension SECURITY
[   316.689] (II) Initializing extension XFIXES
[   316.689] (II) Initializing extension RENDER
[   316.689] (II) Initializing extension RANDR
[   316.690] (II) Initializing extension COMPOSITE
[   316.690] (II) Initializing extension DAMAGE
[   316.690] (II) Initializing extension MIT-SCREEN-SAVER
[   316.691] (II) Initializing extension DOUBLE-BUFFER
[   316.691] (II) Initializing extension RECORD
[   316.691] (II) Initializing extension DPMS
[   316.691] (II) Initializing extension Present
[   316.691] (II) Initializing extension DRI3
[   316.691] (II) Initializing extension X-Resource
[   316.692] (II) Initializing extension XVideo
[   316.692] (II) Initializing extension XVideo-MotionCompensation
[   316.692] (II) Initializing extension SELinux
[   316.692] (II) SELinux: Disabled on system
[   316.692] (II) Initializing extension GLX
[   316.692] (II) AIGLX: Screen 0 is not DRI2 capable
[   317.759] (II) IGLX: Loaded and initialized swrast
[   317.759] (II) GLX: Initialized DRISWRAST GL provider for screen 0
[   317.759] (II) Initializing extension XFree86-VidModeExtension
[   317.759] (II) Initializing extension XFree86-DGA
[   317.760] (II) Initializing extension XFree86-DRI
[   317.761] (II) Initializing extension DRI2
[   317.761] rdpCreateScreenResources:
[   317.881] (II) Using input driver 'XRDPMOUSE' for 'xrdpMouse'
[   317.882] (**) Option "CorePointer"
[   317.882] (**) xrdpMouse: always reports core events
[   317.882] rdpmousePreInit: drv 0x55d0786640 info 0x55d098b310, flags 0x0
[   317.882] (II) XINPUT: Adding extended input device "xrdpMouse" (type: Mouse, id 6)
[   317.882] rdpmouseControl: what 0
[   317.882] rdpmouseDeviceInit:
[   317.882] rdpmouseCtrl:
[   317.882] rdpRegisterInputCallback: type 1 proc 0x7fb8075cc4
[   317.882] (**) xrdpMouse: (accel) keeping acceleration scheme 1
[   317.882] (**) xrdpMouse: (accel) acceleration profile 0
[   317.882] (**) xrdpMouse: (accel) acceleration factor: 2.000
[   317.882] (**) xrdpMouse: (accel) acceleration threshold: 4
[   317.882] rdpmouseControl: what 1
[   317.882] rdpmouseDeviceOn:
[   317.882] (II) Using input driver 'XRDPKEYB' for 'xrdpKeyboard'
[   317.882] (**) Option "CoreKeyboard"
[   317.882] (**) xrdpKeyboard: always reports core events
[   317.882] rdpkeybPreInit: drv 0x55d07865b0 info 0x55d0afae90, flags 0x0
[   317.882] (II) XINPUT: Adding extended input device "xrdpKeyboard" (type: Keyboard, id 7)
[   317.882] rdpkeybControl: what 0
[   317.882] rdpkeybDeviceInit:
[   317.915] rdpkeybChangeKeyboardControl:
[   317.915] rdpkeybChangeKeyboardControl: autoRepeat on
[   317.915] rdpRegisterInputCallback: type 0 proc 0x7fb80626e0
[   317.915] rdpkeybControl: what 1
[   317.915] rdpkeybDeviceOn:
[   317.935] (II) config/udev: Adding input device vc4 (/dev/input/event0)
[   317.936] (II) AutoAddDevices is off - not adding device.
[   317.937] (II) config/udev: Adding input device vc4 (/dev/input/event1)
[   317.937] (II) AutoAddDevices is off - not adding device.
[   317.949] rdpDeferredRandR:
[   317.949] rdpResizeSession: width 1024 height 768
[   317.949]   calling RRScreenSizeSet
[   317.949] rdpRRScreenSetSize: width 1024 height 768 mmWidth 271 mmHeight 203
[   317.949] rdpRRGetInfo:
[   317.949]   screen resized to 1024x768
[   317.953]   RRScreenSizeSet ok 1
[   317.953] rdpResizeSession: width 1920 height 1080
[   317.954]   calling RRScreenSizeSet
[   317.954] rdpRRScreenSetSize: width 1920 height 1080 mmWidth 508 mmHeight 286
[   317.954] rdpRRGetInfo:
[   317.954]   screen resized to 1920x1080
[   317.966]   RRScreenSizeSet ok 1
[   317.966] rdpClientConGotConnection:
[   317.966] rdpClientConGotConnection: g_sck_accept ok new_sck 7
[   317.966] rdpClientConGetConnection: idle_disconnect_timeout set to non-positive value, idle timer turned off
[   317.966] rdpAddClientConToDev: adding first clientCon 0x55d0b30fe0
[   317.966] rdpClientConProcessMsgVersion: version 0 0 0 1
[   317.967] rdpClientConProcessScreenSizeMsg: set width 1920 height 1080 bpp 32
[   317.967] rdpClientConProcessScreenSizeMsg: shmemid 0 shmemptr 0x7f8582a000
[   317.967] rdpClientConProcessMsgClientInput: invalidate x 0 y 0 cx 1920 cy 1080
[   317.967] KbdSync: toggling num lock
[   317.967] rdpkeybChangeKeyboardControl:
[   317.967] rdpkeybChangeKeyboardControl: autoRepeat on
[   317.968] rdpClientConProcessMsgClientInfo:
[   317.968] expected xrdp client_info version 20210723, got 32
[   317.968] (EE)
Fatal server error:
[   317.968] (EE) Incompatible xrdp version detected  - please recompile(EE)
[   317.968] (EE)
Please consult the The X.Org Foundation support
         at http://wiki.x.org
 for help.
[   317.968] (EE) Please also check the log file at ".xorgxrdp.10.log" for additional information.
[   317.968] (EE)
[   317.968] rdpmouseControl: what 4
[   317.968] rdpkeybControl: what 4
[   317.968] rdpLeaveVT:
[   317.970] (EE) Server terminated with error (1). Closing log file.

The problem is here:-
[   317.968] (EE) Incompatible xrdp version detected  - please recompile(EE)

The version of xorgxrdp on your system is not compatible with the version of xrdp. xorgxrdp is a separate package on Debian which contains the Xorg module.
Can you check the version of xorgxrdp with dpkg -l xorgxrdp? I've got these versions which seem to work together:-
$ dpkg -l xorg xorgxrdp
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version               Architecture Description
+++-==============-=====================-============-===============================================
ii  xorg           1:7.7+22              amd64        X.Org X Window System
ii  xorgxrdp       1:0.2.17-1~bpo11+rpt1 amd64        Remote Desktop Protocol (RDP) modules for X.org

+++-==============-=====================-============-===============================================
ii  xorg           1:7.7+22              arm64        X.Org X Window System
ii  xorgxrdp       1:0.2.17-1~bpo11+rpt1 arm64        Remote Desktop Protocol (RDP) modules for X.org

What do you get for xrdp -v?

pi@raspberrypi:~ $ xrdp -v
xrdp 0.9.12
  A Remote Desktop Protocol Server.
  Copyright (C) 2004-2018 Jay Sorg, Neutrino Labs, and all contributors.
  See https://github.com/neutrinolabs/xrdp for more information.

  Configure options:
      --enable-ipv6
      --enable-jpeg
      --enable-fuse
      --enable-rfxcodec
      --enable-opus
      --enable-painter
      --enable-vsock
      --build=aarch64-linux-gnu
      --prefix=/usr
      --includedir=${prefix}/include
      --mandir=${prefix}/share/man
      --infodir=${prefix}/share/info
      --sysconfdir=/etc
      --localstatedir=/var
      --disable-silent-rules
      --libdir=${prefix}/lib/aarch64-linux-gnu
      --libexecdir=${prefix}/lib/aarch64-linux-gnu
      --disable-maintainer-mode
      --disable-dependency-tracking
      --with-socketdir=/run/xrdp/sockdir
      build_alias=aarch64-linux-gnu
      CFLAGS=-g -O2 -fdebug-prefix-map=/build/xrdp-k2mnlg/xrdp-0.9.12=. -fstack-protector-strong -Wformat -Werror=format-security
      LDFLAGS=-Wl,-z,relro -Wl,-z,now -Wl,--as-needed
      CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2
      PKG_CONFIG_PATH=/build/xrdp-k2mnlg/xrdp-0.9.12/pkgconfig

  Compiled with OpenSSL 1.1.1k  25 Mar 2021

I'm getting the following:-
$ xrdp -v
xrdp 0.9.17
  A Remote Desktop Protocol Server.
  Copyright (C) 2004-2020 Jay Sorg, Neutrino Labs, and all contributors.
  See https://github.com/neutrinolabs/xrdp for more information.

  Configure options:
      --enable-ipv6
      --enable-jpeg
      --enable-fuse
      --enable-rfxcodec
      --enable-opus
      --enable-painter
      --enable-vsock
      --build=x86_64-linux-gnu
      --prefix=/usr
      --includedir=${prefix}/include
      --mandir=${prefix}/share/man
      --infodir=${prefix}/share/info
      --sysconfdir=/etc
      --localstatedir=/var
      --disable-silent-rules
      --libdir=${prefix}/lib/x86_64-linux-gnu
      --libexecdir=${prefix}/lib/x86_64-linux-gnu
      --disable-maintainer-mode
      --disable-dependency-tracking
      --with-socketdir=/run/xrdp/sockdir
      build_alias=x86_64-linux-gnu
      CFLAGS=-g -O2 -ffile-prefix-map=/build/xrdp-7MIBo2/xrdp-0.9.17=. -fstack-protector-strong -Wformat -Werror=format-security
      LDFLAGS=-Wl,-z,relro -Wl,-z,now -Wl,--as-needed
      CPPFLAGS=-Wdate-time -D_FORTIFY_SOURCE=2
      PKG_CONFIG_PATH=/build/xrdp-7MIBo2/xrdp-0.9.17/pkgconfig

  Compiled with OpenSSL 1.1.1k  25 Mar 2021
$ dpkg -l xrdp
Desired=Unknown/Install/Remove/Purge/Hold
| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name           Version             Architecture Description
+++-==============-===================-============-====================================
ii  xrdp           0.9.17-2~bpo11+rpt1 amd64        Remote Desktop Protocol (RDP) server

It looks like your xrdp update hasn't been successful for some reason.

I know that. When i use 0.9.17, i can login in with Xorg, as i mentioned: "The response to my mouse pointer was particularly slow, so was typing", and when i used Xvnc, it was just a blank screen, log showed that there was error while loading "libvnc.so".
So i searched libvnc.so in "https://www.debian.org/distrib/packages", and downloaded this version https://packages.debian.org/bullseye/xrdp. then i found that i can login in with Xvnc, and it doesn't respond slowly like Xorg in xrdp 0.9.17. At least it's better than none.
Now, i just want them to use it at the same time, no matter which version, just in case.

You'll need to remove the version of xorgxrdp you have installed and install the version which matches xrdp v0.9.12. Based on what you've said, you'll need this one.
https://packages.debian.org/bullseye/xorgxrdp

it works, thank you!

I was running buster, and this was easily fixed by downloading the latest version of VNC from realvnc directly. Running the .deb as root allowed it to overwrite the older built in version.
sudo dpkg -i _DEBFILEYOUDOWNLOADED_

I was having problems when accessing the vnc viewer to control my raspberry pi and i found this article: https://picockpit.com/raspberry-pi/tigervnc-and-realvnc-on-raspberry-pi-bookworm-os/ where i was able to control my raspberry py using a vnc viewer alternative called tigervnc. I just follow the tutorial (setting up the tigervnc in the raspberry py and in the pc) and it worked flawlessly.


Merge Sort [CPP]

Hey, C and CPP wil have the same code. So I think we should reconsider this. See https://github.com/iiitv/algos/blob/master/CONTRIBUTING.md#standalone-points

What shall we do in case of vectors and stuff?

But that would not be applicable in this case. Closing as duplicate.

What shall we do in case of vectors and stuff?

Makes sense. I agree that people can use vectors in place of arrays. But when doing quicksort or mergesort, any programmer would prefer arrays over vectors as vec[n] is O(n).
But suppose you are creating a Binary Indexed Tree, then implementations in C++ and C should differ. C programmer will have to do with functions and pointers whereas C++ one would use a class.
So I guess this really depends on the algorithm.

Correct. We will know when we are going to need CPP implementation too.


Problem to generate a random number in go tour

I'm trying to execute this code in play.golang. But the same just return 0.
package main
import (
"fmt"
"math/rand"
"time"
)
func main() {
s1 := rand.NewSource(time.Now().UnixNano())
r1 := rand.New(s1)
fmt.Println(r1.Intn(10))
}

time.Now() is always the same in the playground as described in the first lesson.

Yep, the playground environment is deterministic. Closing here.


acceptSuggestionOnCommitCharacter

VSCode Version: 1.27.0
OS Version: macOS

this, super, console isn't finished after pressing .

The issue was also with 1.26.1 version.

@mjbvz As I know you know how this setting works. I really like it and I'm trying to use it as much as possible instead of Tab or Enter cuz' it will reduce some typing, but it kinda annoying when it doesn't work in such cases...

Second part (from comment) is upstream issue https://github.com/Microsoft/TypeScript/issues/25691

You're my hero! 😄


Add swipe gesture to switch between tabs

What's the Problem?
Hi !
It would nice to scroll through the tabs easily.
Solution/Idea
It can be possible to swip with two fingers to scroll through the tabs.
Alternatives
It can be combined with the fact of typing on keys at the same time like the tab key for example.
Priorities

Capability
Priority

This proposal will allow developers to accomplish W
Must

This proposal will allow end users to accomplish X
Should

This proposal will allow developers to accomplish Y
Could

This proposal will allow end users to accomplish Z
Won't

Files Version
v<IP_ADDRESS>
Windows Version
Windows 11 21H2 22000.282
Additional comment
No response

@Pein57 thank you for your feedback, can you share more about your use case for this gesture and how it would be an improvement over clicking the tab?


Custom scripts for kafka connect

What changes were proposed in this pull request?

Added CUSTOM_SCRIPT_PATH_PRE_RUN script to kafka connect to be run before kafka connect. This is important when you want to install custom connector JARs (e.g. Debezium)
Fixed command section in kafka connect deployment.yaml. In the official images from Confluent, /etc/confluent/docker/run exits when kafka brokers are not ready within a timeout (default: 40 seconds). Now, in the official images, once the timeout expires, the /etc/confluent/docker/run script exits and the pod reboots. When using a custom script, however, /etc/confluent/docker/run is run in background and if it exits, it is goes unnoticed. For this reason, it's necessary to deal with that timeout within the CUSTOM_SCRIPT_PATH script, otherwise the custom script could hang forever.

How was this patch tested?
Helm install on Kubernetes on AWS.

@gAmUssA rebased on master


C:/HaxeToolkit/haxe/lib/polymod/git/polymod/backends/FlixelBackend.hx:50: characters 25-30 : openfl.utils.IAssetCache has no field cacheBug Report:

C:/HaxeToolkit/haxe/lib/polymod/git/polymod/backends/FlixelBackend.hx:50: characters 25-30 : openfl.utils.IAssetCache has no field cache
How to fix that?

fixed it
https://github.com/ThatDisabledKid/polymod

fixed


Files stored in tmp directory not being deleted

gocfl writes .tiff and .png files to the tmp directory but does not delete them once the OCFL object has been created

the temp files are generated by migration and thumbnail extension.
v2.0.3-beta4 has solved this issue


Federated identities put into my user pool (React Native)

I'm in the progress of building a React Native app and want to utilize Cognito for handling my users.
Problem:
When a user registers themselves via e.g. Facebook/Google in the app, I also want their information to be put in my User Pool.
What I'm after:
The flow I'm wishing for (which I haven't been able to find documentation for):

I utilize the Facebook SDK in the app and gets their Facebook token.
The token is sent off to Cognito which auth the user and puts them into the user pool.
An access token is sent back.

Question:
It that possible? And if so how?

@Soren-Knudsen, currently our federation support is in React with HOCs and is done with Cognito Federated Identities. We are in the process of building similar native modules for React Native. We are hoping to complete this next sprint which will allow you to do this. It requires native modules in React Native.

Any update on the ETA for federated login HOC for React Native?


Version of blender and bpy

Hi, thanks for your great work. However, I want to know the version of blender api you used and could you provide the link to the corresponding pypi release link?  Currently, the version of blender is v2.9.1. However, while running dataset/fbx2joint3d.py, it throws errors:

File "dataset/fbx2joints3d.py", line 202, in
main()
File "dataset/fbx2joints3d.py", line 124, in main
set_homefile(HOME_FILE_PATH)
File "dataset/fbx2joints3d.py", line 58, in set_homefile
bpy.data.objects['Lamp'].data.energy = 2
KeyError: 'bpy_prop_collection[key]: key "Lamp" not found'
Segmentation fault (core dumped)

Hi! The blender version we used is about 2.79 or 2.78. According to your posted errors, I think you can just delete this line bpy.data.objects['Lamp'].data.energy = 2 because just converting .fbx to .npy does not use object Lamp at all.

Thanks for your quick reply. Similar error is also occurs:

bpy.data.worlds['World'].use_sky_blend = True # shaded / dynamic background for animations
bpy.data.worlds['World'].horizon_color = (1, 1, 1)
bpy.data.worlds['World'].zenith_color = (1, 1, 1)

Is use_sky_blender necessary in the script? I am new to blender and bpy scripts. Is there any tutorials which can help me find how to use the api?
Thanks in advance.

I think you can also remove this three lines. Those are for rendering.
As for tutorial, you can refer to blender's official document https://docs.blender.org/api/current/info_quickstart.html. However, I don't feel it's well written. I feel it's better to search for blender scripts according to your purpose and learn from others' examples.

Thanks for your reply. I will have a try.


[.Net][Proposal]: Provide a merged result as the final chunk in streaming API

Observation
It's challenging to merge message chunks on client side when using streaming API, especially when the agent supports function call and the response message type might be text message or tool call message
Solution
In IStreamingAgent, provides the merged result as the final message chunk and put it in MergedResultEnvelope<T>
The MergedResultEnvelope enables user client to detect the merged message using pattern match, or skip that chunk if they decide to merge the chunk on their own.
Before

After (The orange block is the final chunk with merged result)

Usage case
With the consistence of streaming and non-streaming api, user no long need to manually merge the result from streaming api.
IStreamingAgent agent;
IMessage? reply = null;

await foreach(var res in agent.GenerateStreamingResponse())
{
     if (res is MergeChunkEnvelope<IMessage> mergedMessage)
     {
         reply = mergedMessage.Message;
         break;
     }
    // process chunk, for example, print to console
     // ...
}

@DavidLuong98  Let me know what you think about this proposal

I am in favor of this proposal.
One question, will we also have the flexibility to let the user consume deltas or unmerged chunks?
I'm wondering, what if some applications would like to incrementally re-render their Ui and manually merge the chunks themselves.

@DavidLuong98 the original chunks would still be return so the client can decide whether they merge those chunks by themselves or simply pick the merged result from the streaming response.

This is a really good idea @LittleLittleCloud

This isn't easy to define using the class and interface definition because it is a behavior hidden behind an implementation.  It is something that isn't a compile check and only during runtime would any bugs in an implementation be caught.
Does this mean that when a client is using the streaming API and performing the merging finalTextMessage.Update(textMessage), two merges will happen: the client's merge and the internal API's merge?  Adding additional overhead to clients merging the chunks.
So, the prettyprint would only be available when the client gets the final message, and it would not do anything on each chunk?
If this was the behavior, there would be no point in using the stream, right?
What side effects might this have on other middleware components that process the streaming data in real-time but are not aware of the other message type or have to duplicate their computation on the final message?
What if there was a different approach, where the call is similar to the non-streaming and returns a single item, but internally, it calls the streaming API, constructs the final message, and returns this to the caller?  The implementation could be done using an extension method.
Some behaviors are not compatible with streaming approaches.  You could approach the problem where the PrettyPrintMiddleware is built to support the stream first.  What if the input for the pretty print were a System.IO.Stream type, it would not matter if the data was streamed or not streamed.  The stream handling would unwrap the message chunks and write the content to the stream.

I feel like your concern can be devided into two parts

should non-streaming and streaming API be consistent with each other
Putting merged result in MergeMessage<T> sounds more like an agreement rather than something that can be checked by compiler, therefore it may be fragile to bug.

For the first concern, I feel like the answer is yes? Besides, since all the chunks are returned in streaming api, the option of constructing final messages is still available for those who don't want to use the merged result chunk.
For the second conern, the reason of returning the merged result as a separate chunk is to keep the maximum compatibility with existing IStreamingAgent and IStreamingMiddleware. Because the merged result is yet another chunk, you can simply choose to take or drop that chunk. The decision is on the user side.

What if there was a different approach, where the call is similar to the non-streaming and returns a single item, but internally, it calls the streaming API, constructs the final message, and returns this to the caller? The implementation could be done using an extension method.

I feel like this is quite similar to what PrettyPrintMessageMiddleware does, which uses GenerateStreamingResponse if the inner agent is IStreamingAgent, and use non-streaming api (GenerateResponse) if the inner agent is IAgent? My concern is it might be challenging to merge chunks for an agent because the streaming response/chunks returned from different LLM providers has slightly difference.

Hi @LittleLittleCloud ,
The proposal is strongly focused on solving the problem of an application consuming replies from an IAgent and IStreamingAgent.  It also tries to solve an implementation problem with specific middleware.  I don't have a reference to the source code of the PrettyPrintMessageMiddleware to evaluate its implementation or the use case of the consuming application that would like to use similar middleware for both IAgent and IStreamingAgent
There are other approaches to solving the problems this proposal is trying to resolve.  It would help to see the code that is expected to benefit from the proposal.
Because the final TextMessage has been added to the stream. The proposal has a problem around when the final TextMessage is constructed.  The proposal does not appear to address when the final TextMessage should be constructed. The current implementation creates it at the beginning of the stream when the reply is created.  This means middleware implementations must perform their work twice, for each TextMessageUpdate and the final TextMessage
It also complicates things and increases the potential for bugs at runtime. With this feature, there is now a combination of factors. The client can choose either TextMessageUpdate or TextMessage, and middleware may support either TextMessageUpdate or TextMessage or both.  This information is not known at compile time, and the developer will only discover this at runtime or if they read the source code of all middleware components to verify which object type it supports.
It complicated the client because if they are streaming with the TextMessageUpdate why do they need to support the final TextMessage?
If they want to use middleware suited for non-streaming, they can wrap the streaming reply into a non-streaming reply.  I'll see if I can prototype the idea.
It has made building streaming middleware more complicated.  A middleware developer for streaming will have to support both message types which significantly complicates things when building streaming logic.  That means state that may evolve during the stream will have to be handled differently for the TextMessage because the logic will have to start at the beginning again to reprocess the TextMessage
Let's look at middleware that will uppercase the response stream. The following code demonstrates the use of the middleware and two ways an end user may consume the streams. The first is processing the stream and using Update to build a complete message. The second is discarding TextMessageUpdate until it gets the TextMessage from the stream.
The two approaches will get different results from the ToUpperMiddleware added to the streaming agent.
The code will check if the final TextMessage contains all uppercase text.  The first one will pass because the TextMessageUpdate will contain the uppercase text.  The second will fail because the final TextMessage will not contain the uppercase text.
The ToUpperMiddleware could solve the problem by performing the work twice. Execute ToUpper on both TextMessageUpdate and TextMessage types.
Putting the generation of the final TextMessage at the end of the stream may not be as elegant.  Code that consumes the client stream using the TextMessageUpdate will not observe the TextMessage.  Any middleware that only processes the TextMessage will not work with a stream that builds the message at the end.
        public static async Task StreamingMiddleware(string modelName)
        {
            Console.WriteLine("");
            Console.WriteLine("Streaming Middleware");

            using var httpClient = new HttpClient()
            {
                BaseAddress = new Uri("http://quorra.homelan.binaryward.com:11434"),
            };

            var ollamaAgent = new OllamaAgent(
                httpClient: httpClient,
                name: "ollama",
                modelName: modelName,
                systemMessage: "You are a helpful AI assistant.")
                .RegisterMessageConnector()
                .RegisterStreamingMiddleware(new ToUpperMiddleware());

            #region Streaming chat where the client constructs the final message from the streamed content
            Console.WriteLine("Streaming chat where the client constructs the final message from the streamed content");
            var userTextMessage = new TextMessage(Role.User, "Hello AI!");
            var reply = ollamaAgent.GenerateStreamingReplyAsync([userTextMessage]);
            var finalTextMessage = new TextMessage(Role.Assistant, string.Empty, from: ollamaAgent.Name); // updating this message based off the reply stream
            await foreach (var message in reply)
            {
                if (message is TextMessageUpdate textMessage)
                {
                    finalTextMessage.Update(textMessage); // add the streamed content to the final message
                }
                else if (message is TextMessage txtMessage) // the final message from the stream
                {
                    // discard the stream's generated final message
                }
            }

            // finalTextMessage will contain the final message with all the streamed content
            if (finalTextMessage == null)
            {
                throw new Exception("finalTextMessage is null");
            }
            var finalContent = finalTextMessage.GetContent()!;
            Console.WriteLine("is upper: " + finalContent.Equals(finalContent.ToUpper()));
            Console.WriteLine(finalContent);
            Console.WriteLine("<done>");
            #endregion

            Console.WriteLine("----------------------------------------------------------------------");
            #region Streaming chat where the client expects the streaming method to return the final message as the last message in the stream
            Console.WriteLine("Streaming chat where the client expects the streaming method to return the final message as the last message in the stream");
            reply = ollamaAgent.GenerateStreamingReplyAsync([userTextMessage]);
            finalTextMessage = null;
            await foreach (var message in reply)
            {
                if (message is TextMessageUpdate textMessage)
                {
                    // discard the streamed content
                }
                else if (message is TextMessage txtMessage) // await the final message from the stream
                {
                    finalTextMessage = txtMessage;
                }
            }
            // finalMessage will contain the final message with all the streamed content
            if (finalTextMessage == null)
            {
                throw new Exception("finalTextMessage is null");
            }
            finalContent = finalTextMessage.GetContent()!;
            Console.WriteLine("is upper: " + finalContent.Equals(finalContent.ToUpper()));
            Console.WriteLine(finalContent);
            Console.WriteLine("<done>");
            #endregion
        }

The ToUpperMiddleware that will ToUpper each TextMessageUpdate.  It does not support the TextMessage because, in this example, one of the following reasons could be used to explain why.

the cost of performing the middleware logic is expensive (cpu, memory, resource) and performing it once.
the developer didn't know it should be handled
the end user is not handling the TextMessage from the stream.

    public class ToUpperMiddleware : IStreamingMiddleware
    {
        public string? Name => nameof(ToUpperMiddleware);

        public async IAsyncEnumerable<IMessage> InvokeAsync(MiddlewareContext context, IStreamingAgent agent, [EnumeratorCancellation] CancellationToken cancellationToken = default)
        {
            // streaming middleware input, (maniuplate the input messages)

            // the transition from the input to the output happens when the reply is created

            // streaming middleware output (manipulate the output messages)
            var streamingReply = agent.GenerateStreamingReplyAsync(context.Messages, context.Options, cancellationToken);

            await foreach (var message in streamingReply)
            {
                if (message is TextMessageUpdate textMessage)
                {
                    textMessage.Content = textMessage.Content?.ToUpper();
                }
                yield return message;
            }
        }

        public Task<IMessage> InvokeAsync(MiddlewareContext context, IAgent agent, CancellationToken cancellationToken = default)
        {
            throw new NotImplementedException();
        }
    }

This version of the middleware only supports TextMessage.  When this middleware is used the first method will fail to get all uppercase.  The second method will get all uppercase.
    public class TextMessageToUpperMiddleware : IStreamingMiddleware
    {
        public string? Name => nameof(ToUpperMiddleware);

        public async IAsyncEnumerable<IMessage> InvokeAsync(MiddlewareContext context, IStreamingAgent agent, [EnumeratorCancellation] CancellationToken cancellationToken = default)
        {
            // streaming middleware input, (maniuplate the input messages)

            // the transition from the input to the output happens when the reply is created

            // streaming middleware output (manipulate the output messages)
            var streamingReply = agent.GenerateStreamingReplyAsync(context.Messages, context.Options, cancellationToken);

            await foreach (var message in streamingReply)
            {
                if (message is TextMessage textMessage)
                {
                    textMessage.Content = textMessage.Content.ToUpper();
                }
                yield return message;
            }
        }

        public Task<IMessage> InvokeAsync(MiddlewareContext context, IAgent agent, CancellationToken cancellationToken = default)
        {
            throw new NotImplementedException();
        }
    }

The only middleware implementation that will support both methods to process the stream is the following.  The middleware has to handle both TextMessage and TextMessageUpdate

    public class AllToUpperMiddleware : IStreamingMiddleware
    {
        public string? Name => nameof(ToUpperMiddleware);

        public async IAsyncEnumerable<IMessage> InvokeAsync(MiddlewareContext context, IStreamingAgent agent, [EnumeratorCancellation] CancellationToken cancellationToken = default)
        {
            // streaming middleware input, (maniuplate the input messages)

            // the transition from the input to the output happens when the reply is created

            // streaming middleware output (manipulate the output messages)
            var streamingReply = agent.GenerateStreamingReplyAsync(context.Messages, context.Options, cancellationToken);

            await foreach (var message in streamingReply)
            {
                if (message is TextMessage textMessage)
                {
                    textMessage.Content = textMessage.Content.ToUpper();
                }
                else if (message is TextMessageUpdate textMessageUpdate)
                {
                    textMessageUpdate.Content = textMessageUpdate.Content?.ToUpper();
                }
                yield return message;
            }
        }

        public Task<IMessage> InvokeAsync(MiddlewareContext context, IAgent agent, CancellationToken cancellationToken = default)
        {
            throw new NotImplementedException();
        }
    }

Hi @MarkWard0110,  thanks for the response, I think now I have a better picture of what you are worried about. I also remove this issue from the next milestone until a conclusion has been made here.
I understand that adding a merged chunk as the final result seems to increase the complexity of developing streaming middleware: With the final merged-result, the middleware needs to process one more message type. However, the complexity is optional and arguable. There are two types of middlewares that may interact with streaming api.

IStreamingMiddleware
IMiddleware that invokes streaming api in non-streaming call

Having the final merged block adds complexity to the first case only when the IStreamingMiddleware needs to process the final chunk. But also make it easy for the second case by removing the overhead of constructing merged result from chunks, which is not easy to do correctly given the lacking information to infer the supported message type from an agent.

There are other approaches to solving the problems this proposal is trying to resolve.

Please suggest any, would love to see the proposal other than appending merged result chunk.

What scenario would benefit from the merged result as the last chunk

Good question, my answer is nearly all scenarios that consume streaming-api because in most cases, they need the merged result, not the intermediate chunks, to proceed to the next step. But merging the final result from chunks is not easy to do correctly, so it's better to have the agent provide the merged result instead.

It complicated the client because if they are streaming with the TextMessageUpdate why do they need to support the final TextMessage?

The client can choose not to support the final TextMessage, in this case, the client code can drop the final chunk and won't be complicated by this proposal.

The proposal stated

use streaming api, because user needs to merge the final result manually

I'm not sure if this exclusively refers to code that consumes the IAgent or IStreamingAgent or if it includes the perspective of developing middleware.
Is there a situation where the code can't call GenerateReplyAsync and is forced to use GenerateStreamingReplyAsync?
The IStreamingAgent interface inherits IAgent interface.  Any Agent that is an IStreamingAgent will have the GenerateReplyAsync method.  An IStreamingAgent that does not implement GenerateReplyAsync will fail Liskov Substitution Principle (LSP)
The nature of streaming is different, but that is okay.  An abstract class or extension method could easily convert any GenerateStreamingReplyAsync into a GenerateReplyAsync by building the TextMessage and returning when the stream has completed.
If GenerateReplyAsync was wrapped with a GenerateStreamingReplyAsync, it would be a stream with a single TextMessageUpdate
The API differences should not affect code that consumes the IAgent or IStreamingAgent.
Middleware development concerns follow into proposal situation 2.

develope middleware that can make the most use of streaming api (for example, PrettyPrintMiddleware should be able to use Streaming API if available, however, since the response from streaming is not consistent with non-streaming api and PrettyPrintMiddleware needs to manually merge results from streaming response, which adds tons of complexity to development work.

I feel like this is quite similar to what PrettyPrintMessageMiddleware does, which uses GenerateStreamingResponse if the inner agent is IStreamingAgent, and use non-streaming api (GenerateResponse) if the inner agent is IAgent? My concern is it might be challenging to merge chunks for an agent without knowing its LLM because the streaming response/chunks returned from different LLM providers has slightly difference.

Where would I find the PrettyPrintMiddleware source code and how it is being used?
Could you clarify why the middleware chooses the generate method based on the agent's type? Doesn't the middleware already know if the initial invocation was GenerateReplyAsync or GenerateStreamingReplyAsync?
There might be reasons why a developer calls GenerateReplyAsync on an IStreamingAgent and would expect internal calls to follow the same pattern—for example, a network performance reason or some other reason that is not predictable to the internal code.
With specifics, it is easier to assess why a streaming-first implementation can't be used for a non-streaming use case. The non-streaming use case is a stream with one message.  The alternative is more straightforward to assess.  A non-streaming implementation being used for streaming will always produce a stream with one message.

Sorry, typo on PrettyPrintMessageMiddleware. It's actually PrintMessageMiddleware and here's the link to code: https://github.com/microsoft/autogen/blob/96146aa9b6533ee67ab7048d328548941df491d0/dotnet/src/AutoGen.Core/Middleware/PrintMessageMiddleware.cs#L16

Could you clarify why the PrintMessageMiddleware chooses the generate method based on the agent's type?

Because the PrintMessageMiddleware streams the output to print to console if possible in its non-streaming invoke API so that you can see the immediate output even in the non-streaming agent call. This is also a good example of invoking streaming api from non-streaming scenario.
For example
IStreamingAgent agent;
agent = agent.RegisterStreamingMiddleware(new PrintMessageMiddleware());
var reply = await agent.SendAsync("tell me a long joke"); // non-streaming api call, with streaming response printed to console.

The proposal stated

use streaming api, because user needs to merge the final result manually

I'm not sure if this exclusively refers to code that consumes the IAgent or IStreamingAgent or if it includes the perspective of developing middleware.

It refers to the client code that consumes IStreamingAgent, including the user client or the non-streaming IMiddleware.InvokeAsync which makes uses of the possible streaming call api from its inner agent.
By providing a merged result as the final chunk, the client code has an option to take the final merged result from IStreamingAgent instead of merging them on their side or drop the final merged result and merging chunks by themselves.
Intrinsicly speaking, providing a merged result as the final chunk moves the complexity of merging chunks from client to developing either IStreamingAgent or IStreamingMiddleware. Which is worthwhile from my point of view. Because it's not easy to merge chunks correctly from the client without knowing which LLM the agent is using.

The nature of streaming is different, but that is okay. An abstract class or extension method could easily convert any GenerateStreamingReplyAsync into a GenerateReplyAsync by building the TextMessage and returning when the stream has completed.

Merging chunks might be easy if the IStreamingAgent only returns TextMessage, but it getting much more complex when the tool call steps in. Not only because client needs to handle the merging situation for both ToolCall and TextMessage, but also the streaming behavior for ToolCallUpdate is slightly different in different LLMs. So it's very challenging to merge chunks correctly without knowing the specific type of LLM in this situation.
Let's take PrintMessageMiddleware as an example, in the non-streaming invoke implementation, it needs to merge TextMessage and ToolCallUpdate based on the chunks type the inner agent returns. However, if the merged result is provided by the inner agent, PrintMessageMiddleware no longer needs to merge those chunks and it will be easier to develop and more robust to use.

There might be reasons why a developer calls GenerateReplyAsync on an IStreamingAgent and would expect internal calls to follow the same pattern—for example, a network performance reason or some other reason that is not predictable to the internal code.

I feel like in most cases, what developers need from an agent is the final result. They invoke GenerateStreamingReply mostly because they want to get intermediate responding/update from LLM. Therefore, providing a merged result as the last chunk is meaningful because it saves the effort for client to merge result.

Does AutoGen Python expose streaming up to the ConversableAgent, AssistantAgent, UserProxy', and GroupChat`?
It looks like in the Python client code, the chunking is contained within the client.  In some cases, the client does not support streaming.
It looks like some clients call streaming or non-streaming API endpoints based on "stream" configuration.  When streaming is called, the client will loop until all chunks have been read.  It appears to me AutoGen Python does not expose chunking through the whole framework but rather during the LLM API calls.  The "stream" configuration plays a role in how the AutoGen API works when a stream object is given for the message.  But it does not appear to expose stream's chunking up through the AutoGen library like the AutoGen .NET does.
Processes chunking in the client
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/client.py#L218
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/cohere.py#L185
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/groq.py#L164
Partial support
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/together.py#L118
Not supported
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/anthropic.py#L95
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/gemini.py#L172
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/mistral.py#L149
The implementation of conversable_agent.py does not appear to handle chunking.  I believe chunking is abstracted away in AutoGen Python.

What if the .NET client abstracts away the LLM API? The client can be configured to use the stream API, but during the call, it will chunk and return one complete message.  Basically, the call returns the same thing for both stream and non-streaming.   Then, the rest of the AutoGen .NET API could leverage this assumption and focus on a single object type, not being exposed to LLM API details.
You will not get the awesome to demonstrate streaming per token output. Perhaps AutoGen's nature does not promote the use of chunking internally.
The chunking might be important in certain situations.  request/response size limits, HTTP timeouts.  non streaming when there are API rate limits
When handling binary media types or large text messages, this might be best designed using a .NET type better suited for streaming than the LLM API chunking.

It looks like some clients call streaming or non-streaming API endpoints based on "stream" configuration. When streaming is called, the client will loop until all chunks have been read. It appears to me AutoGen Python does not expose chunking through the whole framework but rather during the LLM API calls. The "stream" configuration plays a role in how the AutoGen API works when a stream object is given for the message. But it does not appear to expose stream's chunking up through the AutoGen library like the AutoGen .NET does.

This is because python allows you to return different types in the same function, so you can return either one completed message or an async iterator in the same function. But CSharp doesn't allow you to do this, which is why in AutoGen.Net, we seperate streaming and non-streaming API.

It looks like some clients call streaming or non-streaming API endpoints based on "stream" configuration. When streaming is called, the client will loop until all chunks have been read. It appears to me AutoGen Python does not expose chunking through the whole framework but rather during the LLM API calls. The "stream" configuration plays a role in how the AutoGen API works when a stream object is given for the message. But it does not appear to expose stream's chunking up through the AutoGen library like the AutoGen .NET does.

This is because python allows you to return different types in the same function, so you can return either one completed message or an async iterator in the same function. But CSharp doesn't allow you to do this, which is why in AutoGen.Net, we seperate streaming and non-streaming API.

The AutoGen Python is not writing the LLM API chunks to the stream.  That is the difference between AutoGen.NET and AutoGen Python.  I don't think the Python stream is used in the internal components of AutoGen.  It appears to be used to obtain the stream of content but the rest of AutoGen is using the single object is the sum of chunks or the response of a non-streaming llm API request.
When streaming, the client processes the chunks and returns a single object. It does not yield or return a list.
OpenAIClient
The check if it is streaming is here
https://github.com/microsoft/autogen/blob/6ef5c3027dee7093b49ef62e2bd11ae1760dd3c1/autogen/oai/client.py#L205
It will process each chunk here https://github.com/microsoft/autogen/blob/6ef5c3027dee7093b49ef62e2bd11ae1760dd3c1/autogen/oai/client.py#L218
Where stream is doing some magic is if there is a provided stream to the agent it is writing the content to the stream.  This is not a returned type.  The program using AutoGen provides the stream that the client will be writing into.  This would be similar to using a System.IO.Stream in .NET.    The stream has each chunk's content written to it.  But it also creates a single response object that for each choice has the content appended to it, constructing a single message of the content.
Writing to the stream https://github.com/microsoft/autogen/blob/6ef5c3027dee7093b49ef62e2bd11ae1760dd3c1/autogen/oai/client.py#L263
Appending the chunk content https://github.com/microsoft/autogen/blob/6ef5c3027dee7093b49ef62e2bd11ae1760dd3c1/autogen/oai/client.py#L264
The CohereClient's chunk handling
If streaming
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/cohere.py#L182
for each chunk
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/cohere.py#L185
appending the chunk to an internal variable
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/cohere.py#L187
building the return object and assign the content
https://github.com/microsoft/autogen/blob/5292b816ad739be5406b34b0c36165b7aba62272/autogen/oai/cohere.py#L227
The AutoGen.NET could do the same.
First phase - streamline the internal API so that there is only one style of message processing.  Add the ability for the agent implementation to be configured if it will call the streaming or non-streaming LLM API.
Delete the IStreamingAgent interface and related interfaces and methods
Add configuration to each agent to support LLM API streaming (chunking) endpoints.
In the implementation of IAgent the GenerateReplyAsync will, if streaming call the LLM API for streaming (chunking) and process all of the chunks before returning.  If not streaming the call is the non-streaming LLM API call.   Both paths will return the same object type with the complete content of the response.
This will encapsulate the specific chunking implementation for a specific LLM API. It will not leak LLM API chunking implementation details up through AutoGen .NET like it is now.
AutoGen.NET will then work as is relative to the code based on GenerateReplyAsync.
Second phase - feedback streaming support
The main goal of the streaming is to provide feedback token by token.  This makes sense and appears to meet the requirements when libraries have allowed the consuming program provide a stream type via parameters to the library.  Then as it receives the data it writes it to the stream.  The stream will provide the token feedback while the agent is able to leverage a object structure that is easier to work with.
Third phase -
Third phase would be addressing something different from the feedback streaming.  huge data processing where the request and response is far too big to be held as a single object in memory and is best to be streamed.  This would be streamed relative to the AutoGen .NET and in each agent implementation the stream would be converted into the appropriate chunking LLM API.
if there is a need for the whole AutoGen .NET process to large data and still perform Agent like behavior.  I'd look into an approach that leverages something like System.IO.Stream or similar that can stack stream readers and writers on top of each other.  The stream would be content only and the metadata about the message would be held outside of the stream.  It would be a pipeline approach.  This would not have the intent to be compatible with the GenericAsync and would result in the original delima you have now with two APIs.  However, this would be for a completely different use case and would probably require a different set of middleware.
In conclusion,  maybe we can hack a prototype of phases 1 and 2 and see how it works.

 It's designed intentionally to make the most use of IAsyncEnumerable syntax sugar for streaming api as well as to follow the pattern recommended by dotnet, which is one API on one task. And it's by design to expose the difference of streaming/non-streaming response instead of hiding them between streaming and non-streaming API. Because the client code for dealing with streaming VS non-streaming is so different, I don't see benefits of attempting to combine and eliminate the difference. Combining streaming/non-streaming APIs into one seems to make the agent interface more clear, but it converts the complexity of handling the streaming/non streaming messages to the client.

@LittleLittleCloud , I don't think that matches what I was pointing out in AutoGen Python and how their approach works as an adaptation for .NET.

@LittleLittleCloud , I don't think that matches what I was pointing out in AutoGen Python and how their approach works as an adaptation for .NET.

Would be helpful if you can provide some code examples for your proposal.

Hi @LittleLittleCloud ,
I have begun a prototype. It is a proof of concept based on what I think AutoGen Python is doing regarding feedback streaming and supporting LLM API chunking/non-chunking (streaming)
AutoGen Python does not stream the LLM API chunks through the internal workflow/middleware/callables. Each LLM API client is written for each LLM API. Each LLM API client contains the code for streaming and non-streaming requests to the LLM API. The method performing this call returns a single object. It does not return an enumeration/list. The function does not yield. It does not yield when streaming is enabled. It chunks all the response data together and returns a single message similar to a non-streaming request.
The "streaming" in AutoGen Python, I believe is treated differently. It is a feedback mechanism for the user to see the generated tokens appear as a stream.  It means how this stream is used matters and it would not make sense to have all agents configured with a stream.  This stream is totally different from AutoGen .NET because it would not have middleware manipulating it.  (this could be explored)  But the stream is not used by the internal system to perform the agent chat.
It does not address the streaming design feature in AutoGen.NET. I wanted to examine things from this perspective before evaluating a streaming approach.
I have yet to test or validate agent workflows or middleware.
I have yet to look at the console print middleware and see how it might work differently.
It is the first attempt to look at AutoGen .NET as if it were like AutoGen Python.
To set the stage, all of the IStreamingAgent and IStreamingMiddleware are commented out. In some areas, IStreamingAgent was replaced with IAgent and IStreamingMiddleware with IMiddleware
First, was getting an IAgent supporting streaming and non-streaming LLM API calls.
Like AutoGen Python the Ollama OllamaAgent was modified to support calling either the streaming or non-streaming API endpoint and return a single object.
https://github.com/MarkWard0110/fork.autogen/blob/5f9648d9ad44bf1d0a34cb55995d33f4a2ca4add/dotnet/src/AutoGen.Ollama/Agent/OllamaAgent.cs#L45-L102
The streaming flag and the stream itself were added to the class constructor for this prototype.
https://github.com/MarkWard0110/fork.autogen/blob/5f9648d9ad44bf1d0a34cb55995d33f4a2ca4add/dotnet/src/AutoGen.Ollama/Agent/OllamaAgent.cs#L30-L34
A Chat_With_LLaMA_Streaming sample was created to call using the new feature
https://github.com/MarkWard0110/fork.autogen/blob/5f9648d9ad44bf1d0a34cb55995d33f4a2ca4add/dotnet/sample/AutoGen.Ollama.Sample/Chat_With_LLaMA_Streaming.cs#L13-L39
https://github.com/MarkWard0110/fork.autogen/blob/5f9648d9ad44bf1d0a34cb55995d33f4a2ca4add/dotnet/sample/AutoGen.Ollama.Sample/Program.cs#L7
I hope this shows how I understand AutoGen Python is handling "streaming".  The idea may not yet show ideas around merging  your intent to stream within the autogen API.   I was curious to see how AutoGen Python addressed this and it appears they settle on the non-streaming approach when working with internal agent chat workflows.
The feature enables:
Per agent the ability to configure if it will call the LLM API with streaming or not streaming.  The control given to the developer.  The internal AutoGen .NET would not be impacted by the mixed use of streaming and non-streaming LLM API.
Perhaps the stream is a feature for end users to see feedback.  In that case the "System.IO.TextWriter" -> which otherwise should be a generic IO.Stream to support binary, gives the ability for the developer to obtain the realtime feedback without affecting the internal orchestration.  However, the feedback might need to come from the workflow of the agents in the chat, the stream from the AutoGen chat abstraction.  I have not had time to explore this further to create sample code.

@MarkWard0110
How can user early-stop the generating process using your proposal.
A common scenario for early stopping is to periodically check the generating result to see if it contains sensitive/unsafe/unexpect information and cut out the generating process if there is any. IAsyncEnumerable allows developer to deal with this situation in a really flexible and elegant way.

Oh man, I guess I'll have to prioritize a more real-time design for agent workflow.
I wish I had your knowledge of the use cases to be better prepared.
I am thinking you want to be able to move data between the agents as quickly (real-time) as possible.  In the agent conversation would be continuous data streams between the agents.   Which would enable faster responding AI agents.  If the LLM API call supports streaming into the LLM? Is there an LLM API that supports this?
How does AutoGen Python provide that feature?  The stop quickly/early?
I know you are thinking how would it work with IMiddleware. - Let's pretend to use the hackery I just posted.
It would be possible to observe the IO.Stream given to the OllamaAgent for content.  That code would then cancel the CancellationToken which should begin to halt agent chats and workflows (haven't thought this through) -
The idea may have a mix of chat workflow the IMiddleware would work with the classes and would be able to manipulate data
What I am going to test out is a rough idea around the System.IO.Stream.  This type works well for middleware / pipelines because you can layer the streams.  A middleware could be registered and it wraps the stream so that as the stream is written and read from the data will pass through the middleware's stream.  Giving it the ability to perform manipulations or perform sidechannel actions like - signal the chat to end in some way.

How does AutoGen Python provide that feature? The stop quickly/early?

Python AutoGen doesn't have this feature for what I know. I think it's doable using async iterator pattern? But that would introduce a lot of break change I guess. Besides, to your point, the agent workflow (group chat) won't benefit a lot from this feature, because you always need to merge chunks to proceed to the next step.
Dotnet AutoGen allows you to interrupt agent stream, In the agent workflow though, the non-streaming api is still the only option for now. However, it's interesting to think about how to stream the response from an agent workflow.
For streaming the response from an agent workflow, I think there are two levels: streaming on each response, or streaming on each chunks. Where the first case, non-streaming API will be used to generate response, and in the second case, streaming API.
Here is a digram on the difference of streaming an agent workflow on each response VS each chunks

This proposal would also benefit the second scenario, where the orchestrator/group chat needs to merge the chunks from the previous agent before invoking the next one.

It would help to have an example or description of the group chat and the user's intent in using the responses. Something describing the intention of the group chat, how the request/response needs to be used in the group chat, and what the user is using them for.
Is the user a human or code consuming the dotNet AutoGen?
in the diagram, is this the same implementation but called with a stream/non-stream approach, or are you describing a whole new idea based on an async iterator pattern?
Is the request the conversation initiation request?
Would the User in the diagram be able to identify turns from the stream? The request/response pair for a turn that occurred in the group chat?

There are times in the dotnet AutoGen that the use of IAgent is not clear.  For example, IStreamingAgent inherits from IAgent.  If I want to hold a reference to either interface type, I can use IAgent.   If I only want to work with GenerateReplyAsync invocations, I only need to use the IAgent interface for my variables and functions.  If I only want to work with GenerateStreamingReplyAsync I would need to make sure my references use IStreamingAgent or else type checking is required to go from IAgent to IStreamingAgent.
When working with middleware.  IMiddleware references IAgent.  This does not mean it supports adding middleware to any IAgent.  It only supports adding middleware for the GenerateReplyAsync method defined in IAgent.  This does not help if I wanted to use GenerateStreamingReplyAsync.  In this context the IAgent is used to mean agents that are using GenerateReplyAsync, not a type for holding any reference of a descendant of IAgent
When working with IStreamingMiddleware because it inherits from IMiddleware, the middleware supports both GenerateReplyAsync and GenerateStreamingReplyAsync.  In this context the IStreamingAgent is used to mean agents that support GenerateReplyAsync and GenerateStreamingReplyAsync
When I am looking at the RolePlayOrchestrator it references IAgent.  Does this mean it is used for any descendant of IAgent or indicate the RolePlayOrchestrator is only used for GenerateReplyAsync use, because IAgent?  In the class there is calls on GenerateReplyAsync.  This pushes the use of IAgent towards a context for the GenerateReplyAsync and not for a higher reference of agent types.
In one case the PrintMessageMiddleware a IStreamingMiddleware crosses request types for example in a GenerateReplyAsync -> InvokeAsync is calling the streaming InvokeAsync
Is this a general understanding of how the code uses IAgent?  IAgent is used to be any IAgent type or to indicate the use of GenerateReplyAsync use?

This issue is about adding merged result in streaming api but the discussion seems to be a bit off the topic. Would you move some questions to discussion or create  another issue, I am happy to answer there


Mutations?

I cannot find an example of a mutation here. Does anyone have a fragment that show how it's done?

        const mutation = (`
            mutation {
                createItem {
                    name
                    description
                }
            }
        `);
        request(url, mutation).then((data) => console.log(data))

I was surprised to not see an example on this either. Seems so basic

Same here, mutations do not work for me. It sends { query: "mutation ..." }  over the line.

Same here, mutations do not work for me. It sends { query: "mutation ..." }  over the line.

Can I see your input?

Got it to work with the following:
const mutation = `mutation Add($Id: String!) {
        add(id: $Id) { data { result } }
}`
const variables = {
      id : 'a'
}
await new GraphQLClient(url).request(mutation, variables)

closed by #197