[00:02] <powersj> ananke, https://github.com/hashicorp/packer/issues/2639
[00:02] <powersj> workaround in the last comment
[00:03] <ananke> powersj: indeed, that's exactly what I found earlier today and used in my implementation
[00:03] <powersj> there's also this one https://www.packer.io/docs/other/debugging.html#issues-installing-ubuntu-packages
[00:03] <ananke> not as clean as I'd like, but it does give one ability to set a timeout if things go south
[00:06] <ananke> my eventual goal is to use gitlab CI to kick off an EC2 instance, deploy packer on it with cloud-init, then use packer and cloud-init to build an AMI based on a vanilla one + our add-ons
[00:33] <ananke> so I'm trying to debug why a very basic set of runcmd directives do not appear to be working. I've gone as far as cleaning out everything from a basic user data and using the example section from https://cloudinit.readthedocs.io/en/latest/topics/examples.html
[00:34] <ananke> cloud-init analyze show indicates that it should have been executed: |`->config-runcmd ran successfully @17.71300s +00.00100s
[00:34] <ananke> yet the log file shows this: cloud-init.log:2020-01-07 00:27:04,770 - cc_runcmd.py[DEBUG]: Skipping module named runcmd, no 'runcmd' key in configuration
[00:38] <powersj> ananke, common thing to do is to validate your YAML to ensure the syntax is good
[00:59] <ananke> sorry, had to deal with a small emergency
[01:01] <ananke> powersj: YAML seems to be correct. multipass accepts it, and the running instance has said YAML in /var/lib/cloud/instances/test/user-data.txt
[01:03] <ananke> http://dpaste.com/28XMSTQ shows the input YAML and the resulting YAML
[01:05] <ananke> I also tried running it explicitly such as:
[01:05] <ananke> $ sudo cloud-init single --name runcmd --frequency always
[01:06] <ananke> the CLI output appears without any issues: Cloud-init v. 19.3-41-gc4735dd3-0ubuntu1~18.04.1 running 'single' at Tue, 07 Jan 2020 01:04:43 +0000. Up 2283.14 seconds.
[01:06] <ananke> but the log still has: 2020-01-07 01:04:43,096 - cc_runcmd.py[DEBUG]: Skipping module named runcmd, no 'runcmd' key in configuration
[01:07] <ananke> this is output of tail -f in /var/log after said execution: http://dpaste.com/2H1NDZN
[13:04] <ananke> powersj: well, color me surprised. after testing the yaml with cloud-init devel schema I learned that the YAML was actually broken. seems double quotes were removed from one of the runcmd items, which meant the http:// URL was breaking YAML
[13:05] <ananke> now the question is, what removed said quotes. cloud-init or multipass. seems they were removed on one line, but not on another, wtf
[14:02] <ananke> so it appears to be due to yaml-cpp used by multipass, https://github.com/canonical/multipass/issues/1097
[14:16] <Odd_Bloke> Huh, weird.
[14:16] <Odd_Bloke> Let me go and chat to the multipass folks and see if we can (in the long-term) improve that.
[14:36] <meena> Odd_Bloke: hello, good morning
[15:31] <Odd_Bloke> o/
[15:47] <ananke> k, I think I've given up. there seems to be no way to quote things with multipass for runcmd to function correctly. semi-related, I am surprised that cloud-init doesn't have a 'fetch file from url' module
[15:47] <Odd_Bloke> You can do includes, let me find the docs.
[15:48] <Odd_Bloke> ananke: https://cloudinit.readthedocs.io/en/latest/topics/format.html#include-file
[15:49] <ananke> Odd_Bloke: that appears to be only for fetching its own configs, not some random data
[15:49] <Odd_Bloke> Oh, in that case I don't think I understand what you're looking for, could you expand?
[15:51] <ananke> yes, I'm trying to fetch a binary from URL, since it's not available in repos as a package. my hope was to use wget/curl, but that's mangled with the existing multipass problems
[15:54] <blackboxsw> ananke: some more complex runcmd examples (could use sh -xc "something" ) or single quote around the outside of the curl etc https://cloudinit.readthedocs.io/en/latest/topics/examples.html#run-commands-on-first-boot and https://www.digitalocean.com/community/questions/help-with-cloud-init-syntax-for-runcmd
[15:55] <ananke> blackboxsw: doesn't work, because yaml-cpp used by multipass mangles that. see https://github.com/canonical/multipass/issues/1097
[15:57] <ananke> it's a bit ironic, considering cloud-init's documentation recommends multipass as means for validating YAML :) things were going too well for me with multipass & cloud-init, figures I'd have to run into something eventually
[16:16] <roberto-sanchez> Hello guys, got a question, I'll try to make it short: can I format and mount an N number of EBS raw volumes (may be 1, may be 20, externally defined) with cloud-init? As in: it would need to check for unforrmatted disks (lsblk, fdisk, or parted), and if found, format them with mkfs.ext4 and mount them to /data[N] (/data01, /data02 and so on). If
[16:16] <roberto-sanchez> someone reads this and can help me, thank you.
[16:16] <roberto-sanchez> I know I can do this in the final stage, my question was thinking of the fs_setup and disk_setup modules.
[16:17] <rharper> roberto-sanchez: yes, those are the right modules
[16:17] <roberto-sanchez> but how can I get it to iterate on the unformatted disks? That's what I don't get.
[16:18] <roberto-sanchez> and to dynamically act on the N amount of volumes, so it makes N amount of dirs (data01, data02).
[16:18] <roberto-sanchez> and then to mount them.
[16:20] <rharper> oh I understand, an instance may launch with variable mount of volumes but you want a consistent process
[16:20] <rharper> those modules for now, require direct mapping in the config, it supports some abstraction (ephemeral0, 1, 2) and it will look up the name, but you really want a pattern matching ...
[16:21] <roberto-sanchez> right, sorry for my poor explanation, you got it clearly
[16:22] <Odd_Bloke> ananke: Yeah, it's frustrating!
[16:22] <roberto-sanchez> so this is better resolved by standard bash scripting then, @rharper?
[16:25] <blackboxsw> ananke: hrm nice issue will peek a bit today to see if I can reproduce the issue today
[16:28] <ananke> blackboxsw: you can see some more attempts here: https://paste.ofcode.org/6vQAzp2LncyVRuh98Gf6qq
[16:38] <rharper> roberto-sanchez: likely;  we don't currently have support for a syntax that would pattern match N devices by some attribute; which is the first part.  if you know the number ahead of time, it's possible to generate a cloud-config using disk_setup + fs_setup and mount config to do what you're wanting to do; for now, it's likely more portable to write a program to do it and you can execute it early via bootcmd: or later in boot via run_cmd cloud-config
[16:39] <roberto-sanchez> thank you very much for your help and time rharper, logging off; have a nice day everyone!
[17:01] <meena> Odd_Bloke: have you had any time to look at cloudinit/net?
[17:03] <Odd_Bloke> meena: Not yet, I'm afraid!
[17:27] <usrdev> hey all! curious, is it intended behavior that at reboot that cloud-init would reject/refuse to see particular partitions sizes and resize the boot partition?
[17:30] <blackboxsw> looks like i's that time again. +15 :)
[17:30] <blackboxsw> #startmeeting Cloud-init bi-weekly status
[17:30] <meetingology> Meeting started Tue Jan  7 17:30:28 2020 UTC.  The chair is blackboxsw. Information about MeetBot at http://wiki.ubuntu.com/meetingology.
[17:30] <meetingology> Available commands: action commands idea info link nick
[17:30] <blackboxsw> #chair Odd_Bloke
[17:30] <meetingology> Current chairs: Odd_Bloke blackboxsw
[17:30] <blackboxsw> #chair rharper
[17:30] <meetingology> Current chairs: Odd_Bloke blackboxsw rharper
[17:31] <blackboxsw> Welcome to the first cloud-init community status meeting of 2020. cloud-init upstream uses this meeting as a platform for community updates, feature/bug discussions, and an opportunity to get some extra input on current development.
[17:31] <Odd_Bloke> usrdev: I'm not 100% sure from that description, could you file a bug using the link in the topic and attach the output of `cloud-init collect-logs` on an affected instance?
[17:31] <blackboxsw> We generally have this meeting ever 2 weeks (outside of intermittent holidays)... You can always find the next scheduled meeting in the topic of this channel
[17:31] <blackboxsw> Let
[17:31] <blackboxsw> Let
[17:32] <blackboxsw> Let's schedule the next meeting now as well
[17:32] <blackboxsw> Any objections to Jan 21 ?
[17:33] <robjo> Look I'm not late ;)
[17:33] <blackboxsw> ok topic set for next meeting
[17:33] <blackboxsw> nope, just me robjo :) welcome to the party
[17:33] <blackboxsw> as always previous meeting minutes are here.
[17:33] <blackboxsw> #link https://cloud-init.github.io/status-2019-12-10.html#status-2019-12-10
[17:34] <blackboxsw> topics for this round: Feel free to interject/suggest other topics at any time. Our typical format is the following: Previous Actions, Recent Changes, In-progress Development, Community Charter, Upcoming Meetings, Office Hours (~30 mins).
[17:34] <robjo> The move to Tuesday creates a conflict for me for the last 15 minutes of the meeting. Generally I don't think that's an issue as we are often done in less than 1 hour, just pointing out that usually I have to leave 15 minutes early
[17:34] <robjo> not today ;)
[17:36] <blackboxsw> +1 robjo. We'll try to keep it snappy :) and if others have conflicts we can certainly touch on shifting the schedule a bit. We generally have a conflict at 1 hr before this meeting, which is the only reason it isn't 1 hr earlier
[17:36] <blackboxsw> #topic Previous Actions
[17:37] <blackboxsw> last round: rharper to confirm https://github.com/canonical/cloud-init/pull/42 can land. COMPLETED
[17:37] <blackboxsw> action2:  upstream core-devs to decide about whether a PR can land if any upstream dev still has 'requested changes'
[17:38] <blackboxsw> Odd_Bloke: started writing up a spec/procedure for PR review and he is currently working on adding a documentation addition PR to http://cloudinit.readthedocs.io that will describe the workflow for a PR to get from proposed -> merged.
[17:39] <blackboxsw> that PR should likely be up this week for review if folks are watching our review queue
[17:39] <blackboxsw> #link https://github.com/cloud-init/cloud-init/pulls
[17:39] <blackboxsw> No other actions from the previous meeting in December.
[17:39] <blackboxsw> #topic Recent Changes
[17:40] <blackboxsw> recent commits that made it into tip: found via git log --since 12-10-2019
[17:41] <blackboxsw> let's see if I get throttled for spam
[17:41] <blackboxsw>     - freebsd: fix create_group() cmd (#146) [Gonéri Le Bouder]
[17:41] <blackboxsw>     - doc: make apt_update example consistent (#154)
[17:41] <blackboxsw>     - doc: add modules page toc with links (#153) (LP: #1852456)
[17:41] <blackboxsw>     - Add support for the amazon variant in cloud.cfg.tmpl (#119)
[17:41] <blackboxsw>       [Frederick Lefebvre]
[17:41] <blackboxsw> heh
[17:41] <blackboxsw>     - freebsd: fix create_group() cmd (#146) [Gonéri Le Bouder]
[17:41] <blackboxsw> 10:41     - doc: make apt_update example consistent (#154)
[17:41] <blackboxsw> 10:41     - doc: add modules page toc with links (#153) (LP: #1852456)
[17:41] <blackboxsw> 10:41     - Add support for the amazon variant in cloud.cfg.tmpl (#119)
[17:41] <blackboxsw> 10:41       [Frederick Lefebvre]
[17:41] <blackboxsw> 10:41     - ci: remove Python 2.7 from CI runs (#137)
[17:41] <blackboxsw> 10:41     - modules: drop cc_snap_config config module (#134)
[17:41] <blackboxsw> 10:41     - migrate-lp-user-to-github: ensure Launchpad repo exists (#136)
[17:41] <blackboxsw> 10:41     - docs: add initial troubleshooting to FAQ (#104) [Joshua Powers]
[17:41] <blackboxsw> 10:41     - doc: update cc_set_hostname frequency and descrip (#109)
[17:41] <blackboxsw> 10:41       [Joshua Powers] (LP: #1827021)
[17:41] <blackboxsw>     - ci: emit names of tests run in Travis (#120)
[17:41] <blackboxsw> 10:41     - Release 19.4 (LP: #1856761)
[17:41] <blackboxsw> 10:41     - rbxcloud: fix dsname in RbxCloud [Adam Dobrawy] (LP: #1855196)
[17:41] <blackboxsw> 10:41     - tests: Add tests for value of dsname in datasources [Adam Dobrawy]
[17:41] <blackboxsw> 10:41     - apport: Add RbxCloud ds [Adam Dobrawy]
[17:41] <blackboxsw> 10:41     - docs: Updating index of datasources [Adam Dobrawy]
[17:41] <blackboxsw> 10:41     - docs: Fix anchor of datasource_rbx [Adam Dobrawy]
[17:41] <blackboxsw> 10:41     - settings: Add RbxCloud [Adam Dobrawy]
[17:41] <blackboxsw> 10:41     - doc: specify _ over - in cloud config modules
[17:41] <blackboxsw> 10:41       [Joshua Powers] (LP: #1293254)
[17:42] <blackboxsw>    - tools: Detect python to use via env in migrate-lp-user-to-github
[17:42] <blackboxsw>       [Adam Dobrawy]
[17:42] <blackboxsw>     - Partially revert "fix unlocking method on FreeBSD" (#116)
[17:42] <blackboxsw>     - tests: mock uid when running as root (#113)
[17:42] <blackboxsw>       [Joshua Powers] (LP: #1856096)
[17:42] <blackboxsw>     - cloudinit/netinfo: remove unused getgateway (#111)
[17:42] <blackboxsw>     - docs: clear up apt config sections (#107) [Joshua Powers] (LP: #1832823)
[17:42] <blackboxsw>     - doc: add kernel command line option to user data (#105)
[17:42] <blackboxsw>       [Joshua Powers] (LP: #1846524)
[17:42] <blackboxsw>     - config/cloud.cfg.d: update README [Joshua Powers] (LP: #1855006)
[17:42] <blackboxsw>     - azure: avoid re-running cloud-init when instance-id is byte-swapped
[17:42] <blackboxsw>       (#84) [AOhassan]
[17:42] <blackboxsw>     - fix unlocking method on FreeBSD [Igor Galić] (LP: #1854594)
[17:42] <blackboxsw>     - debian: add reference to the manpages [Joshua Powers]
[17:42] <blackboxsw>     - ds_identify: if /sys is not available use dmidecode (#42)
[17:42] <blackboxsw>       [Igor Galić] (LP: #1852442)
[17:42] <blackboxsw>     - docs: add cloud-id manpage [Joshua Powers]
[17:42] <blackboxsw>     - docs: add cloud-init-per manpage [Joshua Powers]
[17:42] <blackboxsw>     - docs: add cloud-init manpage [Joshua Powers]
[17:42] <blackboxsw>     - docs: add additional details to per-instance/once [Joshua Powers]
[17:42] <blackboxsw>     - Merge pull request #96 from fred-lefebvre/master [Joshua Powers]
[17:42] <blackboxsw>     - Update doc-requirements.txt [Joshua Powers]
[17:42] <blackboxsw>     - doc-requirements: add missing dep [Joshua Powers]
[17:42] <blackboxsw> Ok that should do it.
[17:42] <blackboxsw> maybe best to just pastebin next time
[17:43] <robjo> yup
[17:43] <blackboxsw> lots of doc changes as you can see. dropping python 2.7 automatic testing
[17:44] <blackboxsw> some additional FreeBSD enablement work landed too (thanks Goneri && meena )
[17:44] <blackboxsw> total changelog since last meeting:
[17:44] <blackboxsw> #link https://paste.ubuntu.com/p/Cwnn3SbmWQ/
[17:44] <blackboxsw> much better
[17:44] <blackboxsw> #topic In-progress Development
[17:45] <blackboxsw> We've dusted off our shoes and will get back into using our Trello board more frequently for the immediate updates for what we are currently working.
[17:45] <blackboxsw> New Year's resolution and all
[17:45] <blackboxsw> #link https://trello.com/b/hFtWKUn3/daily-cloud-init-curtin
[17:46] <blackboxsw> expect to see more cloud-init cards migrating through the lanes of the board. Expectation as well is that we'll drop the backlog and ideas lanes and keep the board a simple kanban of what is in progress, review and done
[17:47] <blackboxsw> Also note I'm going to drop the community charter lane and create bugs for each item, tagging them 'bitesize' so that quick drivebys of developers that want to contribute can search bugs for those straightforward tasks
[17:47] <blackboxsw> that said, some high level goals upstream is working:
[17:48] <blackboxsw> - cloud-init one-shot daemon work
[17:48] <blackboxsw> - cloud-init network hotplug handling
[17:48] <blackboxsw>  - boot performance  improvements
[17:49] <blackboxsw> - github automation and tooling improvements for expedited reviews and process
[17:50] <blackboxsw> I think that plus reviewing the PR active review queues will keep folks busy for the next 2 weeks :)
[17:51] <blackboxsw> we will likely be adding a cloud-init SRU into xenial, bionic, disco, eoan into the mix as well
[17:51] <blackboxsw> #topic Community Charter
[17:52] <blackboxsw> So generally I'd be pointing to the trello lane "Community low hanging fruit" but I hope to convert those cards to bugs today. So let's say community ongoing efforts fall into two camps"
[17:53] <blackboxsw> 1. add json schema validation to missing cloudinit/config/cc_*py modules.  ( I think there are about 45 remaining modules that need json schema for syntax validation)
[17:53] <blackboxsw> 2. doc scrub and update for datasources in read the docs
[17:54] <blackboxsw> All of these items can easily be worked in parallel, which is why they are a good set of tasks for the greater community
[17:55] <blackboxsw> Expect to find them by searching cloud-init bugs for bitesize tag
[17:55] <robjo> With bugs remaining in launchpad, would it be a good idea to have things like the schema validation not as bugs but issues in GitHub?
[17:55] <robjo> that would make them more visible IMHO
[17:55] <robjo> and those are not really bugs nor is it pressing
[17:56] <blackboxsw> #link https://bugs.launchpad.net/cloud-init/?field.tag=bitesize
[17:57] <blackboxsw> robjo: good suggestion.  I think we were trying to avoid the confusion of having two places for bugs (launchpad bugs and github issues) That is a good point though, and maybe it's worth a mailing list discussion to get others to weight in.
[17:58] <Odd_Bloke> I would be -1 on enabling issues, we would spend our entire lives telling people to report in Launchpad instead.
[17:59] <Odd_Bloke> I totally understand wanting to separate "bugs" and "development tasks", though.
[18:00] <Odd_Bloke> But I don't think we have a _great_ way of doing that which doesn't end up with a confusing experience for bug reporters.
[18:00] <robjo> True that people will equate issues in GitHub with bugs and thus file problems there rather than launchpad, it's a two edged sword
[18:00] <blackboxsw> right, I think designation is there. We could also add a link to community charter bugs to the top-level README.md for the github project. Just so there is a close breadcrumb in github to get to those items
[18:01] <Odd_Bloke> Our plan is to assess how this is working in a month or two, so if it's not working well then we can figure something else out.
[18:01] <blackboxsw> I think the designation of "community development tasks" is there by using bitesize tag or some equivalent
[18:02] <blackboxsw> #ACTION bbsw seed initial community charter bitesize bugs
[18:02] <meetingology> ACTION: bbsw seed initial community charter bitesize bugs
[18:02] <blackboxsw> #topic Office Hours (next ~30 mins)
[18:02] <robjo> Well, "community development tasks" is a bit mis-leading, after all the core team should be part of the "community" right?
[18:03] <robjo> So everything is really a  "community development tasks", just that some things are easier than others ;)
[18:03] <blackboxsw> robjo: yes absolutely. right... I've seen some projects use 'goodfirstbug' or something like that  too
[18:04] <blackboxsw> just something to reduce the barrier to involvement for anyone wanting to contribute
[18:05] <blackboxsw> and yes, core team should be accountable to work on some of those community charter tasks when time permits
[18:05] <robjo> Yes, I think it is important to label the "easy" stuff to help people find a place to get started
[18:05] <blackboxsw> so that hopefully next cloud-init summit we can set a charter for something else
[18:06] <robjo> just based on experience there are a lot of people that are sensitive to wording and we don't really want to get into the bikeshedding that comes along with such situations
[18:07] <blackboxsw> for those reading, office hours is a time of open and unstructured discussion. core cloud-init devs will have eyes on the channel to field questions, concerns, feature or bug discussions. Participate at will. In the absence of any ongoing discussions, upstream will groom/review the active review queue @  https://git.io/JeVed |
[18:09] <Odd_Bloke> Honestly losing my mind over this bug: https://bugs.launchpad.net/cloud-init/+bug/1858615
[18:09] <Odd_Bloke> The board reboots if you use dmidecode!
[18:10] <Odd_Bloke> smoser: As you said, that's a regression.  Do you think it follows that the fix should be in cloud-init?
[18:10] <Odd_Bloke> Because I don't know how you deal with something that broken from where we are in the stack. :/
[18:10] <Odd_Bloke> (Unless we think this is enough evidence that we can't reliably use dmidecode on aarch64, then I guess it is on us to stop doing that. :( )
[18:10] <robjo> This was probably in the e-mail by rharper I have not yet read, but I'll ask anyway ;)
[18:11] <robjo> I think I had some pending merge proposals in launchpad and patches, did these "magically" make their way into GitHub? DO I need to sort out where hings were?
[18:11] <smoser> i've heard "board reboots if you use dmidecode" before.
[18:12] <smoser> and maybe even cloud-init skipped calling dmidecode on aarch64 to avoid that.
[18:12] <smoser> but that is sheer non-sense
[18:12] <Odd_Bloke> Very glad that boards like this are going to be in the walls of every building in 5 years. ;)
[18:12] <smoser> umm..... fix your hardware ?
[18:13] <blackboxsw> other dmidecode issues on other hardware here too https://bugs.launchpad.net/qemu/+bug/1243287
[18:13] <smoser> its more forgivable because dmidecode is priviledged but i swear that all it does is *read* /dev/mem
[18:16] <blackboxsw> robjo: for your pending merge proposals we'd like to see you propose against github if possible. Looking for a run of ./tools/migrate-lp-user-to-github robjo <your_GITHUB_USERNAME> to get your github user included as a CLA signer
[18:17] <blackboxsw> then we have Conributor License Agreement accountability and can start merging those branches on the github side
[18:17] <robjo> Yesh I haven't migrated to the GitHub repo.... even in 2020 the 24 hour/day limitation remains, darn it ;)
[18:17] <robjo> I'll get at least my migration to GitHub done this week, possibly even this afternoon
[18:18] <blackboxsw> heh, absolutely, and actually I mistyped your migrate cmd:  ./tools/migrate-lp-user-to-github rjschwei <YOUR_GITHUB_USERNAME>
[18:39] <blackboxsw> ok think that about wraps the meeting for today. Happy new year folks! Thanks for dropping in!
[18:39] <blackboxsw> #endmeeting
[18:39] <meetingology> Meeting ended Tue Jan  7 18:39:55 2020 UTC.
[18:39] <meetingology> Minutes:        http://ubottu.com/meetingology/logs/cloud-init/2020/cloud-init.2020-01-07-17.30.moin.txt
[19:43] <meena> smoser: didn't we have an actual dmidecode fix that would be a better candidate for causing this? given how much care we have taken that my code never gets executed on Linux
[19:46] <smoser> meena: yeah, i incorrectly blamed you i think
[19:46] <smoser> b
[19:47] <smoser> but on such a system ds-identify would probably run dmidecode
[19:47] <meena> does that System have /sys/class?
[19:47] <meena> then, no
[19:48] <meena> well, rather: then, no, it shouldn't
[19:50] <meena>         # if `/sys/class/dmi/id` exists, but not the object we're looking for,
[19:50] <meena>         # do *not* fallback to dmidecode!
[19:50] <meena>         return
[19:56] <meena> we should ask the person if that path exists
[20:45] <anarcat> hello!
[20:45] <Odd_Bloke> Hi!
[20:46] <anarcat> i'm trying to figure out if cloud-init is a good fit for improving our install processes here
[20:47] <anarcat> we have an heterogenous environment, with machines distributed across "cloud" (Hetzner) and baremetal hosts, in different locations, usually without local network authority/trust
[20:48] <anarcat> we have a common trunk of commands we run (by hand!!) on each machine when we set it up, and a set of hosting-specific setup procedures, of course
[20:48] <anarcat> i'm thinking the common set might be done better by cloud-init than trying to write my own installer
[20:48] <anarcat> i also looked at FAI, to give you an idea
[20:48] <anarcat> and i'm wondering if I shouldn't just write Ansible stuff instead
[20:49] <Odd_Bloke> How do the bare metal hosts get provisioned?
[20:50] <blackboxsw> Is this fai? https://fai-project.org/
[20:50] <anarcat> blackboxsw: yes
[20:50] <anarcat> Odd_Bloke: blood, sweat and tears
[20:50] <anarcat> it depends.
[20:51] <anarcat> on Hetzner "robot" (bare metal) it's https://help.torproject.org/tsa/howto/new-machine-hetzner-robot/
[20:51] <anarcat> it's horrendous
[20:51] <anarcat> cloud is https://help.torproject.org/tsa/howto/new-machine-hetzner-cloud/
[20:51] <anarcat> and the common trunk is in https://help.torproject.org/tsa/howto/new-machine/
[20:51] <anarcat> my MVP is to replace the common trunk
[20:53] <Odd_Bloke> anarcat: Are you looking to configure Puppet as part of this, or remove Puppet entirely?
[20:53] <anarcat> Odd_Bloke: we keep puppet
[20:54] <blackboxsw> most of your https://help.torproject.org/tsa/howto/new-machine/  look like they could be replaced with existing cloud-config modules (puppet support, hostname setting package installation etc
[20:54] <Odd_Bloke> OK, cool.
[20:54] <Odd_Bloke> I figured, but just in case.
[20:54] <anarcat> blackboxsw: yeah, that's what i gathered as well
[20:54] <anarcat> blackboxsw: how about the partitionning stuff in https://help.torproject.org/tsa/howto/new-machine-hetzner-robot/ ?
[20:54] <anarcat> does cloud-init support LUKS and raid?
[20:54] <anarcat> and lvm? and btrfs and zfs and weirdofs? :)
[20:55] <anarcat> the only compromise i'd be ready to do on puppet would be to run ansible to bootstrap puppet
[20:55] <anarcat> we have too much puppet stuff written to switch away
[20:55] <meena> Odd_Bloke: https://github.com/canonical/cloud-init/pull/143 merge merge merge
[20:55] <Odd_Bloke> Yeah, I totally understand!
[20:56] <blackboxsw> anarcat: some basic disk formatting and config https://cloudinit.readthedocs.io/en/latest/topics/modules.html#disk-setup
[20:56] <Odd_Bloke> meena: done done done
[20:57] <meena> anarcat: if luks had documentation, we could consider supporting it
[20:57] <blackboxsw> With MAAS maas.io the curtin project also handles bare metal disk formatting and provisioning  and hands the configured machine over to cloud-init
[20:57] <blackboxsw> anarcat: ^
[20:57] <Odd_Bloke> MAAS isn't really practical for Hetzner though?
[20:58] <blackboxsw> good pt.
[20:58] <anarcat> blackboxsw: this doesn't seem to say which filesystem types are supported...
[20:59] <anarcat> i must say i find the documentation a bit confusing :/
[20:59] <anarcat> meena: luks is definitely documented... i am not sure what you mean
[20:59] <anarcat> i mean its documentation isn't great either
[20:59] <anarcat> but it's documented
[21:00] <Odd_Bloke> So I don't know specifics of LUKS and raid off the top of my head.
[21:00] <anarcat> maas looks something like FAI that assume you control DHCP and have a PXE server, not our case
[21:00] <anarcat> right
[21:00] <anarcat> cloud-init is, after all, ... er... cloud :p
[21:00] <anarcat> RAID is abstracted away
[21:00] <meena> https://dev.glitch.social/@hirojin/103382178381991566
[21:01] <anarcat> meena: heh
[21:01] <anarcat> meena: then i think you meant "good" documentation :p
[21:01] <blackboxsw> anarcat:  here are a couple of other contextual examples for disk setup filesystem type etc: https://cloudinit.readthedocs.io/en/latest/topics/examples.html#create-partitions-and-filesystems  and  https://cloudinit.readthedocs.io/en/latest/topics/examples.html#create-partitions-and-filesystems
[21:01] <anarcat> most documentation sucks, to be honest :p
[21:01] <meena> anarcat: does that answer your question
[21:01] <Odd_Bloke> But it has fairly decent support for multiple filesystems, and partitioning
[21:02] <anarcat> Odd_Bloke: yeah, i saw those thanks
[21:02] <anarcat> meena: i guess it does
[21:02] <meena> apparently not. go read some openbsd documentation
[21:03] <anarcat> meena: sorry, say again?
[21:03] <anarcat> meena: did you just RTFM me?
[21:03]  * anarcat confused
[21:04] <meena> also, luks vs luks2 feels like a php function bad, use this new php function
[21:04] <meena> anarcat: no, i meant just for comparison
[21:04] <meena> openbsd docs are really good
[21:04] <anarcat> from here on, you can assume i have read other documentation than the cryptsetup manpage
[21:05] <anarcat> meena: actually, i find the FreeBSD docs much better than openbsd :p
[21:05] <anarcat> but i agree openbsd has good docs :)
[21:05] <Odd_Bloke> anarcat: So I think cloud-init can do a chunk of what you want natively.
[21:05] <anarcat> Odd_Bloke: right :)
[21:05] <anarcat> it doesn't seem reasonable to expect it to do funky disk partitionning though
[21:05] <Odd_Bloke> And when it comes down to it, you can pass shell scripts as additional configuration.
[21:05] <anarcat> right
[21:06] <Odd_Bloke> So hopefully it lets you structure some of the stuff, and only have to have scripting for the Complicated Bits.
[21:06] <anarcat> maybe i could reuse FAI's setup-storage thing to do my funky stuff
[21:06] <anarcat> i created those monstrosities already https://gitweb.torproject.org/admin/tsa-misc.git/tree/installer
[21:06] <anarcat> so maybe those could be hooked up
[21:07] <meena> anarcat: yeah, like, you can read man make in freebsd top to bottom and actually understand it, and you can read gnu's make manual, and not understand anything
[21:07] <anarcat> bsd make is simpler than gnu make, though
[21:07] <anarcat> but yeah, the info system might have been a mistake
[21:08] <Odd_Bloke> Oh, I just recite the four software freedoms three times while thinking about a GNU make option, and the help text for it just enters my head.
[21:09] <anarcat> Odd_Bloke: that's surprisingly disturbing
[21:09] <Odd_Bloke> Is that not how it works for everyone else?
[21:09] <anarcat> no
[21:09] <anarcat> i start by using make
[21:09] <anarcat> then end up writing a shell script in make
[21:09] <anarcat> then realize what the fuck am i doing
[21:09] <anarcat> and rewrite it in python
[21:09] <meena> i have never once successfully read anything in info
[21:09] <Odd_Bloke> i always yell about variable assignment for a while
[21:09] <anarcat> god that's insane
[21:09] <Odd_Bloke> definitely some pacing up and down
[21:10] <anarcat> oh, i have another actual question :)
[21:10] <anarcat> so it seems cloud-init is designed to be automatically hooked up in cloud images
[21:11] <meena> i used to use make before i knew puppet, cuz it's idempotent!
[21:11] <anarcat> so it's kind of pre-installed or something... e.g. on the youtube video in the frontpage, you just copy-paste the config in a text field when you create an amazon instance
[21:11] <Odd_Bloke> Yep, that's how it's expected to be installed.
[21:11] <anarcat> but i'm a weirdo that hate clouds and love the sun
[21:11] <anarcat> so i'm bare bones
[21:11] <anarcat> if i'm lucky, i have a decent debian install to start from
[21:11] <anarcat> how does that work from there?
[21:11] <anarcat> apt install cloud-init?
[21:11] <anarcat> then call cloud-init do-magic?
[21:13] <Odd_Bloke> That's a good question.  We would _generally_ expect cloud-init to be included in the image mastering (or customisation) process, and cloud-init is generally invoked by systemd units in a few phases.
[21:13] <Odd_Bloke> I'm not 100% sure what happens when you install it on a system without it, because there are no such Ubuntu systems and I work on Ubuntu. :p
[21:14] <anarcat> uh!
[21:14] <Odd_Bloke> But I see no reason why it wouldn't work, off the top of my head.
[21:14] <anarcat> so an Ubuntu desktop comes with cloud-init pre-installed?
[21:14] <Odd_Bloke> Oh no, I meant servers, sorry.
[21:14] <blackboxsw> anarcat: ubuntu cloud-images all come with cloud-init installed.
[21:15] <anarcat> how do you pass the config in that context? https://cloudinit.readthedocs.io/en/latest/topics/datasources/nocloud.html ?
[21:15] <anarcat> blackboxsw: yeah i figured as much, but i meant more generically
[21:15] <anarcat> assume that i'm building a cloud
[21:15] <anarcat> can cloud-init be used to setup the cloud? :)
[21:15] <anarcat> maybe i'm just picking the wrong hammer here, that would be fine too :)
[21:16] <Odd_Bloke> anarcat: So the NoCloud data source looks for a CD drive/ISO with a particular label attached to the system, and reads data off of it.
[21:16] <meena> what *is* the cloud?
[21:16]  * blackboxsw just tested in a debian stretch (9)  lxc that didn't have cloud-init apt-get install cloud-init (0.7.9 waaay old) worked fine and ran automatically on reboot
[21:16] <anarcat> meena: other people's computers
[21:16] <Odd_Bloke> That data is the stuff that on, for example, EC2 would be fetched from the metadata service.
[21:17] <anarcat> Odd_Bloke: a *CD* drive?
[21:17] <anarcat> like the plastic coasters you put your beer on? :)
[21:17] <Odd_Bloke> A virtual one!
[21:17] <anarcat> yuck!
[21:17] <anarcat> it must be all sticky! ;)
[21:17] <anarcat> okay, got it
[21:17] <Odd_Bloke> :)
[21:17] <anarcat> can i get away with not doing that? :)
[21:17] <Odd_Bloke> Things like hostname, network configuration, and the user-data/custom data that you would type into a box.
[21:17] <anarcat> like fetch over https://i-trust-this-domain-because-i-love-dnssec.com?
[21:18] <Odd_Bloke> That's a good question, I believe you can.
[21:19] <anarcat> is that the "nocloud" data source?
[21:19] <meena> https://github.com/canonical/cloud-init/pull/144 merge merge merge
[21:19] <Odd_Bloke> That would be the nocloud-net data source.
[21:19] <rharper> No cloud also reads from /var/lib/cloud/seed/nocloud-net/{user-data,meta-data,network-config}
[21:19] <rharper> or from any filesystem with the label 'cidata'
[21:22] <Odd_Bloke> anarcat: So if you're going to be installing cloud-init yourself, you could consider having a simple script which populates /var/lib/cloud/seed/nocloud-net/... from https://its-not-paranoia-if-theyre-really-out-to-get.you/ before cloud-init runs.
[21:23] <blackboxsw> robjo: https://github.com/canonical/cloud-init/pull/158 will land when CI passes thanks
[21:23] <Odd_Bloke> Or you could write the appropriate configuration to /etc/cloud/cloud.cfg.d/... to point at your server.
[21:24] <anarcat> i see
[21:24] <anarcat> you can hook datasources in /etc/cloud?
[21:25] <blackboxsw> anarcat: datasources can be set with something like this:
[21:25] <blackboxsw> root@dev-x:~# cat /etc/cloud/cloud.cfg.d/90_dpkg.cfg
[21:25] <blackboxsw> # to update this file, run dpkg-reconfigure cloud-init
[21:25] <blackboxsw> datasource_list: [ ConfigDrive, NoCloud, OpenNebula, DigitalOcean, Azure, AltCloud, OVF, MAAS, GCE, OpenStack, CloudSigma, SmartOS, Bigstep, Scaleway, AliYun, Ec2, CloudStack, Exoscale, None ]
[21:25] <Odd_Bloke> anarcat: Yeah, so you would just hard-code the specific datasource you want, and the config to kick it off.
[21:25] <blackboxsw> if your image provides just one datasource type in the list, cloud-init will default to use that datasource. otherwise ds-identify script will try detecting the datasource
[21:26] <Odd_Bloke> That long list is intended for images which will run across multiple platforms; cloud-init performs local-only discovery to find which of those data sources are applicable, and then uses that one.
[21:27] <anarcat> okay
[21:27] <anarcat> i don't quite understand the details, but it seems i'll be able to do the magic i need
[21:27] <anarcat> do i need to reboot for cloud-init to do its stuff, or can i run it interactively?
[21:28] <Odd_Bloke> I expect you can run it interactively (and depending on what the units do on installation, may have to do something to stop it happening :p), but a reboot might give you a more consistent experience with other cloud-init users.
[21:30] <blackboxsw> anarcat: right and you can optionally provide other specific datasource config params with a /etc/cloud/cloud.cfg.d file that looks like  https://paste.ubuntu.com/p/kD4SgnMm7J/
[21:31] <blackboxsw> I *think* though I haven't played with overriding NoCloud config in a while.(just looked over the ds_cfg reading in cloudinit/sources/DataSourceNoCloud.py
[21:33] <blackboxsw> and yes you *could* run it directly without reboot during development with sudo cloud-init init --local; cloud-init init; sudo cloud-init modules --mode=config; sudo cloud-init modules --mode=final;
[21:34] <anarcat> how do you remember all that stuff :)
[21:34] <anarcat> thanks!
[21:34] <blackboxsw> it's what the systemd units/jobs do for each cloud-init boot stage per https://cloudinit.readthedocs.io/en/latest/topics/boot.html
[21:34] <blackboxsw> anarcat: I don't remember it. I fgrep -r cloud-init /lib/systemd/ | grep Exec
[21:35] <blackboxsw> that told me the individual cloud-init commands that are run by systemd during boot :)
[21:35] <blackboxsw> fgrep -ir to save swap space in my head
[21:35] <anarcat> aweome
[21:35] <anarcat> you guys rock
[21:37] <Odd_Bloke> rharper: I'd be interested to hear your thoughts on the LUKS/RAID storage configuration anarcat discussed above.
[21:37] <blackboxsw> btw anarcat if you are creating your own debian image, I'd recommend building/getting a more recent version of cloud-init than 0.7.9 it's about 4 years old.
[21:38] <blackboxsw> ubuntu 16.04 (Xenial) and later is currently running cloud-inig 19.3 (released late in 2019)
[21:39] <anarcat> versions in debian: https://tracker.debian.org/pkg/cloud-init
[21:39] <blackboxsw> buster & bullseye look good
[21:39] <anarcat> blackboxsw: yeah, don't worry, we're on stable and up for new machines, so 18.3 at least
[21:39] <blackboxsw> +1
[21:39] <anarcat> i'm trying to avoid building images, because i don't always control the initial setup
[21:39] <anarcat> i am trying to avoid building an installer
[21:39] <anarcat> building images mean writing an installer
[21:40] <anarcat> building an image is like installing debian
[21:40] <anarcat> i wish we would just converge
[21:41] <rharper> Odd_Bloke: right, anarcat : there is limited support in cloud-init for filesystem creation and partitioning;  for advanced storage; I would point toward  curtin, https://curtin.readthedocs.io/en/latest/index.html ; which has extensive support for creating customizing storage, include, bcache, raids, luks, zfs, filesystems, lvm, etc;
[21:42] <anarcat> curtin, intersting thanks
[21:43] <anarcat> not in debian though
[21:43] <rharper> it's an installer, and under the MAAS product, it runs curtin from cloud-init to install ubuntu or other OSes to physical (or virtual) systems;  its all command line based, so it can be incorporated into whatever workflow you have;  it's ubuntu centric, so on the debian path, there are some known issues where it doesn't quite match debian (like kernel package names and othee defaults)
[21:44] <rharper> no
[21:46] <anarcat> so MAAS does the basic PXE bootstrap, then calls cloud-init which calls curt?
[21:46] <blackboxsw> MAAS does PXE bootstrap to call curtin which pre-provisions, then calls cloud-init
[21:48] <anarcat> oic
[21:55] <rharper> blackboxsw: not quite, PXE boot into a live image, which runs cloud-init, pulling cloud-config from MAAS, which invokes curtin with config provided by maas;
[21:57] <rharper> MAAS boots the cloud image ephemerally; and uses that environment to run curtin to deploy any target OS to the platform, curtin customizes the storage layout, unpacks the payload, then runs a "post-install" we call curthooks to enable additional packages, configurations, bootloader etc;
[21:58] <anarcat> such a mess
[22:00] <rharper> bare metal provisioning is overkill for customizing images; for sure.
[22:01] <rharper> that said, re-using curtin to customize storage may be helpful depending on how much you want to do youself vs generating config for curtin to consume;
[22:08] <meena> can we get cloud-init or something else sensible, like pxe for installing android phones?
[22:11] <blackboxsw> harper: doesn't curtin's post-install step pass reporting #cloud-config to properly configure the node to talk to maas etc. Or is that earlier?
[22:12]  * blackboxsw was thinkimg curthooks: handle_cloudconfig function
[22:13] <blackboxsw> but maybe that is only for during ephemeral provisioning  process as you stated
[22:15] <rharper> blackboxsw: right, so we configure the baremetal target machine to use cloud-init as well, it will contact maas for further configuration; this first boot on baremetal is just like a cloud instance; first time booting, finds  a datasource, fetches config etc
[22:15] <rharper> we emit cloud-config to set the datasource to the MAAS controller with creds to talk to MAAS, etc


[07:43] <apw> SturmFlut, i am sure someone was somewhere in the world
[07:44] <sturmflut> Haha
[07:51] <sturmflut> I've debugged most of the problem and filed a bug report in the meantime, looks like the original problem is in ModemManager, but might affect the kernel as well in the end.
[07:53] <ogra_> sturmflut, but not a kernel anyone in here maintains :)
[07:55] <sturmflut> ogra_: Shouldn't the cdc_acm module of my desktop kernel blacklist that specific USB ID, because it will never work anyways and just confuse the russians?
[07:55] <ogra_> dont underestimate the cleverness of the russians :)
[07:56] <ogra_> and i'm not sure its a kernel thing ... did you check that modemmanager doesnt ship udev rules that cause the loading ?
[07:57] <sturmflut> Hmm, now how does this udev stuff work with systemd...
[07:58] <ogra_> heh, te same as with other init systems :)
[07:58] <ogra_> (i dont think systemd comes into play here ... but i guess modemmanager ships udev rules that react to uevents related to serial)
[08:08] <sturmflut> ModemManager ships some rules, not to load the module for this device, but to tell itself to blacklist a lot of similar cases. So the simple solution would be to at least add the device there, then cdc_acm gets still loaded and fails, but ModemManager does no longer claim the device.
[08:08] <sturmflut> Not pretty, but I can live with that.
[12:41] <rtg> apw, ogasawara_ upstream discussion on CONFIG_VM86 http://thread.gmane.org/gmane.linux.kernel/1991020
[12:42] <rtg> kees suggests we turn it off
[12:48] <apw> rtg, yeah someone claims we have it off already, but it looks to be on to me
[12:48] <rtg> yup, their claim was erroneous
[12:48] <apw> rtg, it sounds like a security disaster in the making, so ripping it in wily and seeing if anything explodes seems good
[12:48] <ogasawara_> rtg: ack, do you already have a patch to push to wily to turn it off?
[12:48] <rtg> ogasawara_, I was just disabling it in unstable
[12:49] <rtg> ogasawara_, I can do wily master-next as well
[12:49] <ogasawara_> rtg: ack
[12:50] <infinity> Didn't we just turn that on in an SRU a few months back or something?
[12:50] <infinity> It seems awfully familiar.
[12:53] <infinity> Maybe I'm thinking of something else with a similar-sounding name.
[12:55] <rtg> infinity, not finding any CONFIG_VM86 changes in trusty
[12:55] <infinity> rtg: Ahh, yeah, I was thinking of CONFIG_X86_16BIT, nevermind.
[12:56] <infinity> So, VM86 seems scary and evil as all hell, I'm inclined to agree, but we need to make sure we don't need it for any old skool X drivers (vmware, vbox, etc)
[12:58] <infinity> Oh, but if it doesn't even work on x86_64, my carefactor just went down.
[12:58]  * infinity catches up on the rest of the thread.
[12:59] <rtg> infinity, it was 32 bit only in our configs
[12:59] <infinity> rtg: Yeah.  I'd be inclined to say we should disable it across all stable kernels too, then, as a pre-emptive security feature.
[13:00] <infinity> rtg: If no one was hunting for holes in it before, you can bet that thread will make grey hats start looking.
[13:01] <infinity> rtg: There are one or two X drivers that are i386-only.  If we can verify that they don't need VM86, I'd say the impact is minimal-to-none and worth the risk.
[13:01] <rtg> infinity, agreed, though the attack surface is much smaller given that it is only applicable to 32 bit machines.
[13:02] <rtg> infinity, I'll run this by jjohansen and tyhicks as well
[13:03] <rtg> perhaps 32 bit cloud VMs are vulnerable
[13:04] <infinity> rtg: 32-bit still has an astoundingly large install base.
[13:05] <infinity> rtg: And certainly true going back to precise, where it was still our recommended image download.
[13:05] <rtg> yeah, I'd be willing to bet there are a lot of 32 bit instances in the cloud
[13:06] <infinity> rtg: Probably fewer than there should be (ie: I'd argue that most "tiny" and "small" instances should be i386, even if backed by amd64 hardware), but yeah, there are certainly some.
[13:07] <rtg> infinity, working on an SRU
[13:08] <infinity> rtg: I think we need a userspace impact analysis before we SRU it, but my hope is the impact is "none".
[13:09] <infinity> rtg: My only concern is ancient X drivers that haven't seen significant work in the last decade, and I think we only ship a couple of those.
[13:10] <rtg> infinity, it seems our choice is to orphan some vanishingly rare drivers, or expose possible vulnerabilities to every 32 bit user.
[13:11] <rtg> orphan -> regress
[13:11] <infinity> rtg: Hrm.  And vbetool.  That might need a looking at as well.
[13:11] <rtg> I'll make that part of the SRU discussion
[13:11] <infinity> rtg: But yes, we're on the same page.  I think the goal should be disabling VM86, we should just sort out the impact and see if we can minimize damage.
[13:14] <infinity> rtg: http://codesearch.debian.net/results/sys%2Fvm86.h
[13:15] <infinity> rtg: and http://codesearch.debian.net/results/asm%2Fvm86.h
[13:16] <infinity> rtg: Some might be false positives that we don't enable, others are probably build-time detected and would be broken if we regress the kernel support.
[13:18] <rtg> infinity, I guess Linus will howl if Lutomirski starts breaking user space
[13:20] <infinity> rtg: Well, it's never been a required kernel option, so it doesn't break userspace to disable it by default, by Linus's definition of break.
[13:21] <infinity> rtg: But distributions need to be aware of impact on already-shipped software if they disable features they used to enable.  That's where it gets sticky.
[13:21] <rtg> infinity, is userspace supposed to fall back into an emulation mode ?
[13:22] <infinity> rtg: I'd have to look at the glibc syscall wrapper.  It's possible that people using the abstracted sys/vm86.h are safe.
[13:22] <infinity> rtg: People using asm/vm86.h directly, though, are definitely broken if they don't have their own fallbacks in place.
[13:24] <infinity> rtg: I highly doubt we have an 8086 emulator in glibc though (in fact, I'm sure we don't), so in either case, if people are relying on the syscall without a fallback option, they'd explode regardless, I think.
[13:25] <rtg> infinity, would they explode at runtime or build time, or both ?
[13:25] <infinity> rtg: I'd assume just runtime, if the headers are still exposed when the option is off.
[13:26] <infinity> rtg: If the headers go away completely, then build time, but given the number of things that include the headers, I suspect that's not intended to happen.
[13:26] <rtg> thank you LP for blowing up and _not_ accepting my bug report.
[13:27] <infinity> rtg: LP is suffering some major infra mojo right now.
[13:27] <rtg> infinity, bug #1473447
[13:28] <rtg> infinity, we've got it disabled in Wily (4.1+), so we can experiment with the fallout
[13:29] <infinity> rtg: Yeahp, that seems like the safe approach for now.  We can abuse wily userspace a bit, cross-referencing with codesearch (which should have 100% coverage for what we care about, I'd be VERY shocked if we ship anything Ubuntu-only that uses vm86), and then sort out conclusions.
[13:31] <infinity> rtg: codesearch's only failure here is that it's only search unstable, so doesn't give us a clear view of the possibility that precise might be more broken than trusty or wily.
[13:31] <infinity> rtg: But it's a start anyway.
[13:32] <infinity> I really think we need to set up an Ubuntu codesearch instance that covers all supported series'... It's crazy useful.
[13:32] <rtg> infinity, seems like we should certainly have it disabled by 16.04, so experimenting on 15.10 makes sense
[13:33] <infinity> rtg: Yeahp, and in 15.10/16.04, I have zero issues with the idea that if some bits of userspace break without it, we can just deprecate/remove them if they're not fixable.
[13:33] <infinity> rtg: But that's a harder pill to swallow for precise and trusty.
[13:33] <rtg> agreed
[13:48]  * cking wonders why the watchdogs are going nuts: https://pastebin.canonical.com/135040/
[14:02] <infinity> cking: That's... Impressive.
[14:02] <infinity> cking: I've never seen a kernel try so hard to starve userspace.
[14:04] <cking> infinity, it did have some repercussions: https://pastebin.canonical.com/135045/
[14:10] <pointertonullval> Hey guys I'm building my own embedded device and it will connect over usb to a computer. How I can start writing a linux driver? Is a kernel level problem or more high level? Thanks
[14:11] <apw> usb is interesting because there is a userspace library interface iirc, so you can write things out of the kernel
[14:12] <TJ-> pointertonullval: Look at the USB 'gadget' interface
[14:15] <pointertonullval> apw, TJ- thanks! something like this? http://www.linux-usb.org/gadget/
[14:34] <unixabg> apw: it appears that casper does not like the multiple squashfs files on boot. But i am fairly
[14:35] <unixabg> confident that is sees them since it does make the mount points and then fails (I think on
[14:35] <unixabg> getting the overlay right).
[14:39] <apw> it is possible that casper is trying to lay down multiple layers of overlayfs and not using the multiple layer support
[14:39] <apw> for which there likely is no way to figure out if the functionality exists in the kernel ... hmm
[14:41] <infinity> apw: Does casper have support for multiple layers at all?  That would seem odd, if it's a new kernel feature.
[14:43] <apw> infinity, it has some support in the sense they are layerable, ie mount the bottom two together, then mount the results with the next layer up
[14:43] <apw> though you soon hit the stack depth limit in the kernel, so i think we bust on overlayfs at two mount layers, ie 3 layers
[14:43] <apw> unixabg, how many layers are there ?
[14:45] <infinity> apw: Anyhow, sorting out if functionality exists should be a non-issue, casper isn't the sort of tool one uses "blindly".  It should just do what you ask it to do, and if you're using a kernel that can't support that, you get to keep both pieces.
[14:46] <apw> thoug in a sense we have some way to do it two ways with differnet limits in each case
[14:47] <apw> and the second way only if you have a newer kernel, which doesn't tell you it can do it, though i guess for casper at least we kinda know what its built with
[14:48] <infinity> apw: If it's a question of mount options changing to support way 1 and way 2, and there's a distinct upstream version cutoff to go from 1 to 2, you could case uname -r.
[14:49] <infinity> Or, not case, but convert-and-compare, probably.
[14:49] <unixabg> apw: on my test with wily it fails by just adding a second layer. It may be helpful if I outline
[14:49] <unixabg> how I am testing, I basically am doing a quick remaster and I am generating my partial squashfs
[14:50] <apw> infinity, ugg you are not putting me in my happy place
[14:50] <unixabg> files with live-partial-squashfs-updates:
[14:50] <unixabg> http://live.debian.net/gitweb/?p=live-tools.git;a=blob_plain;f=bin/live-partial-squashfs-updates;hb=refs/heads/debian-next
[14:51] <infinity> apw: Do I ever?
[14:51] <unixabg> once I generate a partial-squashfs-update (psu) against the filesystem.squashfs then I button
[14:51] <apw> infinity, those days you offer to take me to the pub, yes
[14:51] <unixabg> the image back together and boot it. But it is not working. I did test one layer was working on
[14:52] <unixabg> linuxmin17.1 I think, mom to check that,
[14:53] <unixabg> yes that is correct, but any more than filesystem.squashfs and psu1.squashfs caused failure.
[14:54] <unixabg> Let me run one more quick test, since late yesterday and I had many tests going I do believe that now
[14:54] <unixabg> I can not even stack on one psu.squashfs. Be back in a short bit...
[14:58] <unixabg> apw: ok wily failed when I tried to add only one psu.squashfs along with filesystem.squashfs, next testing
[14:58] <unixabg> linuxmint17.2
[15:07] <apw> unixabg, can't look at it right now, but it would be good to get a dmesg if you can to see if it is throwing a depth limit in there somewhere
[15:07] <apw> report any testing here and i'll see if i can figure it out
[15:13] <unixabg> apw: humble appreciation. Ok now I just booted linuxmint17.2 + 1psu.squashfs and it is ok.
[15:14] <unixabg> s/and/and remastered and booted live,/g
[15:16] <apw> unixabg, whihc kernel does that have in it
[15:18] <unixabg> apw: uname -r produces 3.19.0-20-generic
[15:20] <unixabg> apw: dmesg in an image: http://fyeox.com/downloads/20150710-loop1fail-wily.png
[15:22] <unixabg> It could be that it does not have the correct kernel version or something. Anyhow I am willing to
[15:22] <unixabg> assist however I can. I want the multiple squashfs to work well.
[15:44] <brainwash> cking: hey, I noticed bug 1470845 too. should this be forwarded somewhere upstream?
[15:45] <brainwash> installed forkstat some minutes ago
[15:45] <cking> brainwash, I was hoping it would get some triaging and attention on the ubuntu side of things, it maybe the way it's configured at the moment and is a temporary issue
[15:48] <brainwash> cking: it? what could be misconfigured?
[15:48] <cking> brainwash, i don't know, that's why I filed the bug
[15:49] <brainwash> ok :) sadly, I only have access to ubuntu wily right now
[15:49] <brainwash> and I've never used forkstat before
[15:49] <infinity> cking: Did that happen before 222?
[15:49] <cking> infinity, I don't know either
[15:50] <brainwash> 222 was pushed just recently
[15:50] <cking> it's the "oh, I've noticed something that looks suspect and I don't know why or when it happened" kinda bug
[15:50] <brainwash> yeah, I don't understand this, but it made me wonder
[18:25] <kees> (rtg offline?) apw: in the past, usplash needed VM86, but it shouldn't be hard to deal with if that still an issue
[20:26] <infinity> kees: usplash was many, many years ago. :P
[20:36] <mjg59> Yeah sorry about that
[20:38] <ogra_> lol
[20:40] <jdstrand> ogasawara: hey
[20:40] <jdstrand> $ cat /proc/version_signature
[20:40] <jdstrand> Ubuntu 4.1.0-1.1~dogfoodv1-generic 4.1.0
[20:40] <jdstrand> ogasawara: mic does not work
[20:41] <jdstrand> ogasawara: for a data point, 4.0 that is in wily with the two commits I mentioned yesterday does work
[20:42] <smoser> hey. question on https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/1473527
[20:43] <smoser> is there any reason that that should happen ? (IO error on /dev/console)
[20:43] <smoser> where cmdline: console=ttyS1,115200
[20:43] <smoser> and the ttyS1 is seemingly fine (watching it from an ipmi)
[20:44] <smoser> anyone think of some reason that such a thing would occur
[20:44] <ogasawara> jdstrand: care to file me a bug for tracking?  I think that sforshee on my team (possibly even ppisati) have your same system.  I'll see if they can reproduce.  If not, I'll have jsalisbury walk through a bisect with you when he's back on Mon.
[20:47] <jdstrand> ogasawara: ok
[20:47] <jdstrand> ogasawara: ubuntu-bug linux?
[20:47] <ogasawara> jdstrand: yes please
[20:48] <jdstrand> ogasawara: is it worth updating linux-firmware to what is in wily too?
[20:48] <ogasawara> jdstrand: I don't believe so, has it even been updated for wily?
[20:49] <jdstrand> yes
[20:53] <jdstrand> it doesn't look like there is anything in there that would help this, but there are some other things. I'll update, reboot, retry and file
[21:01] <jdstrand> ogasawara: hrmm, it said it wasn't an official Ubuntu package. how do people normally report against these ppa kernels?
[21:02] <ogasawara> jdstrand: ah shoot, we don't typically have high volume of bug reports against the kernels we push to that ppa
[21:03] <ogasawara> jdstrand: I'd say run the 4.0 kernel which was working to run ubuntu-bug linux (just to gather the system details)
[21:03] <ogasawara> jdstrand: and in the description note the regression with 4.1
[21:03] <jdstrand> actually, I see there are some files in /etc/apport for firefox. let me try a few things
[21:04] <jdstrand> well, my 4.0 would fail too I think (I recompiled it with those two patches)
[21:04] <jdstrand> but yeah, I'll figure out how to report this thing :)
[21:04] <ogasawara> oh right
[21:05] <ogasawara> jdstrand: if all else fails, manually file and I'll still make sure we get eyes on it
[21:06] <jdstrand> ok, thanks
[21:11] <jdstrand> ogasawara: heh, after reading the apport source: APPORT_DISABLE_DISTRO_CHECK=1 ubuntu-bug linux
[21:12] <jdstrand> ogasawara: it still throws up a message about how it is unofficial, but it opens the browser to create the report
[21:13] <ogasawara> jdstrand: oh nice, I'll have to remember that.  I assume it does indeed attach the standard logs to the bug?
[21:14] <jdstrand> it says it will. I'll know in a moment once I finish the description
[21:16] <jdstrand> ogasawara: looks like it: https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1473560
[21:17] <ogasawara> jdstrand: great, thanks.  I'll take it from here.
[21:18] <jdstrand> ogasawara: thanks!
[21:39] <kees> infinity: yeah, totally, but just in case there were other sneaky things like that still floating around
[21:40] <infinity> kees: As noted in the bug, codesearch.debian.net certainly shows a fair few references to vm86.h, the question is how many have fallback modes or are ifdeffed to oblivion, etc.
[21:40] <infinity> kees: So, it needs a bit of investigation.
[21:48] <kees> infinity: yeah. fwiw, dosemu is fine. ;)
[21:55] <lamont> http://paste.ubuntu.com/11858310/ <-- lazy brain question... is that trace consistent with a "DO NOT BLOCK" kmalloc call?
[21:57] <lamont> trusty kernel
[22:34]  * lamont stands answered


[01:56] <duflu> robert_ancell, could you explain to people in bug 1825544 what's happened?
[03:06] <robert_ancell> duflu, I don't know of anything that might be causing that!?
[03:07] <duflu> robert_ancell, I thought user switching was intentionally removed?
[03:07] <robert_ancell> duflu, no, that still shows if you click your name in the top-right menu.
[03:07] <duflu> Oh, guest mode. I am confused
[03:10] <duflu> robert_ancell, It's guest mode we knowingly can't support right now, right?
[03:10] <robert_ancell> duflu, correct
[03:10] <robert_ancell> the other removed option was a suspend menu entry
[03:12] <duflu> Suspend? You can still hold down the power button in the GUI right?
[03:12] <duflu> Yes, confirmed
[04:55] <didrocks> good morning
[05:17] <duflu> Morning didrocks
[05:23] <didrocks> hey duflu
[06:45] <oSoMoN> good morning desktoppers, happy Friday!
[06:48] <didrocks> hey oSoMoN
[06:49] <oSoMoN> salut didrocks
[06:49] <oSoMoN> ça va?
[06:51] <duflu> Morning oSoMoN
[06:51] <didrocks> ça va oSoMoN, et toi ?
[06:58] <Wimpress> o/
[07:05] <oSoMoN> didrocks, très bien, plutôt en forme pour un vendredi :)
[07:05] <oSoMoN> hey duflu
[07:05] <oSoMoN> morning Wimpress
[07:16] <duflu> Hi Wimpress
[07:52] <Gargoyle> Hi all. On https://wiki.ubuntu.com/DesktopTeam under the contributing section, the "Work on bitesize..." and "Help with work items.." links are both dead.
[07:56] <oSoMoN> Gargoyle, good catch, thanks!
[07:56] <oSoMoN> not sure what has replaced these things, let's see if we can find replacement links
[07:59] <oSoMoN> https://wiki.ubuntu.com/Harvest suggests harvest.ubuntu.com has been offline since 2016
[08:00] <marcustomlinson> morning all
[08:00] <oSoMoN> morning marcustomlinson
[08:01] <marcustomlinson> happy friday!
[08:02] <Gargoyle> oSoMoN: NP. I'll keep an eye out for updated links. Want to start contributing! :-)
[08:02] <Laney> booooooooooooooooooooom
[08:03] <willcooke> toot toot - morning all
[08:04] <Laney> moin willcooke
[08:04] <Laney> brb trying to find the deep heat
[08:04] <willcooke> ruh roh, that sounds bad
[08:04] <willcooke> climbing injury?
[08:05] <oSoMoN> good morning Laney, willcooke
[08:06] <willcooke> aye up oSoMoN
[08:06] <Laney> moin oSoMoN
[08:06] <marcustomlinson> morning oSoMoN Laney willcooke and Gargoyle :)
[08:07] <Laney> i went to a morris dancing practice on wednesday
[08:07] <willcooke> hi marcustomlinson, how's the frozen north today?  We're promised thunderstorms down here, but doesnt feel like it will come to much at the moment
[08:07] <Laney> and it injured me
[08:07] <Laney> hey marcustomlinson
[08:07] <willcooke> Laney, bad luck, but also, I love it
[08:07] <marcustomlinson> willcooke: nah, sunny here
[08:09] <Laney> https://www.youtube.com/watch?v=dZ-VZpuxOCI
[08:09] <Laney> this is them
[08:09] <marcustomlinson> willcooke: I think the "bad weather in Scotland" is just a conspiracy created to keep the English from moving in ;)
[08:09] <willcooke> marcustomlinson, :DD good idea
[08:09] <oSoMoN> Gargoyle, I removed the dead links for now, couldn't find any suitable replacements right away
[08:09] <willcooke> Laney, awesome!   Thanks. I've added it my auto join. Dual booting between windows, so will pop in and out! :-)
[08:11] <willcooke> welcome Gargoyle!
[08:12] <oSoMoN> cool, welcome to the desktop team's lair :)
[08:12] <didrocks> hey Laney, willcooke, marcustomlinson
[08:12] <marcustomlinson> yo didrocks
[08:13] <willcooke> hi didrocks
[08:14] <Laney> willcooke: yeah, it's quite cool, looking forward to assembling one
[08:14] <Laney> if they don't cripple me first
[08:14] <Laney> hey didrocks, happy friday to you
[08:18] <oSoMoN> I added a link to discourse on https://wiki.ubuntu.com/DesktopTeam
[08:19] <Laney> nice one
[08:19] <Laney> maybe trello?
[08:19] <Laney> if there's a useful index to actually link to
[08:19] <oSoMoN> ah, that would be useful too, indeed
[08:21] <oSoMoN> can't see an index of all desktop-related boards, we would have to link to individual boards, and update the page at the start of every cycle when we add a new board
[08:23] <Laney> sounds error prone
[08:23] <Laney> you can make collections but i can't see how you can link to one of those
[08:24] <Laney> and it might even be private anyway since we use the company account now
[08:25] <oSoMoN> there's https://trello.com/ubuntudesktop
[08:26] <Laney> ya, the one we used before starting to use the company one
[08:26] <Laney> BUT there's a link to make it join the enterprise group, whatever that is
[08:26]  * Laney clicked it
[08:27] <Laney> maybe if that's good we could move the boards back there
[08:57] <duflu> Morning marcustomlinson, willcooke, Laney, others?
[08:57] <marcustomlinson> others is on holiday today
[08:57] <marcustomlinson> :)
[08:58] <marcustomlinson> hey duflu
[09:01] <Laney> there's a Didier
[09:01] <Laney> hey duflu
[09:01] <Laney> and presumably a Till hiding away somewhere too
[09:01] <Laney> maybe even a Marco who I believe is back in Europe now
[09:02] <willcooke> afternoon duflu
[09:04] <duflu> Maybe too early if Marco is back to normal
[09:04] <didrocks> yeah, was supposed to swap today but $something_happened with the house builder
[09:07] <Laney> :(
[09:22] <marlinc> Are there known issues with the 4.15.0-51 kernel release? It can't unmount the LUKS encrypted volume on boot. Booting to 4.15.0-50 works fine
[09:26] <lis> didrocks: hi :)
[09:28] <lis> didrocks: where's your house gonna be?
[09:31] <didrocks> lis: hey! Staying new Lyon, just a little bit outside of the city
[09:32] <lis> didrocks: very nice.  building your own house is very grown-up :)
[09:39] <willcooke> marlinc, try in #ubuntu-kernel, I think you'll have more luck in there
[09:55] <Trevinho> Laney: yeah... Eeupean Marco here too
[09:56] <oSoMoN> hey there European Marco!
[09:57] <Trevinho> oSoMoN: hi Oliver!
[09:57] <oSoMoN> +i :)
[10:00] <Laney> hey Eurovinho
[10:00] <Laney> not back in your actual home though are you?
[10:11] <duflu> Is there such a thing?
[10:12] <duflu> Home is where the heart is
[10:12] <duflu> or whichever country Marco has spent a lot of time in recently
[10:13] <Trevinho> Laney: nope, in Madrid right now
[10:13] <Trevinho> duflu: home is where my thinkpad is :-D
[10:13] <duflu> Ha
[10:16] <duflu> Home is where I will be crashing for the night when I finish this MP
[13:26] <xnox> didrocks:  jibel: is it intentional to have custom livecd-rootfs in the canary ppa? and is it intentional that it is out of date with eoan? E.g. the one in the eoan archive doesn't have "add_package minimal zsys-install openssh-server"
[13:26] <xnox> didrocks:  jibel: it would be nice to merge all the canary stuff into archive livecd-rootfs, to avoid pointless merging all the time.
[13:30] <didrocks> xnox: it's a different package set to have curtin + some ppa experimental zfs packages in. We did diverge when disco was frozen (and didn't want to ask for FFe for those, as it's not going to impact). Indeed, we know that it's out of date and we will fix this in a new upload soon (after pushing most of the stuff in the archive anyway)
[13:30] <didrocks> xnox: this is orthogonal though to the breakage we discussed, I think, as the package is out of date?
[13:49] <xnox> didrocks:  it is orthogonal, just mentioning it.
[13:49] <xnox> didrocks:  if you want to keep livecd-rootfs as a delta; you could setup a ppa daily build recipy to pull in/merge lp:livecd-rootfs and do a daily build into canary ppa. that may work to keep things up to date, most of the time automatically.
[14:03] <didrocks> xnox: thx for the tip, but we'll go the easy path: uploading to distro I guess
[14:07] <xnox> i think i diagnosed this, but not sure how to best fix it
[14:14] <xnox> didrocks:  jibel: email sent
[14:32] <didrocks> xnox: thx, will read this on Tuesday (Monday is off and we are fighting with tests right now)
[16:15] <oSoMoN> okay, that's all folks for this week, have a good week-end and see you on Wednesday (I'm off Monday and Tuesday)!
[17:00] <willcooke> night all
[17:00] <willcooke> have a good (long) weekend
[18:35] <Irc2k_> Hello Everyone :)
[18:35] <Irc2k_> I had a quick question.
[18:35] <Irc2k_> IS anyone around?
[18:37] <sarnold> Irc2k_: irc works best if you just ask questions :)
[18:37] <Irc2k_> Lol Thanks
[18:38] <Irc2k_> Sarnold does Ubuntu still collect data from the computer in the background? Even when privacy is turned off?
[18:38] <mdeslaur> Irc2k_: collected data?
[18:39] <Irc2k_> Yes I just read an article on this a while back
[18:39] <Irc2k_> on OMGubuntu
[18:40] <Irc2k_> so was concerned
[18:40] <Irc2k_> I left windows for this
[18:40] <mdeslaur> could you point to the article? I'm not sure what data you're referring to
[18:40] <mdeslaur> right after install, it asks you if you want to submit some hardware information, but it's optional
[18:41] <Irc2k_> Yes even I thought like that I also watched a video on this
[18:41] <Irc2k_> one moment
[18:41] <Irc2k_> https://www.omgubuntu.co.uk/2018/02/ubuntu-data-collection-opt-out
[18:42] <mdeslaur> Irc2k_: this channel is about desktop development, this isn't really the place to ask questions. You can try #ubuntu
[18:42] <Irc2k_> Thank you
[18:42] <mdeslaur> Irc2k_: yes, it still does that.
[18:42] <mdeslaur> Irc2k_: check the "no" box if you don't want it to
[18:44] <Irc2k_> I know but in the video they mention that still in the backgroud they do collect I'm kind of concered about this and will seek help in the channel ubuntu
[18:45] <mdeslaur> if you check "yes", it will collect hardware info in the background, if you check "no", it won't
[18:45] <mdeslaur> it's as simple as that
[18:46] <Irc2k_> got it thanks


[07:28] <Parsi> hey Melissa-
[07:31] <Melissa-> Parsi: hey Parsi.how's it going?
[07:32] <Parsi> Melissa-: nothing much
[17:35] <Mehrdad12> hi