[13:23] <djszapi> Why does upstart not print the Process ID of an application launched from a script section ?
[13:23] <djszapi> What is the advantage of this way ?
[13:42] <Keybuk> it does
[13:43] <djszapi> w00t ?
[13:44] <djszapi> Script part can have tens of commands and those are not printed. Only if there is problems like this one /proc/self/fd/6: /etc/pulse/pulseaudio_pre-start.sh: line 53: amixer: not found pulseaudio start/running, process 599. /proc/self/fd is where it runs script part of the .conf files
[13:45] <Keybuk> process 599 is the script itself
[13:45] <Keybuk> and the shared process group of all of those tens of commands
[13:46] <djszapi> so it does not print then.
[13:51] <djszapi> well, what is the solution way if I need that sort of information, Keybuk ?
[13:52] <djszapi> I have just read your mail and I am still not sure the next version (that I do not know when will be released) is useful for my purpose.
[13:52] <djszapi> and for sure I do not have time to wait for that.
[13:53] <djszapi> audit kernel stuff I thought can be a hackaround for now, but let me know, I am newbie in this area.
[13:53] <Keybuk> I don't know what your purposes are
[13:53] <djszapi> well
[13:54] <Keybuk> (so it's hard to me to answer questions where you state "for my purpose")
[13:54] <djszapi> I need the configuration file name and pid of the application that was run by upstart.
[13:54] <djszapi> as it happens quite perfectly with 'exec' entries in the /etc/init/*.conf files.
[13:55] <Keybuk> but it does give you that, it gives you the pid of the script
[13:55] <Keybuk> which is all that was run by Upstart ;-)
[13:56] <djszapi> nope
[13:57] <Keybuk> what do you mean "nope"
[13:57] <Keybuk> I'm pretty sure only one of the two of us *wrote* Upstart, so knows what he's talking about :p
[13:57] <djszapi> hwclock:329,sh:328,init:1
[13:57] <djszapi> for example this is the procedure, init -> sh (upstart) -> hwclock
[13:58] <Keybuk> right, Upstart ran "sh"
[13:58] <djszapi> I need the pid of the hwclock and the configuration file name which started it.
[13:58] <Keybuk> so status in that example will give you the pid of sh
[13:58] <Keybuk> not of hwclock
[13:58] <Keybuk> ok, that's *NOT* what you said above
[13:58] <Keybuk> you said "id of the application that was run by upstart"
[13:58] <Keybuk> upstart ran sh
[13:58] <djszapi> no
[13:58] <Keybuk> what sh did is entirely up to sh
[13:58] <Keybuk> are you disputing what you typed?
[13:58] <Keybuk> because I just copy & pasted it
[13:59] <djszapi> okay, you do not still understand it :)
[13:59] <djszapi> let me give you an example:
[13:59] <Keybuk> no, I understand far more than you
[13:59] <Keybuk> you're just not explaining yourself
[13:59] <Keybuk> and probably you don't actually know what you want
[13:59] <Keybuk> and don't understand how processes work on UNIX
[14:00] <djszapi> do not insult me, please.
[14:00] <Keybuk> I believe what you want is:
[14:00] <Keybuk>   script
[14:00] <Keybuk>     foo &
[14:00] <Keybuk>     bar &
[14:00] <Keybuk>     baz &
[14:00] <Keybuk>   end script
[14:00] <djszapi> as said, WAIT
[14:00] <Keybuk> to give you the pids of foo, bar and baz ?
[14:00] <djszapi> I am gonna give you an example, patient pls...
[14:01] <djszapi> I need to boot the device up.
[14:01] <Keybuk> is that not what you want?
[14:01] <djszapi> seems you cannot be patient :)
[14:01] <Keybuk> no
[14:01] <Keybuk> I'm busy
[14:02] <Keybuk> I'm answering you rather than working
[14:02] <Keybuk> because I'm nice ;-)
[14:02] <Keybuk> but I don't have much time
[14:02] <djszapi> work now and ask when I provide info ?
[14:02] <djszapi> and to avoid the guesswork and wasted time till that ?
[14:02] <Keybuk> sure, will you hang around here for, say, 6 hours?
[14:03] <djszapi> please do not be arrogant for hours, if I may ask...
[14:03] <Keybuk> I'm not being arrogant, I am, right now, using my lunch hour to try and help you
[14:03] <Keybuk> and you're not answering my question
[14:04] <djszapi> because I still cannot ?
[14:04] <Keybuk> why?  you can answer questions?
[14:04] <djszapi> http://pastebin.com/GjeJiy7n
[14:05] <djszapi> 14th line for instance.
[14:05] <Keybuk> that's a shell script
[14:05] <djszapi> I would like to get this configuration file back and hwclock pid
[14:05] <Keybuk> hwclock has no pid at the point that script has finished
[14:05] <Keybuk> because the shell runs hwclock, waits for it to finish, and reaps it
[14:05] <djszapi> weird it has.
[14:05] <djszapi> every process has pid
[14:05] <Keybuk> tell you what
[14:05] <Keybuk> why not read what I typed?
[14:06] <Keybuk> AT THE POINT THAT SCRIPT HAS FINISHED
[14:06] <djszapi> even tho. the same sometimes than the parent process (setuid e.g.)
[14:06] <Keybuk> the shell does something like:
[14:06] <Keybuk>   fork()
[14:06] <Keybuk>     (child:
[14:06] <Keybuk>       exec hwclock)
[14:06] <Keybuk>     (parent:
[14:06] <Keybuk>       wait)
[14:06] <Keybuk>   // now at the next line of shell
[14:07] <Keybuk> so there's no backgrounding going on here
[14:07] <djszapi> k, so I cannot get this information by upstart....
[14:07] <Keybuk> the fact that hwclock has a separate pid is basically an implementation detail
[14:07] <djszapi> how can I get then ?
[14:07] <Keybuk> for example, a particularly over-the-top shell could have hwclock as a built-in
[14:07] <Keybuk> which means it would have no pid
[14:07] <Keybuk> get it when?
[14:08] <djszapi> nope
[14:08] <Keybuk> the pid (if there even is one) will appear and vanish pretty quickly
[14:08] <Keybuk> stop saying "nope" to me
[14:08] <Keybuk> I know what I'm talking about
[14:09] <djszapi> let me wait for someone else to help then...
[14:09] <Keybuk> they'll either tell you the same thing
[14:09] <Keybuk> or they'll give you incorrect information
[14:09] <djszapi> seems you are in god-mode.
[14:10] <Keybuk> no, I'm trying to help you
[14:10] <Keybuk> but you're not actually listening to anything I say
[14:10] <Keybuk> which is exasperating
[14:10] <djszapi> by screaming at me ?
[14:10] <Keybuk> I'm not screaming at you
[14:10] <Keybuk> I'm telling you how things work
[14:10] <djszapi> and screaming if I am saying sth ?
[14:10] <Keybuk> and you're refusing to believe me
[14:10] <djszapi> interesting help, never tried
[14:10] <Keybuk> which is insulting
[14:10] <Keybuk> you're wrong
[14:10] <Keybuk> do you accept that there is no need for hwclock to have a separate pid at that point?
[14:11] <Keybuk> (feel free to relink /bin/sh -> /bin/busybox and run that script, if you need physical proof)
[14:12] <djszapi> no idea whether it has got pid that point, but I need the 1) conf file name 2) PID when it ran.
[14:12] <Keybuk> you could add strace -f to the shell invocation
[14:13] <djszapi> exactly as it happens in case the exec line in the configuration files.
[14:13] <Keybuk> that way you'd trace all the system calls, and follow forks, so you'd get the pids that way
[14:13] <Keybuk> you could use the proc connector kernel interface to follow the forks/execs of the shell and again get the pids
[14:14] <Keybuk> proc connector is racey though, because the pid may vanish before you have an opportunity to read /proc/PID/cmdline and actually figure out what it is
[14:14] <Keybuk> (strace at least blocks)
[14:14] <djszapi> pid is not enough, as said also the config file where it was run frm.
[14:14] <Keybuk> audit, ftrace, etc. any of those kernel tracing hooks
[14:14] <djszapi> * from
[14:15] <djszapi> well, my original idea was the audit stuff.
[14:15] <Keybuk> you could mount a subsystem-less cgroup hierarchy and place that script into a new cgroup within it
[14:15] <Keybuk> that would collect the pids run in the tasks file
[14:15] <Keybuk> but again, that only really tells you background processes
[14:15] <Keybuk> hwclock would come and go, so unless you polled the tasks file *really hard* you may not even see the pid
[14:16] <Keybuk> (and even then, /proc/PID/cmdline would probably be gone by the time you found the pid)
[14:16] <djszapi> yeah, but audit should not be racey.
[14:16] <Keybuk> audit is racey
[14:16] <djszapi> or "slow".
[14:16] <Keybuk> and slow
[14:16] <djszapi> lol
[14:16] <Keybuk> sadly
[14:16] <Keybuk> audit doesn't block the shell from doing its work
[14:17] <djszapi> which is nifty.
[14:17] <Keybuk> so when the shell forks off hwclock, it doesn't actually stop anything
[14:17] <Keybuk> so by the time that the audit system is told about the new pid, it could have already finished running
[14:17] <djszapi> hard to imagine.
[14:17] <Keybuk> which means there won't be any information about it - not even the fact it was hwclock at all
[14:17] <Keybuk> happens all the time
[14:17] <djszapi> as long as the audit follows the syscalls.
[14:17] <Keybuk> you'd have to follow the fork and the exec calls
[14:17] <Keybuk> and extract the strings from the exec call (which is actually an array of strings)
[14:18] <Keybuk> that's what then makes it slow
[14:18] <djszapi> well, not really.
[14:18] <djszapi> it depends on the kernel configuration.
[14:18] <Keybuk> no, really ;(
[14:18] <djszapi> you are not obligate to enable the syscall tracing capability.
[14:19] <Keybuk> no, I mean it's just slow when enabled
[14:19] <Keybuk> it can be nearly free when turned off
[14:19] <Keybuk> your best bet, really, is to get the shell to tell you the pids ;)
[14:19] <Keybuk> for example, you could write that script as
[14:19] <Keybuk>   hwclock -s -u &
[14:19] <djszapi> yeah, but that option is just one of the available audit options, you can still use audit with no syscall trace.
[14:19] <Keybuk>   # you now have the PID in $!
[14:19] <Keybuk>   wait $!
[14:19] <Keybuk> sure, but audit with no syscall trace means you won't get the pids ;)
[14:20] <djszapi> mmh
[14:20] <djszapi> so the easiest way is to modify ALL the configuration files existing.
[14:20] <Keybuk> well, it depends yeah
[14:21] <Keybuk> a better question is why do you need to know the PID of that hwclock invocation?
[14:21] <djszapi> I cannot talk about that, sorry. Company project and NDA.
[14:21] <Keybuk> do you need to know the pid of the date and test invocations?
[14:21] <djszapi> if it was run directly by upstart shell then yeah.
[14:22] <Keybuk> there's no such thing as "upstart shell"
[14:22] <Keybuk> that's what I was referring to earlier
[14:22] <djszapi> wtf ? :D
[14:22] <Keybuk> everything in that script block - upstart thinks is a big string
[14:22] <Keybuk> you can put whatever you want in there
[14:22] <Keybuk> all upstart does is exec /bin/sh and pipe that string to it
[14:22] <djszapi> I mean in the process queue, there is a shell process after the init process and I meant that shell as an upstart shell.
[14:22] <Keybuk> which is what I mean about "the application that Upstart runs" *is* /bin/sh
[14:23] <Keybuk> interesting example:
[14:23] <Keybuk>   script
[14:23] <Keybuk>     exec hwclock -s -u
[14:23] <Keybuk>   end script
[14:23] <Keybuk> what will Upstart say the pid of that is? :-)
[14:23] <Keybuk> (hwclock)
[14:23] <djszapi> no idea
[14:23] <Keybuk> without knowing your project
[14:24] <Keybuk> one option would be for you to patch /bin/sh itself
[14:24] <djszapi> lol
[14:24] <Keybuk> why lol ?
[14:24] <djszapi> funny
[14:24] <Keybuk> let's say you have this upstart job
[14:24] <Keybuk>   exec /lib/foo/some-binary
[14:25] <Keybuk> if that some-binary is that shell script you paste-binned
[14:25] <Keybuk> do you *still* need the pids from it?
[14:25] <djszapi> no
[14:25] <Keybuk> interesting, why the distinction?
[14:25] <djszapi> :)
[14:25] <Keybuk> *shrug*
[14:26] <Keybuk> seems pretty arbitrary to me
[14:26] <Keybuk> btw, that script is *completely* bat shit insane
[14:26] <Keybuk> you don't need to run hwclock like that
[14:26] <djszapi> no we need.
[14:26] <djszapi> and it was just an example if you do not like, the point is you got the idea.
[14:26] <Keybuk> but I don't get the idea
[14:27] <djszapi> do not make it with me, pleae :)
[14:27] <djszapi> * please
[14:27] <Keybuk> I don't get why you're treating scripts pasted into an upstart conf file differently to those elsewhere
[14:27] <djszapi> do not do it to me :)
[14:27] <Keybuk> let's give a hypothetical example:
[14:27] <djszapi> you do not need to understand that bit.
[14:27] <Keybuk> let's say you have a system which is entirely an upstart-native boot
[14:27] <Keybuk> no, I *do* to help you
[14:27] <djszapi> our system is that.
[14:27] <Keybuk> ok
[14:28] <Keybuk> so have upstart as the last thing run a shell
[14:28] <djszapi> well, I can tell you why tbh.
[14:28] <Keybuk> the PIDs run by Upstart are "1 thru the PID of that shell:"
[14:28] <Keybuk> no?
[14:28] <djszapi> but it requires a more in-depth understanding.
[14:28] <djszapi> and not 1 min
[14:28] <djszapi> that is why I would like to avoid it for now.
[14:28] <djszapi> I would not like to bother you.
[14:28] <Keybuk> if, after booting, the shell's pid is 1456 then Upstart has resulted in 1,456 processes being run
[14:29] <djszapi> yeah, a sheel after the jobs, daemons.
[14:29] <djszapi> * shell
[14:29] <djszapi> yeah, it is trivial
[14:29] <djszapi> but what is your gist ?
[14:29] <Keybuk> no gist, I'm trying to understand what you're trying to do
[14:30] <djszapi> well
[14:30] <Keybuk> for example, if you were trying to measure the overhead of using shell scripts for things rather than writing them in C
[14:30] <djszapi> there are more shell scripts in fact, not just one.
[14:30] <djszapi> depends on the amount of configuration file stuff
[14:30] <djszapi> well, put it that way.
[14:30] <djszapi> do you know Maemo, MeeGo ?
[14:30] <Keybuk> of course
[14:30] <djszapi> define "of course".
[14:30] <Keybuk> it's hard to work out, because - for example - what if hwclock forked off an extra child?
[14:31] <Keybuk> do you want to know two pids or one?
[14:31] <djszapi> are you a nokian ?
[14:31] <Keybuk> of course I know of Maemo/MeeGo
[14:31] <Keybuk> no, but I know lots of people who are
[14:31] <djszapi> excellent
[14:31] <djszapi> so we use upstart in this project.
[14:31] <djszapi> right ?
[14:31] <Keybuk> indeed
[14:31] <djszapi> well
[14:31] <Keybuk> well, on/off
[14:31]  * Keybuk checks whether there's a "u" in the day
[14:32] <djszapi> ?
[14:32] <Keybuk> sorry, possibly an English expression
[14:32] <djszapi> nvm...
[14:32] <Keybuk> Maemo/MeeGo uses Upstart on Sunday, Tuesday, Thursday and Saturday
[14:32] <Keybuk> but something else on Monday, Wednesday, Friday
[14:32] <djszapi> so you know maemo, meego security framework
[14:32] <Keybuk> a joke about the fact that they change their mind every minute
[14:32] <Keybuk> no, I don't know about that I'm afraid
[14:32] <djszapi> it is a bit different compared to the traditional posix capability stuff.
[14:32] <djszapi> it is more fine tuned stuff, like selinux, smack
[14:32] <djszapi> and the like
[14:33] <Keybuk> ok
[14:33] <djszapi> btw, do not starve because of me :)
[14:33] <Keybuk> I've eaten :)
[14:33] <djszapi> *Nifty, googles*
[14:33] <djszapi> *giggles*
[14:33] <djszapi> typo king, so
[14:34] <djszapi> we use manifest files for fine-tune capability management
[14:34] <djszapi> and upstart and daemon maintainer did something wrong :)
[14:34] <djszapi> no manifest stuff, and all the run jobs, daemon, upstart have all the capabilities.
[14:34] <djszapi> We wrote a debugger kernel module which gives a line like that:
[14:35] <djszapi> [    9.701293] captrace:allow=25|hwclock:329,sh:328,init:1
[14:35] <djszapi> 25 obv. comes from /usr/include/linux/capabili.*
[14:35] <djszapi> it is a capability
[14:35] <djszapi> that is the output during the boot-up
[14:36] <djszapi> and I do some kind of reverse engineering with the perl script
[14:36] <djszapi> to generate the manifest files for upstart and jobs, daemons maintainers.
[14:36] <djszapi> can you follow ?
[14:36] <Keybuk> yup
[14:36] <djszapi> okay, great I need the conf path to put into the manifest file.
[14:36] <djszapi> and just the daemon binary path in case daemons
[14:36] <djszapi> that is why it does not matter in that case.
[14:37] <djszapi> so the jobs, daemons capabilities are separated.
[14:37] <djszapi> because they are different debian packages.
[14:37] <djszapi> or rpm, whateva'
[14:38] <djszapi> e.g. /usr/bin/pulseaudio
[14:38] <djszapi> but the stuff run by upstart.
[14:38] <djszapi> will be dropped by the upstart package.
[14:38] <djszapi> and we are still speaking about credentials.
[14:38] <djszapi> is it clear ?
[14:39] <djszapi> so [    9.701293] captrace:allow=25|hwclock:329,sh:328,init:1 is a good line, but I need to pass it with a configuration file
[14:39] <djszapi> from which hwclock was run actually.
[14:40] <djszapi> hwclock pid is there, 329
[14:40] <djszapi> and that is what I should match with upstart debug output, if any
[14:40] <djszapi> and get the related configuration file name
[14:40] <djszapi> that is all about, feel free to ask if something is vague.
[14:44] <Keybuk> ah, I usee
[14:44] <Keybuk> so if you had a line saying date:330,sh:328,init:1
[14:44] <Keybuk> you'd just ignore it?
[14:47] <Keybuk> so there's an easy way to map <blah>:<pid>,init:1 to an Upstart conf name
[14:47] <Keybuk> boot with --verbose, in the system logs upstart will list all the pids along with the job
[14:47] <Keybuk> so you could compare the syslog to that log, and extract :PID,init:1 and map all instances of those to a .conf
[14:48] <Keybuk> as for the things like hwclock and soforth run by the scripts, audit is *probably* the right place
[14:49] <Keybuk> since in those cases, hwclock's ppid remains 328, init never finds out about it - the only time init really cares about a pid is when the ppid dies and it gets reparented to init, you see
[14:49] <djszapi> usee ?
[14:50] <Keybuk> "see" sorry
[14:50] <djszapi> I would not ignore a line, why do you think that ?
[14:50] <Keybuk> well, date doesn't need any extra capabilities? :-)
[14:50] <Keybuk> or maybe it does
[14:50] <djszapi> gotcha
[14:50] <djszapi> you are right
[14:50] <djszapi> the common linux commands do not.
[14:51] <djszapi> but it is just a whitelisty parsy thingy.
[14:51] <djszapi> this is not the main question here for now.
[14:51] <djszapi> so there's an easy way to map <blah>:<pid>,init:1 to an Upstart conf name
[14:51] <djszapi> nope ^
,sh:xxx,init:1
[14:52] <Keybuk> --verbose will give you the log messages to do that
[14:52] <djszapi> the blah process can most likely be "advanced" cmd.
[14:52] <djszapi> advanced = extra credentials for now
[14:52] <djszapi> most likely the one left to that in the line is gonna be a common tool as you have just mentioned.
[14:52] <djszapi> it is a kinda process tree tho.
[14:52] <Keybuk> --verbose will give you:
[14:53] <Keybuk> Jan 14 14:52:49 deathspank init: tty6 main process (8557)
[14:53] <Keybuk> log messages
[14:53] <Keybuk> that means you know that all
[14:53] <djszapi> boot with --verbose, in the system logs upstart will list all the pids along with the job -> nope
,sh:8557,init:1
[14:53] <djszapi> as said, we do not need jobs
[14:53] <djszapi> just commands from the scripts
[14:53] <Keybuk> no, not ALL
[14:53] <djszapi> jobs means by me the daemons.
[14:53] <Keybuk> but you don't need all
[14:53] <Keybuk> Jan 14 14:52:49 deathspank init: tty6 main process (8557)
,sh:8557,init:1
[14:53] <Keybuk> see how the 8557 lines up there
[14:53] <djszapi> so ?
[14:54] <Keybuk> every <blah> run by that script will still end ",sh:8557,init:1"
,sh:8557,init:1
[14:54] <Keybuk> hwclock:8558,sh:8557,init:1
[14:54] <Keybuk> date:8559,sh:8557,init:1
[14:54] <djszapi> give me 2-3 mins to read your posts again.
[14:54] <Keybuk> mkdir:8560,sh:8557,init:1
[14:54] <Keybuk> hwclock:8563,sh:8557,init:1
[14:54] <Keybuk> for example
[14:54] <Keybuk> if that script ran at least those four commands, that's what you'd see, no?
[14:54] <Keybuk> (the ppid of each of those remains 8557)
[14:55] <Keybuk> and, in fact, the pgid and sid of even hwclock is 8557
[14:55] <Keybuk> so that tells you that the shell script run by upstart for the "tty6.conf" ran those four commands
[14:55] <djszapi> since in those cases, hwclock's ppid remains 328, init never finds out about it - the only time init really cares about a pid is when the ppid dies and it gets reparented to init, you see
[14:55] <Keybuk> right
[14:56] <djszapi> -> yes, I mentioned an example in the beginning for that.
[14:56] <Keybuk> unless a process deliberately attempts to shed its session, the pgid and sid will remain the same as the shell
[14:57] <djszapi> 16:57 < Keybuk> Jan 14 14:52:49 deathspank init: tty6 main process (8557)
[14:57] <djszapi> well it is not enough
[14:57] <Keybuk> why not?
[14:57] <djszapi> it is just about the shell script, not the application run by that.
[14:57] <Keybuk> why is it just about the shell script?
[14:58] <Keybuk> from what you've told me, that'd exactly what you want, isn't it?
[14:58] <djszapi> nope
[14:58] <djszapi> okay, give me a better example and not hwclock
[14:58] <Keybuk> no, what _do_ you want then?
[14:58] <djszapi> * let me give you
[15:01] <djszapi> sorry Keybuk
[15:01] <Keybuk> maybe I should have picked a better test command than "stop gdm" :p
[15:01] <djszapi> :P
[15:01] <djszapi> you were right
[15:01] <Keybuk> http://people.canonical.com/~scott/forkwatch.c.txt
[15:01] <djszapi> I did not understand what I mean
[15:02] <djszapi> * You mean
[15:02] <djszapi> * meant..erm
[15:02] <Keybuk> if you need something that *really* can watch all the forks, that C program should help
[15:02] <djszapi> wait
[15:02] <Keybuk> though you'll see what I mean about the cmdline race there
[15:02] <Keybuk> audit might be better
[15:02] <djszapi> we have just discussed the verbose stuff is ok'ish
[15:02] <Keybuk> the verbose stuff should get you back to an upstart .conf file, yeah
[15:03] <Keybuk> and you can turn it off after boot with "initctl log-priority warn"
[15:03] <djszapi> well, the pid idea was about the identification
[15:03] <djszapi> but I need the config file only at last.
[15:04] <djszapi> http://people.canonical.com/~scott/ -> is it your stuff ?
[15:04] <djszapi> Keybuk: well not an issue
[15:04] <Keybuk> djszapi: yup
[15:04] <Keybuk> my stuff
[15:04] <djszapi> as you got the point this task is because of a wrong maintainer past
[15:04] <djszapi> so hopefully it is not gonna require continous running.
[15:04] <djszapi> during the lifecycle of the device, os.
[15:05] <Keybuk> *nods
[15:05] <Keybuk> you can turn on --verbose btw with "initctl log-priority info"
[15:05] <Keybuk> so you don't even need to reboot to enable it
[15:05] <djszapi> but for confirmation, let me give you a better example than hwclock was.
[15:05] <djszapi> [    8.318542] captrace:allow=6|aegis-loader:191,sh:188,init:1
[15:06] <djszapi> aegis-loader is an internal application
[15:06] <djszapi> run by upstart's script section
[15:06] <djszapi> right ?
[15:06] <Keybuk> right
[15:06] <Keybuk> I assume from that that aegis-loader still has pgid/sid=188
[15:07] <Keybuk> ?
[15:07] <djszapi> highly doubt
[15:07] <Keybuk> is it a background process?
[15:08] <djszapi> RM680-02-2_RD_001:/etc/init# grep -rni aegis-loader .
[15:08] <djszapi> ./aegis.conf:14:    if [ -d /sys/kernel/security/credp -a -x /usr/bin/aegis-loader ]; then
[15:08] <djszapi> ./aegis.conf:15:        /usr/bin/aegis-loader || echo "Failed to load aegis"
[15:08] <Keybuk> right, so doesn't look backgroundy to me
[15:08] <Keybuk> so it should still be easy to match up
[15:08] <djszapi> righty
[15:08] <djszapi> k
[15:09] <djszapi> one thingy I do not understand about your posts.
[15:09] <Keybuk> what's that?
[15:09] <djszapi> 16:52 < Keybuk> as for the things like hwclock and soforth run by the scripts, audit is *probably* the right place
[15:10] <Keybuk> right, to find out the pid of hwclock in the first place, rather than the shell
[15:10] <Keybuk> you know the pid of the shell, upstart knows the pid of the shell, upstart logs it, etc.
[15:10] <Keybuk> but upstart doesn't know (or care) what the shell does
[15:10] <Keybuk> so you need something to get the pid of the things the shell runs
[15:10] <Keybuk> so then you map those things back to the .conf file
[15:10] <Keybuk> basically you'll end up with a process tree
[15:11] <Keybuk> pid 1 forked pid 57 forked pid 134
[15:11] <Keybuk> 134 might be hwclock
[15:11] <Keybuk> so you look up its parent, 57, in the log - and that tells you its wibble.conf
[15:11] <Keybuk> wibble main process (57)
[15:12] <djszapi> well
[15:12] <djszapi> sh:pid,init:1
[15:12] <Keybuk> right
[15:12] <djszapi> shS are the shell scripts coming from upstart (during the boot-up for instance), aren't they ?
[15:12] <Keybuk> not always
[15:12] <Keybuk> that's why I asked about exec /lib/blah/blah style
[15:12] <Keybuk> that can show up as just "sh" too
[15:13] <Keybuk> (if blah is a shell script)
[15:13] <djszapi> because that way I would say we do not need anything else than the process id of the shell if it is quite enough information to figure out the configuration file.
[15:13] <djszapi> let us grep for exec lines, momment
[15:14] <djszapi> well I do not see any shell script in the output.
[15:15] <djszapi> getty, syslogd, sshd, udevd, internal binaries...
[15:15] <djszapi> no shell script.
[15:15] <djszapi> mostly /usr/(s)bin stuf.
[15:15] <djszapi> and /sbin ofc.
[15:16] <Keybuk> cool
[15:16] <Keybuk> you're using dash then? ;-)
[15:16] <djszapi> your mean ?
[15:16] <Keybuk> readlink /bin/sh
[15:17] <Keybuk> probably /bin/dash or /bin/bash
[15:17] <djszapi> busybox
[15:17] <Keybuk> oh right
[15:17] <djszapi> ofc embedded mobile phones
[15:17] <Keybuk> that's usually a bugger for not setting proctitles ;-)
[15:17] <djszapi> and some netbook
[15:17] <djszapi> we are talking about in case Maemo, MeeGo.
[15:17] <djszapi> 'that's usually a bugger for not setting proctitles ;-)' -> can you be a bit more expressive, please ?
[15:18] <Keybuk> oh, I mean that busybox often doesn't bother setting the proctitle
[15:18] <Keybuk> (the thing you see in ps)
[15:18] <Keybuk> so when it gets a shell script, you often just see 'sh' rather than '/bin/foo'
[15:18] <Keybuk> but that, of course, depends on its configuration when built
[15:19] <Keybuk> doesn't matter
[15:19] <Keybuk> since it obviously does the right thing for you
[15:21] <djszapi> y
[15:23] <djszapi> so it seems we do not need audit and kernel hacking then
[15:23] <djszapi> but let me know the truth (aka. fixme).
[15:29] <djszapi> After this command: 'initctl log-priority warn' -> it works perfectly after every reboot that way until I set it back, right ?
[15:31] <Keybuk> right
[15:32] <djszapi> right what ?
[15:33] <Keybuk> right :)
[15:34] <djszapi> well, I said/asked two things, both ?
[15:35] <Keybuk> both
[15:35] <djszapi> k, ty.
[15:38] <djszapi> btw, are you an upstart developer ?
[15:38] <Keybuk> I wrote Upstart
[15:38] <djszapi> alone ?
[15:39] <Keybuk> pretty much
[15:39] <djszapi> lol
[15:39] <djszapi> you are crazy :D
[15:39] <Keybuk> yes
[15:40] <djszapi> CONFIG_CMDLINE="init=/sbin/preinit root=0xB302 rootfstype=ext4 rw console=ttyS0,115200n8 omap3_die_id" -> this is the default, so just -> CONFIG_CMDLINE="init=/sbin/preinit root=0xB302 rootfstype=ext4 rw console=ttyS0,115200n8 omap3_die_id verbose" ?
[15:40] <djszapi> or what is the correct syntax ?
[15:40] <Keybuk> --verbose
[15:41] <djszapi> weirdy
[15:42] <djszapi> well, it does not print anything in fact, I have just tried it out.
[15:43] <Keybuk> no, it logs it so syslog
[15:43] <djszapi> highly doubt
[15:43] <djszapi> we redirect it to console and watch in minicom
[15:44] <djszapi> there is no syslog here.
[15:45] <djszapi> mmh....
[15:46] <djszapi> no idea I guess.
[15:47] <djszapi> ok I gave it up :/
[15:47] <Keybuk> I've no idea what your configuration is wrt syslog
[15:47] <Keybuk> but that's where upstart sends its log messages
[15:48] <djszapi> by default: /var/log/syslog ?
[15:49] <Keybuk> no, syslog
[15:49] <Keybuk> man 3 syslog
[15:49] <djszapi> there is no /var/log/messages*
[15:49] <Keybuk> syslog is a socket interface
[15:49] <Keybuk> where they go after that, depends entirely on your setup
[15:49] <Keybuk> (or even if anything listens)
[15:49] <djszapi> man 3 -> developer man pages.
[15:50] <djszapi> I do not develop with syslog.
[15:50] <Keybuk> I do :)
[15:50] <djszapi> by default: -> there is not "depends entirely on your setup" thingy :)
[15:51] <djszapi> anyways...no idea where the log should be then
[15:53] <Keybuk> you need to speak to the maemo/meego developer who knows where syslog() messages go on meego
[15:53] <djszapi> again: minimum, serial port
[15:54] <djszapi> at least my printk goes there...
[15:54] <djszapi> and printk is supposed to go to the syslog if nothing else h appens.
[15:54] <djszapi> * minimum = minicom
[15:55] <djszapi> seems verbose does not work
[15:55] <djszapi> maybe the command line option struggles.
[15:55] <djszapi> is there any way to check the command line with which the system was booted up ?
[15:56] <djszapi> some sysfs or procfs heuristics ?
[15:57] <djszapi> RM680-01-2:~# initctl log-priority warn
[15:57] <djszapi> RM680-01-2:~#
[15:57] <djszapi> this does not help either after the rebootl.
[15:57] <djszapi> * reboot
[15:58] <djszapi> the upstart might not work on this boot at all ??
[15:58] <djszapi> I do not see any other upstart related entries either.
[15:58] <djszapi> should they occur at all ?
[16:04] <sam-_-> so i want to disable an upstart job. would i just uncomment the "start on" line? what's the preferred way to do this?
[16:14] <djszapi> how can that be Keybuk, I do not get any upstart messages, nor the debug statements about the stopped, started processes ?
[16:45] <JanC> sam-_-: I think you mean *comment* the "start on" line?  (and yes, that should prevent it from getting started automatically)
[16:45] <sam-_-> JanC, is the preferred way?
[16:46] <sam-_-> JanC, y. that's what i meant.
[16:46]  * sam-_- hits himself in the head
[16:46] <JanC> ☺
[16:46] <Stevee> if you are using upstart 0.6.7, there is a stanza called "manual" add this entry to any jobfile to prevent upstart to start it by any event
[16:47] <sam-_-> Stevee, ah ok. thx
[16:48] <JanC> in case you are using Ubuntu, Maverick still has 0.6.6
[16:48] <sam-_-> i'm on 0.6.6 though :-(
[16:48] <JanC> so 0.6.7 is very new
[16:48] <sam-_-> yes
[16:48] <sam-_-> so the comment will have to do it :-)
[16:48] <sam-_-> so the commenting will have to do it :-)
[16:49] <JanC> for now at least
[16:49] <sam-_-> y. no problem.
[16:50] <JanC> even natty still has 0.6.6 it seems
[16:50] <JanC> (but I guess that will change?)
[16:51] <Keybuk> natty has 0.6.7
[16:51] <JanC> hm, no
[16:51] <JanC> package says 0.6.7, --version says 0.6.6
[16:52] <JanC> initctl --version
[16:53] <JanC> so probably it's 0.6.7 but doesn't know that itself  ;)
[16:54] <Keybuk> huh
[16:57] <SpamapS> Has there very been a discussion of an 'expect listen' ?
[16:57] <Keybuk> yes
[16:59] <Keybuk> though it goes something more like "why not use 'start on socket'" :p
[16:59] <SpamapS> Yeah thats where I get to as well.. the example I'd give is mysql .. which may take 20 minutes to start if it has to recover large innodb transaction logs.
[17:00] <Keybuk> yeah, and on a server you might want mysql always running
[17:00] <Keybuk> not just started on first activation
[17:00] <Keybuk> though even then, it would have the listening socket passed to it
[17:01] <Keybuk> so it's a bit of a tricky one
[17:01] <Keybuk> the other side of the question is - does the kernel even notify userspace of new sockets?
[17:01] <Keybuk> I had a vague idea there might be something netlink-y but I've not looked
[17:02] <djszapi> Keybuk: gotta go, ty for your help today :)
[17:03] <djszapi> my virtual beer is yours, heh
[17:30] <JanC> Keybuk: the source package for upstart 0.6.7 in natty says 0.6.6 in configure.ac & configure, but that seems to be correct/fixed in upstream 0.6.7
[17:30] <Keybuk> weird
[17:33] <Keybuk> wonder why
[20:47] <SpamapS> t
[20:48] <SpamapS> 1
[20:49] <SpamapS> oops wrong window
[22:44] <bencc> can you give an example of ubuntu package that uses upstart?
[22:45] <bencc> I'm trying to package a server and want to see how to include the upstart script
[22:48] <JanC> bencc: you mean a server package?
[22:49] <bencc> JanC: a web server
[22:49] <bencc> what do you mean by a server package?
[22:49] <JanC> well, there are many packages that use upstart jobs, but starting a server isn't the same as starting a TTY of course  ;)
[22:50] <bencc> ok
[22:50] <bencc> I have a basic upstart job that start the server
[22:50] <bencc> I want to see how to include it in a .deb package
[22:50] <JanC> I see mysql & squid have upstart jobs
[22:51] <bencc> thanks, checking
[22:54] <bencc> JanC: dh-make produces example files like init.d.ex
[22:54] <bencc> I don't see an example for upstart
[22:56] <JanC> bencc: maybe file a (wishlist) bug about that then?
[22:56] <bencc> JanC: ok but I don't know how to use it
[22:57] <bencc> JanC: if there is init.d.ex the packaging tool probably handle it in a special way
[22:57] <bencc> so I'm not sure if I need to put the upstart on the root or maybe treat it as a normal file
[23:51] <tgm4883> When looking at upstart services, what is the meaning of +, -, and ?. for example   [ ? ]  acpi-support
[23:51] <tgm4883> I've been querying the info, but + and ? tend to be the same


[12:39] <mup> Bug #2006598 opened: Storage tag not showing in the filter <MAAS:New> <https://launchpad.net/bugs/2006598>
[12:42] <mup> Bug #2006598 changed: Storage tag not showing in the filter <MAAS:New> <https://launchpad.net/bugs/2006598>
[12:45] <mup> Bug #2006598 opened: Storage tag not showing in the filter <MAAS:New> <https://launchpad.net/bugs/2006598>


[14:58] <Ddorda> hey
[20:52] <g4br13l> hey
[20:52] <g4br13l> what cloak /ubuntu/member give?
[20:53] <g4br13l> set me
[20:53] <Pici> g4br13l: Are you an Ubuntu Member?
[20:53] <g4br13l> ¡!Pici!¡: yes
[20:53] <Pici> g4br13l: What is your launchpad id?
[20:53] <g4br13l> [Pici]: of website?
[20:54] <Pici> g4br13l: Yes, from launchpad.net
[20:54] <g4br13l> [Pici]: g4br13l
[20:54] <Pici> g4br13l: It looks like you are not part of the Ubuntu Members group.
[20:55] <Pici> !membership
[20:55] <Pici> Please see the link above for more information.
[20:58] <g4br13l> Pici hey, i'm logged site... launchpad.
[20:58] <Pici> g4br13l: You are not a member of the ubuntumembers team.
[20:59] <g4br13l> https://launchpad.net/ ?
[20:59] <Pici> g4br13l: You need to be an approved Ubuntu Member to qualify for an ubuntu/member cloak.
[21:00] <g4br13l> |Pici|: ah
[21:04] <g4br13l> Pici hey


[01:24] <nichos> hi all, has anyone noticed high RAM usage in myth .23? http://pastebin.com/u9Fym3Cg
[02:38] <nichos> hi all, has anyone reported high RAM usage when using myth .23? http://pastebin.com/u9Fym3Cg
[02:49] <dardack>  hi can anyone help me, i have an older nvidia 5500 card in my backend, only the 173 drivers work for this, i just upgraded to 9.10 mythbuntu, .22 mythtv, and now mythbackend is complaing it can't find the libvdpau.so.1 library
[02:59] <dardack> find it in some guys repos:  http://www.avenard.org/media/Ubuntu_Repository/Ubuntu_Repository.html, ty
[02:59] <Zinn> [www.avenard.org] Ubuntu Media Playing Packages
[03:00] <dardack> zinn, yea found it on my own, mythbackend now works
[03:00] <Zinn> Hi dardack, something I can help you with today?  I am a bot, use !help to see what I can do.
[03:00] <dardack> nvm
[03:18] <nichos> battery died:)
[03:18] <nichos> anyone answer my ? :)
[14:49] <PimP> is there a known hardware list that works with mythbuntu
[14:50] <rhpot1991> PimP: not really, if it works with linux it should work with mythtv
[14:50] <rhpot1991> you can see what the developers are using here: http://mythbuntu.org/developer-hardware
[17:02] <markl_> anyone here familiar with the mythtv svn revision that is going out on 10.04?
[17:03] <markl_> they fixed a nasty HD Homerun bug a week or two ago and i am hoping that the bug doesn't make it out into the wild
[17:04] <markl_> any 0.22 dual tuner HDHR users who get this bug will be hating life
[17:04] <rhpot1991> markl_: whats the bug?
[17:05] <mrand> markl_: if it was fixed in the -fixes branch of 0.23, then it'll be included.
[17:05] <mrand> The snapshot will be created today I believe.
[17:05] <rhpot1991> if it is fixed today that is, our final build is going to happen today
[17:06] <mrand> markl_: I assume this is what you're talking about: http://svn.mythtv.org/trac/changeset/24024
[17:06] <Zinn> [svn.mythtv.org] Changeset 24024 – MythTV
[17:06] <mrand> If so, no worries.
[17:07] <markl_> let me get some numbers here really quick, brb
[17:08] <markl_> it was #8193
[17:10] <markl_> not a changeset though, let me get a url :)
[17:12] <mrand> http://svn.mythtv.org/trac/ticket/8193 points to the above changeset
[17:12] <Zinn> [svn.mythtv.org] #8193 (HDHR fails to record on both tuners simultaneously) – MythTV
[17:13] <markl_> ah ok ty, that is the bug
[17:13] <markl_> someone in #mythtv rolled back the problematic HDHR code a week or two ago
[17:14] <markl_> i'm still using the custom mythbuntu .deb that I built to test the fix
[17:14] <markl_> just wondering when it is safe to apt-get upgrade again :)
[17:14] <mrand> markl_: yep... that changeset shows iamlindoro did it.  Thanks for pinging us though!
[17:14] <markl_> so the final release will be ok then?
[17:15] <markl_> 0.23-fixes?
[17:15] <mrand> markl_: yes.  And auto-builds are ok right now.
[17:15] <markl_> fantastic, thanks.
[17:15] <mrand> Thank you!
[17:15] <markl_> finally i actually contributed some help instead of just leeching .debs :)
[17:16] <mrand> markl_: heheh.  There's always little things like that which I think many people don't realize they can help with.
[17:28] <markl_> yep, scratching your own personal itch in this case :)
[17:33] <PimP> i have to allow a specific ip to be allowed by the url filtering
[17:33] <PimP> this is going to require a policy change
[17:33] <PimP> someone made threats at school on myspace
[17:33] <PimP> now administration wants to look at them
[17:34] <PimP> first step is getting the ip address of the computer they are using


[00:02] <bac> thumper, rockstar, stub, mwhudson, stevek: reviewers meeting
[00:05] <mwhudson> something in the bugs page seems to make firefox go insane :(
[00:05] <mwhudson> js-wise
[01:00] <lifeless> jml: another recipe bug for you - https://bugs.launchpad.net/launchpad-code/+bug/687623
[01:00] <_mup_> Bug #687623: No ability to manually trigger a build 'like the daily build system' <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/687623>
[01:08] <spm> thumper: spiv: would appreciate your thoughts here: we have ~ 10 requests against BRANCH:%2Bbranch/tsumego20k/1.0 but https://code.launchpad.net/~branch says no code?
[01:09] <spiv> spm: %2B is +, not ~
[01:09] <spm> Ah
[01:09] <spm> wrong group. that's a start.
[01:10] <spm> https://code.launchpad.net/tsumego20k looks more like it
[01:11] <spiv> But there's no 1.0 series.
[01:11] <spiv> There's a 'stable' series (with a 1.0 release).
[01:12] <spm> hrm. and their branch is only 472K as well.
[01:12] <spiv> So, requests for that branch are presumably wrong, and ought to give "branch not found".
[01:12] <spm> lp:tsumego20k  as in
[01:12] <spiv> they probably mean /stable rather than /1.0?
[01:13] <spm> well. it gets worse. that, and another branch are eating memory. just got a swap alert. I'm going to have to big stick shortly.
[01:13] <spiv> Oh and for added fun the stable branch appears to have a broken stacked-on target.
[01:14] <spm> would that be causing the pain that codehost is experiencing?
[01:14] <spiv> I wouldn't *think* so
[01:14] <spm> ah. they have another: BRANCH:~goadict/tsumego20k/release-1.0
[01:15] <spiv> But it could be... it's hard to be sure.
[01:16] <spm> mwhudson: so gist is we have one, maybe 2 or 3 branches that are spinning doing "something"; and eating memory and CPU
[01:16] <spiv> I don't know of any open bugs that would cause this sort of spinning now that codehosting is running bzr 2.2
[01:16] <mwhudson> ah ok
[01:16] <spiv> Er, 2.2.2
[01:17] <mwhudson> spm: have you whacked them with that gdb thing?
[01:17] <spm> no, wasn't game to just yet.
[01:17] <spm> haven't done any whacking - just watching/gathering.
[01:17] <mwhudson> k
[01:17] <spm> https://pastebin.canonical.com/40699/
[01:18] <spm> https://pastebin.canonical.com/40700/ for a process listing
[01:19] <spm> the db-devel ones, while alarming to have a few, don't seem to be problematic.
[01:19] <spiv> Huh.
[01:20] <spiv> Some say "+branch", some say "%2Bbranch".  I wonder why.
[01:20] <spiv> Probably a red herring for this issue.
[01:20] <spm> the cpu/load spike seems to be coming down tho. so I'm strongly tempted to just let things resolve on their own.
[01:20] <spm> well... load. not cpu. that's still flat out.
[01:23] <mwhudson> that's a lot of processes
[01:23] <mwhudson> i wonder which user 2978124 is
[01:23] <mwhudson> probably launchpad-pqm
[01:23] <spm> how do we find out? select * from person?
[01:24] <mwhudson> y
[01:24] <spm> too obvious that. if even a sysadmin can guess correctly. /me files a big for greater obscurity.
[01:24] <spm> bug, too.
[01:25] <spm> launchpad-buildbot
[01:25] <mwhudson> ah ok, that makes some kind of sense too
[01:29] <spm> so something baout those branches tho is driving codehost to 100% cpu for ages, which is bad. :-(
[01:29] <spm> worth a pygdb on one?
[01:30] <spm> haha. on codehost itself, and it's easier to copy the branch from my local pc up.
[01:32] <mwhudson> hee hee
[01:32] <mwhudson> i guess you could run bzr get /srv/.../00/0x/xx/xx if you coudl be bothered
[01:32] <mwhudson> yeah, i reckon pygdb one
[01:32] <mwhudson> the user on the other end must be bored already by now
[01:35] <spm> or afk
[01:39] <spm> oh that's a bad sign... the backtrace.py is not showing a whole lotta activity...
[01:39]  * spm lets it be for a bit longer
[01:40] <mwhudson> :/
[02:06] <spm> mwhudson: I gave up. it never progressed beyond 'Thread 1' as output. trying a gcore.
[02:06] <mwhudson> spm: :(
[02:06] <mwhudson> spm: maybe just attach with gdb and run 'bt'
[02:07] <mwhudson> the process will only be single threaded
[02:07] <spm> sure
[02:07] <spiv> mwhudson: not necessarily (but probably)
[02:08] <spm> err. pebkac? $ gdb 23558 ; (gdb) bt ==> No stack.
[02:08] <spiv> mwhudson: the insert_stream verb in the server spawns a thread as that was easiest...
[02:08] <spiv> spm: gdb -p 23558 ?
[02:08] <spm> bah
[02:08] <spm> that's better. ta
[02:09] <mwhudson> spiv: whee
[02:09] <spm> oh wow. it looks almost like recursion hell
[02:09] <spm> pls to save me from rtfm - the command to disable gdb paging?
[02:10] <spm> set pagination off
[02:10] <mwhudson> i guess that might explain why pygdb didn't work so well
[02:11] <mwhudson> though it prints the stack as it goes
[02:11] <spm> holy crap. I'm copping 50k/s of gdb output. and it's not stopping.....
[02:11] <spiv> spm: be careful what you wish for!
[02:12] <spm> reckon
[02:12] <spm> glad I don't have "infinite" buffers any more, or this'd suck
[02:12] <mwhudson> well this sounds like fin
[02:12] <mwhudson> fun
[02:13] <spm> https://pastebin.canonical.com/40701/ for 3 pages of so far
[02:14] <mwhudson> whisky tango foxtrot!!
[02:15] <spm> https://pastebin.canonical.com/40702/ next 4 pages (i think...)
[02:15] <spm> i call busticated.
[02:15] <spiv> Huh, this is an lp-serve process?
[02:15] <spm> codehost?
[02:16] <spm> yes
[02:16] <mwhudson> spiv: yes
[02:16] <spiv> Ah, I see, it veers off into Twisted via lp.codehosting.vfs.transport.SynchronousAdapter
[02:17] <spiv> So this is a bit disheartening :(
[02:17] <spm> spiv: so it's your fault?
[02:17]  * spm establishes random pin-the-blame-game early
[02:17] <spiv> spm: well, I thought bzr 2.2.2 fixed this bug. :/
[02:18] <spm> :-(
[02:18] <spiv> Or rather, fixed one of the two issues that combined to cause it.
[02:18] <mwhudson> oh, it's stringifying a traceback inside twisted's failure code
[02:18] <spm> ah luverly, so we've maybe just discovered #3?
[02:18] <spiv> It appears to be a error-handler-falls-over-and-recurses-into-the-error-handler death sprial.
[02:18] <spiv> mwhudson: right
[02:18] <spm> spiv: so stabby time, or more info would be useful?
[02:19]  * spiv digs up the issue he thought he fixed.
[02:19] <spiv> spm: hmm.  The traceback is probably enough.  Can you give me a minute to ponder?
[02:19] <spm> sure. it's not wonderful, but getting this fixed is worth the pain
[02:20] <mwhudson> spm: can you paste some more traceback actually?
[02:20] <spm> easily
[02:20] <spm> well.. easily but boringly. just sayin'
[02:20] <mwhudson> just a couple more pages
[02:21] <spiv> Argh, this really looks like the bug I fixed!
[02:21] <mwhudson> i'm not sure if it's infinite recursion or just stupidly deep recursion
[02:21] <spiv> repr of NotBranchError
[02:21] <spm> https://pastebin.canonical.com/40703/
[02:22] <spm> mwhudson: I don't believe there's a difference outside academia there :-)
[02:22]  * spiv hmms
[02:23] <spiv> Oh, *blah*
[02:23]  * spiv thinks out loud
[02:23] <spiv> NotBranchError guards against open_repository failing with unexpected exceptions
[02:23] <spm> you think "*blah*"?? really!?!?!
[02:23] <mwhudson> spiv: let me guess, it's partly twisted's fault?
[02:23] <spiv> mwhudson: (yes)
[02:23] <spiv> But open_repository is calling off into the lp VFS stuff
[02:23] <spiv> Which calls into Twisted stuff
[02:24] <mwhudson> anything traceback that has __getstate__ in it instantly makes me want to cry
[02:24] <mwhudson> and punch glyph in the face in about 2001
[02:24]  * spm hands mwhudson a tissue box, and starts playing a small violin
[02:24] <spiv> Which thanks to the crumminess of Twisted's Failure class stringifies errors and tracebacks
[02:25] <spiv> Which happens inside the lp codehosting VFS stuff, so my NotBranchError guard doesn't matter: Twisted tries to stringify a NotBranchError (whose bzrdir fails badly on open_repository)
[02:26] <spiv> So off we go into the recursion.
[02:26] <lifeless> [02:26] <lifeless>     Hard / Soft  Page ID
[02:26] <lifeless>      146 / 5425  Archive:+index
[02:26] <lifeless>       62 /  400  Distribution:+bugs
[02:26] <lifeless>       61 /  233  BugTask:+index
[02:26] <lifeless>       59 /    2  Person:EntryResource:retractTeamMembership
[02:26] <lifeless>       38 /  419  Distribution:+bugtarget-portlet-bugfilters-stats
[02:26] <lifeless>       20 /   28  DistroArchSeries:+index
[02:26] <lifeless>       11 /   16  DistroSeries:+queue
[02:26] <lifeless>       11 /    2  Cve:+index
[02:26] <lifeless>       10 /   95  ProjectGroupSet:CollectionResource:#project_groups
[02:26] <lifeless>        9 /    6  ProjectGroup:+milestones
[02:26] <spiv> Roughly: repr(NotBranchError) -> open_repository -> codehosting VFS -> Twisted gunk -> repr(NotBranchError)
[02:26] <wgrant> The fix for the top one should be on qastaging in a few minutes, hopefully.
[02:27] <spiv> mwhudson: does that sound plausible?
[02:27] <mwhudson> spiv: yes
[02:27] <spiv> mwhudson: ok, now what can we do about it? :(
[02:27] <mwhudson> spiv: make Failure throw away the traceback entirely?
[02:27] <spiv> mwhudson: I *think* perhaps this isn't something bzrlib can sensibly fix.
[02:28] <mwhudson> perhaps with a monkey patch?
[02:28] <mwhudson> spiv: how can we reproduce this locally?
[02:28] <spiv> mwhudson: IIRC the condition is something like a lookup on +branch/something/something causes a strange VFS error
[02:29] <mwhudson> spiv: i don't quite understand why NotBranchError is getting raised in the smart server
[02:29] <mwhudson> oh no, maybe that makes sense
[02:29] <wgrant> mwhudson: Stacking, maybe?
[02:29] <spiv> Because the codehosting VFS is returning that from a transport operation, or something odd like that?
[02:29] <spiv> thumper may remember the details
[02:29] <mwhudson> wgrant: stacking is mostly a client side thing though
[02:29] <spiv> I'll see if I can dig out the relevant conversation from my logs
[02:30] <mwhudson> spiv: is NotBranchError raised by BzrDir.open or something maybe?
[02:30] <spiv> It's definitely an issue relating to how the server-side vfs translates stuff, maybe in weird cases like a dev focus stacked on itself, or something.
[02:31] <spiv> IIRC it's at the point of trying to open the bzrdir that in LP would have the auto-stacking config
[02:31] <spiv> IIRC +branch might be involved?  Or something?
[02:31] <spiv> mwhudson: yeah, probably
[02:31] <mwhudson> +branch shouldn't be involved unless someone has been editing .bzr/branch/branch.conf by hand
[02:32] <spm> i'm about to break for lunch - need anything more before i run?
[02:32] <lifeless> noone would be silly enough to do that :O
[02:32] <spiv> spm: I think it's probably ok to kill before you run, if you want
[02:33] <spm> spiv: okidoki
[02:33] <mwhudson> spm: yeah, as spiv says
[02:33] <spiv> mwhudson: see logs for this channel for Oct 11
[02:33] <spiv> mwhudson: I think that's when I chatted with thumper about this initially
[02:33] <spiv> just reviewing now
[02:34] <spm> ugh. looks like the owner is kicking the branch/whatever operation(s) off again
[02:35]  * spm has applied the sledgehammer :-(
[02:38]  * mwhudson pokes around with sftp
[02:38] <spiv> mwhudson: so in thumper's situation (which involved a patch he may have abandoned, so there may some red herrings here)
[02:40] <mwhudson> i can't see anything unusual :(
[02:40] <mwhudson> https://code.launchpad.net/~goadict/tsumego20k/trunk
[02:40] <spiv> mwhudson: he encountered this problem on an open_branch HPSS call to ~user/proj (trying 'bzr revno ~user/proj/no-such-branch', I think)
[02:40] <mwhudson> isn't a real branch
[02:40] <mwhudson> hang on, let me check what it _is_
[02:41] <mwhudson> spiv: hm
[02:41] <spiv> mwhudson: default_stack_on = /~goadict/tsumego20k/trunk
[02:41] <mwhudson> there's a bzrdir there, but no branch
[02:41] <mwhudson> spiv: yeah, which is what one would expect
[02:41] <spiv> mwhudson: in sftp://bazaar.launchpad.net/~goadict/tsumego20k/.bzr/control.conf
[02:41] <mwhudson> spiv: right
[02:42] <spiv> That is a branch AFAICT over SFTP?
[02:42] <spiv> and not a stacked branch.
[02:42] <mwhudson> spiv: parse error
[02:43] <spiv> mwhudson: lftp bazaar.launchpad.net:~goadict/tsumego20k/trunk/.bzr> cat branch/format
[02:43] <spiv> Bazaar Branch Format 7 (needs bzr 1.6)
[02:43] <spiv> Why do you say that ~goadict/tsumego20k/trunk isn't a real branch?
[02:43] <mwhudson> spiv: by mistake, sorry
[02:43] <spiv> Ah.
[02:44] <mwhudson> spiv: https://code.launchpad.net/~goadict/tsumego20k/stable
[02:44] <mwhudson> spiv: ^ that one is just a bzrdir
[02:44] <mwhudson> lftp mwhudson@bazaar.launchpad.net:/~goadict/tsumego20k/stable/.bzr> ls
[02:44] <mwhudson> -rw-r--r--    1 1001     1001          141 Dec 09 02:21 README
[02:44] <mwhudson> -rw-r--r--    1 1001     1001           35 Dec 09 02:21 branch-format
[02:44] <mwhudson> drwxr-xr-x    2 1001     1001         4096 Dec 09 02:21 branch-lock
[02:45] <mwhudson> the gdb trace is talking about ~goadict/tsumego20k/release-1.0/.bzr/repository though
[02:45] <mwhudson> so maybe this stable branch is a red herring
[02:46] <spiv> I don't think it is.
[02:47] <spiv> My recollection is that accessing non-existent paths triggered this codehosting vfs issue.
[02:48] <spiv> But maybe I'm wrong :/
[02:48] <mwhudson> yeah, maybe that's it
[02:48] <mwhudson> it's striking that all the problem processes are accessing paths that shouldn't work at all, like
[02:48] <mwhudson> BRANCH:%2Bbranch/tsumego20k/1.0
[02:48] <mwhudson> codehost
[02:49] <mwhudson> the thing is of course that maybe the user is getting confused and flailing around creating and deleting things in the webapp
[02:49] <spiv> mwhudson: also, the gdb trace includes stuff like:
[02:49] <spiv> <Deferred at 0xa99a878  current result: <twisted.python.failure.Failure <class 'bzrlib.errors.NoSuchFile'>>>
[02:49] <mwhudson> spiv: that's not so surprising really
[02:50] <mwhudson> the synchronous launchpad transport wraps the asynchronous one
[02:52] <spiv> mwhudson: argh, the data is changing from under us :)
[02:52] <spiv> mwhudson: now ~goadict/tsumego20k/stable has a repo
[02:56] <spiv> A bit odd:  _cache={'//~goadict/tsumego20k/release-1.0': ('BRANCH_TRANSPORT', {'writable': True, 'trailing_path': '.bzr/repository/format', 'id': 410572}, <float at remote 0x34d4cd0>)}
[02:59] <mwhudson> writeable=True suggests that branch existed in the db at the time
[03:07] <mwhudson> wow, i was a bit confused there -- was looking at launchpad.net not launchpad.dev :)
[03:08] <mwhudson> of course launchpad.dev is impossible to use
[03:09] <mwhudson> because of 1 miiiiiiiiiiiiiillion js files
[03:09] <wgrant> devmode off
[03:09] <wgrant> problem solved.
[03:09] <mwhudson> GET /+icing/rev10399/yui/datatype/lang/datatype-date-format_it-IT.js
[03:09] <mwhudson> GET /+icing/rev10399/yui/datatype/lang/datatype-date-format_it.js
[03:09] <wgrant> Yes.
[03:09] <mwhudson> COME ON!
[03:09] <wgrant> And half of them OOPS.
[03:09] <wgrant> So /var/tmp/lperr fills up.
[03:09] <mwhudson> mostly 404ing
[03:10] <mwhudson> where is devmode set?  configs/development/launchpad.conf?
[03:10] <wgrant> That.
[03:11] <mwhudson> $ du -sh /var/tmp/lperr/2010-12-09/
[03:11] <mwhudson> 4.6M	/var/tmp/lperr/2010-12-09/
[03:11] <wgrant> Yeah.
[03:15] <mwhudson> spiv: well i don't know what's going on :/
[03:17] <spiv> mwhudson: hmm
[03:19] <mwhudson> spiv: i guess it might be interesting to see the bottom 100 lines or so of the traceback
[03:19] <spiv> mwhudson: yeah
[03:20] <spiv> mwhudson: also translateVirtualPath maybe should have a catch-all errback
[03:21] <mwhudson> spiv: sounds plausible, without looking at the code or thinking terribly hard
[03:21] <spiv> mwhudson: to avoid unexpected errors leaking out and confusing callers that expect only transport-related errors.
[03:21] <spiv> I don't see any obvious routes for those to happen
[03:21] <spiv> And I've just spent the last 30 min or so looking for one :)
[03:23] <spm> bleh, they're still trying to get the code, or update or whatever. have sent the team members emails asking them to stop.
[03:25] <mwhudson> spm: it would be good to know exactly what they're trying to do
[03:25] <spm> mwhudson: spiv: would a copy of the raw branch(s) off crowberry be of any use to you guys in debugging? seeing as they're able to kill it time and again.
[03:25] <spm> mwhudson: sure - do we have a formal bug yet?  I'll go with that
[03:27] <spm> hahaha
[03:28] <spiv> (A slow, painful doom...)
[03:28] <mwhudson> weird-shit-of-doom sounds like a good bug tag
[03:28] <spm> +1
[03:29] <spm> mwhudson: the only one that works is tiny. http://bazaar.launchpad.net/~goadict/tsumego20k/trunk/files
[03:29] <spiv> spm: for added laughs the user has been actively changing stuff while we've been looking
[03:30] <spiv> Replacing an empty bzrdir with one with a repo etc.
[03:30] <spm> well. trying to. I keep killing their processes off as it hurts
[03:34] <spm> shame. codehost has dropped to #2 spot in firefox quick find for "filebug"
[03:34] <wgrant> Does Soyuz win?
[03:35] <spm> ha. no. most of my bugs are generically filed against launchpad these days. never quite sure where they go.
[03:35] <lifeless> from mid jan 'launchpad'
[03:35] <lifeless> one bug tracker. \o/
[03:36] <wgrant> We'll see how that goes.
[03:37] <spiv> wgrant: more successfully than lifeless' holiday, I hope... ;)
[03:37] <wgrant> Indeed.
[03:37] <spiv> lifeless: I'd suggest you visit some remote island with minimal internet access
[03:38] <spm> spiv 1, lifeless 0
[03:38] <spiv> lifeless: but you already moved to NZ ;)
[03:38] <spm> spiv 3, lifeless 0
[03:38]  * lifeless is on wow atm
[03:38] <spiv> lifeless: that's more like it :)
[03:38] <lifeless> so, I claim success.
[03:38] <spm> spiv 3, lifeless -1
[03:38] <lifeless> oi!
[03:39] <spiv> Some people are so judgemental!
[03:39] <spm> spm 1, spiv 0, lifeless -1
[03:39] <lifeless> s/ w+!/sysadmins!/
[03:39] <lifeless> wow
[03:39] <lifeless> found the stone core
[03:39] <spm> someone has to be. it's a burden that we take upon ourselves to spare the lesser folks aka developers
[03:39] <lifeless> first quest in there...
[03:40] <lifeless> 115K xp
[03:43] <spm> https://bugs.edge.launchpad.net/launchpad-code/+bug/687653
[03:43] <_mup_> Bug #687653: codehost error handling infinite recursion death spiral of doom <canonical-losa-lp> <Launchpad Bazaar Integration:New> <https://launchpad.net/bugs/687653>
[04:18] <wgrant> lifeless: https://qastaging.launchpad.net/~yavdr/+archive/stable-vdr/+packages is promising. 4s.
[04:19] <wgrant> Ah, 2s with hot cache.
[04:19] <wgrant> Not too bad.
[04:20] <wgrant> OOPS-1804QS12
[05:17] <lifeless> wgrant:
[05:17] <lifeless> SQL time: 16441 ms
[05:17] <lifeless> Non-sql time: 4324 ms
[05:17] <lifeless> Total time: 20765 ms
[05:17] <lifeless> Statement Count: 50
[05:18] <lifeless> wgrant: also there are two repeats of a 1000ms query
[05:20] <wgrant> Hmm.
[05:20] <wgrant> A COUNT()?
[05:20] <lifeless><PHONE_NUMBER>017SQL-launchpad-main-slave(SELECT SourcePackagePublishingHistory.archive, SourcePackagePublishingHistory.component, SourcePackagePublishingHistory.datecreated, SourcePackagePublishingHistory.datemadepending, SourcePackagePublishingHistory.datepublished, SourcePackagePublishingHistory.dateremoved,
[05:20] <lifeless> ...
[05:20] <lifeless> SourcePackagePublishingHistory.id, DistroArchSeries.architecturetag) EXCEPT (SELECT SourcePackagePublishingHistory.archive,
[05:20] <wgrant> Oh, that one.
[05:38] <LPCIBot> Project db-devel build (206): FAILURE in 13 min: https://hudson.wedontsleep.org/job/db-devel/206/
[08:59] <bigjools> morning all
[08:59] <wgrant> Morning bigjools.
[09:00] <wgrant> We has a regression.
[09:00] <wgrant> Which I already had a one-line branch to fix...
[09:00] <wgrant> Without knowing about it.
[09:00] <bigjools> wgrant: do tell
[09:00] <wgrant> bigjools: Bug #687662
[09:00] <_mup_> Bug #687662: Upload processor attempts to verify hashes against expired files <Soyuz:New> <https://launchpad.net/bugs/687662>
[09:01] <bigjools> how did that happen?
[09:01] <wgrant> Hmm?
[09:01] <bigjools> oh, that change for getFile
[09:01] <wgrant> Yeah.
[09:01] <bigjools> do you think more pairs of eyes would have caught that?
[09:02] <wgrant> Who knows.
[09:03] <bigjools> wgrant: what affect elsewhere will your fix cause?
[09:04] <wgrant> bigjools: It'll fix the +files/foo.orig.tar.gz 404. It could also impact other A.getFileByName callsites, but I doubt any rely on this behaviour, what with expired LFAs being useless and everything.
[09:06] <bigjools> wgrant: right
[09:06] <bigjools> wgrant: whack up an MP and let's get it landed
[09:06] <wgrant> bigjools: MP is already there.
[09:06] <wgrant> Oh. Branch is not linked.
[09:07] <bigjools> yeah
[09:07] <wgrant> Fix-ed.
[09:07] <bigjools> woo JS regression, the bug page says "remove assignee" when there isn't one
[09:07] <bigjools> wgrant: you've got all pt_br
[09:08] <wgrant> bigjools: WFM
[09:08] <bigjools> s/got/gone/
[09:09] <bigjools> wgrant: the test should not be expanding the doctest, would you mind making a unit test?
[09:10] <bigjools> test_archive.py
[09:11] <mrevell> Hello
[09:13] <wgrant> bigjools: k
[09:17] <wgrant> bigjools: Should I take out the whole doctest while I'm at it? It's short enough.
[09:17] <bigjools> wgrant: be my guest
[09:18] <bigjools> the test style in test_archive should be obvious
[09:23] <wgrant> Yeah, I've already added a few to it.
[09:26] <adeuring> good morning
[09:28] <henninge> Hallo adeuring!
[09:28] <adeuring> moin henninge
[09:32] <henninge> So, I am trying to update sample data that is used in tests.
[09:32] <wgrant> Uhoh.
[09:33] <henninge> I know, I should rewrite the test to not use sample data, but ...
[09:33] <henninge> ... I want to finish this branch today. ;-)
[09:34] <henninge> Anyway, I ran the migration script locally that updates the data the way I need it.
[09:34] <wgrant> You need to run it on launchpad_ftest_playground.
[09:34] <henninge> Did "make newsampledata" and updated current.sql and current-dev.sql.
[09:34] <wgrant> Then make newsampledata.
[09:34] <henninge> oh
[09:35] <henninge> hm
[09:35] <henninge> what do I get then?
[09:36] <wgrant> newsampledata.sql and newsampledata-dev.sql, or something like that.
[09:36] <wgrant> Then you clobber current(-dev).sql with those.
[09:52] <henninge> Interesting. I just checked the (changed) current(-dev).sql that I have and what I did only changed current-dev.sql but not current.sql.
[09:52] <wgrant> Right, you ran it on launchpad_dev, not launchpad_ftest_playground.
[09:52] <henninge> I did not know that.
[09:52] <wgrant> You may need to run with LPCONFIG=test-playground
[09:52] <henninge> How do I ...
[09:52] <henninge> .. thanks ;)
[09:53] <wgrant> Oh yeah, thanks for running those three branches last night.
[09:53] <wgrant> They all passed and landed first time!
[09:53] <wgrant> This hasn't happened in many months.
[09:55] <henninge> :-D
[09:59] <henninge> Looking good, let's see if my tests passes now ...
[10:10] <henninge> Hooray! ;)
[10:10] <wgrant> Excellent.
[10:22] <bigjools> wgrant: I had a thought
[10:22] <bigjools> your change to getFileByName will revert us back to the old situation
[10:22] <wgrant> Not entirely.
[10:22] <bigjools> well pretty much, as soon as a file expires you can re-upload it
[10:22] <wgrant> Right.  And it's as good as we can get without changing the data model.
[10:26] <wgrant> I really wish people would think about what they're doing before they start irrevocably deleting data.
[10:27] <bigjools> hahaha
[10:28] <bigjools> LP has a great history of not deleting anything
[10:28] <wgrant> Yes.
[10:28] <bigjools> The librarian has a girth that rivals Ron Jeremy
[10:29] <wgrant> Sure, and we need to delete things from it.
[10:29] <wgrant> But there are better ways.
[10:29] <wgrant> Like not deleting the hashes.
[10:29]  * bigjools is filing a bug about that right now
[10:29]  * wgrant makes a note to check it in a decade.
[10:30] <bigjools> https://bugs.edge.launchpad.net/launchpad-foundations/+bug/687752
[10:30] <_mup_> Bug #687752: It's impossible to check a librarian file's hash once it's expired <Launchpad Foundations:New> <https://launchpad.net/bugs/687752>
[10:30]  * bigjools notices that gary_poster is not a Foundations bug contact :)
[10:31] <wgrant> Does he read the ML, perhaps?
[10:38] <wgrant> Oh I hate doctests.
[10:58] <lifeless> wgrant: so your fix looks good?
[10:58] <bigjools> wgrant: I guessed it wouldn't be long before the "no ui" whingers started on the ppa stats bug
[10:59] <wgrant> lifeless: For which?
[10:59] <lifeless> Archive:+index
[10:59] <wgrant> lifeless: It seems to behave OK on qastaging now.
[11:00] <lifeless> cool
[11:00] <wgrant> Not great.
[11:00] <bigjools> lifeless: you do a really bad impression of someone who's on holiday :)
[11:00] <wgrant> But I will look at that more when I don't have to pester too many people.
[11:00] <lifeless> bigjools: I spent all day playing WoW :)
[11:00] <bigjools> yes I know someone else who took holiday to do that :)
[11:00] <lifeless> bigjools: the reason I take big blocks of holiday is it takes me ages to spin down and forget about work.
[11:00] <lifeless> bigjools: 2 weeks in is about where I actually start to feel like I'm on leave
[11:00] <bigjools> maybe disconnecting from IRC would help? :)
[11:01] <lifeless> bigjools: I have many non work interests I wish to be on IRC for
[11:01]  * wgrant searches for the banhammer.
[11:09] <wgrant> bigjools: New tests are up, this time without conflicts.
[11:09] <bigjools> bonus
[11:14] <jml> lifeless: last time I went on leave, I also stayed on IRC but deliberately left many work-related channels.
[11:14] <jml> lifeless: I found that helped
[11:15]  * jml isn't really here
[11:45] <henninge> Does anybody have an idea what this is about?
[11:45] <henninge> http://paste.ubuntu.com/541413/
[11:48] <maxb> traceback header with no traceback? thats a bit odd
[11:50] <wgrant> maxb: Is there a librarian running?
[11:50] <wgrant> Er, henninge ^^
[11:51] <henninge> wgrant: oh, you mean an old process?
[11:51] <henninge> yes :(
[11:51] <wgrant> Hmm.
[11:51]  * henninge slaps head
[11:51] <wgrant> Looks like the new librarian fixture needs work.
[11:52] <henninge> yes, possibly. But I just remembered that I had this issue before after forcibly terminating a test run.
[11:53] <henninge> but I agree that something like this should be detected
[11:54] <henninge> Tests are running happily now :)
[12:08] <deryck> Morning, all.
[14:15] <gary_poster> bigjools: I get those emails somehow or other
[14:16] <bigjools> gary_poster: maybe you're subscribed to lp bug
[14:16] <bigjools> s
[14:16] <gary_poster> maybe so
[15:31] <benji> mthaddon: I'm working on the script to run the LP tests in the new tarmac-based world and would like to write up how to set up the environment it needs; do you know of any server setup documentation for the current buildbot machines?
[15:31] <mthaddon> benji: otp
[15:31] <benji> mthaddon: thanks, no rush
[15:47] <EdwinGrubbs> allenap: hi
[15:56] <allenap> EdwinGrubbs: Hi there.
[15:57] <allenap> EdwinGrubbs: I wrote down a kind of rationale for BugSubscriptionInfo: http://pastebin.ubuntu.com/541503/
[15:57]  * EdwinGrubbs looks
[15:59] <EdwinGrubbs> allenap: by "snapshot value factory", do you mean that getDirectSubscriptions() might return an existing value?
[16:00] <EdwinGrubbs> allenap: what do you think the difference is between this solution and pre_iter_hook? Are you planning on adding more stuff to the BugSubscriptionInfo, which I just don't get to see yet?
[16:02] <allenap> EdwinGrubbs: BugSubscriptionInfo is designed to work with a bug snapshot, so that (Zope) subscribers can find out deltas in subscriptions without having to issue many queries.
[16:03] <EdwinGrubbs> allenap: do you think that there would be a case in the future that you would need to batch the results of getDirectSubscriptions?
[16:03] <allenap> EdwinGrubbs: Some big queries are being hit 7 or more times in a single page request, when they only need to be executed twice.
[16:03] <allenap> EdwinGrubbs: I doubt we'll ever have that many subscribers.
[16:03] <allenap> direct subscribers.
[16:05] <allenap> EdwinGrubbs: What would the pre_iter_hook do in this case?
[16:08] <EdwinGrubbs> allenap: oops, I meant that pre_iter_hook would eliminate the need for BugSubscriberSet, not BugSubscriptionInfo. pre_iter_hook would take the place of the self.subscribers call made before returning BugScriptionSet.sorted.
[16:10] <EdwinGrubbs> btw, those are two sentences, although it looks like the period is just separating a class and an attribute.
[16:14] <allenap> EdwinGrubbs: Okay :) Sorry for being slow, my other half is badgering me for things at the same time.
[16:15] <EdwinGrubbs> no problem
[16:21] <allenap> EdwinGrubbs: A pre-iter hook when fetching the subscriptions would work, so it would mean 2 queries when fetching subscriptions. Might not be a bad thing though, because the person/subscriber is almost always referenced.
[16:22] <allenap> EdwinGrubbs: That would make the implementation of BugSubscriptionSet and StructuralSubscriptionSet a bit simpler, but only by the same amount that BugSubscriptionInfo gets more complex.
[16:23] <allenap> EdwinGrubbs: I don't think it removes the usefulness of BugSubscriberSet, because that's useful for its sorted property.
[16:32] <EdwinGrubbs> allenap: sorry, the standup got delayed to just now.
[16:33] <allenap> No worries :)
[16:35] <EdwinGrubbs> allenap: the potential benefit of pre_iter_hook that may not apply here is that it only runs the main query and the pre_iter_hook query when it is iterated over. The BugSubscriptionSet runs the first query immediately to turn it into a frozenset, and then it runs the second query if the BugSubscriptionSet.sorted is used.
[16:36] <EdwinGrubbs> allenap: I'm wondering what the benefit is to putting the results in a set and then putting them into a sorted list. Do you use the set directly someplace else?
[16:39] <allenap> EdwinGrubbs: It's necessary to identify different subsets of subscribers. There's the obvious ones like direct and duplicate subscribers, but then there are duplicate-only subscribers (which I've added in a later branch), also notified subscribers, indirect subscribers. Many of these are remixes of the simpler subscription/subscriber sets.
[16:40] <allenap> EdwinGrubbs: I could do these as UNION/EXCEPT queries, but the complexity shoots up.
[16:42] <allenap> The reason it shoots up is because there are BugSubscription records which are not compatible with StructuralSubscription records, and because some of the queries involve joining through bugtask to target (i.e. product or productseries or ...).
[16:51] <EdwinGrubbs> allenap: ok, I don't think I have any more questions about the sets. It just was bothering me to have an unusual implementation solve such a similar problem.
[16:52] <EdwinGrubbs> allenap: I have a branch in pqm that effectively implements the same thing as load_people(). I'll not that in the mp with a few other small items. Thanks for explaining everything to me.
[16:53] <EdwinGrubbs> s/not/note/
[16:54] <allenap> EdwinGrubbs: Cool. I agree it's weird. I've tried something like this refactor before and a lot of subtle issues seem to crop up, so I started from the premise of having a sets-based API so that I could understand myself what was going on as I went along.
[16:56] <gary_poster> jml, ping
[16:57] <jml> gary_poster: hi
[16:58] <gary_poster> hey.  Could matsubara and I pick your brain on mumble for a few minutes about some subunit/ec2 remote.py details?
[16:58] <gary_poster> sorry, jml ^^^
[16:59] <jml> gary_poster: sure. gimme a few seconds first.
[16:59] <gary_poster> of course, thanks
[17:05] <benji> mthaddon: when you get a minute; I have the script ready that will be the driver for the tests in the new tarmac-based SMM world; I think you're the one I should talk to about getting it installed
[17:06] <mthaddon> benji: I'm past EOD I'm afraid - can you add the details to the RT?
[17:07] <benji> mthaddon: gladly; I haven't done anything with RTs yet, how do I do that?  (is there something I should read on the RT system?)
[17:10] <mthaddon> benji: I've added you as a cc to the ticket, and have sent you a reply - you should be able to just reply to the email
[17:10] <benji> mthaddon: great, thanks
[17:17]  * jml gone
[19:03] <lifeless> gary_poster: jml has run, can I help ?
[19:27] <gary_poster> lifeless, he was able to get in touch with us before he ran.  thank you very much!
[19:31] <lifeless> ok, cool
[19:32] <lifeless> if you're doing subunit stuff, it would be cool to wire up SubunitStream ( the table ) to the API for uploads.
[19:33] <lifeless> I'd just love ec2test to add a stream to the branch it tests.
[20:16] <LPCIBot> Yippie, build fixed!
[20:16] <LPCIBot> Project db-devel build (207): FIXED in 3 hr 35 min: https://hudson.wedontsleep.org/job/db-devel/207/
[20:41] <james_w> deryck, hi, I'm interested in what you say about not liking testing pages. You say that you have talked about this previously. Was that on a public mailing list?
[20:41] <deryck> james_w, yes, several threads (2-3 maybe?) on launchpad-dev.  about ui, ajax, and testing all this.
[20:42] <james_w> deryck, ah, specifically where js gets involved?
[20:42] <deryck> james_w, mostly about that, yes.  though I do have some criticisms of our page tests in an old proposal thread of mine
[20:43] <deryck> but that's probably less about testing web pages and more about bad tests generally
[20:43] <deryck> :-)
[20:43] <james_w> down with bad tests!
[20:43] <james_w> deryck, have you had a look at soupmatchers that is now available in the LP testing infrastructure thanks to rockstar?
[20:44] <deryck> james_w, I haven't yet, no.  though rockstar did rave to me about it.
[20:44] <james_w> deryck, cool. I'd be interested in what you make of it if you find time to have a play with it
[20:45] <deryck> james_w, ok, cool.  I will definitely look at it and let you know
[20:45] <james_w> it's static html only, but I think it could be a lot better than the current approach of extracting text from elements and printing it etc.
[21:43] <LPCIBot> Project devel build (285): FAILURE in 3 hr 12 min: https://hudson.wedontsleep.org/job/devel/285/
[21:43] <LPCIBot> * Launchpad Patch Queue Manager: [r=jml][ui=none][no-qa] Update testtools to r151.
[21:43] <LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none][no-qa] Use the stdlib multiprocess module to
[21:43] <LPCIBot> run WADL generation in parallel subprocesses.
[21:43] <LPCIBot> * Launchpad Patch Queue Manager: [r=adeuring][ui=none][bug=638924] Fixed timeout on ubuntu milestone
[21:43] <LPCIBot> +index page.
[21:43] <LPCIBot> * Launchpad Patch Queue Manager: [r=gmb][ui=none][bug=677511] Fix "invited members" link created by
[21:43] <LPCIBot> javascript.
[21:43] <LPCIBot> * Launchpad Patch Queue Manager: [r=allenap][ui=none][bug=193870] The bug subscriptions portlet for
[21:43] <LPCIBot> StructuralSubscriptionTargets has been updated to improve readability.


[02:33] <bignewb> Hi.  I have a usb audio interface and one of the things im interested in doing is automatically controlling mixers from a midi interface. For example one scenario I want is, a single mic input, and two separate buttons which selects two different outputs, or another button will change my headphones to split two different channels on left/right. Is this something Ardour can do for me? Thanks!
[02:34] <bignewb> lua maybe?
[14:29] <akik> hi, do you have somewhere a list of all audio packages that are included in ubuntu studio?
[15:26] <learnOPENtechBEC> OK Eickmeyer , on behalf of learnOPENtech, we put some cash in the Ubuntu Studio pot to help support the project!
[15:49] <Eickmeyer> learnOPENtechBEC: Thanks!
[15:49] <Eickmeyer> akik: `sudo apt-cache depends ubuntustudio-audio ubuntustudio-audio-core ubuntustudio-audio-plugins` would give the answer.
[15:50] <Eickmeyer> Unfortunately, it can change to frequently to keep a list.
[15:51] <akik> Eickmeyer: i don't have it installed, but i just read your web page and it didn't have that list
[15:51] <Eickmeyer> akik: If you have Ubuntu installed you can do that command.
[15:51] <akik> Eickmeyer: ok i have one vm
[15:51] <akik> thanks
[15:52] <Eickmeyer> akik: Any flavor of Ubuntu can do that command and get the list. No flavor of Ubuntu is allowed to have anything outside of what's already in the Ubuntu repositories.
[15:53] <Eickmeyer> Website is largely out of date. Time is an issue.
[15:54] <akik> Eickmeyer: ok i get the list. do you know if kx.studio has some packages that are not included in ubuntu studio?
[15:54] <Eickmeyer> They do, but....
[15:54] <Eickmeyer> !kxstudio
[15:55] <akik> Eickmeyer: i used it with ubuntu mate earlier and didn't notice any problems
[15:55] <Eickmeyer> YMMV. We just don't support it and immediately stop helping people if they have problems after installing it.
[15:55] <akik> ok thanks for the info
[17:26] <bignewb> ardour mixer scenes are pretty cool


[05:02] <JoseeAntonioR> jcastro: announcement made: http://joseeantonior.wordpress.com/2012/07/17/3-2-1-live/
[18:30] <czajkowski> http://blog.launchpad.net/general/beta-test-asynchronous-ppa-package-copies
[18:32] <JoseeAntonioR> czajkowski: would you like me to include it in the UWN?
[18:40] <czajkowski> JoseeAntonioR: most lp blog posts end up there
[18:40] <czajkowski> thanks
[19:11] <jono> mhall119, would you mind redditing my recent blog entry?
[19:19] <mhall119> JoseeAntonioR: sure
[19:19] <JoseeAntonioR> mhall119: jono has already quit
[19:19] <mhall119> meant for jono, tab-complete fial
[19:21] <IdleOne> fail fail
[19:21] <mhall119> upvotes appreciated: http://www.reddit.com/r/Ubuntu/comments/wrwv4/ubuntus_jono_bacon_discusses_steam_on_ubuntu/
[22:23] <bkerensa> mhall119: jono is here with me do you need to get a message to him?
[22:25] <mhall119> bkerensa: um, no
[22:25] <mhall119> bkerensa: oh, yes, let him know I sent his blog to reddit like he asked
[22:26] <bkerensa> kk
[22:26] <bkerensa> mhall119: I won a helicopter
[22:26] <bkerensa> :D
[22:26] <mhall119> a real one?
[22:27] <mhall119> and does it run Ubuntu?
[22:28] <bkerensa> mhall119: uhh an electric one... and no but Im gonna paint it orange and put a ubuntu logo sticker on it and fly it at oscon
[22:28] <bkerensa> ;)
[22:29] <mhall119> You should write an app for Ubuntu that will let you control it with jkl; or awsd
[22:29] <mhall119> if jkl;, you can re-name it the vimcopter
[22:31] <balloons> bkerensa, you have to put wireless video on it!
[22:32] <pleia2> delayed out of SFO, perhaps I should start walking :)
[22:32] <bkerensa> pleia2: your stuck at SFO?
[22:32] <bkerensa> =/
[22:33] <pleia2> yeah, was supposed to leave at 4, now they're saying 5:30
[22:33] <pleia2> but I found power, and there is wifi here :)
[22:34] <bkerensa> pleia2: 5:30pm so you will be here at 7pm?
[22:34] <bkerensa> :D
[22:34] <pleia2> bkerensa: "here" as in the airport, then get back, check in my hotel (which is also at the airport)
[22:35] <pleia2> I need to get up at like 6 to switch hotels to something downtown, so I am turning in early
[22:35] <pleia2> s/get back/get bag
[22:36] <bkerensa> pleia2: ok well if u change your mind there are three parties tonight :)
[22:36] <pleia2> maybe tomorrow night :)
[22:36] <pleia2> (parties aren't my thing anyway)


[06:50] <lunatiq> whats a method to protect against DDoS on ubuntu 22.04 LTS
[07:13] <lunatiq> https://devanswers.co/how-to-get-php-mail-working-on-ubuntu-16-04-digitalocean-droplet/ It was suggested to install this to enable mail() in PHP
[07:13] <lunatiq> do I setup a domain for "internet site"
[07:54] <rfm> lunatiq, honestly, what are you trying to accomplish?  You surely do not appear to be clueful enough to run a wide-open mail server, shouldn't you just let google or somebody do it for you?  I do.
[16:31] <rbasak> blackboxsw: o/ on https://github.com/canonical/cloud-init/commit/8f569d45111dce368f92883e35bfbb40bbb86091, shouldn't this also be the default behaviour any time the instance-id changes, rather than being limited to a CLI command? IOW, shouldn't there be a module on this, just like cc_ssh.py?
[16:31] -ubottu:#ubuntu-server- Commit 8f569d4 in canonical/cloud-init "clean: add param to remove /etc/machine-id for golden image creation"
[16:31] <rbasak> IOW, AIUI "golden image creation" should continue to be the default and automatic cloud-init behaviour - golden images shouldn't need to be special or declared as such, cloud-init allows any image to be a golden image without special configuration.
[17:32] <blackboxsw> rbasak: instance-id changes could be used to indicate some smaller config change such as network config has changed due to new routes being added or a device hotplug event due to LXD system config changes or any number of cloud-native config changes. At some point instance-id changes will indicate whether  storage is added/removed. Some of these cases don't warrant rewriting /etc/machine-id (it's a pretty big hammer).
[17:38] <blackboxsw> This specific corner case which warranted this parameter was someone manually preparing an image installed via the Ubuntu Desktop Live installer and Server Live installer (subiquity-based) for re-use as a golden image in other cloud platforms (VMWare). Specific to the reused of the live installer, the 'golden image' may be running on bare metal or an env with no instance metadata, or inability to change instance-id in config.
[17:39] <blackboxsw> In those cases, we suggest people redact /etc/machine-id to avoid surprises on cloned image boot so allow systemd to take care of that setup.
[17:42] <rbasak> blackboxsw: isn't the hammer already used with ssh host key regeneration?
[17:42] <rbasak> My impression was that it's a fundamental design principle of cloud-init that every image is a golden image. Is that no longer the case?
[17:43] <rbasak> I get that running outside a cloud environment that can identify to cloud-init that it's a clone will require manual intervention.
[17:44] <rbasak> I'd just like to see the behaviour when it *is* in such an environment not disappear.
[17:44] <blackboxsw> that said, I believe most cloud images already handle /etc/machine-id configuration across the "clone my vm" operation to ensure that machine-id is regenerated across clone image reboot.
[17:44] <rbasak> What's the mechanism used for that? That's what I couldn't find.
[17:44] <rbasak> So I assumed that it's not happening, but perhaps I'm wrong?
[17:45] <blackboxsw> blackboxsw: isn't the hammer already used with ssh host key regeneration? Yes true, when instance-id delta is triggered by IMDS, hostkeys are regen'd and the instance is considered new and configured as such.
[17:48] <blackboxsw> in cases where IMDS is not longer available after reboot (sandboxing a VM, bare-metal with no longer a NoCloud or DataSourceNone configuration available, ConfigDrive no longer attached) cloud-init won't be activated at systemd-generator timeframe so this cloud-init clean --machine-id helps in that environment to allow reuse of those images across that tranansition.
[17:48] <blackboxsw> My impression was that it's a fundamental design principle of cloud-init that every image is a golden image. Is that no longer the case?   (Correct in that every first boot with new instance-id detected is a golden image that needs full configuration)
[17:52] <blackboxsw> And currently that's how LXD datasource behaves on Jammy++ for many `lxc config set` configuration changes. the instance re-runs and regenerates SSH host-keys across that instance-id delta. Though, I guess I'm trying to present cases where I don't think cloud-init should re-configure everything and reset SSH host keys, create users, set passwords. I think we'll need to define granularity there to avoid cfg re-run where unneccessary
[17:55] <rbasak> blackboxsw: what you just said matches my expectation I think, but then shouldn't machine-id be reset any time ssh host keys are?
[17:56] <blackboxsw> rbasak: sorry was talking out of both sides of my mouth. The future desired case w/ scoped reaction to only subsets of changed IMDS data, vs current state of the world.
[17:59] <blackboxsw> rbasak: so, long conversation short. Yes, I think that is the case. The an instance-id delta from IMDS is currently equivalent to the IMDS indicating that this is a full new instance/machine/configuration and that all cloud-init setup needs to be performed as if it were a brand new image.
[18:00] <blackboxsw> rbasak: so yes, I think you are right, we want to build in that logic as well for EventType.NEW_INSTANCE_ID  (IMDS instance-id delta) to also clean/redact the /etc/machine-id
[18:02] <rbasak> blackboxsw: OK! Would you like a bug? :)
[18:06] <blackboxsw> rbasak: also note the above behavior of removing /etc/machine-id is actually buggy because on some non-systemd distros will regenerate /etc/machine-id when the file is absent, but most systemd environments rely on   systemd-machine-id-commit.service which depends on the presence of `ConditionPathIsMountPoint=/etc/machine-id
[18:06] <blackboxsw> `
[18:06] <blackboxsw> here's a PR I hope to land today tomorrow related to this bug https://github.com/canonical/cloud-init/pull/1953
[18:06] -ubottu:#ubuntu-server- Pull 1953 in canonical/cloud-init "machine-id: set to uninitialized to trigger regeneration on clones" [Open]
[18:07] <blackboxsw> rbasak: I think a separate bug is good thanks Robie. I'll followup with a separate fix to generalize that functionality and use it across instance-id deltas
[19:08] <lemoi> is there a way to statically IP ubuntu server 20.04.5? I'm trying to do an automated install via packer and I'm lost
[19:08] <lemoi> I need to statically IP the installation medium and then point it to meta-data + user-data
[19:11] <sarnold> lemoi: maybe this? https://cloudinit.readthedocs.io/en/latest/reference/network-config-format-v2.html
[19:19] <lemoi> I hope it works in 20.04.x
[19:20] <lemoi> ideally I'd want to use vanilla install media and IP it through command line parameters. Is this doable?
[19:21] <sarnold> you can certainly run the ip command in your own scripts if you wish, but netplan is right there..
[21:19] <rbasak> blackboxsw: I filed bug 2003121, and also found a case where the bug is quite severe - cloud-init "golden images" don't at all on Ubuntu + cloud-init + libvirt at the moment.
[21:19] -ubottu:#ubuntu-server- Bug 2003121 in cloud-init "machine-id is not reset when instance-id changes" [Undecided, New] https://launchpad.net/bugs/2003121
[21:19] <rbasak> don't *work* at all
[21:49] <falcojr> blackboxsw: I thought we decided the other day that we DON'T want cloud-init doing golden image creation
[21:50] <falcojr> there's a lot more than just resetting the instance id / machine id, and we don't do any of it currently
[21:55] <rbasak> falcojr: my understanding was that part of the principle of cloud-init's design was to do all the things necessary. Did that change?
[21:56] <falcojr> rbasak: I'm not aware of that expectation
[21:57] <Odd_Bloke> cloud-init's instance ID behaviour enables the use of ~any image as a "golden" image, without a "creation" step.  It's not incompatible, IMO, to say cloud-init shouldn't create golden images but should reset machine-id when it detects a new instance.
[22:02] <Odd_Bloke> falcojr: To clarify: you aren't aware of the expectation that cloud-init should enable launching fresh instances from images(/instance snapshots) which have not been specifically prepared for that purpose (i.e. non-golden images), or you aren't aware of an expectation that cloud-init should support preparing golden images?
[22:03] <falcojr> Odd_Bloke: that last one...support preparing golden images
[22:03] <Odd_Bloke> (I believe the former to be a crucial part of cloud-init's utility, and I don't think any commitment to the latter has been made except that `clean` might be helpful if someone is doing that.)
[22:04] <falcojr> I agree about the former
[22:04] <rbasak> What's the difference between the two cases?
[22:04] <rbasak> In the former case, every VM that uses cloud-init is a golden image.
[22:05] <rbasak> The only benefit of a specific "make this a golden image" command is to clean up for size purposes.
[22:05] <falcojr> blackboxsw had a list of things we don't want to do the other day that I don't remember at the momemt. One example I can think of is we don't clean the journal logs on cloud-init clean
[22:05] <falcojr> (but we do clean cloud-init's logs)
[22:05] <rbasak> Or for an environment where instance-id isn't available I suppose, sorry.
[22:05] <falcojr> ((if you pass --logs))
[22:11] <blackboxsw> sry catching up. falcor, Correct per cloud-init doing golden image creation, we don't want cloud-init to boil the ocean and be "the tool" to perform golden image creation because there are a lot of config or first boot artifact to cope with outside the scope of cloud-init.
[22:12] <blackboxsw> We want cloud-init to be able to provide some semblance of tooling to help support folks who may be trying to perform some semblance of golden image creation themselves to prepare such an image for reuse as a golden image.
[22:14] <blackboxsw> cloud-init also can't easily clean up all artifacts on the system due to first boot (nor should it).   Things like user creation, ssh keys imports, host keys laid down, any runcmd or bootcmd artifacts executed during initial vm/instance  boot.
[22:15] <Odd_Bloke> rbasak: To my mind, an image is only golden if it's been specifically prepared for use as an image.  So using cloud-init enables you to launch _non-golden_ images in a similar fashion to golden images.  But if an instance has, say, Postgres installed with some databases, it's ~impossible for cloud-init to know if "golden" for that means keep the databases or trash them etc.
[22:16] <rbasak> Odd_Bloke: sure. That sounds like a difference in the definition of "golden" only.
[22:16] <rbasak> I think we agree, but to be clear, my view is that a golden image is a golden image if the person who created it says it is. cloud-init helps with that by dealing with all the pitfalls on the OS side, such as ssh host keys and (when the bug is fixed) /etc/machine-id.
[22:17] <blackboxsw> I recall end of last week we chatted about potentially starting a discussion with CPC team as they have a lot more goodness, tooling and knowledge when it comes to golden image creation required by official cloud images. The intent is to see if we might work with them to help provide a utility that may have more broad usefulness that could help assist community members interested in creating their own images .
[22:17] <rbasak> OTOH, a golden image is not a golden image if, when booted, it's broken, such as with a duplicate ssh host key or /etc/machine-id.
[22:18] <rbasak> So maybe I should correct myself and say that a golden image is a golden image if the person who created it says it is, _and_ it isn't otherwise broken like that. cloud-init's job IMHO is to ensure the latter for all cases where the OS does something that would make a clone non-golden.
[22:19] <Odd_Bloke> Right, I agree that we agree: I guess my point is that this isn't a problem with _golden_ images, it's a problem with _all_ images.
[22:20] <rbasak> Only if you clone though, right? I can't imagine a reason to clone in a non-golden case.
[22:20] <falcojr> I think there's still a distinction I'm not understanding, but I don't see how having a stale ssh host key is any more broken than a stale username and user key and cloud-init doesn't clean those up on new instance creation
[22:21] <falcojr> and I'm also not sure we intend "new instance id" == "golden image"
[22:21] <Odd_Bloke> An SSH host key is necessarily stale on a new instance, a username/key are not necessarily stale on a new instance.
[22:22] <rbasak> falcojr: a stable ssh host key is a security vulnerability. For example, a well known hosting company provided OS images that all had the same ssh host key when they launched, meaning that their customers could MITM each other.
[22:22] <falcojr> not necessarily on a new instance...in fact it's one of the biggest source of frustration for our users
[22:22] <blackboxsw> So, for cloud-init's role we can provide simple entry points that help support one-off cases for manual image creation and reuse. If someone is using cloud-init and already knows they want to trigger cloud-init to manually re-run 'from scratch' on next boot, they are already told to run `sudo cloud-init clean --logs`.
[22:22] <falcojr> they get assigned a new instance id when the machine didn't change and now they have new host keys
[22:22] <rbasak> OTOH, a known user with a given authorized_keys might be _intentionlly_ part of a golden image.
[22:23] <sarnold> just in case [redacted] needs to debug an ubuntu-core system in the field? :)
[22:23] <rbasak> If the instance id changed but the user didn't intend the machine to change, then perhaps instance id from that particular cloud provider is the wrong identifier to use to detect when the machine has been cloned, but the principle remains.
[22:24] <rbasak> sarnold: :)
[22:24] <rbasak> sarnold: but seriouly, within a particular org it might be a perfectly reasonable thing to do.
[22:24] <blackboxsw> If they want to ensure that image will come up with an unique machine-id, it's easy to extend cloud-init for that element of "preparing your image" but cloud-init will probably not grow a full suite of features to tackle all  systemd recommendations for https://systemd.io/BUILDING_IMAGES/
[22:24] <sarnold> *nod*
[22:25] <rbasak> blackboxsw: I think maybe it should
[22:26] <rbasak> I haven't reviewed each of those in detail, but the principle behind cloud-init's design is that it is supposed to take care of those things, AIUI.
[22:26] <rbasak> The principle being that if its identity has changed, it's a clone and it should adapt correctly.
[22:28] <rbasak> If instance-id changes when network config changes, that's either a) wrong; or b) an environment where instance-id is no longer an indicator of identity, so cloud-init cannot work as designed any more unless that's provided some other way.
[22:28] <rbasak> I mean: "instance identity" should be a clue :-P
[22:29] <rbasak> "This instance has a different identity" -> "OK, so the ssh host keys should be different then" seems entirely reasonable.
[22:29] <rbasak> And same for /etc/machine-id, etc.
[22:31] <falcojr> rbasak: that's not how it works today, and I think that would need to be a larger discussion
[22:31] <falcojr> I don't see why cloud-init couldn't grow that functionality, but I disagree that that is the principle behind cloud-init's design
[22:31] <falcojr> (unless smoser wants to pop in and correct me XD )
[22:33] <Odd_Bloke> `/var/lib/cloud/instances` exists for this reason: so that the same filesystem can be used as multiple different instances, as identified by the underlying cloud platform.
[22:36] <blackboxsw> rbasak: I think the broader conversation of whether  cloud-init needs to prune things well beyond the scope of cloud-init needs to be had w/ CPC and Foundations before cloud-init signs up for that level of feature/scope for "golden image" preparation.  Maybe this is something that needs broader adoption as a separate tool/script delivered in ubuntu-dev-tools?  Also, are there images which don't have cloud-init installed that would
[22:36] <blackboxsw> need a more broad tool/utility to support golden image setup?
[22:38] <rbasak> blackboxsw: for the "every image with cloud-init should be able to safely and correctly boot with a new identity" use case, your statement sounds odd to me because I've taken that to *be* the scope of cloud-init all along.
[22:39] <rbasak> That doesn't include cleaning everything, to be clear. Just the things that would be broken to carry forward, such as security keys, machine-id, etc.
[22:40] <rbasak> For images that don't have cloud-init installed, my answer would be "use cloud-init; that's what it's for".
[22:52] <blackboxsw> rbasak falcojr, Odd_Bloke I'm not sure if we are talking past each other or maybe falcojr responded to an earlier aspect of the discussion when he said "I think that would need to be a  larger discussion").   Do we think that instance-id delta from a cloud datasource is sufficient to indicate that this is a new system "enough" that we don't want cloud-init to also uninitialize machine-id when instance-id changes?
[22:54] <blackboxsw> the case I'm wondering about is services like Canocial livepatch  or Ubuntu Pro which may take into account the current machine-id on the filesystem, if there are clones launched from that filesystem (which would receive a new instance-id) I'd potentially like cloud-init to perform the work to purge/regenerate that machine-id during boot to ensure other services which key on /etc/machine-id are aware of such an identity change
[22:55] <rbasak> blackboxsw: I'm not sure I follow your question. I think that if instance-id changes, machine-id should be reset, although with anything else needed.
[22:56] <rbasak> If there's a case where instance-id changes where you don't think that should happen, then I'd like to hear more about that case.
[22:56] <rbasak> *along* with anything else needed
[23:02] <blackboxsw> I think when an instance-id changes in metadata cloud-init loks at, host keys should be regenerated (already done), and I do think we should think about regenerating machine-id across that boundary.  Your "anything else needed" comment I'd like to limit to only things attributed strictly to identity of that host.
[23:07] <rbasak> blackboxsw: from your link, would you include /var/lib/systemd/random-seed and /var/lib/systemd/credential.secret?
[23:18] <blackboxsw> rbasak: I think this is asking a lot that cloud-init would have to grow into if we decide it's the right place to do this. Ideally I'd want cloud-init to invoke an  external tool or script to perform this "image clean" there was a product cpaelzer/holmanb referenced virt-sysprep as a tool for performing 'full clone image creation' and this really can only be done safely on an inert image with the host stopped.
[23:24] <blackboxsw>  https://www.libguestfs.org/virt-sysprep.1.html#copying-and-cloning as a point of reference.
[23:28] <blackboxsw>  But, cloud-init shouldn't be that broad a utility at runtime to ensure we fix  everything related to a hosts identity, cleanse various service logs, etc by default. I think we *can* do a best effort in scope of cloud-init where there is friction because a cloud is telling us via instance-id that identity needs to change. It would be nice though, if cloud-init could leverage some external tool in the OS itself.
[23:29] <blackboxsw> I want to note as well that pulling in libguestfs-tools  as a dependency of cloud-init is a non-starter due to the sheer number of dependencies it also pulls in
[23:29] <rbasak> blackboxsw: cleaning service logs is scope creep. I'm not asking for that.
[23:30] <blackboxsw> +1 thx
[23:30] <blackboxsw> yet so is /var/lib/systemd/random-seed and /var/lib/systemd/credential.secret :)
[23:30] <rbasak> I agree that's work, but I think they fit in the same category as ssh host keys.
[23:31] <blackboxsw> yes, I think it's worth a spec for broader discussion w/ CPC/Foundations too. We'll tag you on it and can reflect the conversation on that here too for others.
[23:33] <blackboxsw> I think we have to be certain what we think cloud-init should automatically be in charge of as far across instance-id delta because this is processed after reboot, not before reboot so certain services may be in flight trying to consume those artifacts that cloud-init is changing out from under them
[23:34] <blackboxsw> the machine-id uninitializing is something safely performed prior to the eventual reboot of that machine image, so we aren't pulling the rug out from under any service which may have just started
[23:34] <rbasak> Yeah the DHCP case is an interesting one
[23:34] <rbasak> With networkd using machine-id to derive the DHCP client identifier.
[23:35] <blackboxsw> sorry to clarify: the cloud-init clean --machine-id cmd is performed pre-reboot. So it's safer than at runtime
[23:35] <blackboxsw> right on DHCP
[23:35] <rbasak> That's not the use case I'm interested in though.
[23:35] <rbasak> I want the "just boot it and cloud-init will do the right thing" use case to work, without a pre-boot step.
[23:40] <falcojr> That category feels very squishy to me. What makes a new instance broken? Ask 10 different people and you'll probably get 10 different answers.
[23:45] <rbasak> Like all bugs :)


[14:11] <DiogoConstantino> hi all
[14:24] <lotuspsychje> welcome DiogoConstantino
[14:32] <Fallen> Good to see you Diogo! :)
[14:34] <arraybolt3[m]> o/
[14:54] <rs2009> hey DiogoConstantino :)
[20:16] <Fallen> Do we have the criteria for becoming an Ubuntu Community Council documented somewhere? I thought I saw this on a page, but I cannot find it anymore.
[21:19] <Eickmeyer[m]> Fallen: If no criteria is listed, then the default is Ubuntu Membership. However, as you'll see on the governance, pages, Community Council is special in that it's also shortlisted by Mark.
[21:19] <Eickmeyer[m]> s/,//
[21:20] <Eickmeyer[m]> https://discourse.ubuntu.com/t/governance/332
[21:22] <Eickmeyer[m]> Technically, the nomination is up to Mark, but in the previous cycle he wanted the members to nominate and he'd shortlist.
[21:23] <Eickmeyer[m]> Hence, we're using that as precedent.