[00:00] <infinity> hloeung: Apparently, nihal needs filecache-default scrubbed.
[00:00] <infinity> hloeung: However...
[00:00] <infinity> hloeung: Before you do that, a copy of it might be nie.
[00:00] <infinity> nice, too.
[00:00] <hloeung> infinity: just filecache-default? or the build-*'s as well?
[00:01] <infinity> hloeung: Are there more than one build-*?
[00:01] <hloeung> infinity: yes, there's 3 actually
[00:01] <infinity> Oh.
[00:01] <infinity> Fun.
[00:02] <infinity> hloeung: In that case, yeah.  Stop lp-buildd, clear out the whole mess.  But I'd like a backup of filecache-default, if you could?
[00:03] <hloeung> infinity: yep, taring up filecache-default
[00:03] <infinity> hloeung: On lillypilly or chinstrap, or somewhere I can get to it readily.
[00:06]  * infinity wonders if anyone in the Bay Area wants to bring him a power cord for his laptop, since he stupidly forgot his at home.
[00:06] <infinity> Don't all speak up at once.
[00:19] <smoser> is there a reason that language-pack-en-base Recommends firefox-locale-en ?
[00:19] <smoser> its really only an annoyance, but installing a language pack on a server then pulls in firefox packages
[00:19] <smoser> (well, not really firefox packages, but firefox-locale-xx)
[00:20] <infinity> You could install with --no-install-recommends
[00:20] <JanC> you mean you didn't configure that as the default?  ;)
[00:21] <infinity> But the relationships for langpacks have always been a bit sketchy, because it's hard to adequately represent what we want to happen there.
[00:21] <cjwatson> Partly it's because IIRC that used to be in language-pack-en but was split out.
[00:22] <infinity> Mozilla locales have always been separate.  Unless you meant the dependency used to be elsewhere.
[00:22] <infinity> At least, I think they always have been...
[00:23] <infinity> But really, no amount of Enhances or Recommends magic can make it DTRT for all people.
[00:23] <cjwatson> I may just be misremembering.
[00:23] <infinity> The per-package mini langpack spec would solve the issue, but I so think it would cause more problems than it solves. :P
[00:24] <smoser> cjwatson, you pointed at python-apt earlier, do you know if i can use it to tell me if a package is installed without having cache available ?
[00:24] <cjwatson> You generally need to instantiate a cache to find out much of anything.  Why don't you want to?
[00:24] <infinity> You could just use dpkg...
[00:24] <infinity> (or dpkg-query)
[00:25] <cjwatson> dpkg's own interfaces for this are actually kind of crappy, so I understand preferring to ask apt.
[00:25] <infinity> Forking dpkg is going to still be less painful than spinning up an apt cache, if you didn't need the latter.
[00:25] <cjwatson> $ time python -c 'import apt; cache = apt.Cache()'
[00:25] <cjwatson> real    0m0.900s
[00:26] <infinity> Though it would be nice for libapt/python-apt to be able to give dpkg status info with no cache built...
[00:26] <cjwatson> $ time dpkg-query -W man-db
[00:26] <cjwatson> man-db  2.6.1-1
[00:26] <cjwatson> real    0m1.882s
[00:26] <cjwatson> I'd use apt if I were you :-P
[00:26] <infinity> cjwatson: I'm unconvinced that benchmark is linearly true on slower hardware.
[00:26] <cjwatson> Possibly, but dpkg has never been very quick to parse its own status file.
[00:26] <infinity> No, true.
[00:27] <infinity> apt takes nearly a minute to rebuild a cache here, though.
[00:27] <infinity> (assuming a fresh invalidation, ie: apt-get update)
[00:28] <infinity> Oh, actually, most of that is probably unzipping lists.  So, I guess it depends on smoser's definition of "no cache".
[00:28] <smoser> "no cache" really meant no /var/lib/apt/lists
[00:29] <cjwatson> I suppose you could use python-debian to parse /var/lib/dpkg/status if you were desperate
[00:29] <smoser> and i'd really hope to be able to query "is a package installed" in less than a second.
[00:29] <infinity> smoser: Well, nothing there at all?  In that case, building a cache as per cjwatson's code above would just build one with dpkg/status.
[00:29] <cjwatson> Or python-apt might have a way to do that too which might be faster, I forget
[00:29] <smoser> infinity, right. thats what i'm seeing.
[00:29] <cjwatson> smoser: Do you need to query more than one?
[00:30] <smoser> in this caase i'm only after if a package is installed. dont eed to query more than once.
[00:30] <cjwatson> Then none of the options will be significantly faster as far as I can think of.
[00:31] <cjwatson> If you're prepared to hardcode things about the packages you could check for a file they're known to ship.
[00:31] <infinity> Ew?
[00:31] <cjwatson> Or possibly check for the existence of /var/lib/dpkg/info/$pkg.list.  But if you do that you need to be very clear on how/whether multiarch will affect you; that's an implementation detail.
[00:31] <cjwatson> I mean, .list files are an implementation detail.
[00:32] <infinity> Surely, higher level abstractions of dpkg/status are the sane way to go...
[00:32] <cjwatson> Yes.  As I say, I'd use python-apt.
[00:33] <smoser> so.. the general idea here is this (feel free to tell me any portion of this is wrong)
[00:33] <cjwatson> Because normally it's already done the work of parsing the status file into a more quickly loadable cache format, that being a lot of the point of the cache.
[00:33] <smoser>  * bug https://bugs.launchpad.net/ubuntu/+source/cloud-init/+bug/859814 shows the issue we're trying to address (and links to other bugs there too)
[00:34] <smoser>  * cloud-images have not in the past had a language pack installed, resulting in stderr output when someone ssh's in with other than en_US.UTF-8
[00:34] <smoser>  * cloud-init has long had this support for setting a default host locale, and only now have we found that that didn't really  make sense without installing language-pack-XX
[00:35] <smoser>  * so we're looking to install language-pack-en in the default images, and if the user asks cloud-init for "configure my default locale to french" we'll want to install language-pack-fr
[00:35] <cjwatson> If you're trying to test whether a given locale is installed, the right way to do it is:
[00:35] <cjwatson> import locale
[00:35] <smoser> i want to ask "do i have language-pack-fr" installed before doing an apt-get update and 'apt-get install language-pack-fr'
[00:35] <infinity> smoser: Now, this is a slightly different direction, but do these VMs typically actually need a langpack installed?  Cause if it's just about annoying error messages from shell/perl/etc, you could localegen the world. :P
[00:35] <cjwatson> try:
[00:36] <smoser> infinity, you can't just use localegen.
[00:36] <cjwatson>     locale.setlocale(locale.LC_ALL, '')  # or a specific locale as the second argument if you don't want to take it from the environment
[00:36] <smoser> unfortunately. :)
[00:36] <cjwatson> except locale.Error:
[00:36] <cjwatson>     # take steps to install the locale
[00:36] <smoser> https://bugs.launchpad.net/ubuntu/+source/postgresql-9.1/+bug/969462
[00:37] <infinity> smoser: Since when?  I do it all the time.  Doesn't buy you translations (obviously), bet gets you a valid locale.
[00:37] <cjwatson> so you could easily touch the apt cache only in the locale.Error slow path there
[00:37] <smoser> well, ok. maybe you can do that i guess.
[00:37] <cjwatson> infinity is correct too
[00:37] <smoser> hm..
[00:37] <infinity> cjwatson: And, of course, your code is tripped up by people like me, since it's testing for a locale as a key to installing a langpack, the two are only loosely related.
[00:37] <cjwatson> generating locales manually is explicitly supported
[00:38] <smoser> so it is OK to localgen even if i dont have the language-pack-xx installed ?
[00:38] <cjwatson> sure
[00:38] <infinity> Of course.
[00:38] <cjwatson> there're special arrangements to remember which ones you've locale-gen'ed manually vs. via language packs
[00:39] <cjwatson> the manual ones will end up in /var/lib/locales/supported.d/local
[00:40] <infinity> smoser: Of course, if you really do want translations as well as valid locales, you need to go the langpack route.
[00:40] <smoser> infinity, yeah. so i think i'll go the quick 'imporet locale' route to test, and if thats not there, then install the package.
[00:40] <GrueMaster> soren: ping - need some assistance with preseeding.  I have the same preseed for 3 different systems, and I am getting two different errors in partman.
[00:40] <smoser> allowing for the case when i've been trick'ed by you, infinity
[00:40] <infinity> smoser: And I really see no reason that can't just be a shell 2-liner of dpkg-query && apt-get install.
[00:40] <smoser> s/allowing/ignoring/
[00:41] <GrueMaster> Or anyone else that wants to jump into my preseed nightmare.
[00:41] <infinity> smoser: My trickery is fine.  If I'm generating locales by hand, it's because I don't want the langpack, so that seems a sane thing.
[00:41] <smoser> infinity, yeah, the reason i'm concerned is this happens on first boot.
[00:41] <cjwatson> infinity: or even just apt-get install --no-upgrade
[00:41] <smoser> and its generallyhappening serially.
[00:42] <cjwatson> but, indeed, I don't really see what installing the package is going to buy you, from your description.
[00:42] <cjwatson> it's strictly more than you actually want.
[00:42] <GrueMaster> http://paste.ubuntu.com/912292/ for anyone that wants to look.
[00:42] <smoser> so i dont really want to postpone boot by .5 seconds or more.
[00:42] <cjwatson> and language-pack-en take significantly longer, since it will generally have to generate rather more locales.
[00:42] <cjwatson> *will take
[00:42] <smoser> right
[00:42] <cjwatson> even discounting having to download and unpack it and its -base pair.
[00:42] <infinity> Yeah, the localgen in langpack-en is painful.
[00:43] <smoser> so maybe doing localgen is enough.
[00:43] <infinity> I just had a perverse idea.
[00:43] <infinity> pam module that checks the starting ENV and forks localegen for uninstalled locales.
[00:44] <infinity> Please pretend I didn't just say that.
[00:44] <infinity> Thanks.
[00:44] <smoser> yeah. i sort of looked there.
[00:44] <infinity> (It's quite easily doable...)
[00:44] <smoser> what we have now is a profile.d message that informs the user how to make the environment usable if they do want their locale
[00:45] <infinity> And would once and for all end the "I sshed from my desktop to my server, and Perl whines at me, HALP" bugs.
[00:45] <smoser> well, thtas sort of what i was addressing here.
[00:46] <cjwatson> GrueMaster: suggestion for you in bug 971608, assuming that this is the same thing.
[00:46] <infinity> It's not really any uglier than pam_userdir and other such one-time violence.
[00:46] <smoser> but i thikn i'm good enough at this point to just locale-gen, and inform the user on how to fix the issue.
[00:46] <infinity> In fact, less so, because it doesn't need to keep track of priv bouncing, it would just be a pre-session root deal.
[00:47] <infinity> Anyhow, not suggesting this as the solution for your bug in the next few weeks before release.
[00:47] <infinity> But now the gears are turning in my mind. :P
[00:47] <GrueMaster> cjwatson: Yes, I have that in my current preseed for both panda and armada systems.  Pandas are now complaining about not mounting /boot, armada is complaining about blank lvm.
[00:47] <GrueMaster> Preseeds are identical.
[00:47] <smoser> the "locale_warn" branch is at https://code.launchpad.net/~utlemming/cloud-init/cloud-init.profile.d/+merge/100262
[00:47] <cjwatson> GrueMaster: I need the precise text of the complaints in order to help
[00:47] <cjwatson> paraphrasing means I can't grep for it
[00:48] <smoser> infinity, just in case you care. and it might be updated with what i learned here. thank you cjwatson and infinity
[00:48] <GrueMaster> panda:  The attempt to mount a file system with type ext3 in SCSI1 (0,0,0), partition
[00:48] <GrueMaster> #1 (sda) at /boot failed.
[00:48] <GrueMaster> Armada:  No file system is specified for partition #1 of LVM VG eilt0, LV Test1.
[00:48] <cjwatson> I don't see method_only in http://paste.ubuntu.com/912292/ or in your bug report from earlier, by the way.
[00:48] <infinity> GrueMaster: Not ext4?
[00:49] <GrueMaster> I added the method_only after filing that bug report (found it at noon PST).
[00:49] <GrueMaster> infinity: Which?  /boot?  u-boot doesn't understand ext4.
[00:50] <cjwatson> GrueMaster: And the message you quote for Armada will be silenced by preseeding partman-basicmethods/method_only.  Please check again that you really preseeded it.
[00:51] <cjwatson> The u-boot case might want to be something like  method{ format } use_filesystem{ } filesystem{ uboot }  instead.
[00:52] <cjwatson> Oh, except partman-uboot is still dove-only, unless ogra fixed that ...
[00:52] <cjwatson> So maybe not.  That's unrelated to the blank partition problem, anyway, I think
[00:52]  * cjwatson -> really crash
[00:54] <GrueMaster> partman-uboot was dove only.
[00:56] <smoser> ok..
[00:56] <smoser> i'm doing an automated install.
[00:56] <smoser> i watch the installer get up and do some downloading of the installer packages.
[00:56] <smoser> and i'm seeding this to use a squid-deb-proxy
[00:58] <smoser> heres a snippet of my squid proxy logs
[00:58] <smoser>  http://paste.ubuntu.com/912304/
[00:58] <smoser> why is the installer doing 10 requests for Pakcages.gz ?
[00:58] <GrueMaster> groan.  To add to my issues, it would appear that my most recent preseed was clobbered due to stale vi swap.  Fortunately, I have a copy on another system.  retrying.
[00:58] <smoser> and then it dies complaining about it being corrupt
[00:58] <smoser> but i'm validating it manually locally, and it has identical checksum as reported by Release
[00:59] <infinity> hloeung: Did you restart lp-buildd on nihal when you were done tidying?
[01:00] <smoser> earlier today we thought this issue was https://bugs.launchpad.net/ubuntu/+source/squid-deb-proxy/+bug/971820 , but that change has been applied locally
[01:00] <hloeung> infinity: I've stopped it now. Am going through cleaning up build-* and the filecache-default
[01:00] <hloeung> and will start it back up when done
[01:00] <infinity> hloeung: Ahh, okay.  I was just being impatient then, nevermind. ;)
[01:01] <GrueMaster> Well, this is...different.  cjwatson:  New pastebin.  http://paste.ubuntu.com/912305/
[01:01] <hloeung> infinity: heh sorry, I'm also uploading a copy of the filecache which doesn't help. Each command is taking quite some time to send over
[01:01] <Shinobi> What is the local port for mount.cifs?
[01:01] <infinity> hloeung: No rush, I'm about to head out for food anyway, and one dead Panda won't kill the rebuild.
[01:02] <infinity> hloeung: Just poke me on IRC when it's all done, and I'll look at things when I get back.
[01:02] <GrueMaster> Now I still get the same failures as above, but on different systems.  2 pandas and armada fail to mount /boot, 1 panda now fails with "!! ERROR: No root file system
[01:02] <GrueMaster> "
[01:02] <hloeung> filecache-default.tar.bz2                      31%  137MB  94.2KB/s   53:55 ETA
[01:02] <hloeung> infinity: ^
[01:02] <hloeung> so it'll probably be done when you return...
[01:02] <infinity> hloeung: Err, are you bouncing it from your home connection?
[01:02] <hloeung> infinity: yep
[01:03] <infinity> hloeung: You could have just put it in ~buildd/public_html/ and grabbed it from another host in the DC. :P
[01:03] <hloeung> errr...   why didn't you tell me that earlier? ;-)
[01:03] <infinity> Well, not from just any host, but some have firewall rules to allow it (nusakan, lillypilly, etc)
[01:04] <infinity> And lillypilly would be a fine choice.
[01:04] <hloeung> infinity: ah, I'll remember that for next time.
[01:05] <hloeung> anyways, launchpad-buildd bounced on nihal
[01:05] <infinity> Danke.
[01:12] <infinity> Design Capacity: 22 Wh, Energy when Full: 27 Wh.
[01:13] <infinity> This is one heck of a battery.
[01:16] <micahg> infinity: any chance of helping me to get https://launchpad.net/~ubuntu-security-proposed/+archive/ppa/+build/3377300 going on roseapple?
[01:23] <infinity> micahg: You expect it to magically not fail there?
[01:23] <micahg> infinity: hoping :)
[01:24] <micahg> infinity: I got it to build locally in a 32 bit chroot
[01:25] <infinity> Yeah, the 64-bit kernel might "fix" it.
[01:25] <infinity> But, uhm.  That's not really fixing it.  Just sayin. ;)
[01:26] <micahg> infinity: yeah, there's something weird going on in that build, <= natty takes a lot of memory/space, oneiric+ takes very little (relatively)
[01:26] <micahg> infinity: I'd rather just get it built now and figure that out later
[01:27] <infinity> Already on it.
[01:28] <infinity> It's ridiculous how long that PNG optimising stuff takes.
[01:28] <infinity> pitti: ^-- Is it really that hard, or is the code just that bad?
[01:29] <micahg> infinity: also, would you mind copying chromium-browser lucid, maverick, and oneiric from ubuntu-security-proposed to -proposed
[01:30] <infinity> micahg: Can you ask me to do that after I've eaten? ;)
[01:30] <infinity> micahg: (the build is on roseapple, though)
[01:30] <micahg> infinity: thanks, sure, I'm heading out for a bit, will ping when I return
[01:30] <infinity> Kay.
[01:31]  * infinity goes to wander the streets of SF aimlessly until he smells something good.
[02:13] <psusi> bug #919281 appears to be the result of some kind of error spinning the server cd and leaving out kernel modules.  What's the correct package that should be assigned to?
[02:14] <smoser> lifeless, around ?
[02:16] <lifeless> smoser: hi, sure.
[02:16] <smoser> https://bugs.launchpad.net/ubuntu/+source/squid-deb-proxy/+bug/971820
[02:16] <smoser> i'm desparate
[02:17] <smoser> i'im hoping that you could tell me some magic "the right way" to run a squid proxy for apt.
[02:18] <smoser> i had an issue earlier today, we opened that bug, and hoped it had been addressed, but with the suggested lines:
[02:18] <smoser> refresh_pattern \/(Packages|Sources)(|\.bz2|\.gz)$ 0 0% 0
[02:18] <smoser> refresh_pattern \/Release(|\.gpg)$ 0 0% 0
[02:18] <lifeless> you can at best work around things; apt archives are not HTTP safe.
[02:18] <smoser> I've still got a squid proxy that has cached a bad version of http://archive.ubuntu.com/ubuntu/dists/precise/main/binary-i386/Packages.gz
[02:19] <smoser> well, i'm very interestedin your input on the best possible configuration.
[02:19] <lifeless> the thing is that the constant-named files have no safe cache semantics should of must-revalidate
[02:19] <lifeless> s/should/short/
[02:21] <smoser> well, honestly, i'd greatly prefer horrendous waste of network badwidth to the pain of a bad cache'd value.
[02:21] <lifeless> blacklist Releases/Sources/Packages then
[02:22] <lifeless> sorry, I don't have a good canned answer for you
[02:22] <lifeless> caching debs is fine, anything that we replace the content on has a race window *if* any other file is coordinated with it
[02:22] <hloeung> infinity: chinstrap:~hloeung/filecache-default.tar.bz2
[02:23] <lifeless> at scale, any size race is exploitable
[02:23] <smoser> lifeless, as in 'cache deny PACKAGES'
[02:23] <lifeless> yah
[02:23] <lifeless> IIRC you can force must-revalidate as well, which would be a little more efficient
[02:24] <smoser> "force must-revalidate" would be doc'd here http://www.squid-cache.org/Doc/config/refresh_pattern/ ?
[02:24] <lifeless> *note* though that you will *still* hit these races: our mirror update scripts reduce the window down to subsecond, *but* different mirrors update up to 15 minutes apart, or more.
[02:25] <lifeless> smoser: IIRC last we looked at this we determined your squid was using multiple upstreams ?
[02:25] <smoser> this one is not.
[02:26] <smoser> we have had that issue in other cases.
[02:26] <smoser> but this one has a pinned IP somewhere for me.
[02:26] <lifeless> smoser: what version of squid are you using ?
[02:26] <smoser> 3.1.19-1ubuntu1 (precise archive)
[02:26] <lifeless> so - re the upstream bug
[02:26] <lifeless> http://www.squid-cache.org/Doc/config/balance_on_multiple_ip/
[02:27] <lifeless> I think the thing is that squid by default is much more sticky
[02:27] <smoser> lifeless, fwiw, the mirror update scripts window is not subsecond.
[02:27] <smoser> even in a single mirror, its well above that.
[02:27] <lifeless> but that probably it only takes a single failure to trigger a rotate
[02:27] <lifeless> might be worth adding that to the bug
[02:27] <lifeless> smoser: really? How long does the critsection take?
[02:27] <smoser> its bad.
[02:28] <smoser> i dont have numbers in front of me, but we hit it way too often for it to be sub second.
[02:28] <lifeless> smoser: and the mirrors are using the three-run rsync approach ?
[02:28] <smoser> they're canonical run mirrors
[02:28] <smoser> i dont really know.
[02:28] <lifeless> hmm
[02:28] <smoser> https://code.launchpad.net/~smoser/+junk/check-archive/
[02:28] <lifeless> I can look into that
[02:28] <smoser> that script, run it in a loop (on canonistack if you'd like)
[02:28] <smoser> you'll hit single inconsistent mirrors way to often
[02:29] <smoser> (and by "in a loop", i mean with a 1 or 2 minute sleep)
[02:29] <lifeless> smoser: so the thing is, that either a) apt needs to send appropriate paranoid cache-busting headers, or b) apt needs to retry on inconsistent results (with appropriate headers) or c) we have to change the apt disk format.
[02:29] <smoser> lifeless, yeah.
[02:29] <lifeless> smoser: you're really at the edge of the fixes that can be done for dealing with live repositories
[02:29] <smoser> i have a list of apt greivences against apt.
[02:30] <lifeless> other mirroring scripts might be able to force it to subsecond, but even then, as discussed, at scale, it will be hit.
[02:30] <smoser> and server team hopes to have some session on that at uds-q, and we would love to have your expertise there.
[02:30] <smoser> i'd like to address it where it can be fixed for good.
[02:31] <smoser> we want to tell people to install packages and provision things in an automated fashion.
[02:31] <smoser> and when doing so, at least for the development release, the archive is the most likely cause of failure.
[02:31] <lifeless> smoser: the root cause is to fix apt's disk format: it needs to depend on only a single file being updated atomically at a time, not two files.
[02:32] <smoser> well, N files.
[02:32] <lifeless> 1
[02:32] <smoser> multiple Packages.gz and Source.gz have to be in sync with Release
[02:32] <lifeless> at the moment, Packages.gz <- Release
[02:32] <lifeless> and separately
[02:32] <lifeless> Sources.gz <- Release
[02:33] <lifeless> but this is a meaningless difference in model; the key thing is that to be safe in the absence of caching, the archive has to be fully readable at any given state, including when only one file of a group has been updated.
[02:33] <smoser> right.
[02:34] <lifeless> To be safe in the presence of caching the archive has to be fully readable with the old versions of a file at each url.
[02:34] <lifeless> the way I would approacho it, if I were to work on it, would be to ditch Packages.gz and Sources.gz and the other similar fixed-name metadata files;
[02:35] <lifeless> use hash or otherwise uniquely generated names for them and reference those from Release
[02:35] <lifeless> and inline-sign Release [if its not already]
[02:36] <lifeless> this would make mirror pushes atomic - write the new debs & source packages, new packages/sources/contents, new releases, and as each release is written it becomes visible.
[02:37] <lifeless> until a client sees the new releases, it will happily read the old packages.gz and debs etc
[02:38] <smoser> sign is separate now, but i'm told that in-line is done or soon to be done.
[02:38] <lifeless> smoser: I'd be delighted to write up something somewhere, but I can't commit to implementing :)
[02:38] <lifeless> we could publish both styles in parallel quite cheaply, and new clients would be more robust.
[02:40] <smoser> above, how would you atomically handle Releases ?
[02:40] <smoser> ie, that would still be a single path, right?
[02:42] <infinity> You don't need to, you just make sure you mirror Releases last.
[02:43] <infinity> (In lifeless's scenario)
[02:43] <infinity> The old Release file will remain valid for the old hashed Packages files, you get new hashes on disk, mirror new Release, maintain consistency.
[02:43] <infinity> It's elegant enough.  Probably not wildly difficult to implement.  Does make archives less human-navigable, though.
[02:44] <infinity> Perhaps a small price to pay.
[02:44] <infinity> (And fixable by making Packages.gz a symlink to the most recent hash, for people who prefer the fixed names for non-apt uses)
[02:45] <smoser> yeah.
[02:45] <infinity> lifeless: Can you write this up as an apt bug?  I suspect mvo and I would both be interested in looking into it further.
[02:45] <infinity> lifeless: Doing it on the LP side is borderline trivial (or possibly even "for free", with an apt-ftparchive upgrade), but the apt side will take some thought.
[02:46] <lifeless> sure
[02:54] <psusi> slangasek, why is plymouth not in the initramfs normally?  I could have sworn it was supposed to be.. that would eliminate the gui vs text console early boot failure problem woudln't it?
[02:55] <slangasek> psusi: it's not in the initramfs normally because it slows down the boot when it is, since you can't get out of the initramfs then until the video driver has loaded
[02:55] <slangasek> (so the system waits, making no use of either cpu nor disk, while the video card is initialized)
[02:55]  * slangasek trades a negation for a vowel
[02:56] <slangasek> psusi: I am increasingly thinking that we should just stick it in the initramfs and try to make up the boot speed impact elsewhere, but it's a bit late to be pondering this for 12.04 :/
[02:56] <psusi> hrm... so it did used to be there?  and to speed boot, it was removed because loading the video driver was wasting too much time?
[02:56] <psusi> video driver is kind of required for text mode console as well isn't it? ;)
[02:56] <Sarvatt> psusi: it was only there for a short short time in the 10.04 dev cycle
[02:57] <Sarvatt> maybe you had the cryptsetup packaged installed previously or something
[02:58] <psusi> how long does it take to init the video card anyhow?  and doesn't it have to do that anyhow when you don't pass the quiet splash boot options?
[02:59] <slangasek> which is certainly possible, as it gets installed in support of encrypted swap if you choose the encrypted homedir option in the installer
[02:59] <slangasek> psusi: depends on the video card and driver and and; but it may be as much as a couple of seconds
[02:59] <slangasek> (in various common cases)
[02:59] <psusi> wow!
[02:59] <slangasek> yes, you have to init the video card anyway, but when you *don't* have to do it in the initramfs, you're not stuck *waiting* for it
[02:59] <RAOF> You need to do all sorts of output probing, running BIOS scripts, etc.
[03:00] <psusi> RAOF, just to load the dri driver and not even switch video modes?
[03:01] <RAOF> Yes, because loading the dri driver generally implies switching video modes.
[03:01] <slangasek> :)
[03:01] <psusi> not any more... we have gfxpayload=keep
[03:02] <slangasek> heh
[03:02] <RAOF> The dri driver doesn't know that it's not going to switch video modes or bring up new outputs until it's probed all the outputs.
[03:02] <psusi> so... the video driver is still in the initramfs and loaded, just doesn't block switching to the real root?  or you're saying it isn't even loaded until after the root is mounted?
[03:02] <slangasek> yeah; gfxpayload=keep just means the framebuffer is passed to the kernel, it doesn't mean the drm driver is smart enough to *use* it
[03:03] <slangasek> no, the video driver isn't included in the initramfs at all
[03:03] <slangasek> unless you've asked for it to be
[03:03] <psusi> umm... then how is initramfs rescue recovery ever possible at all?
[03:04] <psusi> ohh, it's only possible when the kernel comes up still in VGA text mode?
[03:04] <psusi> that's no good at all...
[03:05] <psusi> this might tie into something else I was thinking of the last time I worked on ureadahead... why doesn't it start in the initramfs... if you're doing readahead while loading the video driver that should fix the slow down problem without breaking initramfs recovery
[03:06] <slangasek> psusi: yes, precisely - and if you're booting in rescue mode, you *are* coming up in text mode, that's one of the differences between the main boot option and the fallback one
[03:07] <psusi> slangasek, of course, on macs or whatnot that don't have vga text boot and require fbcon, then you're screwed?
[03:07] <infinity> psusi: How do you propose we readahead before we have a rootfs to read ahead on?
[03:08] <slangasek> psusi: what kind of macs are we talking here?
[03:08] <slangasek> UEFI has its own framebuffer
[03:08] <psusi> infinity, you obviously mount the root fs first, but you can start the readahead and video driver load in parallel, all before you actually run-init the real root fs
[03:08] <slangasek> and we're supposed to be using that, but I have certainly seen bug reports about it not working
[03:09] <slangasek> psusi: you can't start the readahead until you have the root filesystem mounted to find the ureadahead pack data
[03:09] <infinity> psusi: If we've found the rootfs, we should be pivoting ASAP, there's no reason to stay in the initrd.
[03:09] <psusi> infinity, if you're still waiting for the video to init, you can at least start the readahead...
[03:10] <infinity> slangasek: To be fair, I'm not sure we can guarantee every device will have a usable text mode.  We're only saved on ARM by building every subarch individually and building in framebuffers, that can't last if we ever have a unified ARM kernel.
[03:10] <psusi> if you do that, then delaying the pivot for a second or two doesn't matter since all you'd be doing after is waiting for the readahead anyhow
[03:10] <slangasek> infinity: sure; we might prefer to special-case the initramfs video handling on different architectures then
[03:11] <slangasek> anyway, as I said, I think we should bite the bullet and put the video in the initramfs - just not for 12.04
[03:11] <slangasek> oh, also, aren't we getting rid of initramfs on arm, making this a moot point ;)
[03:11] <infinity> slangasek: On some ARM machines, perhaps, I doubt we are on all.
[03:11] <Sarvatt> it would be so freaking nice to have the splash up 1.5 seconds into the boot instead of 7 seconds when the desktop starts at 8 so you barely see it, but yeah cairo in the initrd, 20mb+ extra initrd size on proprietary drivers, etc..
[03:12] <infinity> slangasek: And hey, if someone makes uBoot suck less, we won't really need to at all.
[03:12] <psusi> Sarvatt, exactly..
[03:12] <slangasek> psusi: so I think what I would prefer is to put the video drivers in the initramfs unconditionally, but *not* block the initramfs waiting for plymouth to start unless we know cryptsetup is in use
[03:12] <psusi> I thought most of the proprietary driver bulk was the xorg user mode driver part, not the kernel module?
[03:12] <infinity> slangasek: Though, really, any machine that doesn't NEED an initrd shouldn't have one.  uBoot isn't the only bootloader that sucks at loading things from disks.
[03:12] <Sarvatt> the kernel module is like 10mb last i checked for nvidia
[03:13] <psusi> what the crap?
[03:14] <psusi> wow... I guess fglrx_updates.ko is 4.3M... wow...
[03:15] <Sarvatt> 17M /var/lib/dkms/nvidia-current/295.33/build/nvidia.ko
[03:17] <psusi> jesus... what the hell are they shoving in there?  it wasn't that long ago that debian netinst came on two 1.44 MB floppies and that was the whole linux kernel plus root fs
[03:18] <slangasek> no, no... it was that long ago :)
[03:19]  * psusi corrects himself... damn old misnomer... 1440K floppies, which is not 1.44 MB ;)
[03:20] <psusi> slangasek, so wait... can you even flip it back to vga text mode without the driver loaded?
[03:21] <slangasek> well, *I* haven't managed to yet
[03:21] <psusi> I wouldn't think you could...
[03:21] <slangasek> well
[03:21] <slangasek> you should be able to do it as raw VGA
[03:21] <slangasek> not using a kernel fb driver
[03:21] <micahg> infinity: or slangasek: could one of you copy chromium-browser lucid-oneiric from ubuntu-security-proposed to *-proposed*
[03:21] <slangasek> but I haven't found the right tool for it yet
[03:21] <psusi> right, but you need something to modeset it back to vga mode don't you?
[03:22] <infinity> micahg: If you ask really nicely.
[03:24] <slangasek> psusi: maybe? :)
[03:24] <infinity> micahg: (looking at it right now)
[03:25] <micahg> infinity: please.....with visions of bunnies and kittens
[03:31] <infinity> micahg: I assume we don't care about the oneiric build failures?
[03:31] <micahg> infinity: not enough to block the rollout
[03:31] <infinity> Alright.
[03:32] <infinity> LP is grinding, but it'll be done "soon".
[03:40] <infinity> micahg: Finally done.  Double-check after the next publisher that it's all sane?
[03:40] <micahg> infinity: yep, thanks
[03:42] <lifeless> smoser: infinity: https://bugs.launchpad.net/ubuntu/+source/apt/+bug/972077
[03:43] <smoser> thanks.
[03:45] <infinity> lifeless: Danke.
[03:48] <lifeless> infinity: I've elaborated a little, perhaps. Its likely we need to do some dance around mirror pushes to really hammer that down, but thats orthogonal - push mirrors are already pretty tightly synced, and the new format will self-correct as soon as they have all pushed (which the current one does not)
[03:48] <lifeless> smoser: if you have a session on this @ UDS and you want me there - I'm attending remotely.
[03:48] <lifeless> smoser: so feel free to put me in as mandator or whatever, but I can't make the first 2-3 hours of each day
[03:49] <smoser> lifeless, do not pretend to be mortal by implying you have to sleep
[03:49] <smoser> thanks for your help./
[03:49] <infinity> I suspect mvo and I might want to take it upstream a bit, but a UDS session isn't out of the question.
[03:49] <smoser> right. upstream is correct, but we can have a good plan in ubuntu.
[03:49]  * infinity nods.
[03:50] <lifeless> upstream honour patches :)
[03:50] <lifeless> you can call the file Releases.lifeless if you want to give me cred :P
[03:51] <smoser> lifeless, are you opposed to forwarding that to upstream ?
[03:51] <infinity> Well, we already have InRelease (the inline signed version)
[03:51] <infinity> LinRelease?  Lifeless's Inline Release? :P
[03:51] <lifeless> smoser: I think its totally appropriate to be upstream
[03:51] <smoser> right.
[03:52] <infinity> (I'm sure we can do it in the existing InRelease withough adding a new file)
[03:52] <lifeless> smoser: I think we need to resource doing it ourselves. Upstream doesn't move at our pace, nor have our cloud adoption, so won't be seeing it anywhere near as much.
[03:52] <infinity> Given that old clients verify from Packages->Release (not the other way).
[03:52] <smoser> lifeless, do you mind filing upstream?
[03:52] <infinity> lifeless: Some of us *are* upstream.
[03:53] <lifeless> infinity: (InRelease) - cool : I didn't want to make assumptions about all the clients out there
[03:53] <infinity> But yeah, I think we could implement it in the current InRelease without breaking the world.
[03:53] <lifeless> smoser: I think someone likely to be doing it should file it upstream.
[03:54] <infinity> I'll talk to mvo about it a bit, and then we can file upstream.
[03:54] <lifeless> smoser: I'm really just applying my HTTP + doing whacky shit databases with filesystems knowledge here :)
[03:54] <smoser> lifeless, thats fine.
[03:54] <infinity> Or just implement it as a fait accomplis.
[03:54] <infinity> The more I think about it, the more it actually doesn't sound hard.
[03:54] <smoser> its clearly a valid bug wether or not we're going to work on it.
[03:55] <lifeless> smoser: it is, but also the symptoms are already reported upstream
[03:55] <smoser> no, its not that hard. just different filenames.
[03:55] <infinity> smoser: No, I mean the client implementation.
[03:55] <infinity> smoser: It's not just new filenames. :P
[03:55] <lifeless> theres what, a half dozen clients to update
[03:55] <infinity> (Currently, Release serves no purpose but to sign Packages, we'll be turning it into an index)
[03:56] <lifeless> apt, various apt proxies, apt-ftp and so forth.
[03:56] <infinity> lifeless: Maintaining backward compat is cake, so I'm less concerned about clients following suit ASAP.
[03:56] <lifeless> infinity: always an advantage
[03:56] <lifeless> {I wish we did that for more of our transitions}
[03:57] <smoser> infinity, well it largely is just new filenames. you just have to update the name versus path.
[03:57] <smoser> the way i read lifeless bug is that Releases-lifeless has '<sum> Packages-$HASH.gz'
[03:57] <smoser> or something
[03:57] <infinity> smoser: Yes, but then we need to make apt care about that. :P
[03:58] <infinity> smoser: Right now, apt downloads Packages, then checks Release for the sum.
[03:58] <infinity> smoser: So, we're flipping that on its head.
[03:58] <smoser> but i tihnk '<sum> old-school-filename fullpath/new-school/filename' would make sense.
[03:58] <infinity> smoser: And using Release as the index of available Packages files.
[03:58] <smoser> then you dont have to parse the filename out of some token, you're fed it explicitly
[03:58] <infinity> Anyhow.  Implementation details.
[03:58] <infinity> Just saying it's more than "new filenames".
[03:59] <infinity> But not rocket surgery either.
[03:59] <lifeless> hah, this is *all* implementation details :)
[03:59] <smoser> well, my "just filenames" was implying that clients already read the releases, and get the path relative to that
[03:59] <infinity> smoser: They don't.
[03:59] <smoser> no? my client does
[03:59] <smoser> :)
[03:59] <smoser> what do they do?
[03:59] <infinity> smoser: It finds Packages from Release?
[04:00] <smoser> why wouldnt it?
[04:00] <infinity> smoser: apt finds Packages on its own, then verifies it with Release.  Not so subtly very different.
[04:00] <lifeless> heheh, I *knew* there would be clients out there doing that
[04:00] <smoser> infinity, yeah,that makes sense.
[04:00] <infinity> And yes, if clients are doing that backward, we need to rename InRelease for the new implementation, or they explode.
[04:01] <smoser> for a mirror (my client is "check-mirror") it is easier to just get Releases and walk thorugh it.
[04:01] <infinity> Though I'm not sure how much I care about !apt...
[04:01] <infinity> Anyhow.  We'll discuss more later.  Lots more.
[04:01] <infinity> And yeah, maybe do a UDS session, though I think this is more of a pair-programming "hammer on it til it works" thing, not something that needs a committee to argue about it for an hour.
[04:02] <lifeless> infinity: right, not encouraging you to have a session, just noting that *if* you have one *and* you want me, those are the constraints involved
[04:03] <infinity> lifeless: I think your bug (and the above discussion) probably outlines things well enough anyway.
[04:03] <infinity> Pretty clear problem and pseudo-atomic solution.  Just need to sort out transition, and argue about things that don't matter.
[04:03] <lifeless> yup
[04:03] <lifeless> if you want more chatter of any sort, just ping me
[04:03] <lifeless> otherwise I'll assume that Q will be perfect.
[04:04] <infinity> Of course it will be.
[04:04] <infinity> Every release is.
[04:04] <infinity> And I won't hear another word on that topic.
[04:11] <StevenK> infinity: Explain Edgy, then?
[04:11] <StevenK> :-P
[04:17] <infinity> StevenK: We don't talk about edgy.
[04:18]  * micahg wonders if Maverick broke the LTS+1 streak
[04:19] <StevenK> micahg: Intrepid wasn't that bad.
[04:20]  * micahg wasn't around for edgy
[04:20] <StevenK> Like infinity says, we don't talk about edgy.
[04:25] <lifeless> forget edgy. Talk warty.
[04:25] <lifeless> *warty*
[04:26] <StevenK> I missed warty, my first release was breezy.
[04:26] <infinity> lifeless: warty doesn't count.
[04:26] <lifeless> infinity: sif
[04:27] <infinity> And hoary was actually pretty good, given it was only the second try.
[04:31] <StevenK> I liked breezy as a release. Ran it off a Live CD for two days.
[04:33] <JontheEchidna> I started using Kubuntu intermittantly at feisty, more often in gutsy, and started contributing around the time hardy was released
[04:34]  * ScottK started using a Dapper a bit before it's release and started contributing mid-cycle in Feisty.
[04:35] <ScottK> Edgy wasn't so bad.
[04:38] <StevenK> ScottK: Compare it to Dapper, though.
[04:38] <ScottK> What?  You mean released on time?
[04:38] <ScottK> I know what you mean though.
[04:39] <ScottK> Edgy felt better after I'd been through Hardy, Intrepid, and Jaunty.
[04:39] <StevenK> ScottK: It was the first LTS, I was willing to wait and see and not complain. And it was worth it, IMO.
[04:39] <ScottK> Sure.  Makes sense.
[04:40] <ScottK> I used Dapper pre-release and was happy with it.
[04:48] <smoser> pitti, you're last-touched lp:~ubuntu-core-dev/ubuntu/precise/apport/ubuntu . i'd appreciate a review and merge of https://code.launchpad.net/~smoser/ubuntu/precise/apport/ec2-metadata-timeout/+merge/100552
[04:48] <smoser> (but anyone who wants to can do it, i can, but just owuld  like the peer review, and havne't touched apport before).
[04:48] <smoser> goot night.
[04:53] <pitti> Good morning
[04:53] <nigelb> Good Morning pitti :)
[04:55] <pitti> infinity: we can parallelize the PNG optimizing stuff, that's on my TODO somewhere; each individual PNG does not actually take long (subseconds), just packages with tons of PNGs will take a while
[04:55] <pitti> infinity: SVGs take a tad longer
[04:56] <pitti> smoser: looks nice, thanks
[04:56] <micahg> pitti: you mean it might be possible to build openclipart in a reasonable amount of time :)
[04:58] <pitti> if that causes so much trouble, you can also build it with NO_PNG_PKG_MANGLE=1
[04:59] <micahg> for the 2 or 3 times a year we build it, it's not that bad
[05:00] <micahg> it just seems crazy for clip art :)
[06:41] <dholbach> good morning
[08:57] <pitti> jibel: our meeting is where? G+?
[08:57] <pitti> jibel: oh, mumble
[08:57] <cjwatson> GrueMaster: Nothing jumps out at me; I'd need to see syslog/partman to look for errors there.
[08:57] <jibel> pitti, mumble but we can do it on g+ if you prefer
[08:59] <cjwatson> lifeless: there's a bug for inline-signing Release, but implementing it before we've switched to a new signing key opens the ability to exploit old versions of apt
[09:00] <lifeless> cjwatson: well, we don't want that :)
[09:00] <lifeless> cjwatson: however, the main part of the discussion wasn't that, but getting to a less cache-incoherent layout
[09:01] <cjwatson> sure; it's an important piece though
[09:01] <hrw> hi
[09:01] <cjwatson> bug 804252, FWIW
[09:01] <hrw> lifeless: is subunit your project?
[09:03] <lifeless> hrw: I'm its BD
[09:03] <lifeless> hrw: and I founded it
[09:03] <hrw> lifeless: sorry to bother but it ftfbs without python 2.6 ;(
[09:05] <lifeless> works for me
[09:06] <hrw> lifeless: http://people.ubuntuwire.org/~wgrant/rebuild-ftbfs-test/test-rebuild-20120328-precise.html lists it as broken on all 3 archs in precise.
[09:07] <lifeless> bad testtools version
[09:07] <lifeless> testtools had a release that broke subunit
[09:07] <lifeless> the next release fixes it
[09:07] <lifeless> nothing to do with python version
[09:09] <hrw> thanks
[09:10] <lifeless> jodh: btw, please include the OOPS id if you're reporting a failed page in launchpad
[09:10] <lifeless> jodh: the oops id lets us tell whats going on without having to guess.
[09:11] <lifeless> jodh: saying 'if you do x you will get a timeout' does not work because timeouts are often usercode specific, due to team memberships.
[09:12] <lifeless> jodh: I've taken a guess in this case, but including the oops id in future will make it a lot easier; thanks.
[09:33] <smb> cjwatson, I got the feeling a recent upgrade to grub2 may be causing problems when using block lists (I know there are deprecated but just so useful). bug 972250
[09:52] <cjwatson> smb: I had a Debian report of a problem as well
[09:52] <cjwatson> I expect it's the 4K sector backport, sigh
[09:52] <cjwatson> can you debug it?
[09:53] <smb> cjwatson, I have an affected system sitting around. So I should be able to check things (not yet sure where exactly to look)
[09:55] <smb> Great, since a bit ago and now, the error changed from the non-sector aligned data to "cannot read `/boot/grub/core.img' correctly"
[09:56] <cjwatson> Otherwise it'll need to wait until I can set up a suitable VM and look for myself.
[10:00] <smb> cjwatson, The fs /boot/grub is on is ext4 which has 4k sectors. Partition looks reporting 4k block size as well and 512 sector size
[10:02] <cjwatson> I hope it's not actually dependent on having logical 4K sectors since I can't test that.
[10:03] <smb> Never tried to change things, but blockdev has an option to modify the block size
[10:07] <doko_> hmm, updating to the new grub:
[10:08] <doko_> Setting up grub-pc (1.99-20ubuntu1) ...
[10:08] <doko_> /usr/sbin/grub-setup: warn: Attempting to install GRUB to a partitionless disk or to a partition.  This is a BAD idea..
[10:08] <doko_> /usr/sbin/grub-setup: warn: Embedding is not possible.  GRUB can only be installed in this setup by using blocklists.  However, blocklists are UNRELIABLE and their use is discouraged..
[10:08] <doko_> /usr/sbin/grub-setup: error: non-sector-aligned data is found in the core file.
[10:08] <doko_> cjwatson, ^^^
[10:08] <smb> doko_, See bug I just talked to him about. :)
[10:08] <doko_> ohh
[10:09] <cjwatson> I'll look at it once I've finished with the thing I'm currently on
[10:10] <smb> A bit weird is that blockdev --getbsz returns 4k but all the locical and physical block sizes values in sysfs/queue are 512...
[10:10] <cjwatson> Hopefully this is a red herring.
[10:11] <cjwatson> It might just plain be broken with blocklists.
[10:12] <smb> cjwatson, I could offer an strace of the run, but for some reason my error has changed now. :(
[10:13] <cjwatson> Don't bother; I'm not convinced that would be terribly helpful.  I'll see if I can reproduce it first.
[10:14]  * cjwatson curses blocklists.
[10:16] <smb> cjwatson, Ok, got it back (it was not helpful to have had a peek with debugfs while being mounted). At least that way I could make a note of the message I get on boot: "error: out of partition" (press any key to continue). But it does come up
[10:17] <cjwatson> Some systems will still work as they might just be using the old core image, but there was an ABI change in the disk structure so I expect it will depend on the complexity of the partitioning of /boot.
[10:18] <cjwatson> Anyway that isn't too interesting.
[10:18] <cjwatson> Best to debug the first error.
[10:20] <smb> OK, unfortunately have to run now, but in case you need someone doing something on an affected system, let me know
[10:23] <pitti> jibel: so, turns out that the upower autopkgtest actually discovered a crash in upowerd :)
[10:25] <cjwatson> jibel: it was a lot quicker for me to use autopkgtest's schroot runner, which did reproduce an xvfb-related failure in ubiquity; hopefully it's the same one
[10:25] <cjwatson> (admittedly I had to fix a bug in the schroot runner first ...)
[10:51] <Daviey> Okay.. I've been handed a machine that when i create files, they get permission d-w---x--- root:root .. but the same action on a supposedly equal machine are created with drwxr-xr-x .. umask is normal, both are ext4 with no special mount flags.  Indeas?
[10:57] <cjwatson> pitti: to save you debugging the same thing: Xvfb currently breaks if TMPDIR is set
[10:57] <cjwatson> or rather if it's set to something which isn't on the same mount as /var/lib/xkb
[11:03] <pitti> cjwatson: ah, thanks; I would have hit that in jockey
[11:03] <pitti> cjwatson: I copy and run the tests to $TMPDIR as otherwise python uses the local ones from teh package source instead of the system
[11:03] <pitti> s/package/source tree/
[11:04] <cjwatson> pitti: bug 972324
[11:04] <pitti> cjwatson: thanks
[11:05] <cjwatson> I'll probably work around it by something akin to env -u TMPDIR xvfb-run env TMPDIR="$TMPDIR"
[11:30] <pitti> jibel: so, I'm not really sure what https://jenkins.qa.ubuntu.com/view/Precise/job/precise-adt-postgresql-common/lastFailedBuild/ARCH=amd64,label=albali/console is
[11:30] <pitti> it doesn't seem to find the .deb it just built
[11:36] <pitti> cjwatson: did you see failures from Python tempfile operations?
[11:37] <pitti> I'm gettin
[11:37] <pitti>     fd = _os.open(file, flags, 0600)
[11:37] <pitti> OSError: [Errno 13] Permission denied: '/tmp/tmp.FecYZsCFYt/ubtree0-build/tmpdir/tmpkLfNk4'
[11:38] <cjwatson> I haven't seen anything like that as yet
[11:38] <cjwatson> but I'm using the schroot runner for convenience so perhaps that's a bit different
[11:38]  * cjwatson <3 schroot
[11:38]  * pitti tries to replicate this in his local kvm
[11:39] <cjwatson> ubiquity's problem is part that X bug and part me being desperately stupid.
[11:39]  * pitti seriously doubts the latter
[11:41] <cjwatson> smb: reproduced
[11:45] <penguin42> cjwatson: I put you as a reviewer on the fix for the procps/watch bug we discussed the other night
[11:46] <cjwatson> penguin42: oh, yeah, I saw that in my mailbox thanks
[12:20] <pitti> jibel: could you please re-run the upower adt test, and perhaps udisks with 1.5 GB RAM?
[13:11] <cjwatson> jibel: OK.  I think the combination of autopkgtest 2.0.1ubuntu3 and ubiquity 2.10.6 should fix precise-adt-ubiquity.
[13:16] <jibel> cjwatson, k, I re-run the test when it is in the archive. I'll add an option to run in a schroot when the test doesn't need the level of isolation of a vm
[13:16] <jibel> pitti, tests are running with 2GB
[13:18] <cjwatson> jibel: watch out that the schroot runner is a bit broken right now though - I filed a Debian bug with a fix, which I guess will show up on bugs.debian.org sooner or later
[13:18] <cjwatson> so no rush on that, I can easily just do something like 'sudo adt-run ../ubiquity_2.10.6.dsc --- ~/src/ubuntu/autopkgtest/git/autopkgtest/virt-subproc/adt-virt-schroot precise-i386-sbuild'
[13:38] <ochosi> kenvandine: ping
[13:39] <kenvandine> ochosi, pong
[13:39] <ochosi> kenvandine: hey, i wanted to inquire about indicator-messages-gtk2 a bit, are you the right guy to talk to? (lp suggested that)
[13:40] <kenvandine> yup
[13:40] <ochosi> ok, i wanted to ask about this bug (i
[13:40] <ochosi> 've been going in circles trying to figure out what's wrong)
[13:40] <ochosi> https://bugs.launchpad.net/ubuntu/+source/xubuntu-artwork/+bug/956147
[13:41] <ochosi> (afaik there shouldn't even be any icons)
[13:41] <kenvandine> there shouldn't be
[13:41] <kenvandine> there should be a thunderbird icon
[13:41] <ochosi> right, so there's nothing for me (xubuntu artwork-dev) to fix
[13:42] <ochosi> yeah, that's there, and then two text-only entries for addressbook and compose
[13:42] <kenvandine> oh
[13:42] <ochosi> it works as long as tb is closed, but as soon as it's running a "missing icon"-icon is shown
[13:42] <kenvandine> looking at his screenshot, that isn't for thunderbird
[13:42] <kenvandine> that is for evolution
[13:43] <ochosi> right, well i can confirm it for thunderbird :)
[13:43] <ochosi> i didn't test it with evolution
[13:43] <ochosi> but thanks, i didn't notice that at first
[13:43] <kenvandine> interesting, with the gtk3 build of it, it doesn't even try to add icons tehre
[13:43] <kenvandine> there
[13:44] <ochosi> is there anything i can do to debug this?
[13:44] <sconklin> @pilot in
[13:45] <kenvandine> ochosi, i am triaging it so the indicator-messages upstream sees it
[13:46] <ochosi> kenvandine: thanks!
[13:46] <ochosi> it'd be pretty great to get this fixed in time for precise, it's a highly obvious and annoying bug
[13:47] <ochosi> and it affects ubuntu-studio and mythbuntu as well
[13:47]  * astraljava emerges
[13:47] <astraljava> What's that?
[13:47] <pitti> jibel: hooray, adt-upower succeeded now
[13:47] <ochosi> astraljava: https://bugs.launchpad.net/ubuntu/+source/xubuntu-artwork/+bug/956147
[13:48] <ochosi> astraljava: if you can help me following up on that, that'd be great
[13:48] <pitti> jibel: could you please restart jockey, too? fix is in teh archive
[13:48] <kenvandine> tedg, did you see bug 956147?
[13:48] <astraljava> ochosi: Whoopsie... I hadn't even noticed. Sure, I'll do what I can.
[13:50] <tedg> Hmm, there shouldn't be icons there.
[13:50] <tedg> They should be blank space.
[13:51] <ochosi> tedg: as long as thunderbird is not running, there are no icons (i.e. everything is fine)
[13:51] <kenvandine> tedg, right... so something weird with the gtk2 builds
[13:51] <kenvandine> tedg, and notice in the screenshot it is evolution
[13:51] <kenvandine> so it seems to affect both
[13:52] <tedg> kenvandine, I think you have your default mail client set to Evolution :-)  "Mail" is default client
[13:52] <kenvandine> oh
[13:52] <kenvandine> ok
[13:52] <kenvandine> :)
[13:54] <kenvandine> tedg, i reproduced it with the indicator-loader
[13:54] <tedg> larsu, Have you seen bug 956147 ?  Seems to happen in the GTK2 version.
[13:54] <kenvandine> evolution has the missing icons now
[13:54] <kenvandine> tedg, so does "Update Status" under gwibber
[13:55] <larsu> tedg, I'll have a look at it in a bit (doing the blacklist thing right now)
[13:57] <apw> @pilot in
[14:13] <bigon> slangasek: hey, I'm looking at your multiarch patch for libcanberra, and it seems that libcanberra-gtk-module and libcanberra-gtk3-module are not really m-a
[14:16] <bigon> as you are installing files in no m-a paths
[14:32] <smoser> lool-, around ?
[14:34] <PaoloRotolo> Hi all!
[14:35] <smoser> lool- so i don't forget.  if/when you see this, my query was about linaro use of s3 mirrors, and if they've had issues.  we believe we've found and fixed the last issue in bug 948461
[14:49] <tgardner> cjwatson, does the debian installer need to be rebuilt before linux-firmware udeb updates are incorporated into the ISO ?
[14:54] <slangasek> bigon: hi, how do you mean, "not really m-a"?
[14:57] <bigon> slangasek: well it was a misunderstanding that you can actually have bitwise identical files outside of /usr/share/doc/<pkgname>
[14:58] <slangasek> bigon: ah, ok
[14:58] <cjwatson> tgardner: scsi-firmware yes, nic-firmware no
[14:58] <tgardner> vjwk, thanks
[14:58] <tgardner> cjwatson, ^^
[15:15] <lool-> smoser: I seem to recall repsonding to an earlier query on the topic, but I don't remember who had asked me; in any case, I have been moderately using the S3 mirrors and didn't have any issue with them; they worked fast and reliably in my personal experience
[15:55] <cjwatson> smb: OK, so I have a bit of a problem here
[15:56] <cjwatson> smb: I've backported an upstream patch which I'm pretty sure fixes this problem, and it certainly makes grub-install work again
[15:56] <smb> cjwatson, ok...
[15:56] <cjwatson> smb: however, when I install that, it doesn't boot
[15:56] <cjwatson> smb: now normally this would make me give up and revert the whole thing
[15:57] <cjwatson> smb: except, when I install 1.99-18ubuntu1, that doesn't boot for me either in this environment
[15:57] <cjwatson> smb: so I'm pretty sure this is a local problem.  Could you try out the packages I've built?
[15:57] <smb> cjwatson, sure.
[15:57] <smb> where ?
[15:57] <GrueMaster> cjwatson: Morning.
[15:58]  * smb meant where can I find the packages...
[15:58] <GrueMaster> Here is a syslog from this morning's attempt.  The system is sitting at a prompt failing to mount /boot.  http://paste.ubuntu.com/913180/
[15:59] <cjwatson> smb: just copying them up now
[16:00] <cjwatson> GrueMaster: could I have the current recipe to go with that?
[16:00] <cjwatson> preseed file, I mean
[16:01] <GrueMaster> cjwatson: http://paste.ubuntu.com/913187/
[16:02] <cjwatson> smb: http://people.canonical.com/~cjwatson/tmp/grub2/ - let me know if you need amd64 and I can build that
[16:02] <smb> cjwatson, Yes, the box here is 64bit
[16:02] <cjwatson> smb: OK, ETA <30m
[16:02] <smb> ack
[16:12] <GrueMaster> cjwatson: Where would I find what options partman is trying to mount /boot with?  That appears to be the failure (according to syslog).
[16:15] <cjwatson> GrueMaster: mm, something's odd here, I want to trace this a bit
[16:15] <GrueMaster> Let me know what I can run to help.  I have a system sitting at that failure, and I have ssh access.
[16:17] <cjwatson> it looks for all the world like an attempt to mount ext4 as ext3
[16:17] <cjwatson> what I don't yet know is why
[16:18] <cjwatson> oh, blah
[16:18] <cjwatson> GrueMaster: sorry about this - change "format { }" to "format{ }"
[16:19] <cjwatson> as a result of that it wasn't formatting /dev/sda1, so you got whatever was there beforehand, which was apparently ext4
[16:19] <cjwatson> it's not the world's smartest parser
[16:20] <GrueMaster> Ah.  Let me respin with that.  Good eye.  I've been staring at partman recipes since Friday, getting a little buggy.  :P
[16:25] <GrueMaster> YAY!  Seems to be going through.  Now installing base system (which means partman, while still grumpy as all hell, is satisfied for the moment).
[16:25] <cjwatson> phew
[16:25] <cjwatson> sorry that was opaque!
[16:25] <cjwatson> the parser badly needs (a) a test suite (b) a rewrite, in that order.
[16:26] <GrueMaster> And (c) Decent documentation.
[16:26] <cjwatson> That wouldn't hurt.
[16:26] <GrueMaster> Thanks for your help though.  I really appreciate it.
[16:27] <cjwatson> You're welcome
[16:27] <cjwatson> Hopefully that will cover the problems on all your machines ...
[16:27] <GrueMaster> Now to figure out why 2 of the 3 systems is complaining about base system installation (3rd is almost finished).
[16:27] <cjwatson> subarch-specific?
[16:28] <cjwatson> though base-installer should support all the subarches in question.
[16:28] <GrueMaster> All three are pandas.  Exact same makeup.  And they are all pulling netboot & preseed from the same location.
[16:28] <GrueMaster> Only diff is mac address/ip.
[16:29] <GrueMaster> Probably a network glitch.  I've seen it before.  Just need to kick the fenders.
[16:34] <GrueMaster> cjwatson: Of lesser concern is the lvm naming.  In my recipe, I have Test-LVM as a blank area.  The finished product names it Test--LVM.  Not sure where the extra "-" is from, and it doesn't hurt (I plan on renaming that partition before production anyways).
[16:39] <smoser> roaksoax, join #ubuntu-server
[16:40] <roaksoax> lol o'm there :)
[16:41] <cjwatson> GrueMaster: That's a devmapper requirement.  It uses "-" as a separator, so "-" gets escaped to "--".
[16:42] <cjwatson> Though it should generally still be presented as "-", unless you're looking in /dev/mapper/ directly.
[16:44] <GrueMaster> Ok.  Like I said, it isn't really a concern.  I plan on renaming that partition in my local systems anyways, to match the arm server testing environment.
[16:54] <bdrung> doko: bug #972625
[16:55] <doko> ouch
[16:58] <smb> cjwatson, Ok, so for my system the updated grub packages fix the error on running grub-install and boot ok
[16:58] <cjwatson> brilliant
[16:59] <cjwatson> I was just about to tell you that the amd64 binaries were up ...
[16:59] <cjwatson> (I missed them finishing for 20 minutes.)
[16:59] <smb> :) Have been polling for them to appear because I got to leave soon.
[16:59] <cjwatson> OK.  I'll get that uploaded then; it'll actually be -21ubuntu1.
[16:59] <cjwatson> thanks!
[17:02] <smb> Ah, ok. The I get them replaced when they are up. Thanks for resolving that so quickly
[17:04] <cjwatson> Yeah, you would have done anyway actually since apt will reinstall even same version when the checksums differ.
[17:07] <k-rAd-> what is the email address for ubuntu selinux department ?
[17:07] <hrw> fun during oneiric->precise upgrade:
[17:07] <hrw> locale: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.15' not found (required by locale)
[17:07] <k-rAd-> i wish to lend a hand
[17:07] <hrw> locale: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.14' not found (required by locale)
[17:08] <slangasek> hrw: should be a non-fatal warning (which is why we haven't dug into it)
[17:08] <micahg> k-rAd-: there isn't really one, you can try in #ubuntu-security though
[17:08] <k-rAd-> thank you
[17:08] <hrw> slangasek: its non-fatal
[17:14] <dobey> hey all
[17:16] <dobey> if i have a change which is related to a bug, but doesn't fully fix it for all cases, what's the best way to reference that in the changelog so lp doesn't auto-close the bug? is there a regex somewhere for what the server does, which i can match against to check my changelog?
[17:17] <slangasek> dobey: the regexp is applied in dpkg-genchanges, not on the server; you can examine your .changes file prior to upload to see if there are launchpad bug references
[17:17] <slangasek> (I don't remember the field name off the top of my head, but grep for Launchpad :)
[17:19] <dobey> ok
[17:19] <dobey> thanks slangasek
[17:21] <cjwatson> dobey: I usually use 'LP #nnnnnn', omitting the colon after LP.
[17:22] <dobey> cjwatson: thanks, i'll try that
[17:27] <apw> cjwatson, load_video, can i tell if that worked?
[17:27] <apw> cjwatson, in grub2 .cfg files ...
[17:28] <cjwatson> EOD, sorry
[17:28] <cjwatson> suggest you RTFS as I don't know off the top of my head anyway :)
[17:28] <apw> cjwatson, np :)  will do ... have a good one
[17:29] <cjwatson> probably depends what you mean by "worked", given it's graphics
[18:18] <apw> cjwatson, yeah indeed, thats another wrinkle... i think we have enough info to avoid the question and for where we are in the cycle thats safest ... i'll brief you when i have the paired changes ready for thinking about committing
[18:19] <apw> slangasek, did you have a machien which had the broken handoff behaviour ?
[18:21] <apw> @pilot out
[18:22] <rlaager> I maintain various packages in a private repository and a few PPAs. Is there a way to flag them so apport doesn't offer to send a report to Launchpad when, for example, their postinst fails?
[18:23] <dobey> rlaager: it should not be doing that anyway, as they are not official ubuntu packages
[18:23] <rlaager> dobey: I'm on Precise and it just did.
[18:23] <slangasek> apw: remind me which broken handoff behavr we're talkig about?
[18:24]  * slangasek stabs kvm in the face.  Give me back my keyboar interrupts.
[18:24] <dobey> rlaager: is it a bug about the installation of your package, or the removal of an official ubuntu package, while installing your package which replaces the official one?
[18:25] <rlaager> dobey: The postinst for my package failed on installation. This particular package, at least, did not cause any official packages to be removed.
[18:25] <apw> slangasek, there are two triggers, (broken graphics cards in grub, or encrypted root so no way to do graphics), but with nouveau i think and psb graphics crossing from grub in text mode but with 'vt.handoff=7' triggers black screens of doom, double ESC sometimes helps sort of thing
[18:26] <apw> slangasek, i am sure you were bugging me about something similar
[18:26] <slangasek> apw: ah right, that one
[18:26] <apw> slangasek, and i have potential fixes in the works which i'd like to get tested once they are packaged
[18:26] <apw> and you are awake :)
[18:26] <slangasek> apw: I have a box with nvidia graphics here for testing, but I haven't tried to reproduce this yet
[18:27] <tgardner> apw, I've got one for the nVidia problem
[18:27] <apw> tgardner tells me his box hits the issue, with nvidia ... :)
[18:27] <apw> see
[18:27] <apw> ok will start with tgardner and expand from there
[18:27] <dobey> rlaager: ah sorry, i misread. apport will offer, but it will fail because it's not an official package. i don't think there is any way to prevent that in package metadata itself
[18:28] <tgardner> apw, guess I'll get it back in that state again....
[18:30] <rlaager> dobey: Okay. Well, I'll just have to make less buggy packages. ;)
[18:30] <dobey> yes :)
[19:08] <apw> slangasek, in your inbox is an email 'On vt.handoff' which has the details of what I would like to test
[20:13] <infinity> Someone remind me which bit I need to double-check to make sure my headless/remote machine will reboot with a degraded RAID?
[20:14] <infinity> Maybe I should just fix the disk and rebuild the array before I try rebooting. :P
[20:16] <stgraber> infinity: mdadm/boot_degraded
[20:16] <stgraber> infinity: that and make sure you have grub in the mbr of the working disk
[20:18] <infinity> stgraber: grub *should* be correctly installed to both disks.
[20:18] <infinity> stgraber: But thanks for the boot_degraded pointer.  I'm pretty sure I answered that at install, but wanted to be sure.
[20:19] <rlaager> Security Question: /usr/bin/at is currently permissioned "6755 daemon daemon". Doesn't that make it possible, if there's a bug in at, for a regular use to run at, which could be used to overwrite itself, leaving evil code for root to run later on? In other words, wouldn't 6555 be better?
[20:29] <infinity> rlaager: Can you file a bug?
[20:29] <infinity> rlaager: But yeah, potentially true of any non-root suid binary, I suppose.
[20:30]  * infinity wonders how many of those we have...
[20:31] <rlaager> I think at is the only one (in a <= desktop install, which is what I have).
[20:32] <infinity> rlaager: The real bug could be that it should be 2755 root:daemon instead.
[20:32] <rlaager> infinity: The changelog documents the intention to be suid daemon.
[20:32] <infinity> (See: mlocate, wall, ssh-agent, etc)
[20:32] <rlaager> I have some permission limiting code that I've carried forward forever. It was originally from the bastille script. I was just auditing it to see what it actually accomplishes any more. It was basically that, removing setuid from mount/umount (these are servers, so there's no need for users to mount filesystems), and dropping read permissions from ELF binaries (which seems of dubious value).
[20:33] <infinity> rlaager: Sure, but perhaps sgid would be enough.  Dunno.  Either way, bug report please. ;)
[20:35] <rlaager> infinity: Should I subscribe the Ubuntu Security Team or not?
[20:36] <infinity> Ahh, kay, so the changelog details why it needs to run as daemon (to signal atd), but it also claims it's installed 6555.
[20:36] <infinity> So, it might just be a packaging error introduces later that made it 6755.
[20:36] <infinity> I see no mention of the change.
[20:37] <infinity> rlaager: If it's in previous releases as well, yeah.  This would probably warrant an update.  It's not exactly a massive exposure window (or, possibly, not one at all, if at is bug-free), but it's a security bug nonetheless.
[20:38] <infinity> The part where it does everything internally sgid, though, and only needs daemon to signal itself seems like something that cuold be fixed.
[20:38]  * infinity shrugs.
[20:39] <psusi> cjwatson: is installing grub to partitions supposed to be supported in Ubuntu or not?  I thoguht it wasn't, but the debconf screen for grub-pc has a note saying it is possible, but doesn't actually give the partitions as a choice
[21:04] <vanhoof> superm1: ping
[21:05] <superm1> vanhoof: hi
[21:05] <vanhoof> superm1: alberto mentioned a change for dkms that he recently proposed that you ack'd, just curious when we might see a new dkms upload?
[21:05] <vanhoof> superm1: not sure if you have other bits to factor in first :)
[21:06] <superm1> vanhoof: i was hoping to do another release first, but I could just test that patch and do an upload to ubuntu this week in case I don't get around to another release in time for next freeze
[21:06] <vanhoof> superm1: cool, that would be awesome if you could :)
[21:07] <superm1> sure, will plan to do that
[21:10] <vanhoof> superm1: thanks Mario
[21:37] <cjwatson> rlaager: I'm not sure I see how 6555 could help.  A process whose effective UID matches the owner of a file can chmod that file.
[21:37] <cjwatson> Non-owner-writability is more a safety catch than an effective security measure.
[21:40] <cjwatson> That said, writing to the file as daemon will clear the set-user-id bit, IIRC.
[21:45] <nailora> could one of you make  bug #954895 public if possible
[21:49] <Laney> nailora: done
[21:51] <rlaager> cjwatson: Good point. I was actually hoping for "you're missing X" as the answer, so I could drop that from my configuration.
[21:57] <rlaager> cjwatson: Are you confident enough in that answer to (tell me to) close the bug?
[22:04] <cjwatson> rlaager: I'm not confident that it isn't an issue, just that 6555 wouldn't help. :-)
[22:04] <cjwatson> Or at least seems to have some problems as a solution.
[22:31] <rlaager> Can someone please re-open #570542? It has multiple comments confirming the bug is present in Maverick final, Oneiric, and Natty and #658461 has been marked as a duplicate of it.
[22:34] <micahg> rlaager: you should talk to the kernel team (#ubuntu-kernel)
[22:43] <jono> I wonder if anyone can help with this? I am trying to generate a .pot file from a .desktop file that has underscores before each key - I created po/POTFILES.in with the file in it and have run 'intltool-update --verbose --pot --gettext-package=foobar' but it says 'None of the files in POTFILES.in contain strings marked for translation.' - which is odd as the .desktop does have the underscores before the strings
[22:43] <jono> any idea?
[22:44] <cjwatson> I bet there's more output than that
[22:45] <cjwatson> also are you running that in the right directory?  I think you normally want to run it in po/
[22:45] <jono> cjwatson, I am running it in po/
[22:46] <jono> cjwatson, I will pastebin the full output
[22:46] <cjwatson> Maybe it would be quicker to debug from a tarball
[22:47] <jono> cjwatson, http://pastebin.ubuntu.com/913809/
[22:47] <jono> cjwatson, I can put that together if you prefer
[22:48] <cjwatson> yeah, that would be easier
[22:48] <jono> thanks cjwatson, one sec
[22:49] <jono> cjwatson, I emailed it to you
[23:11] <cjwatson> jono: move foo.desktop to foo.desktop.in and make the same change in POTFILES.in
[23:12] <cjwatson> it's not actually a valid .desktop file when it has the translation prefixes in it - it needs to be run through intltool-merge first
[23:12] <cjwatson> sorry, I don't really mean first, you'll do that afterwards
[23:13] <cjwatson> anyway the rename to .in makes it work
[23:13] <doko> bdmurray, do you have a lp script which takes a list of debian bug reports, files corresponding ones in lp, and links the debian reports?
[23:16] <tumbleweed> doko: there's one in ubuntu-dev-scripts import-bug-from-debian (single bug, though, but that's easily scripted around)
[23:16] <tumbleweed> err ubuntu-dev-tools
[23:20] <doko> tumbleweed, thanks, I'll have a look
[23:20] <tumbleweed> oh, it does do multiple bugs
[23:29] <jono> thanks cjwatson, sorry was on the phone
[23:29] <jono> so it requires the .in extension
[23:29] <jono> thanks