[04:20] <rick_h_> jcastro: how do you do a "Show HN"? I only see ask/jobs/submit
[04:24] <Blazeix> rick_h_: pretty sure you prefix the title with "Show HN"
[04:24] <Blazeix> http://www.hnsearch.com/search#request/all&q=SHow+HN&sortby=create_ts+desc
[04:27] <rick_h_> ok yea, I found a big thread discussing making it a tab
[04:27] <rick_h_> but looks like it's shot down
[04:27] <rick_h_> well, don't want to submit and go to bed, so maybe for tomorrow
[13:36] <snap-l> Good morning
[13:54] <rick_h_> morning
[14:29] <rick_h_> 499
[14:29] <rick_h_> views on your busiest day, August 18, 2012
[14:29] <rick_h_> lol, damn one short
[14:33] <snap-l> rick_h_: Did the API url change or something?
[14:34] <snap-l> I can't seem to bookmark
[14:34] <snap-l> via the extension
[14:35] <snap-l> I'm getting a 502 error when bookmarking from the site
[14:36] <rick_h_> snap-l: no, 502 from the site? using 'add bookmark'?
[14:36] <snap-l> Yes
[14:36] <rick_h_> k, sec will look
[14:41] <snap-l> bbl
[14:44] <brousch> I found my old 1 gallon aquarium in the basement and got some fish for it
[14:46] <rick_h_> wtf it works locally
[14:47] <brousch> Hm, yeah 502 for me too
[14:48] <rick_h_> ok yea saw the error that time
[14:48] <rick_h_> whoosh file lock error because I've got import going I bet
[14:49] <rick_h_> there's a huge import going on that's killing me
[14:56] <brousch> so add doesn't go through celery?
[14:56] <rick_h_> the import does
[14:56] <rick_h_> but it's throwing new bookmarks at whoosh like no tomorrow
[14:56] <rick_h_> and the web site gets locked out
[14:57] <brousch> right, so the normal add doesn't get put in the same way?
[14:57] <rick_h_> I just killed celery and got it to work
[14:57] <rick_h_> still slow
[14:57] <brousch> It seems like all adds would go through the same process to be added
[14:57] <rick_h_> no, normal add happens in request. Because just saving a bookmark hits whoosh to index the page title, description, and extended description
[14:57] <rick_h_> they do, sqlalchemy events, but two people are talking now
[14:57] <rick_h_> the web site and the celery importer
[14:57] <rick_h_> so they're fighting over who has lock access to the whoosh index
[14:58] <rick_h_> and this is when you get fun with file based data stores
[14:58] <brousch> Yeah, I'm probably just showing my ignorance
[14:59] <rick_h_> so when the website is adding all is well. It's sharing a single lock for all of it's threads
[14:59] <brousch> I figured the importer would throw each add into celery, then that add would be processed by whoosh as it has resources
[14:59] <rick_h_> whoosh only lets one person hold the lock at a time
[14:59] <rick_h_> so the importer starts up, and it's taking over 30min (keep having to kill/restart his import)
[14:59] <rick_h_> and it grabs the lock
[14:59] <rick_h_> and the website says "hey, I need that..."
[14:59] <brousch> So the web site add would throw it's stuff into celery and it would be processed in the queue with the import
[15:00] <rick_h_> but the web site doesn't toss a single new bookmark to celery, you want to have that stored now
[15:00] <brousch> Sounds like maybe it should
[15:00] <rick_h_> the proper thing to do would be to catch the whoosh index lock exception, and file a background celery process for later
[15:00] <rick_h_> or move fulltext off to something that works with multiple users/locks
[15:01] <brousch> That sounds like extra code. Why not just always do it in celery?
[15:01] <rick_h_> most imports take 2min
[15:01] <jrwren> i was afk until now. congrats rick_h_
[15:01] <rick_h_> and most people aren't saving new bookmarks every 2min so it's not been an issue
[15:01] <jrwren> excellent work.
[15:01] <brousch> But as you scale
[15:01] <rick_h_> jrwren: heh, well it's blowing up in places :)
[15:01] <rick_h_> brousch: yea, as I scalethe right thing to do is move fulltext off of whoosh
[15:02] <brousch> Onto what?
[15:02] <rick_h_> every db actoin can't have to go through celery
[15:02] <rick_h_> brousch: elasticsearch, lucense/blah
[15:02] <brousch> Why not? I thought that was the new hotness
[15:02] <rick_h_> heh
[15:06] <brousch> async all the things!
[15:08] <rick_h_> dammit, something is up. I don't get what.
[15:09] <jrwren> bmark.us feels faster.
[15:10] <jcastro> rick_h_: where you hosting this btw?
[15:11] <rick_h_> jcastro: pair os ec2 instances
[15:11] <rick_h_> /os/of
[15:16] <rick_h_> I think the add new bookmark issue isn't all the whoosh as well. I shut down everything, restarted services, and still getting an issue.
[15:21] <rick_h_> heh, guess I need an account delete button
[15:21] <rick_h_> someone is going through and deleting all their imported bookmarks one by one
[15:28] <rick_h_> yea, whoosh index is melted
[15:28] <rick_h_> there goes that
[15:28] <rick_h_> but adding should work again
[15:32] <rick_h_> snap-l: brousch ^
[17:25] <snap-l> rick_h_: Will check. Thanks!
[17:36] <snap-l> "Xubuntu is not a refugee camp"
[17:36] <snap-l> Oh, this is going to be a good blog post.
[17:37] <rick_h_> .lol
[17:37] <snap-l> http://open.knome.fi/2012/08/19/xubuntu-is-not-a-refugee-camp/
[21:26] <snap-l> Evening (at least, in 35 minutes)
[21:53] <rick_h_> party
[23:32] <brousch> Ridiculous http://twitter.com/N165Mars
[23:42] <Blazeix> brousch++
[23:45] <brousch> It's kind of cool to send Shawn Powers a personal invite to BarCampGR
[23:58] <rick_h_> lol