Planet (Former) Advogato

This is a complement to Advogato, it is an aggregation of blogs of those who used to post on Advogato, but for one reason or another moved their blog from Advogato. It is provided as a service to those who would like to read the "greater Advogato" community.

This site works only as a Planet, it aggregates the post only, to comment on a blog entry, click on the title or time to go to the blog entry on the original site, hopefully it will have a comment facility.

August 22, 2014

Mary Gardiner [hypatia]

Skiing round one: 1998

Me learning to ski a couple of weeks ago is a weirdly long story, beginning in 1998.

In 1998, I was in the final year of high school, but because of my ludicrous and I now think in some ways ill-advised academic program, I had already completed 9 units of study of the required 11 minimum for the Higher School Certificate and was only doing 8 more. (The reason I now think this was ill-advised is beside the point, but in short, I should have risked a slightly lower university entrance score in return for just completing the entire thing in a year early in 1997, and not spent so much energy on doing 1½ times the required courses for absolutely no long-term benefit.) So it was not completely out of the question to head off to New Zealand for a week in winter.

My sister Julia and I were both working retail at the time, and my parents offered us half the price of the trip if we saved the other half. We duly did so and thus embarked on all the mysterious preliminary rituals for a snow trip (getting fitted for gear and such before leaving) and flew to New Zealand with a small group of fellow pupils. It wasn’t my first extended trip away from my family by any means, nor my first plane flight: in the preceeding year, I’d done two fortnight long nerd camps and flown by myself to Sydney a few times to take part in a selective university-level philosophy course for high schoolers. But it was my first international trip, and my first trip between time zones.

The trip was basically a disappointment in several ways. First, I think in retrospect that the supervising teacher, who went every year, must have been frustrated at the social dynamic. There’s good odds that when you take a small group of teenagers out of their usual environment and hierarchies and give them something to do, they behave much more like adults. But it didn’t really work like that. Unless I’m forgetting someone, in terms of age, there was myself in Year 12, Julia in Year 10, and six or seven other girls all in Year 11. All but two of those were part of a group that even I, a year older and not really in need of knowing their class’s dynamics, recognised as the core of a notoriously cliquish group of princesses. We were staying in a lodge in Methven, and they grabbed their own dorm room with unseemly haste and proceeded to have nothing to do with the likes of the rest of us. We made shift for ourselves, but it was still less than ideal.

Second, most importantly, most of us really struggled to learn to ski. The teacher explained the setup to us, and pointed us at the trail guide and the longest beginner run that we were all going to ski with him at the end of the week, and it wasn’t to be. Or at least, I don’t recall how the princesses did, but of my dorm-mates, one was a natural, already turning parallel within a day of starting, one I think wasn’t and other than participating in lessons took to spending most of the day reading in the bus, and Julia and I weren’t much chop either. I think I was the worst. It was the first time in my life that I got pulled aside by an adult to be complimented for trying really really hard, as distinct from succeeding at all. (As I recall, the instructor was quite emphatic about this: he’d never seen anyone work so hard at it. Subtext: at least, not without learning anything.)

With hindsight: here’s what happened. First, I hadn’t even finished growing at this point. (I finished really late for a woman, when I was 18 or 19.) Physically, I was enormously tall and stretched out like gum. My brain and body were not well matched at the time. Second, this was the dying days of non-carved skis. If you were buying yourself skis, they were carved. If you were renting them, at least at Mt Hutt that year, they were still long narrow flat fence-posts. Thirdly, and most importantly, I just didn’t lean forward enough to stop my skis crossing in front. That last the instructor really ought to have picked up: it’s the most common failure mode in beginning skiing. Perhaps he did and I just never learned quite far enough forward to believe him.

The setup was much the usual for beginners: there was a very shallow first day slope and then over to a short but slightly steeper slope to get the technique down. And that’s pretty much where I was done. On, I think, the second last day, still believing that I’d celebrate with a run down the much longer ultimate beginners’ slope the following day, I grit my teeth and just figured that more hours were more better, went higher up a second short beginners’ slope, and went down it, falling at every single turn. I am pretty sure that I spent the best part of two hours snow-plowing cautiously down in one direction, trying to turn, falling over, retrieving my skis (the bindings were pretty loose), pointing myself in the other direction, spending ages knocking snow out of the soles of my ski boots and skiing in the other direction. Two hours, two baby slopes. Not one successful turn. Lots of crying and self-pep talks. Presumably my growing exhaustion and cementing bad technique were hindering me by then.

I don’t even know what got me back on the slopes the last day. Probably the money I’d spent on it. The last day brought the backhanded compliment about my work ethic (albeit true, I am bad at quitting things), and, crucially and a bit cruelly, the actual breakthrough I’d wanted the day before. For whatever reason, I decided to lean forward to what I considered a ludicrous degree, and which was probably barely acceptable, I pointed my skis downhill, I lost all fear, and I skiied to the bottom (if I am remembering correctly, more or less without attempting to turn) and stopped myself. And then I got back on the pommel, rode up, pointed myself down the hill again, and did it again.

It was exhilarating; I can still feel how happy I was about it.

And then there was absolutely no time to do it a third time because it was time to return my skis, get back in the minibus, ride the nailbiting drive back down to Methven, and fly home to Australia knowing that, probably, I was capable of skiing and would find it rather fun.

And then I didn’t return to the slopes until 2003 and, when I did, I made the regrettable decision to switch to snowboarding.

August 22, 2014 10:38 PM

Don Marti [dmarti]

Temporary directory for a shell script

Set up a temporary directory to use in a Bash script, and clean it up when the script finishes:

TMPDIR=$(mktemp -d)
trap "rm -rf $TMPDIR" EXIT

August 22, 2014 06:46 PM

Michael Still [mikal]

Juno nova mid-cycle meetup summary: conclusion

There's been a lot of content in this series about the Juno Nova mid-cycle meetup, so thanks to those who followed along with me. I've also received a lot of positive feedback about the posts, so I am thinking the exercise is worthwhile, and will try to be more organized for the next mid-cycle (and therefore get these posts out earlier). To recap quickly, here's what was covered in the series:

The first post in the series covered social issues: things like how we organized the mid-cycle meetup, how we should address core reviewer burnout, and the current state of play of the Juno release. Bug management has been an ongoing issue for Nova for a while, so we talked about bug management. We are making progress on this issue, but more needs to be done and it's going to take a lot of help for everyone to get there. There was also discussion about proposals on how to handle review workload in the Kilo release, although nothing has been finalized yet.

The second post covered the current state of play for containers in Nova, as well as our future direction. Unexpectedly, this was by far the most read post in the series if Google Analytics is to be believed. There is clear interest in support for containers in Nova. I expect this to be a hot topic at the Paris summit as well. Another new feature we're working on is the Ironic driver merge into Nova. This is progressing well, and we hope to have it fully merged by the end of the Juno release cycle.

At a superficial level the post about DB2 support in Nova is a simple tale of IBM's desire to have people use their database. However, to the skilled observer its deeper than that -- its a tale of love and loss, as well as a discussion of how to safely move our schema forward without causing undue pain for our large deployments. We also covered the state of cells support in Nova, with the main issue being that we really need cells to be feature complete. Hopefully people are working on a plan for this now. Another internal refactoring is the current scheduler work, which is important because it positions us for the future.

We also discussed the next gen Nova API, and talked through the proposed upgrade path for the transition from nova-network to neutron.

For those who are curious, there are 8,259 words (not that I am counting or anything) in this post series including this summary post. I estimate it took me about four working days to write (ED: and about two days for his trained team of technical writers to edit into mostly coherent English). I would love to get your feedback on if you found the series useful as it's a pretty big investment in time.

Tags for this post: openstack juno nova mid-cycle summary
Related posts: Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno nova mid-cycle meetup summary: scheduler; Juno nova mid-cycle meetup summary: ironic; Juno nova mid-cycle meetup summary: DB2 support; Juno nova mid-cycle meetup summary: social issues; Juno nova mid-cycle meetup summary: slots Comment

August 22, 2014 07:47 AM

Rachel Chalmers [rachel]

kittenbloggin’

So how’s your year been? Mine’s been pretty harsh. To be honest, I just wanted to bump that last post out of the top of the blog.

Ahoy!

I gotta say, these here shiny kittenses helped a lot.

Snuggles

 

August 22, 2014 04:42 AM

Michael Still [mikal]

Juno nova mid-cycle meetup summary: the next generation Nova API

This is the final post in my series covering the highlights from the Juno Nova mid-cycle meetup. In this post I will cover our next generation API, which used to be called the v3 API but is largely now referred to as the v2.1 API. Getting to this point has been one of the more painful processes I think I've ever seen in Nova's development history, and I think we've learnt some important things about how large distributed projects operate along the way. My hope is that we remember these lessons next time we hit something as contentious as our API re-write has been.

Now on to the API itself. It started out as an attempt to improve our current API to be more maintainable and less confusing to our users. We deliberately decided that we would not focus on adding features, but instead attempt to reduce as much technical debt as possible. This development effort went on for about a year before we realized we'd made a mistake. The mistake we made is that we assumed that our users would agree it was trivial to move to a new API, and that they'd do that even if there weren't compelling new features, which it turned out was entirely incorrect.

I want to make it clear that this wasn't a mistake on the part of the v3 API team. They implemented what the technical leadership of Nova at the time asked for, and were very surprised when we discovered our mistake. We've now spent over a release cycle trying to recover from that mistake as gracefully as possible, but the upside is that the API we will be delivering is significantly more future proof than what we have in the current v2 API.

At the Atlanta Juno summit, it was agreed that the v3 API would never ship in its current form, and that what we would instead do is provide a v2.1 API. This API would be 99% compatible with the current v2 API, with the incompatible things being stuff like if you pass a malformed parameter to the API we will now tell you instead of silently ignoring it, which we call 'input validation'. The other thing we are going to add in the v2.1 API is a system of 'micro-versions', which allow a client to specify what version of the API it understands, and for the server to gracefully degrade to older versions if required.

This micro-version system is important, because the next step is to then start adding the v3 cleanups and fixes into the v2.1 API, but as a series of micro-versions. That way we can drag the majority of our users with us into a better future, without abandoning users of older API versions. I should note at this point that the mechanics for deciding what the minimum micro-version a version of Nova will support are largely undefined at the moment. My instinct is that we will tie to stable release versions in some way; if your client dates back to a release of Nova that we no longer support, then we might expect you to upgrade. However, that hasn't been debated yet, so don't take my thoughts on that as rigid truth.

Frustratingly, the intent of the v2.1 API has been agreed and unchanged since the Atlanta summit, yet we're late in the Juno release and most of the work isn't done yet. This is because we got bogged down in the mechanics of how micro-versions will work, and how the translation for older API versions will work inside the Nova code later on. We finally unblocked this at the mid-cycle meetup, which means this work can finally progress again.

The main concern that we needed to resolve at the mid-cycle was the belief that if the v2.1 API was implemented as a series of translations on top of the v3 code, then the translation layer would be quite thick and complicated. This raises issues of maintainability, as well as the amount of code we need to understand. The API team has now agreed to produce an API implementation that is just the v2.1 functionality, and will then layer things on top of that. This is actually invisible to users of the API, but it leaves us with an implementation where changes after v2.1 are additive, which should be easier to maintain.

One of the other changes in the original v3 code is that we stopped proxying functionality for Neutron, Cinder and Glance. With the decision to implement a v2.1 API instead, we will need to rebuild that proxying implementation. To unblock v2.1, and based on advice from the HP and Rackspace public cloud teams, we have decided to delay implementing these proxies. So, the first version of the v2.1 API we ship will not have proxies, but later versions will add them in. The current v2 API implementation will not be removed until all the proxies have been added to v2.1. This is prompted by the belief that many advanced API users don't use the Nova API proxies, and therefore could move to v2.1 without them being implemented.

Finally, I want to thank the Nova API team, especially Chris Yeoh and Kenichi Oomichi for their patience with us while we have worked through these complicated issues. It's much appreciated, and I find them a consistent pleasure to work with.

That brings us to the end of my summary of the Nova Juno mid-cycle meetup. I'll write up a quick summary post that ties all of the posts together, but apart from that this series is now finished. Thanks for following along.

Tags for this post: openstack juno nova mid-cycle summary api v3 v2.1
Related posts: Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno nova mid-cycle meetup summary: scheduler; Juno nova mid-cycle meetup summary: ironic; Juno nova mid-cycle meetup summary: conclusion; Juno nova mid-cycle meetup summary: DB2 support; Juno nova mid-cycle meetup summary: social issues Comment

August 22, 2014 12:52 AM

Thomas Thurman [marnanel]

Gentle Readers: to cut a cabbage leaf

Gentle Readers
a newsletter made for sharing
volume 1, number 19
21st August 2014: to cut a cabbage leaf
What I’ve been up to

I've been looking into PhD possibilities. But more of that later.

And we visited the John Rylands Library for the first time, a beautiful place in Victorian Gothic made as a memorial to a local industrialist. (The law students among you may know him as a party to Rylands v Fletcher.) It has an impressive collection of books and manuscripts, including the oldest known fragment of the New Testament, part of John's gospel copied only a few decades after the book was written.

As if that weren't enough, the building is quite breathtakingly beautiful. Here's part of the reading room:

http://thomasthurman.org/pics/rylands-1

The library also contains a dragon named Grumbold. Regular readers who remember Not Ordinarily Borrowable, a story of mine largely about dragons and libraries, may judge of my surprise to discover it coming true.

A poem of mine

STORYTELLING, PART II (T80)

When Merlin looked upon this land,
he knew by magic arts
the anger in the acts of men,
the hatred in their hearts:
he saw despair and deadly things,
and knew our hope must be
the magic when the kettle sings
to make a pot of tea.

When Galahad applied to sit
in splendour at the Table,
he swore an oath to fight for good
as far as he was able.
But Arthur put the kettle on,
and bade him sit and see
the goodness that is brought anon
by making pots of tea.

When Arthur someday shall return
in glory, with his knights,
he'll rout our foes and bless the poor
and put the land to rights.
And shall we drink his health in ale?
Not so! It seems to me
he'll meet us in the final tale
and share a pot of tea.

A picture

http://gentlereaders.uk/pics/caught-the-sun

I was out fishing all day,
and I seem to have caught the sun

Something wonderful

Suppose I asked you to name the world's great heroes? (For example, as you may recall, some talk of Alexander.) Well, in the Middle Ages, a fair amount of thought went into the list. Who was an example of virtue and valour; whose chivalry was worth emulating?

One such list is known in English as the Nine Worthies. It was drawn up in the early 1300s, and remained a popular theme in art for centuries after. Here they are in 1460, looking for all the world like a medieval pack of Top Trumps:

http://gentlereaders.uk/pics/nine-worthies

Even though some of these men had lived (or were supposed to have lived) millennia earlier, they are all drawn wearing armour of the time, and bearing their own coat of arms, as if they lived in that very moment. This is because they are deliberately idealised-- after all, as a careful perusal of the Old Testament will show, not all of them were in fact models of chivalry.

They are divided into three groups of three: three Jewish heroes, three Christian heroes, and three pagan heroes-- that is, pagan in the old sense of not following an Abrahamic religion.

The Jewish heroes are: Joshua the son of Nun, who led the invasion of Canaan; David the son of Jesse, who became king and wrote psalms; and Judas Maccabeus, who led the revolt against the Syrians now commemorated by Hanukkah. (Don't confuse Judas Maccabeus with Judas Iscariot.)

The pagan heroes are: Hector of Troy, a great warrior of the Trojan War; Julius Caesar, the first emperor of Rome; and Alexander the Great.

The Christian heroes are: Arthur, the hero of the Matter of Britain; Charles the Great, also called Charlemagne, the first emperor of the Holy Roman Empire; and Godfrey of Bouillon, who became the first crusader king of Jerusalem but disclaimed the title.

I am particularly interested by the heraldry. How did they make up new and unique coats of arms for people who had been dead for three thousand years? David has a harp because he composed psalms (and not because he was king of Ireland). Julius has an eagle rather like the one on the Roman standard; Charles has the same, appropriately for someone who was also trying to become Emperor of Rome, but combined with the lily pattern known as "France Ancient". Others of them are baffling to me: what is Joshua bearing, for example? I did find a reference to the arms they made up for Alexander in a book, but frustratingly I ran out of time to research this.

I am glad to report that there were also nine female Worthies to balance out the nine men. Unfortunately none of the writers seem to agree about which nine women they were.

Something from someone else

When a certain Charles Macklin claimed he could repeat any sentence he heard, no matter how complex, Samuel Foote allegedly composed this sentence impromptu:

THE GREAT PANJANDRUM
by Samuel Foote

So she went into the garden
to cut a cabbage-leaf
to make an apple-pie;
and at the same time
a great she-bear, coming down the street,
pops its head into the shop.
What! no soap?
So he died,
and she very imprudently married the Barber:
and there were present
the Picninnies,
and the Joblillies,
and the Garyulies,
and the great Panjandrum himself,
with the little round button at top;
and they all fell to playing the game of catch-as-catch-can,
till the gunpowder ran out at the heels of their boots

Colophon
Gentle Readers is published on Mondays and Thursdays, and I want you to share it. The archives are at http://gentlereaders.uk/ , and so is a form to get on the mailing list. If you have anything to say or reply, or you want to be added or removed from the mailing list, I’m at thomas@thurman.org.uk and I’d love to hear from you. The newsletter is reader-supported; please pledge something if you can afford to, and please don't if you can't. Love and peace to you all.

This entry was originally posted at http://marnanel.dreamwidth.org/309388.html. Please comment there using OpenID.

August 22, 2014 12:30 AM

August 21, 2014

Eric Anholt [anholt]

X with glamor on vc4

Today I finally got X up on my vc4 driver using glamor.  As you can see, there are a bunch of visual issues, and what you can't see is that after a few frames of those gears the hardware locked up and didn't come back.  It's still major progress.

2014-08-21 16.16.37
The code can be found in my vc4 branch of mesa and linux-2.6, and the glamor branch of my xf86-video-modesetting.  I think the driver's at the point now that someone else could potentially participate.  I've intentionally left a bunch of easy problems -- things like supporting the SCS, DST, DPH, and XPD opcodes, for which we have piglit tests (in glean) and are just a matter of translating the math from TGSI's vec4 instruction set (documented in tgsi.rst) to the scalar QIR opcodes.

August 21, 2014 11:58 PM

Michael Still [mikal]

Don't Tell Mum I Work On The Rigs




ISBN: 1741146984
LibraryThing
I read this book while on a flight a few weeks ago. Its surprisingly readable and relatively short -- you can knock it over in a single long haul flight. The book covers the memoirs of an oil rig worker, from childhood right through to middle age. That's probably the biggest weakness of the book, it just kind of stops when the writer reaches the present day. I felt there wasn't really a conclusion, which was disappointing.
An interesting fun read however.

Tags for this post: book paul_carter oil rig memoir
Related posts: Extreme Machines: Eirik Raude; New Orleans and sea level; Kern County oil wells on I-5; What is the point that people's morals evaporate?
Comment

August 21, 2014 09:45 PM

August 20, 2014

Michael Still [mikal]

Juno nova mid-cycle meetup summary: nova-network to Neutron migration

This will be my second last post about the Juno Nova mid-cycle meetup, which covers the state of play for work on the nova-network to Neutron upgrade.

First off, some background information. Neutron (formerly Quantum) was developed over a long period of time to replace nova-network, and added to the OpenStack Folsom release. The development of new features for nova-network was frozen in the Nova code base, so that users would transition to Neutron. Unfortunately the transition period took longer than expected. We ended up having to unfreeze development of nova-network, in order to fix reliability problems that were affecting our CI gating and the reliability of deployments for existing nova-network users. Also, at least two OpenStack companies were carrying significant feature patches for nova-network, which we wanted to merge into the main code base.

You can see the announcement at http://lists.openstack.org/pipermail/openstack-dev/2014-January/025824.html. The main enhancements post-freeze were a conversion to use our new objects infrastructure (and therefore conductor), as well as features that were being developed by Nebula. I can't find any contributions from the other OpenStack company in the code base at this time, so I assume they haven't been proposed.

The nova-network to Neutron migration path has come to the attention of the OpenStack Technical Committee, who have asked for a more formal plan to address Neutron feature gaps and deprecate nova-network. That plan is tracked at https://wiki.openstack.org/wiki/Governance/TechnicalCommittee/Neutron_Gap_Coverage. As you can see, there are still some things to be merged which are targeted for juno-3. At the time of writing this includes grenade testing; Neutron being the devstack default; a replacement for nova-network multi-host; a migration plan; and some documentation. They are all making good progress, but until these action items are completed, Nova can't start the process of deprecating nova-network.

The discussion at the Nova mid-cycle meetup was around the migration planning item in the plan. There is a Nova specification that outlines one possible plan for live upgrading instances (i.e, no instance downtime) at https://review.openstack.org/#/c/101921/, but this will probably now be replaced with a simpler migration path involving cold migrations. This is prompted by not being able to find a user that absolutely has to have live upgrade. There was some confusion, because of a belief that the TC was requiring a live upgrade plan. But as Russell Bryant says in the meetup etherpad:

"Note that the TC has made no such statement on migration expectations other than a migration path must exist, both projects must agree on the plan, and that plan must be submitted to the TC as a part of the project's graduation review (or project gap review in this case). I wouldn't expect the TC to make much of a fuss about the plan if both Nova and Neutron teams are in agreement."


The current plan is to go forward with a cold upgrade path, unless a user comes forward with an absolute hard requirement for a live upgrade, and a plan to fund developers to work on it.

At this point, it looks like we are on track to get all of the functionality we need from Neutron in the Juno release. If that happens, we will start the nova-network deprecation timer in Kilo, with my expectation being that nova-network would be removed in the "M" release. There is also an option to change the default networking implementation to Neutron before the deprecation of nova-network is complete, which will mean that new deployments are defaulting to the long term supported option.

In the next (and probably final) post in this series, I'll talk about the API formerly known as Nova API v3.

Tags for this post: openstack juno nova mid-cycle summary nova-network neutron migration
Related posts: Juno nova mid-cycle meetup summary: scheduler; Juno nova mid-cycle meetup summary: ironic; Juno nova mid-cycle meetup summary: conclusion; Juno nova mid-cycle meetup summary: DB2 support; Juno nova mid-cycle meetup summary: social issues; Juno nova mid-cycle meetup summary: slots Comment

August 20, 2014 04:37 AM

August 19, 2014

Michael Still [mikal]

Juno nova mid-cycle meetup summary: slots

If I had to guess what would be a controversial topic from the mid-cycle meetup, it would have to be this slots proposal. I was actually in a Technical Committee meeting when this proposal was first made, but I'm told there were plenty of people in the room keen to give this idea a try. Since the mid-cycle Joe Gordon has written up a more formal proposal, which can be found at https://review.openstack.org/#/c/112733.

If you look at the last few Nova releases, core reviewers have been drowning under code reviews, so we need to control the review workload. What is currently happening is that everyone throws up their thing into Gerrit, and then each core tries to identify the important things and review them. There is a list of prioritized blueprints in Launchpad, but it is not used much as a way of determining what to review. The result of this is that there are hundreds of reviews outstanding for Nova (500 when I wrote this post). Many of these will get a review, but it is hard for authors to get two cores to pay attention to a review long enough for it to be approved and merged.

If we could rate limit the number of proposed reviews in Gerrit, then cores would be able to focus their attention on the smaller number of outstanding reviews, and land more code. Because each review would merge faster, we believe this rate limiting would help us land more code rather than less, as our workload would be better managed. You could argue that this will mean we just say 'no' more often, but that's not the intent, it's more about bringing focus to what we're reviewing, so that we can get patches through the process completely. There's nothing more frustrating to a code author than having one +2 on their code and then hitting some merge freeze deadline.

The proposal is therefore to designate a number of blueprints that can be under review at any one time. The initial proposal was for ten, and the term 'slot' was coined to describe the available review capacity. If your blueprint was not allocated a slot, then it would either not be proposed in Gerrit yet, or if it was it would have a procedural -2 on it (much like code reviews associated with unapproved specifications do now).

The number of slots is arbitrary at this point. Ten is our best guess of how much we can dilute core's focus without losing efficiency. We would tweak the number as we gained experience if we went ahead with this proposal. Remember, too, that a slot isn't always a single code review. If the VMWare refactor was in a slot for example, we might find that there were also ten code reviews associated with that single slot.

How do you determine what occupies a review slot? The proposal is to groom the list of approved specifications more carefully. We would collaboratively produce a ranked list of blueprints in the order of their importance to Nova and OpenStack overall. As slots become available, the next highest ranked blueprint with code ready for review would be moved into one of the review slots. A blueprint would be considered 'ready for review' once the specification is merged, and the code is complete and ready for intensive code review.

What happens if code is in a slot and something goes wrong? Imagine if a proposer goes on vacation and stops responding to review comments. If that happened we would bump the code out of the slot, but would put it back on the backlog in the location dictated by its priority. In other words there is no penalty for being bumped, you just need to wait for a slot to reappear when you're available again.

We also talked about whether we were requiring specifications for changes which are too simple. If something is relatively uncontroversial and simple (a better tag for internationalization for example), but not a bug, it falls through the cracks of our process at the moment and ends up needing to have a specification written. There was talk of finding another way to track this work. I'm not sure I agree with this part, because a trivial specification is a relatively cheap thing to do. However, it's something I'm happy to talk about.

We also know that Nova needs to spend more time paying down its accrued technical debt, which you can see in the huge amount of bugs we have outstanding at the moment. There is no shortage of people willing to write code for Nova, but there is a shortage of people fixing bugs and working on strategic things instead of new features. If we could reserve slots for technical debt, then it would help us to get people to work on those aspects, because they wouldn't spend time on a less interesting problem and then discover they can't even get their code reviewed. We even talked about having an alternating focus for Nova releases; we could have a release focused on paying down technical debt and stability, and then the next release focused on new features. The Linux kernel does something quite similar to this and it seems to work well for them.

Using slots would allow us to land more valuable code faster. Of course, it also means that some patches will get dropped on the floor, but if the system is working properly, those features will be ones that aren't important to OpenStack. Considering that right now we're not landing many features at all, this would be an improvement.

This proposal is obviously complicated, and everyone will have an opinion. We haven't really thought through all the mechanics fully, yet, and it's certainly not a done deal at this point. The ranking process seems to be the most contentious point. We could encourage the community to help us rank things by priority, but it's not clear how that process would work. Regardless, I feel like we need to be more systematic about what code we're trying to land. It's embarrassing how little has landed in Juno for Nova, and we need to be working on that. I would like to continue discussing this as a community to make sure that we end up with something that works well and that everyone is happy with.

This series is nearly done, but in the next post I'll cover the current status of the nova-network to neutron upgrade path.

Tags for this post: openstack juno nova mid-cycle summary review slots blueprint priority project management
Related posts: Juno nova mid-cycle meetup summary: social issues; Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno nova mid-cycle meetup summary: scheduler; Juno nova mid-cycle meetup summary: ironic; Juno nova mid-cycle meetup summary: conclusion; Juno nova mid-cycle meetup summary: DB2 support Comment

August 19, 2014 08:34 AM

August 18, 2014

Thomas Thurman [marnanel]

file upload thing

A thought: I keep running into people who need to schlep large files around, bigger than can be sent by email. (One example is someone I know who runs a dictation service for the blind.) So they have to fit things like yousendit or dropbox into their workflow, and often they don't fit as well as they might.

But there doesn't seem to be a free alternative to run on your own server. Today I realised that this could be done fairly easily as an extension to a bug tracker like Bugzilla-- take out most of the fields on the "create a bug" page, optionally add anonymous uploads and quotas, and you're pretty much there. This would be useful enough for me that I might well have a go.

Update: A friend suggests OwnCloud.

This entry was originally posted at http://marnanel.dreamwidth.org/309205.html. Please comment there using OpenID.

August 18, 2014 08:11 PM

Michael Still [mikal]

Juno nova mid-cycle meetup summary: scheduler

This post is in a series covering the discussions at the Juno Nova mid-cycle meetup. This post will cover the current state of play of our scheduler refactoring efforts. The scheduler refactor has been running for a fair while now, dating back to at least the Hong Kong summit (so about 1.5 release cycles ago).

The original intent of the scheduler sub-team's effort was to pull the scheduling code out of Nova so that it could be rapidly iterated on its own, with the eventual goal being to support a single scheduler across the various OpenStack services. For example, the scheduler that makes placement decisions about your instances could also be making decisions about the placement of your storage resources and could therefore ensure that they are co-located as much as possible.

During this process we realized that a big bang replacement is actually much harder than we thought, and the plan has morphed into being a multi-phase effort. The first step is to make the interface for the scheduler more clearly defined inside the Nova code base. For example, in previous releases, it was the scheduler that launched instances: the API would ask the scheduler to find available hypervisor nodes, and then the scheduler would instruct those nodes to boot the instances. We need to refactor this so that the scheduler picks a set of nodes, but then the API is the one which actually does the instance launch. That way, when the scheduler does move out it's not trusted to perform actions that change hypervisor state, and the Nova code does that for it. This refactoring work is under way, along with work to isolate the SQL database accesses inside the scheduler.

I would like to set expectations that this work is what will land in Juno. It has little visible impact for users, but positions us to better solve these problems in Kilo.

We discussed the need to ensure that any new scheduler is at least as fast and accurate as the current one. Jay Pipes has volunteered to work with the scheduler sub-team to build a testing framework to validate this work. Jay also has some concerns about the resource tracker work that is being done at the moment that he is going to discuss with the scheduler sub-team. Since the mid-cycle meetup there has been a thread on the openstack-dev mailing list about similar resource tracker concerns (here), which might be of interest to people interested in scheduler work.

We also need to test our assumption at some point that other OpenStack services such as Neutron and Cinder would be even willing to share a scheduler service if a central one was implemented. We believe that Neutron is interested, but we shouldn't be surprising our fellow OpenStack projects by just appearing with a complete solution. There is a plan to propose a cross-project session at the Paris summit to cover this work.

In the next post in this series we'll discuss possibly the most controversial part of the mid-cycle meetup. The proposal for "slots" for landing blueprints during Kilo.

Tags for this post: openstack juno nova mid-cycle summary scheduler
Related posts: Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno nova mid-cycle meetup summary: ironic; Juno nova mid-cycle meetup summary: conclusion; Juno nova mid-cycle meetup summary: DB2 support; Juno nova mid-cycle meetup summary: social issues; Juno nova mid-cycle meetup summary: slots Comment

August 18, 2014 04:06 AM

Juno nova mid-cycle meetup summary: bug management

Welcome to the next exciting installment of the Nova Juno mid-cycle meetup summary. In the previous chapter, our hero battled a partially complete cells implementation, by using his +2 smile of good intentions. In this next exciting chapter, watch him battle our seemingly never ending pile of bugs! Sorry, now that I'm on to my sixth post in this series I feel like it's time to get more adventurous in the introductions.

For at least the last cycle, and probably longer, Nova has been struggling with the number of bugs filed in Launchpad. I don't think the problem is that Nova has terrible code, it is instead that we have a lot of users filing bugs, and the team working on triaging and closing bugs is small. The complexity of the deployment options with Nova make this problem worse, and that complexity increases as we allow new drivers for things like different storage engines to land in the code base.

The increasing number of permutations possible with Nova configurations is a problem for our CI systems as well, as we don't cover all of these options and this sometimes leads us to discover that they don't work as expected in the field. CI is a tangent from the main intent of this post though, so I will reserve further discussion of our CI system until a later post.

Tracy Jones and Joe Gordon have been doing good work in this cycle trying to get a grip on the state of the bugs filed against Nova. For example, a very large number of bugs (hundreds) were for problems we'd fixed, but where the bug bot had failed to close the bug when the fix merged. Many other bugs were waiting for feedback from users, but had been waiting for longer than six months. In both those cases the response was to close the bug, with the understanding that the user can always reopen it if they come back to talk to us again. Doing "quick hit" things like this has reduced our open bug count to about one thousand bugs. You can see a dashboard that Tracy has produced that shows the state of our bugs at http://54.201.139.117/nova-bugs.html. I believe that Joe has been moving towards moving this onto OpenStack hosted infrastructure, but this hasn't happened yet.

At the mid-cycle meetup, the goal of the conversation was to try and find other ways to get our bug queue further under control. Some of the suggestions were largely mechanical, like tightening up our definitions of the confirmed (we agree this is a bug) and triaged (and we know how to fix it) bug states. Others were things like auto-abandoning bugs which are marked incomplete for more than 60 days without a reply from the person who filed the bug, or unassigning bugs when the review that proposed a fix is abandoned in Gerrit.

Unfortunately, we have more ideas for how to automate dealing with bugs than we have people writing automation. If there's someone out there who wants to have a big impact on Nova, but isn't sure where to get started, helping us out with this automation would be a super helpful way to get started. Let Tracy or I know if you're interested.

We also talked about having more targeted bug days. This was prompted by our last bug day being largely unsuccessful. Instead we're proposing that the next bug day have a really well defined theme, such as moving things from the "undecided" to the "confirmed" state, or similar. I believe the current plan is to run a bug day like this after J-3 when we're winding down from feature development and starting to focus on stabilization.

Finally, I would encourage people fixing bugs in Nova to do a quick search for duplicate bugs when they are closing a bug. I wouldn't be at all surprised to discover that there are many bugs where you can close duplicates at the same time with minimal effort.

In the next post I'll cover our discussions of the state of the current scheduler work in Nova.

Tags for this post: openstack juno nova mi-cycle summary bugs
Related posts: Juno nova mid-cycle meetup summary: nova-network to Neutron migration; Juno nova mid-cycle meetup summary: scheduler; Juno nova mid-cycle meetup summary: ironic; Juno nova mid-cycle meetup summary: conclusion; Michael's surprisingly unreliable predictions for the Havana Nova release; Juno nova mid-cycle meetup summary: DB2 support Comment

August 18, 2014 03:38 AM

August 17, 2014

Don Marti [dmarti]

Original bug, not original sin

Ethan Zuckerman calls advertising The Internet's Original Sin. But sin is overstating it. Advertising has an economic and social role, just as bacteria have an important role in your body. Many kinds of bacteria can live on and around you just fine, and only become a crisis when your immune system is compromised.

The bad news is that the Internet's immune system is compromised. Quinn Norton summed it up: Everything is Broken. The same half-assed approach to security that lets random trolls yell curse words on your baby monitor is also letting a small but vocal part of the ad business claim an unsustainable share of Internet-built wealth at the expense of original content.

But email spam didn't kill email, and surveillance marketing won't kill the Web. Privacy tech is catching up. AdNews has a good piece on the progress of ad blocking, but I'm wondering about how accurate any measurement of ad blocking can be in the presence of massive fraud. Fraudulent traffic is a big part of the picture, and nobody has an incentive to run an ad blocker on that. The results from the combination of fraud and use of privacy tools are unpredictable. Paywalls are the obvious next step, but there are ways for sites to work with privacy tools, not against them.

What Ethan calls pay-for-performance is the smaller, and less valuable, part of advertising. Online ads are stuck in that niche not so much because of original sin, but because of an original bug. When the browsers of Dot-Com Boom 1.0 came out in a rush with support for privacy antifeatures such as third-party tracking, the Web excluded itself from lucrative branding or signaling advertising. The Web became a direct-response medium like email spam or direct mail. Bob Hoffman said, The web is a much better yellow pages and a much worse television. But that's not inherent in the medium. The Web is able to carry better and more signalful ads as the privacy level goes up. That's a matter of fixing the privacy bugs that allow for tracking, not a sin to expiate.

Recent news, from Kate Tummarello at The Hill: Tech giants at odds over Obama privacy bill. Microsoft is coming in on one side, and a group of mostly surveillance marketing firms calling itself the united voice of the Internet economy is on the other. There's no one original sin here, but there's plenty of opportunity in fixing bugs.

Bonus links

Jeff Jarvis: Absolution? Hell, no

Jason Dorrier: Burger Robot Poised to Disrupt Fast Food Industry

BOB HOFFMAN: Confusing Gadgetry With Behavior

August 17, 2014 12:21 PM

August 16, 2014

Mary Gardiner [hypatia]

July 2014

I took about a week to get over my jetlag from the USA, but it was really rather mild. I would just get on with my day, only as soon as the sun set, the day would be over. The unpleasantness was mostly that this meant that for about a week, I worked and slept and did nothing else.

I met my mother, aunt and sister in Hornsby — where Andrew and I lived for 5 years and where V was born — the Friday afternoon after I got back, which was odd. Of course, most things are exactly the same, but there was also no point to it. We didn’t have friends up there even at the time and there was nowhere to go where people would remember me unless the food sellers in the (very very busy) local shopping centre remember a very tall past customer.

The playground where V played the most was exactly the same, but he’d forgotten it, and it was also verging on being a little young for him. Hornsby Shire Council is not the City of Sydney in terms of devotion to adventure playgrounds. I drove past the hospital to show V where he was born, and realised I’m not sure I’d remember which birth suite it was (even if they were inclined to allow random people to traipse through the delivery ward, which of course they would not be). Hornsby Hospital, which was a state-wide scandal in terms of maintainence, has had some money spent on it in the last few years. The building which I believe contained the old maternity ward my mother was born in has been knocked down and replaced with something in blocky primary colours, looking much like the new Royal North Shore hospital. It shouldn’t surprise me there are trends in hospital design, but it does.

V, of course, was politely puzzled by the idea that he had ever lived in this place or been in that hospital and so on. He also didn’t recognise his old daycare centre.

The block of flats we lived in — we still own the flat — looks exactly the same as when I last saw it more than two years ago, but there’s a speed bump in the street, and a new Thai restaurant where the sad failed grocer was. (Sad both in that any failed small business is sad, and also that they appeared to have sunk a lot of thought and effort into the fitout, trying to set up a slightly fancy deli that was patronised mostly by me and Andrew. They left on what would have been about the first annual review of their lease.) I entirely forgot to check if the Blockbuster franchise was still there; I would assume not.

I had intended to spend most of a day up there remembering things, in the end I drove around and left after 45 minutes.

It was probably that trip that inspired me into a very brief foray into the Sydney property market the following weekend, to wit, inspecting two properties. One we arrived at only to be told by a bored agent it had been bought before its first public inspection. “Yeah, sorry. That’s how it goes!” The other involved real estate agents cornering us to let us know how very motivated (very, very motivated) the seller was, and wanting to have a big discussion about what we were looking for in the market and what we thought about the market and why we thought that and whether they’d be of any help re-aligning our thoughts for us and what kind of finance we might have access to or could be assisted with, and etc, and were not easily put off by “we live just up the street and are having a sticky-beak, and also, this apartment is down two internal flights of stairs and we have a baby in a stroller, so no.” I suppose it could be worse, we could have actually bought the place. But it was surprisingly difficult to get away from them even as entirely unmotivated buyers.

And that was Andrew’s cue to nick off to the USA himself. It was a long and lonely trip at my end, probably much the same as mine for him. As our work trips become increasingly totalising — he was expected to have all three meals a day with work colleagues he needs to know better, I took a baby with me — we’ve dropped off our communications. I spoke to him a couple of times while I was away (and mostly in order to speak to a very bored and slightly bewildered V, at that), I think while he was away we had a couple of abortive attempts at video chat and that was about it. Not much fun having a chat that consists mostly of “… no, I still can’t hear you, oh, I just saw you wave, nope, now you’ve frozen, can you hear me? CAN YOU HEAR ME?” It got even worse when he got to London and didn’t have a local SIM and was impossible to reach at all.

Andrew works in an office, but I don’t, so when he travels I can go for days without having face-to-face interactions with other adults that aren’t transactional. (“Have I paid for V’s dance class this week? No? Here’s the fee!”) So I took V and A to my parents for three nights in the middle of Andrew’s trip. Packing alone for a trip is always really annoying and boring, but the drive that I was dreading (about 4 hours each way) ended up being surprisingly painless. V remains a good and surprisingly non-whingy car traveller and A sleeps even better in cars than he used to. The first morning we were there they had their snowfall of the year; unfortunately we hadn’t brought gloves with us but had a bit of fun anyway, with my parents hauling V around on a tarpaulin “sled”.

Once I was back, I warned Val that I was feeling slightly ill and was having an inexplicably grumpy and sad day. (The amount of emotional work and intimacy required in a small business can be high, but I do like being able to rearrange my day around being grumpy every so often.) It got much more explicable when I realised I was having cellulitis symptoms in my left ankle (an infection of soft tissue under the skin).

I had cellulitis in September 2012 with a slightly unusual and very aggressive presentation: I got a high fever first, about 24 hours before there was any redness or swelling and so on. By the time the redness was even really properly visible, I had been running a 40°C fever for several days, could barely walk due to the painful swelling of lymph nodes, was dehydrated, and was admitted to hospital for 6 days of IV antibiotics (and three days of rehydration, because I refused to take anything by mouth). When I was in there, the infectious diseases registrar asked if she should draw the boundaries of the redness on my leg to check if it was spreading, and the specialist said mildly “I don’t think there’s much point to that.” He was quite right: within a couple more days, the redness had spread all over my left thigh, and I ended up losing two layers of skin from most of my inner thigh, very much (as the specialist pointed out) as if I’d badly burned it. The day before I was discharged, he stopped by my bed alone and remarked that it was cases like this that “remind us that even in the age of antibiotics, these things can be very aggressive, and sometimes even fatal.”

… Thanks.

So, naturally, I panicked that I was having cellulitis symptoms again, only this time with two children in my care and Andrew in London (so, timezone-flipped) and close to unreachable other than by email. It wasn’t, in the end, justified: this time I got the redness and swelling but no fever or systemic illness, and a couple of courses of antibiotics cleared it up without me losing any skin, although I did walk with a cane for a couple of days due to lymph node pain. It was no worse than having twisted an ankle a bit in the end. It was tough on the extended family, as I set up Illness Level Red in case of needing to be hospitalised, unnecessarily in the event. (Andrew and I agreed that he’d arrange to leave London early as soon as I started running a fever, so he ended up leaving as planned.)

As a concrete thing, Andrew and I are going to have to work a bit more about communicating, and being accessible, while each of us travels. I used to talk about emotionally putting our marriage on ice for the duration — which is already much easier for the person who is travelling than the person left behind — but it’s not possibly for parenting, especially if the at-home parent gets taken out of action.

Once Andrew was back, all was well with the world. For the week and a half it took him to incubate the influenza he presumably picked up while travelling, anyway… stay tuned.

August 16, 2014 09:29 PM