Move along, nothing to read

Working on my conference travel plans and wishes for the year. I am downgrading OLS to a maybe, the cost is becoming more of a factor. Likewise, while I had incredible fun at FOSDEM last year, and OSCON in 2006, I cannot justify the airfare/hotel expenses for them. I would like to attend SCALE at some point as well, but uncertain for the same cost reason.

• February 25-27, SCALE 9x @ Los Angeles, CA, USA. [SCALE9x].
• April 11-14, MySQL UC @ Santa Clara, CA, USA [MySQLconf].
• April 13, Embedded Linux Conference @ San Francisco, CA [ELC].

• April 27-30, STS-134 launch @ Kennedy Space Centre, FL, USA [STS134].
• May 26-29, Bowen Island, BC, Canada.
• June 25-26, Mini Maker Faire Vancouver @ Vancouver, BC, Canada.
• August 17-19, LinuxCon 2011 @ Vancouver, BC, Canada [LinuxCon].
• August 25-28, PAX Prime 2011 @ Seattle, WA, USA.

• July 25-29, OSCON @ Portland, OR, USA.
• August 7-11, SIGGRAPH 2011 @ Vancouver, BC, Canada.
• October 19-22, Access 2011, @ Vancouver, BC, Canada.
• October 22-23, Google Summer of Code 2011 Mentor Summit @ Mountain View, CA, USA.

• February 5-6, FOSDEM @ Brussels, Belgium.
• June 13-15, Linux Symposium @ Ottawa, ON, Canada.
• September 7-9, Linux Plumbers @ Santa Rosa, CA, USA.

SCALE9x

Arriving on the 24th actually

MySQLconf

I will be manning the phpMyAdmin booth, like past 5 years.

ELC

Dropped in for just one day for hallway track

STS134

KSC grandstand seats to see the penultimate launch :-)

LinuxCon

Local this year, so no travel costs :-)

2011/04/26

Added GSoC, Access 2011, Bowen Island, STS-134

I apologize for doing this, but recent onslaught of spammers (~35 in the last 5 days) have left me with no choice: I've changed comments from non-friends to be screened by default AND require a captcha.

(This post inspired by Petteri Räty (betelgeuse)'s similar post

For this year's Gentoo GSoC projects, I'm a mentor on two of our suggested ideas (but also interested in totally new ideas that fit my fields):

• upstart on Gentoo
• Distfile Fetcher Intelligence

Do you actually understand the project idea?

This is actually a gap that I didn't expect to exist, but I have seen in previous years. This is mainly a difference of expectations between the proposal and what the potential student sees as what the idea really entails.
Using Upstart as an example, it supports an existing init.d compatibility mode, but we're not interested in that. Instead we want our init.d scripts to be treated just like upstart jobs (located in /etc/init/). The init.5 manpage shipped with upstart gives a good start...

Code maintainability

betelgeuse spoke about long-term maintenance, but you should think about it long ahead of that. Some degrees of abstraction, and avoiding difficult to understand logic should be prevalent here. betelgeuse mentioned spaghetti code, but it's important to realize that even well formatted code can impose a much larger mental workload if not well thought out.

Timezones, Timezones!

Most of your project should not be blocking on asking for mentor advice, as timezones and real world pressures often conspire to prevent easy real world communication. I may live in UTC-7, but my hours drift as needed by work but I tend to be online anywhere between 17h00 UTC and 10h00 UTC. If you're trying to communicate with me on a regular basis, this can be tough, so being able work on a problem independently, ask highly directed questions via email can go a long way.

Sitting in the MirrorBrain talk at FOSDEM, taking notes.

Actively used since ~2007.
Split between the redirector and the tester, explicitly made separate.
SourceForge helped with the ASN/Closest-Network side.
Metalinks and P2P support.
Scans mirrors for filelist to see what's present.
Load limiting by making director support mirrors that are limited to a local network / AS / country etc.
MetaLinks don't have Magnet links presently, but I noted that it should be possible to include it.

Using GeoDNS directly for lookups can cause trouble with partial mirrors. Ideally need to put a MirrorBrain server on each continent/region, and GeoDNS to point to that. Also, from some countries, bandwidth to adjcaent countries that might have a mirror is MUCH worse than bandwidth to a well-connected country elsewhere. Past user experience noted with a user in Mozambique, for whom the fastest mirror was via satellite to Canada. Routing data IS needed to make that best choice.

MirrorBrain mailing lists also have a generic non-project-specific "networkers" list for talk between content providers and mirror admins, non-specific to any app.

My bicycle was stolen earlier this evening. Sufficiently close to see the guy cycling away with it. Drove around a bit with Dave looking for it, but didn't find :-(

• Orange DeVinci St Tropez (large)
• Reward if you return it!
• Extensive scuff damage to the handlebar ends
• Panier Rack
• Rain fenders
• 2x front LED lights
• 2x rear LED lights
• Serial: SA…863

Last time I had my bike stolen I was in the downtown eastside. This time it was stolen from outside my house, NOT visible from the street or alley, around 23h00 at night.

I've been prodding at the concept of the new network script in OpenRC-0.5, and I'm at a loss to try and see why Roy has decided to toss the old network config system away. The new system doesn't have a lot of capabilities, and most significantly totally loses the ability to restart a single interface without affecting the rest of the system. If it's just for a rewrite, then I'm not too worried, but unless all the functionality is still there, I'm worried we are going to move backwards with it.

At the same time, I don't think many people are aware of how powerful the "old" network configuration mechanism is. The net.examples file is only the start, once you start mixing in the pre/post calls, there's a lot of power. It's capable of some feats that I don't see used even in certain parts of the Gentoo documentation^[1]. I've put together some of my gems of conf.d/net, and if you have some, I'd love to hear them. Leave a comment or email me the scripts, along with a description.

Configurations available

• Easy to maintain HE.net (Hurricane Electric) IPv6 tunnels - Download
• Running two ISPs at home (basic multi-homing) - Download
• "Enterprise" multi-homing setup, with 4 paths to the Internet - Download

Hosting

I've also started a bit of storage in my Gentoo webspace for these collected works of network configuration, with a bit more documentation.

Notes

1. The Gentoo docs have this for IPv6: Gentoo IPv6 Router Guide, Tunnel Configuration. You could bring it up manually, or you could just take the IPv6 config above and use it straight with your variables filled in. Volunteers welcome to help merge that config into the Gentoo IPv6 documentation.

To add a new USE flag, that's globally enabled for all Linux profiles, what's the minimum set of profiles that need to change? Deprecated profiles must be handled as well, for users that need to migrate still.

I ran into this today, while working on the USE=modules changes for linux-mod.eclass.

As an attempt to solve this, I munged up some GraphViz work to show profile inheritance, pictures as the end. Both sets have the trailing profiles "/desktop", "/developer", "/server" turned off for the 2008.0 and 10.0 releases, to cut down on the noise.

Graphs and script for download.

Odd observations

• Several Prefix profiles (linux/{amd64,ia64,x86} link to 2008.0 profiles explicitly instead of the generic architecture)
• default/linux does not bring in base. Some profiles at a glance neglect this and might not have base brought in at all.
• "embedded" is all alone in the tree.

Question for any skilled GraphViz users:

If all nodes in a given subgroup/cluster have an edge going to a single destination node, is there any way to get graphviz to replace them with a single fat edge from cluster to destination node?

I wasn't aware of this forthcoming movie until some of my co-workers mentioned it today, and watching the trailers, it wasn't the movie itself that caught me (but it looks promising too), but rather the usage of South African settings and historical footage from the apartheid era. As a South Africa, some of it is just too eerily familiar. It's just really apartheid again, with a twist that it's against "real aliens" now, not the forced alienation of the black population.

• Trailer #1
  Original footage: Starting at 0:35, for 6 seconds.
• Trailer #2
  Same original footage, 0:26 for 2 seconds.
• Original short film
  The SABC logo is too new for 1990, the one used is post-1994.

I really need to get back to writing in this blog. In the meantime, I scoured my email for the last 2 years of fortune submissions that I hadn't compiled together yet, and make a release. Go forth and amuse yourselves with it.

Now for the second set of statistics. These aren't directly useful to mirrors in estimating their traffic, but instead gives a good overview of how our mirroring setup works internally, and now much traffic is involved in the fan-out stage. Distfiles are the main content moved around by this system, but it is also used for the other directories for releases, experimental and snapshots.

A very quick overview of the existing setup:

1. Developer uploads new distfile directly to dev.gentoo.org.
2. The master-distfiles box pulls from dev.gentoo.org hourly.
3. The master-distfiles box checks every ebuild, and downloads missing distfiles from their primary URI if they do not exist. The daily distfile report is also created at this point.
4. Every hour, the cluster master of ftp.osuosl.org pulls the latest content from master-distfiles. (Averages 240MB/day of traffic).
5. The OSL FTP cluster master (in Oregon) pushes to it's slave locations in Atlanta and Chicago.
6. All distfiles mirrors pick up their content from one of the FTP nodes - Internet2-connected hosts are directed via DNS to an Internet2-connected slave for performance.

Each of the distfiles mirrors has about 140-160MB of upstream traffic every day (including both the new files and the rsync overhead for scanning). If there are no files changed, the rsync traffic for a directory scan is 1-2MB. While this isn't a lot of traffic, it's very spiky, as mirrors tend to be on fast links.

The new weekly builds from the Release Engineering team will probably be adding another 1.3GB per week, staggered as one arch per day.

I got a small subset of the logs from the OSU FTP cluster for processing some of these statistics. They cover the 24 hour period of 2008/08/07 UTC. It does not have data of which traffic went via Internet2, and I've grouped the sources by country code (using IP::Country::Fast from CPAN).

As a bit of analysis, I think that more than half of our mirrors (Europe, Middle East, RU) would benefit from having a box to sync against in Europe.