I saw your notes, and I'd like to add my comments to some things, if you don't mind:
> SourceForge helped with the ASN/Closest-Network side.
It was vice versa :-) I helped them, and they use my code for this purpose. I'm not sure if they also use mod_asn, the lookup Apache module, but they definitely use the database backend that I designed for that. http://mirrorbrain.org/news/sourceforge_uses_part_of_mirrorbrain/
> MetaLinks don't have Magnet links presently, but I noted that it should be possible to include it.
The one part that's missing here is a better way to store hashes. Right now, the hashes that MirrorBrain stores are stored in a way that would require MirrorBrain to parse a file to construct a magnet link. I'd rather want to avoid that. So, the hash cache needs to undergo a little redesign (which is planned anyway), and then magnet links will be trivial to implement.
> Using GeoDNS directly for lookups can cause trouble with partial mirrors.
Yes, similar as DNSrr causes trouble with partial mirrors.
> Ideally need to put a MirrorBrain server on each continent/region, and GeoDNS to point to that.
That would be my favourite setup for failover and load balancing, and for minimizing latency for the clients (which matters in particular when clients have to request lots of files).
> Also, from some countries, bandwidth to adjcaent countries that might have a mirror is MUCH worse than bandwidth to a well-connected country elsewhere.
... the best link almost always being either to Central Europe, to the U.S., or both.
> Past user experience noted with a user in Mozambique, for whom the fastest mirror was via satellite to Canada. > Routing data IS needed to make that best choice.
In fact, routing data doesn't help with that. (Or I haven't found a way to make it useful.) Thus, what I use to deal with these cases is manual configuration. In effect, I configure an association of client countries to mirrors where the connection is known to work (always under the condition no local mirror is available).
> MirrorBrain mailing lists also have a generic non-project-specific "networkers" list for talk between content providers and mirror admins, non-specific to any app.
The general mailing list is discuss@mirrorbrain.org (see http://mirrorbrain.org/communication/). The MirrorBrain-specific mailing list is mirrorbrain@mirrorbrain.org.
Thanks again, and I'm looking forward to further discussion! Peter
![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png){kind=small}
Comments to your MirrorBrain notes
Date: 2010-02-16 06:19 pm (UTC)thanks for visiting the talk & the discussion!
I saw your notes, and I'd like to add my comments to some things, if you don't mind:
> SourceForge helped with the ASN/Closest-Network side.
It was vice versa :-) I helped them, and they use my code for this purpose. I'm not sure if they also use mod_asn, the lookup Apache module, but they definitely use the database backend that I designed for that.
http://mirrorbrain.org/news/sourceforge_uses_part_of_mirrorbrain/
> MetaLinks don't have Magnet links presently, but I noted that it should be possible to include it.
The one part that's missing here is a better way to store hashes. Right now, the hashes that MirrorBrain stores are stored in a way that would require MirrorBrain to parse a file to construct a magnet link. I'd rather want to avoid that. So, the hash cache needs to undergo a little redesign (which is planned anyway), and then magnet links will be trivial to implement.
> Using GeoDNS directly for lookups can cause trouble with partial mirrors.
Yes, similar as DNSrr causes trouble with partial mirrors.
> Ideally need to put a MirrorBrain server on each continent/region, and GeoDNS to point to that.
That would be my favourite setup for failover and load balancing, and for minimizing latency for the clients (which matters in particular when clients have to request lots of files).
> Also, from some countries, bandwidth to adjcaent countries that might have a mirror is MUCH worse than bandwidth to a well-connected country elsewhere.
... the best link almost always being either to Central Europe, to the U.S., or both.
> Past user experience noted with a user in Mozambique, for whom the fastest mirror was via satellite to Canada.
> Routing data IS needed to make that best choice.
In fact, routing data doesn't help with that. (Or I haven't found a way to make it useful.) Thus, what I use to deal with these cases is manual configuration. In effect, I configure an association of client countries to mirrors where the connection is known to work (always under the condition no local mirror is available).
> MirrorBrain mailing lists also have a generic non-project-specific "networkers" list for talk between content providers and mirror admins, non-specific to any app.
The general mailing list is discuss@mirrorbrain.org (see http://mirrorbrain.org/communication/).
The MirrorBrain-specific mailing list is mirrorbrain@mirrorbrain.org.
Thanks again, and I'm looking forward to further discussion!
Peter