[Foresight-devel] Foresight (past, present, future) , and the distro world as a whole... - part two

[António Meireles]

with a bit higher latency than predicted here's part two, of latest mail
regarding our collective future.

*-- break for infomercial*
Last week rPath made available on http://hg.rpath.com/mirrorball their
fully automated (as possible) framework to re-wrap any foreign apt/rpm
with conary.
It's clean, it's nice, and it works, being easy to grasp and understand.

*-- on the future, and strategic choices*
Supporting a distro, end to end, is not a lightweight business, it
implies maintaining and tracking several thousands of packages, and a
minimal level of understanding about them and the way they relate
between then, in order to get a consistent set. A distro is a set
several things - a set of choices that are supposed to result in a
consistent vision about how things should be deployed to the end user, a
set of patches to develop on that vision and a community of users and
developers to bring it all possible. Right now, we don't have the
resources to support as many packages as the 'big' ones - they have
dedicated staff, we don't, to start, so we need to focus on what we are
absolutely best, and go from there. This implies to be creative, to make
choices and trade-offs, to be absolutely focused, and to execute flawlessly.

With rPath's release of mirrorball, it's quixotic to try to fight
against what will come due to it, so what we could do is to get the best
of it. Yes, it can wrap (with conary) _any_ rpm/deb based distro, and
can wrap/keep it tracked with relatively low overhead (man power wise).
The bad news - it is _just_ a binary rewrap, with all the limitations it
implies (albeit in the near future, 'automated' refactoring resulting
from straight builds up from spec rpms will be possible which will
mitigate things a bit, one is still limited by rpm's weak semantics),
the good news - we, conary based platforms, get access to the vast array
of ready to use packages on the 'legacy' platforms, on the cheap.

So, let's be smart and creative, and get the best of all worlds.
Foresight evolved, over time, on top of rPL; rPL was started by the very
same people who started RHEL... and fedora, so, if just for consistency,
it makes sense, to make a 'marriage of convenience' with fedora. Yes,
'hooking' on Ubuntu, or OpenSuse, would be possible too, but the
deviation, from current behavior in a lot of fronts, would be far
greater, for no obvious gains.

So, on what would consist this 'marriage of convenience' ?  On two
parallel, symbiotic paths, maintaining and developed under the _same_
umbrella - Two full distributions - one - the 'easy' one, containing the
full  conaryfied fedora repositories, plus a few extra key ones, like
rpm-fusion, lets call this just '*boots*', to honor trademark
restrictions. boots would be time based, trickinh upstream fedora
releases. The 'other' one, a full, from the ground up, fully source, and
conary recipes, based distribution, on good old foresight/rPL fashion,
let's call it *foresight3*.  Regarding 'boots', on itself, mkj will get
deeper in the details... regarding *foresight3* here's what i envision...

   1. as we don't have the resources, nor enough people, to maintain our
      own specific patchsets, we need to follow those who have, and make
      the best advantage of it. At low level, toolchain/major libs, we
      'll track, bug by bug, Fedora ones (just ignoring irrelevant, for
      us, technologies - like selinux and gcj support, in the sake of
      simplicity). At the layers above (with UI exposure), it will be on
      a case by case scenario (see below). The end goal is not to have
      zero bugs, is to have zero bugs that are f3 specific (other than
      ones specific to our tools - our anaconda, conary, etc, our
      packaging). As we are dropping reliance on rPL, this is an
      excellent opportunity to get more people in our community engaged
      in maintenance, and exposed to conary voodoo. And we should also
      make sure that we are being active in fixing/reporting bugs back
      up the chain to the Fedora community as needed. We don't want our
      relationship with Fedora to be purely parasitic.
       
   2. There should be interoperability, as we want, expect, most
      packages, from the boots repositories to install, cleanly on top
      of f3 either to fill gaps, or just for testing. This implies that
      both   sides must follow a given set of conventions to make this
      transparent, and sane. This will not be true in all cases,
      naturally - If one bumps gnome on f3, gnome packages from boots,
      won't install on f3, but _then_ the ones from  boots/rawhide
      should. In general the goal is to avoid *gratuitous* difference to
      enhance the possibility for sharing/scaling, particularly with
      regard to packages that are outside foresight's core vision, such
      as server-related packages. To ease things we expect both sides to
      have a very similar group layout. Where there should be as close
      matching is on groups layouts and package naming, for obvious
      reasons.

   3. this doesn't mean we will have to be tied by RH/Fedora's agenda,
      or schedules. The core purpose of building and maintaining a full
      distro from source is to get full flexibility. It is our full
      belief that one can get more flexibility, compared to anything
      else, by using conary, and co-related tools, so the road-map to f3
      is to get, using a full conary based recipe tree, at same point,
      full parity, with a relevant subset of Fedora (say, toolchain +
      stuff to get toolchain bootstrapped against itself + Xorg + Gnome
      + Kde, etc), and _then_ go from there. (see 5.)

   4. albeit mirrorball is, right now, 'just' a tool to wrap foreign
      distros into conary, at its inner there's lot of cool stuff on it
      already. I envision it to be extended, to become f3's central
      'nervous system', a console that will look at our repositories, at
      foreign ones, and just present us with the options/issues we'll
      have, at any given moment, allowing us to know how we compare
      version wise, patch-wise, bug-wise, against all else, warning of
      new versions, dead patches, update suggestions, availability of
      security fixes, etc, at a micro and macro level. A console that
      will integrate with our bug-tracker, and foreign ones. This new
      tool will be central to our strategy, and ongoing development,
      allowing us to scale better, de-dup efforts, communicate better,
      and ultimately allowing us to provide a best of breed solution to
      our end-users, quickly. (the patch part is not incompatible with
      what was said before, re: tracking fedora patches - we should be
      conservative at low level, for obvious reasons, but above that
      other than keeping consistent with them  re: pkgNaming
      conventions, we 're free).

   5. as soon as we get feature parity, with a given (stable) fedora -
      the show truly begins, since as we won't have the technology
      limitations they have, we will keep updating our tree, with newer
      stuff, we deem as stable (eg. major Gnome releases, Xorg, kde,
      etc), stuff that on their side, will be relegated to their
      unstable repos (as they - in our lingo - only 'promote' to stable,
      at time of every major release). We will keep being _rolling_. (as
      feature wise, ideally, in the packages f3 provides natively, in
      the worst case, they will be the _exactly_ same as boots, modulo
      intentional deviation to track newer packages)

   6. Being close to fedora behavior, lowers learning curve to their
      users and provides a migration path to them. As soon as it becomes
      plain obvious (if isn't already) that a conary (source) based tree
      is insanely easier to maintain than one based in legacy packaging
      technologies, our hope is to attract foreign maintainers, like
      those not on RH's payroll, to make f3 their _primary_ development
      platform.   

   7. there will be straight and clean migration paths either from fl:2
      to boots and fl:3, and from boots to fl:3.


So, and paraphrasing mkj, am i got definitively insane ?

Feedback welcome.

All the best,

*António Meireles*, aka *doniphon*

P.S. 1. next - the *f3* development model in detail.
P. S.  2.  thanks for mkj, smerp and MarkT for helping/commenting on
writing this, and proof-reading, and keeping me with a(t least one) 
foot on land.