It's a Cycle of Life Thing: Managing Linux ReleasesOne of the things I used look forward to about twice a year was a new release of Red Hat (RH) Linux. I'm pretty conservative on my main desktop system, where I write my articles and do my main development, and tend to run a pretty out-of-the-box configuration. I leave my hotrodding for my experimental system, where it doesn't matter if I accidentally trash a disk. Up until recently (say, Red Hat 7.2), upgrading my system wasn't a major hassle, because the overall rate of change of packages was relatively small and the improvements in system performance, as well as the look and feel of the user interface/experience, got to be more fun with each new release.
The release of the 2.4 series kernel made a lot more functionality available to developers, and the Linux community has taken advantage of it with wild abandon. With the release of Red Hat 7.3 (and SuSE 8.0, and most other Linux distributions from about mid-2001), I noticed a sudden bump in the number of applications available and a radical change in the dependencies in any given distribution, release after release.
For many users, upgrading a system or two is not a major hassle; they put in
the CD-ROMs and go. However, the picture gets a bit more interesting if you're
an enterprise using Linux for development or, dare I say it, as the corporate
desktop standard for your users. A small change to a critical library (like
glibc or gtk) can cause applications to break. A
change in application functionality sprung on unwary users can cause support
headaches galore. One of the great things about Linux is how quickly new
software and services are becoming available--but it's also turning into a
major headache for enterprise IT support people who have to keep track of all
of these changes and software versions. Let's face it, the system itself (and
its OS) is in reality only a small part of the Total Cost of Ownership (TCO)
and one reason why end-users would rather fight than switch when it comes to
operating systems and applications.
|
Also in Linux in the Enterprise: Unfinished Business Part 2: Closing the Circle Unfinished Business: The One Missing Piece Linux in the Enterprise at LWE 2003 |
One of the selling points and competitive levers for commercial Linux distributions is how many applications they bring to the table for their various constituencies. For power programmers, it's all about the depth of the development environment. For those deploying web services, it's the degree of support for Apache and related technologies like Tomcat. Most recently, for the desktop user, it's been the ever-growing number of applications available on the desktop, like StarOffice and OpenOffice.org.
It's interesting to note that in the RH7.2 release there were perhaps 75 desktop GUI-based packages and utilities available. Under RH7.3, and now 8.0, a default, standard desktop system includes over 200 (yes, 200!) unique applications installed and available for the GNOME and KDE desktop environments. All of these programs have some level of dependence on other parts of the system, and as the depth of applications grow, so does the level of dependence.
Before Red Hat developed the first serious package manager--the Redhat
Package Manager, or "RPM"--for Linux, there was a list of packages that you
needed to install in order to make a bootable system. This list consisted of
the Linux Kernel, glibc, various "userland" utilities (e.g., the
GNU binutils), networking tools like ftp and telnet,
and a few shells. Everything else was optional. Fast-forward to 2003: the
smallest possible automated installations (in terms of packages) from the major
Linux vendors look like this:
Red Hat8.0 |
SuSE 8.1 |
Debian 3.0 (woody) |
acl, anacron, apmd, aspell, at, attr, authconfig, authconfig, autofs, bash, bc, bdflush, bind-utils, bzip2, cpio, crontabs, cyrus-sasl-plain, dhclient, diffutils, dos2unix, dosfstools, dump, e2fsprogs, ed, eject, ethtool, fbset, file, filesystem, finger, ftp, gash, glibc, gpm, grub, hdparm, hotplug, initscripts, iproute, iputils, irda-utils, jfsutils, kbd, kbdconfig, kernel, kernel-pcmcia-cs, ksymoops, kudzu, kudzu, lftp, lha, libgcc, libtermcap, logrotate, logwatch, lokkit, losetup, lsof, mailcap, man, man-pages, mkbootdisk, mouseconfig, mt-st, mtools, mtr, net-snmp-utils, netconfig, nfs-utils, nss_ldap, ntsysv, openssh-clients, openssh-server, pam_krb5, pam_smb, parted, passwd, pax, pciutils, pinfo, procps, quota, raidtools, rdate, rdist, readline, reiserfs-utils, rp-pppoe, rpm, rsh, rsync, sendmail, setserial, setup, setuptool, slocate, specspo, star, stunnel, sudo, sysklogd, SysVinit, talk, tcp_wrappers, tcpdump, tcsh, telnet, termcap, time, timeconfig, tmpwatch, traceroute, unix2dos, unzip, up2date, utempter, util-linuxvim-minimal, vim-common, vixie-cron, wget, whois, wireless-tools, ypbind, zip |
acl, ash, at, attr, autoyast2-installation, bash, bc, bzip2, cpio, cpp, cracklib, cron, curl, cyrus-sasl, db, devs, dhcpcd, dialog, diffutils, e2fsprogs, ed, efibootmgr, eject, elilo, fbset, file, filesystem, fileutilsfillup, findutilsfinger, gawkgdbm, glibc, gpg, gpm, grep, gruff, grub, gzip, hdparm, heimdal-lib, hotplug, hwinfo, ibmsis, initviocons, iproute2, iptables, iputils, isapnp, jfsutils, kbd, ksymoops, lcs, less, libgcc, libstdc++, libxcrypt, libxml2, liby2util, lilo, logrotate, lsof, lukemftp, lvm, m4, mailx, man, mdadm, mktemp, modutils, ncurses, net-tools, netcat, netcfg, openldap2-client, openssh, openssl, pam, pam-modules, parted, pciutils, pcre, pdisk, perl, permissions, popt, portmap, postfix. Procmail, providers, ps, raidtools, readline, recode, reiserfs, rpm, rsh, s390-tools, sash, sed, sh-utils, shadow, silo, sitar, src_vipa, suse-build-key, SuSEfirewall2, Sysconfig, Syslogd, Sysvinit, Tar, Tcsh, telnet, terminfo, textutils, timezone, unitedlinux-release, usbutils, utempter, util-linux, vim, w3m, xfsprogs, yast2, yast2-backup, yast2-bootloader, yast2-core, yast2-country, yast2-firewall, yast2-inetd, yast2-installation, yast2-ldap-client, yast2-mail, yast2-mouse, yast2-ncurses, yast2-network, yast2-nfs-client, yast2-nfs-server, yast2-nis-client, yast2-nis-server, yast2-nisplus-client, yast2-online-update, yast2-packagemanager, yast2-packager, yast2-pam, yast2-restore, yast2-runlevel, yast2-security, yast2-storage, yast2-support, yast2-sysconfig, yast2-theme-SuSELinux, yast2-transfer, yast2-tune, yast2-update, yast2-users, yast2-xml, zipl, zlib |
adduser, apt, apt-utils, at, base-config, base-files, base-passwd, bash, bsdmainutils, bsdutils, console-common, console-data, console-tools, console-tools-libs, cpio, cron, debconf, debianutils, dhcp-client, diff, dpkg, e2fsprogs, ed, exim, fdutils, fileutils, findutils, gettext-base, grep, groff-base, gzip, hostname, ifupdown, info, ipchains, iptables, kernel, klogd, libc6, libcap1, libdb2, libdb3, libgdbmg1, libident, libldap2, liblockfile1, libncurses5, libnewt0, libpam-modules, libpam-runtime, libpam0g, libpcap0, libpcre3, libpopt0, libreadline, libsasl, libstdc++, libwrap0, lilo, login, logrotate, mailx, makedev, man-db, manpages, mawk, mbr, modconf, modutils, mount, nano, ncurses-base, ncurses-bin, net-tools, netbase, netkit-inetd, netkit-ping, nvi, passwd, pciutils, pcmcia-cs, perl-base, ppp, pppconfig, pppoe, pppoeconf, procps, psmisc, sed, setserial, shellutils, slang1, sysklogd, syslinux, sysvinit, tar, tasksel, tcpd, telnet, textutils, util-linux, whiptail |
Wow, that's a lot of stuff isn't it? That doesn't include X11, or any user-level applications like OpenOffice.org or even freecell. If you
count just the basic packages that make up X11, window managers like GNOME or
KDE, support libraries, and user-level productivity tools, you can add up to 100
more packages to this list. This isn't a bad thing, but it's clear that a
minimum usable, networked system comprises a lot of software.
|
Related Reading 
Learning Red Hat Linux |
Imagine that you run a large enterprise and support not only the base operating system across hundreds or thousands of machines, but also end-user applications ranging from office productivity to CRM, or order management, or customer-service applications. It also makes for a lot of packages for an IT manager to support.
One of the hardest things for any enterprise to deal with is change. Back in the days of the Internet bubble when we thought (wrongly) that old paradigms didn't matter, no amount of disruption to the workflow of an organization was too radical. Now, we understand that change is good, but unexpected change can damage the ability of people to work together and get things done. In deploying IT solutions, the two hardest parts of any deployment are 1) consistent equipment configuration and 2) training the people who will use it in the software packages they need to get their jobs done.
Some people might suggest that this is a non-issue and that services such as the Red Hat Network allow IT managers to tame this complexity with automatic updating. This is true, provided you always want Red Hat's suggestions. What if installing a new version of OpenOffice.org or some other application would mean that your staff needs to be retrained, or a critical feature changes in some way that negatively affects your operations? What happens then?
The ability to upgrade systems in a logical way that minimizes system-level changes (except for bug fixes) and allows the enterprise to keep using older/existing versions of applications to help manage user expectations and constrain retraining is extremely important.
In the days of mainframes and mini-computers, upgrades were a lot more logical and well-contained because an upgrade applied to a system shared by a large number of users. With workstations and PCs, even, a small upgrade can cause issues far beyond the scope of a single package, as anyone who has ever had the pleasure of maintaining even a small gaggle of Microsoft Windows-based systems can attest. One of the things that made the non-PC world manageable was that, for the most part, hardware vendors made a very clean separation between the operating system and the applications.
|
Check out what went on at the O'Reilly booth and find out who the drawing winners are. |
Back in January at the NY LinuxWorld Expo, it was interesting to hear a developer from Dell opine onstage that the current "problem" with Linux distributions was all about the number of releases per year and that "things were settling down to a nice 18-month release cycle." I contend that the release cycle of distributions as a whole is less of a problem than the lack of a clean interface for layering products onto Linux systems. There can be no argument that the speed of Linux development--especially from a security perspective--is a good thing; a cleaner mechanism is needed in order to support multiple versions of software to help IT managers keep end-user-visible changes and disruptions under control.
Chuck Yerkes, a consultant specializing in Unix and sendmail, suggests
that the Linux world could benefit from the experiences of the BSD world, where
almost all non-core functionality is installed in a non-root tree. Instead of
a package like OpenOffice living under /usr/bin, it
would live in /packages or a similar directory that allows
multiple versions of packages to be installed easily. Of course, this
"re-rooting" can be done now with RPM and other package managers, but it has to
be done manually.
One way to look at this is that, in the old parlance of Digital's VAX/VMS, the core of the operating system itself is a separate product from the "layered products" used by end users. It should be possible to install or upgrade the core of the Linux operating system without forcing the upgrade of the layered packages. In fact, it should be possible for a Linux system to support multiple versions of a package along with whatever underlying libraries are needed, without causing application dependencies to break. This might seem to add major complications to Linux distributions, since vendors would have to track and understand more details about every package they include in their distributions, but it would make the adoption of Linux by large and medium-sized organizations a lot easier by making their support mechanisms more clear-cut and predicable.
It seems to me that it's time for Linux distribution vendors, backed by the Linux Standards Base (LSB), to break the core OS out from the rest of the "layered products" that make up the bulk of their offerings. Make all of the end-user applications into a separately-selectable set of installable packages and develop a way to allow for multiple versions of packages to coexist on the same system. In the old days, under mainframes and minicomputers, this was done with "logical names" under VMS and virtual directories under VM and other OSes. The BSD world seems to have navigated this shoal; it shouldn't be to hard for Linux world to do so, either.
Deep thanks to Steve Jones of Crash Computing, Chuck Yerkes of SNEW.com, Stephen Degler, Andrew "Bobcat" Templin, and Richard Rognlie, "Sr. Lackey" for Gamerz.NET Enterprises, for helping me sift through bits and pieces of various Linux distributions and offering suggestions on how maintaining large Linux installations could be better handled.
David HM Spector is President & CEO of Really Fast Systems, LLC, an infrastructure consulting and product development company based in New York
Read more Linux in the Enterprise columns.
Return to the Linux DevCenter.
Copyright © 2009 O'Reilly Media, Inc.