10 April 2003, 06:13:07 The Complete FreeBSD (current.mm), page 581 In this chapter: • FreeBSD releases and CVS • FreeBSD releases • Getting updates from the Net • Creating the source tree 31 Keeping up to date FreeBSD is constantly changing The average time that elapses between changes to the source tree is in the order of a few minutes Obviously you can’t keep up to date with that pace of change In the following three chapters we’ll look at how to keep up to date In this chapter, we’ll look at: • FreeBSD releases: how the FreeBSD project comes to terms with the rapid rate of change, and how it keeps the system stable despite the changes • How the system sources are stored, and how you can update them In Chapter 32, Updating the system software, we’ll look at how to upgrade FreeBSD to a new release, with particular reference to upgrades to FreeBSD Release 5, and in Chapter 33, Custom kernels, we’ll look at building special kernels FreeBSD releases and CVS The FreeBSD project keeps the entire operating system sources in a single master source tree, called a repository, which is maintained by the Concurrent Versions System, or CVS It’s included in most multi–CD-ROM distributions of FreeBSD The repository contains all versions of FreeBSD back to Release 2.0 and the last release from the Computer Sciences Research Group of the University of California at Berkeley, 4.4BSD-Lite, upon which it was based For copyright reasons FreeBSD Release was not included, because at the time, as the result of the lawsuits described on page 8, it was not permitted to distribute it freely That situation changed in early 2002, but it’s now too late to include FreeBSD Release in the repository current.mm,v v4.17 (2003/04/09 19:28:00) 581 10 April 2003, 06:13:07 The Complete FreeBSD ( /tools/tmac.Mn), page 582 582 Chapter 31: Keeping up to date CVS is built on top of the Revision Control System, or RCS RCS keeps multiple versions of files, called revisions, in a single RCS file Each revision has a number indicating its relationship to the other revisions The oldest revision has the number 1.1, the next oldest has the number 1.2, and so on The RCS file contains the most recent revision of the file along with instructions for creating any other revision In addition to this linear sequence, it’s possible to update a specific revision in more than one way The obvious way to update revision 1.2 would create revision 1.3; but it’s also possible to create branches, which get numbers like 1.2.1.1 Updating revision 1.2.1.1 would create revision 1.2.1.2, and so on By contrast, the revisions with a two-part number are collectively called the trunk of the tree Symbolic names or tags In addition to the numeric identifiers, each of which relates only to a single file, RCS allows you to attach symbolic names to specific revisions CVS generally calls these names tags, and that’s the term you’ll see most often FreeBSD uses tags to indicate the revisions corresponding to a particular release For example, in the directory /usr/src/sys/kern, revision 1.13 of kern_clock.c, revision 1.12 of kern_fork.c and revision 1.21.4.1 of kern_exec.c participate in RELENG_2_1_0_RELEASE We’ll look at tags in more detail on page 588 RCS stores its files either in the same directory as the working files it is tracking, or in a subdirectory RCS if it exists To avoid file name conflicts, RCS appends the characters ,v to the RCS file, so the working file main.c would correspond to the RCS file main.c,v For more details of RCS, see the man page CVS is an extension to RCS that allows concurrent access, making it more suitable for projects involving more than one person Unlike RCS, it stores its RCS files in a separate directory hierarchy, called a repository Each directory in the working tree contains a subdirectory CVS with information on the location of the repository, the revisions of the working files and a tag if the revision isn’t on the trunk If you’re a serious developer, there are a number of advantages to keeping a copy of the repository If you’re a casual user, it’s probably overkill FreeBSD releases There are four main versions of FreeBSD, each intended for use by different people: FreeBSD-RELEASE FreeBSD-RELEASE is the latest version of FreeBSD that has been released for general use It contains those new features that are stable, and it has been through extensive testing You can get it on CD-ROM FreeBSD-RELEASEs are given a release number that uniquely identifies them, such as 5.0 There are three or four releases a year A new branch is made for each release of FreeBSD current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD ( /tools/tmac.Mn), page 583 FreeBSD releases 583 FreeBSD-STABLE FreeBSD-STABLE is an updated version of FreeBSD-RELEASE to which all possible bug fixes have been applied, to make it as stable as possible Fixes are made on a daily basis It is based on the same source branch as FreeBSD-RELEASE, so it has all the features and fewer bugs It may contain additional features, but new features are tested in the -CURRENT branch first Due to the frequent updates, FreeBSD-STABLE is not available on CD-ROM Security fix releases Despite the name, FreeBSD-STABLE is subject to some problems Every change to a source tree has the potential to go wrong In many cases, you’re more interested in keeping your system running than you are in getting minor bug fixes FreeBSD also maintains a second ‘‘stable’’ branch consisting of the release and only very important bug fixes, including security updates This branch does not have a well-defined name, but it’s generally referred to as the security branch FreeBSD-CURRENT FreeBSD-CURRENT is the very latest version of FreeBSD, located on the trunk of the tree All new development work is done on this branch of the tree FreeBSD-CURRENT is an ever-changing snapshot of the working sources for FreeBSD, including work in progress, experimental changes and transitional mechanisms that may or may not be present in the next official release of the software Many users compile almost daily from FreeBSD-CURRENT sources, but there are times when the sources are uncompilable, or when the system crashes frequently The problems are always resolved, but others can take their place On occasion, keeping up with FreeBSD-CURRENT can be a full-time business If you use -CURRENT, you should be prepared to spend a lot of time keeping the system running The following extract from the RCS log file for /usr/src/Makefile should give you a feel for the situation: $ cvs log Makefile revision 1.152 date: 1997/10/06 09:58:11; author: jkh; state: Exp; lines: +41 -13 Hooboy! Did I ever spam this file good with that last commit Despite reviewers, we still managed to revoke the eBones fixes, TCL 8.0 support, libvgl and a host of other new things from this file in the process of parallelizing the Makefile DOH! I think we need more pointy hats - this particular incident is worthy of a small children’s birthday party’s worth of pointy hats ;-) I certainly intend to take more care with the processing of aged diffs in the future, even if it does mean reading through 20K’s worth of them I might also be a bit more anal about asking for more up-to-date changes before looking at them ;) This example also shows the list of the symbolic names for this file, and their corresponding revision numbers There is no symbolic name for -CURRENT, because it is current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD ( /tools/tmac.Mn), page 584 584 Chapter 31: Keeping up to date located on the trunk That’s the purpose of the line head:, which shows that at the time of this example, the -CURRENT revision of this file was 1.270 So why use -CURRENT? The main reasons are: • You might be doing development work on some part of the source tree Keeping ‘‘current’’ is an absolute requirement • You may be an active tester, which implies that you’re willing to spend time working through problems to ensure that FreeBSD-CURRENT remains as sane as possible You may also wish to make topical suggestions on changes and the general direction of FreeBSD • You may just want to keep an eye on things and use the current sources for reference purposes People occasionally have other reasons for wanting to use FreeBSD-CURRENT The following are not good reasons: • They see it as a way to be the first on the block with great new FreeBSD features This is not a good reason, because there’s no reason to believe that the features will stay, and there is good reason to believe that they will be unstable • They see it as a quick way of getting bug fixes In fact, it’s a way of testing bug fixes Bug fixes will be retrofitted into the -STABLE branch as soon as they have been properly tested • They see it as the newest officially supported release of FreeBSD This is incorrect: FreeBSD-CURRENT is not officially supported The support is provided by the users If you decide to use -CURRENT, read the suggestions on page 621 Snapshots FreeBSD-CURRENT is available in the form of ISO (CD-ROM) images From time to time, at irregular intervals when the tree is relatively stable, the release team makes a snapshot release from the -CURRENT source tree They are also available on CD-ROM from some vendors; check the online handbook for details This is a possible alternative to online updates if you don’t want the absolute latest version of the system Getting updates from the Net There are a number of possibilities to keep up with the daily modifications to the source tree The first question is: how much space you want to invest in keeping the sources? Table 31-1 shows the approximate space required by different parts of the sources Note that the repository keeps growing faster than the source tree, because it includes all old revisions as well current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD ( /tools/tmac.Mn), page 585 Getting updates from the Net 585 Table 31-1: Approximate source tree sizes Component Repository src/sys Repository src Repository ports Source tree /usr/src/sys Source tree /usr/src Source tree /usr/ports Object tree /usr/obj Size (MB) 250 1000 300 110 450 200 160 The size of /usr/src/sys includes the files involved in a single kernel build You can remove the entire kernel build directory, but if you want to be able to analyze a panic dump, you should keep the kernel.debug file in the kernel build directory This changes the size of /usr/src as well, of course The other object files get built in the directory /usr/obj Again, you can remove this directory tree entirely if you want, either with the rm command or with make clean Similarly, the size of /usr/ports includes a few ports It will, of course, grow extremely large (many gigabytes) if you start porting all available packages If you’re maintaining multiple source trees (say, for different versions), you still only need one copy of the repository CVSup CVSup is a software package that distributes updates to the repository You can run the client at regular intervals—for example, with cron (see page 151) to update your repository To get started with CVSup, you need the following: • A source tree or repository, which doesn’t have to be up to date This is not absolutely necessary, but the initial setup will be faster if you it this way • A copy of the cvsup program Install it with pkg_add from the CD-ROM (/cdrom/packages/Latest/cvsup.tbz) • A cvsupfile, a command file for cvsup We’ll look at this below • A mirror site from which you can load or update the repository We’ll discuss this below as well The cvsupfile contains a description of the packages you want to download You can find all the details in the online handbook, but the following example shows a reasonably normal file: current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD (current.mm), page 586 586 Chapter 31: Keeping up to date *default release=cvs *default host=cvsup9.freebsd.org *default base=/src/cvsup *default prefix=/home/ncvs *default delete *default use-rel-suffix *default compress src-all ports-all doc-all The lines starting with *default specify default values; the lines that not are collections that you want to track This file answers these implicit questions: • Which files you want to receive? These are the names of the collections in the last three lines: all of the sources, ports and documentation • Which versions of them you want? By default, you get updates to the repository If you want a specific version, you can write: *default tag=version version is a release tag that identifies the version you want, or (a period) to represent the -CURRENT version We’ll discuss release tags on page 588 Alternatively, you might ask for a version as of a specific date For example: *default date=97.09.13.12.20 This would specify that you want the version as it was on 13 September 1997 at 12:20 In this case, version defaults to (a period) • Where you want to get them from? Two parameters answer this question: host=cvsup9.freebsd.org specifies the name of the host from which to load the files, and release=cvs specifies to use the cvs release The release option is obsolescent, but it’s a good idea to leave it in there until it is officially removed • Where you want to put them on your own machine? This question is answered by the line *default prefix=/home/ncvs We’re tracking the repository in this example, so this is the name of the repository If we were tracking a particular release, we would use *default prefix=/usr The collections are called doc, ports and src, so we refer to the parent directory in each case • Where you want to put your status files? This question is answered by the line *default base=/src/cvsup In addition, the file contains three other lines *default delete means that cvsup may delete files where necessary Otherwise you run the risk of accumulating obsolete files *default compress enables compression of the data transmitted, and *default userel-suffix specifies how cvsup should handle list files It’s not well-documented, but it’s necessary Don’t worry about it current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD (current.mm), page 587 Getting updates from the Net 587 Which CVSup server? In this example, we’ve chosen one of the backup US servers, cvsup9.FreeBSD.org In practice, this may not be the best choice A large number of servers are spread around the world, and you should choose the one topographically closest to you This isn’t the same thing as being geographically closest—I live in Adelaide, South Australia, and some ISPs in the same city are further away on the Net than many systems in California Look on the web site http://www.FreeBSD.org for an up-to-date list Running cvsup cvsup is a typical candidate for a cron job I rebuild the -CURRENT tree every morning at am To so, I have the following entry in /root/crontab: # Get the latest and greatest FreeBSD stuff * * * /extract-updates The file /root/extract-updates contains, amongst other things, cvsup -g -L2 /src/cvsup/cvs-cvsupfile /src/cvsup/cvs-cvsupfile is the name of the cvsupfile we looked at above The other parameters to cvsup specify not to use the GUI (-g), and -L2 specifies to produce moderate detail about the actions being performed Getting individual releases The example cvsupfile above is useful if you’re maintaining a copy of the repository If you just want to maintain a copy of the sources of one version, say Release 5.0, use the following file instead: *default *default *default *default *default *default *default *default src-all tag=RELENG_5_0_0_RELEASE release=cvs host=cvsup9.freebsd.org base=/usr prefix=/home/ncvs delete use-rel-suffix compress for /usr/doc, /usr/ports, /usr/src Be careful with tags They must exist in the repository, or cvsup will replace what you have with nothing: it will delete all the files In our original cvsupfile, we had two additional sets, ports-all and doc-all These sets don’t have the same release tags, so if you left them in this file, you would lose all the files in the /usr/doc and /usr/ports directory hierarchies current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD (current.mm), page 588 588 Chapter 31: Keeping up to date Creating the source tree If you’re tracking the repository, you’re not finished yet Once you have an up-to-date repository, the next step is to create a source tree By default, the source tree is called /usr/src, though it’s very common for /usr/src to be a symbolic link to a source tree on a different file system You create the tree with cvs Before you check anything out with cvs, you need to know: What you want to check out? You specify this with a module name, which usually corresponds with a directory name (for example, src) There are a number of toplevel modules, including doc, ports, src and www Which version you want to check out? By default, you get the latest version, which is FreeBSD-CURRENT If you want a different version, you need to specify its tag Possibly, the date of the last update that you want to be included in the checkout If you specify this date, cvs ignores any more recent updates This option is often useful when somebody discovers a recently introduced bug in -CURRENT: you check out the modules as they were before the bug was introduced You specify the date with the -D option, as we’ll see below Release tags FreeBSD identifies releases with two or more numbers separated by periods Each number represents a progressively smaller increment in the functionality of the release The first number is the base release of FreeBSD The number is incremented only when significant functionality is added to the system The second number represents a less significant, but still important difference in the functionality, and the third number is only used when a significant bug requires rerelease of an otherwise unchanged release Before Release of FreeBSD, a fourth number was sometimes also used Tags for released versions of FreeBSD follow the release numbers For release x.y.z you would look for the tag RELENG_ x_ y_ z_RELEASE For example, to get the current state of the FreeBSD 5.0 source tree, you would look for the tag RELENG_5_0_0_RELEASE Tags for the -STABLE branch are simpler: they just have the release number, for example RELENG_4 The security branch has an additional number, for example RELENG_4_7 Some tags diverge from this scheme In particular, CSRG and bsd_44_lite both refer to the original 4.4BSD sources from Berkeley If you feel like it, you can extract this source tree as well To find out what tags are available, do: # cd $CVSROOT/src # rlog Makefile,v | less RCS file: /home/ncvs/src/Makefile,v RCS file: /home/ncvs/src/Makefile,v Working file: Makefile current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD (current.mm), page 589 Creating the source tree head: 1.270 branch: locks: strict access list: symbolic names: RELENG_5_0_0_RELEASE: 1.271 RELENG_4_7_0_RELEASE: 1.234.2.18 RELENG_4_7: 1.234.2.18.0.2 RELENG_4_7_BP: 1.234.2.18 RELENG_4_6_2_RELEASE: 1.234.2.12 RELENG_4_6_1_RELEASE: 1.234.2.12 RELENG_4_6_0_RELEASE: 1.234.2.12 RELENG_4: 1.234.0.2 RELEASE_2_0: 1.30 BETA_2_0: 1.30 ALPHA_2_0: 1.29.0.2 bsd_44_lite: 1.1.1.1 CSRG: 1.1.1 keyword substitution: kv total revisions: 179; selected revisions: 179 description: 589 5.0-RELEASE 4.7-RELEASE 4.7 security fixes only branch point for 4.7 4.6.2-RELEASE 4.6.1-RELEASE 4.6-RELEASE 4-STABLE 2.0-RELEASE 4.4BSD-Lite This example shows the same file we saw on page 583 This time we use the rlog command, which is part of RCS, to look at the revision log Normally you’d use cvslog, but that only works in a checked out source tree There are a number of ways to tell cvs the name of its repository: if you already have a CVS subdirectory, it will contain files Root and Repository The name of the repository is in Root, not in Repository When you first check out files, you won’t have this directory, so you specify it, either with the -d option to cvs or by setting the CVSROOT environment variable As you can see in the example above, it’s convenient to set the environment variable, since you can use it for other purposes as well The repository contains a number of directories, usually one for each collection you track In our case, we’re tracking the source tree and the Ports Collection, so: • CVSROOT contains files used by CVS It is not part of the source tree • ports contains the Ports Collection • src contains the system sources The directories ports and src correspond to the directories /usr/ports and /usr/src for a particular release To extract the src tree of the most up-to-date version of FreeBSDCURRENT, the following: # cd /usr # cvs co src 2>&1 | tee /var/tmp/co.log current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD (current.mm), page 590 590 Chapter 31: Keeping up to date To check out any other version, say, everything for Release 4.6, you would enter: # cd /usr # cvs co -r RELENG_4_6_RELEASE src 2>&1 | tee /var/tmp/co.log If you need to check out an older version, for example if there are problems with the most recent version of -CURRENT, you could enter: # cvs co -D "10 December 2002" src/sys This command checks out the kernel sources as of 10 December 2002 During checkout, cvs creates a subdirectory CVS in each directory CVS contains four files We’ll look at typical values when checking out the version of the directory /usr/src/usr.bin/du for Release 4.6, from the repository at /home/ncvs: • Entries contains a list of the files being maintained in the parent directory, along with their current versions In our example, it would contain: /Makefile/1.4.2.1/Sun Jul 10:45:29 2000//TRELENG_4_6_0_RELEASE /du.1/1.15.2.7/Thu Aug 16 13:16:47 2001//TRELENG_4_6_0_RELEASE /du.c/1.17.2.3/Thu Jul 12 08:46:53 2001//TRELENG_4_6_0_RELEASE D Note that cvs prepends a T to the version name • Repository contains the name of the directory in the repository that corresponds to the current directory This corresponds to $CVSROOT/directory In our example, it would contain src/usr.bin/du • Root contains the name of the root of the repository In our example, it would contain /home/ncvs • Tag contains the version tag of the source tree This is the RCS tag prefixed by a T In this case, it is TRELENG_4_6_0_RELEASE cvs co produces a lot of output—at least one line for each directory, and one line for each file it checks out Here’s part of a typical output: U src/usr.sbin/mrouted/rsrr_var.h U src/usr.sbin/mrouted/vif.c U src/usr.sbin/mrouted/vif.h cvs checkout: Updating src/usr.sbin/mrouted/common U src/usr.sbin/mrouted/common/Makefile cvs checkout: Updating src/usr.sbin/mrouted/map-mbone U src/usr.sbin/mrouted/map-mbone/Makefile cvs checkout: Updating src/usr.sbin/mrouted/mrinfo U src/usr.sbin/mrouted/mrinfo/Makefile cvs checkout: Updating src/usr.sbin/mrouted/mrouted U src/usr.sbin/mrouted/mrouted/Makefile cvs checkout: Updating src/usr.sbin/mrouted/mtrace U src/usr.sbin/mrouted/mtrace/Makefile cvs checkout: Updating src/usr.sbin/mrouted/testrsrr U src/usr.sbin/mrouted/testrsrr/Makefile current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD (current.mm), page 591 Creating the source tree 591 The flag at the beginning of the line indicates what action cvs took for the file The meanings are: • U means that cvs updated this file Either it didn’t exist previously, or it was an older version • You won’t normally see P on a local update It implies that cvs patched the file to update it Otherwise it has the same meaning as U • ? means that cvs found the file in the directory, but it doesn’t exist in the repository • M means that cvs found that the file in your working directory has been modified since checkout, but it either didn’t need to change it, or it was able to apply the changes cleanly • C found that the file in your working directory has been modified since checkout, and it needed to change it, but it was not able to apply the changes cleanly You will have to resolve the conflicts manually After checkout, check the log file for conflicts For each conflict, you must check the files manually and possibly recover the contents See the man page cvs(1) for more details Updating an existing tree Once you have checked out a tree, the ground rules change a little Next time you a checkout, files may also need to be deleted Apart from that, there isn’t much difference between checkout and updating To update the /usr/src directory after updating the repository, do: # cd /usr/src # cvs update -Pd Note that this time we can start in /usr/src: we now have the CVS/ subdirectories in place, so cvs knows what to without being given any more information Using a remote CVS tree A CVS tree takes up a lot of space, and it’s getting bigger all the time If you don’t check out very often, you may find it easier to use anonymous CVS, where the tree is on a different system FreeBSD provides the server anoncvs.FreeBSD.org for this purpose For example, to check out the -CURRENT source tree, perform the following steps: $ cd /usr go to the parent directory $ CVSROOT=:pserver:anoncvs@anoncvs.FreeBSD.org:/home/ncvs set the server path $ cvs login You only need to this once Logging in to :pserver:anoncvs@anoncvs.freebsd.org:2401/home/ncvs CVS password: enter anoncvs; it doesn’t echo $ cvs co src cvs server: Updating src U src/COPYRIGHT U src/Makefile U src/Makefile.inc1 (etc) current.mm,v v4.17 (2003/04/09 19:28:00) 10 April 2003, 06:13:07 The Complete FreeBSD (current.mm), page 592 current.mm,v v4.17 (2003/04/09 19:28:00) 592 ... repository CVSup CVSup is a software package that distributes updates to the repository You can run the client at regular intervals—for example, with cron (see page 151) to update your repository To. .. stuff * * * /extract-updates The file /root/extract-updates contains, amongst other things, cvsup -g -L2 /src/cvsup/cvs-cvsupfile /src/cvsup/cvs-cvsupfile is the name of the cvsupfile we looked at... and updating To update the /usr/src directory after updating the repository, do: # cd /usr/src # cvs update -Pd Note that this time we can start in /usr/src: we now have the CVS/ subdirectories