To the Brass 
Cannon Webboard
Established 1986

Kevin Martin - PO Box 82783 - Portland, OR 97282

System (Hardware and OS) Configuration

Lesson One: Our colo service didn't read any of this, even though they said they would. Turns out they had their own policies, which were utterly inflexible, but which I learned about after all the local "QA" machines were set up.

Lesson Two: In the final analysis, it didn't much matter.

QA Architecture

Five Enterprise servers (Sun E250), set up as two web servers, two app servers, one Oracle database server. Dual machines required in order to practice load balancing and get realistic performance figures.

Support Systems

Three Sun Netra T1s procured as "utility" machines and to do QA for miscellaneous software, such as the static content site, mailing list management, search engine, and log analysis.

Three Sparc Ultra 10 desktops were provided for the administrator, the dba, and an outside consultant to do load testing.

All these machines reside on-site.

  1. E-250 QA Servers (Web, Application, Database)

    Each machine has a pair of 18GB drives, which the customer wanted mirrored for reliability.

    My recommendation: Do a "Developer" install with manual layout, breaking out the /usr, /opt, and /var partitions to keep system files, user code, and log files separate from each other and from the root.
    SliceMount Point Size (MB)
    0/200
    1/usr964
    3/opt600
    4/var800
    5swap1000
    6unassigned*remainder of disk
    7unassigned**100

    	*(Referred to as /app below; replace "app" with
    	 the name of the application to which this machine
    	 is dedicated)
    
    	**(reserved for metadb/replica backups - Disksuite)
    
    

    (8/31/00: Colo site announces their preference that /opt remain empty except for Sun patches, and that /opt and /usr both remain within root. After some discussion, they agree to move /home out from under root and make /usr/local a symlink to /home/local; they also agree to create a separate /log slice so that logs won't be written into our binary or content spaces. See second table:)

    Colo's NameMy NameUsage
    /home/local/optNon-Sun binaries
    /app/appContent
    /log/varLogfiles

    Design notes:

  2. Netra Disk Modifications

    The Netras came with pre-installed 2.7 - we needed to reformat the enormous /space partition and move /opt and /var out of the root.

     	Slice	Mount	   Size
    	0	/ 	   1200MB
    	1	/opt	   2GB
    	3	swap	   2GB
    	4	unassigned 4GB (/app)
    	5	unassigned --
    	6	/var       7GB*
    	7	unassigned 200MB (metadata)
    
    

    Saving a small metadata slice allows us to install DiskSuite later. It would be awkward if we didn't have that spare slice available!

    9/6/00: The Virtual Adrian performance monitor tells us that our mirrorring arrangement is impairing disk performance, at least for the root partition. I've unmirrored / and /var on the web servers, and it seems to have helped. We should do that to the Weblogic servers as well. [At this point I provided a live link to the Virtual Adrian results -- on the "Administration" page I'll explain how I did that.]

  3. System identfication - create or check for valid entries in:

  4. Select "NONE" for DNS - we will fix the /etc files (nsswitch.conf, hosts, and resolv.conf) manually.

    At least one machine needs a compiler, perl, and the developer's files including C headers (since we have both Solaris 2.6* and 7, we should have a compiler on two machines, one running each version). Include the packages SUNWbtoo*, SUNWhea, and SUNWarc* on these systems.

    [*The hosting site insisted that the only webserver they would support was Netscape running under Solaris 2.6.]

  5. Install current Recommended* patchcluster from ftp://sunsolve.sun.com/pub/patches
    	* 2.6Recommended.Z for 2.6      7_Recommended.zip for 2.7
    	---------------------------     --------------------------
    	uncompress 2.6Recommended.Z     unzip 7_Recommended.zip
    	                                cd 7_Recommended
    	./install_cluster               ./install_cluster
    
    
    Error codes of 2 or 8 are not significant -- they indicate a patch for a package that was not installed, or one that has already been applied. (An error code of 1 means you're missing something critical, such as a required system utility in your PATH.)

    Applying patches may put services back into /etc/rc2.d and rc3.d even if they have been removed, so apply the patches before doing the next step.

  6. Stop these services with "/etc/init.d/xxxx stop", and move their rc2.d equivalents to /etc/rc2.d/disabled:

  7. Stop insecure inetd services and keep them from restarting:
        # pkill inetd
        # sed -e 's/^/#/' < /etc/inetd.conf > foo
        # mv foo /etc/inetd.conf
    

  8. To force the hme (network interface) to 100Mbit and full duplex, install the S95net-tune script in /etc/rc2.d.

  9. mkdir /root and edit /etc/passwd to make /root the login directory for the root account -- chown root:root /root and make it chmod 700. This keeps the root (/) directory tidy and the root account a tiny bit more secure; every bit helps!

    Yes, this will break the vipw program, which faints in horror if the login directory for /root is not /. I think that's nuts, and Solaris needs to be slapped around a little.

  10. Get these utilities from sunfreeware.com and install with 'pkgadd -d':

    For use on the Netras, I compile on whorfin** (a Sparc 10) under Solaris 7. That's where the Apache/PHP configuration information for the content site resides -- I do a 'make' and 'make install' on whorfin to install it into /opt/apache, then tar that directory and copy it to sabre. But I use an E250 to compile code to be run on an Enterprise hardware -- mustang for Solaris 2.6, and voodoo for 2.7.


**Naming conventions

This is an area where letting your sysadmin exercise a bit of creativity costs nothing and is great for morale. A scheme that has some sort of underlying logic has a psychological basis, as well; people can keep track of far more complex relationships when they fit into a family of some kind.

So, our Enterprise boxes are named for famous fighter aircraft -- prop planes if they are running 2.6, jets for 2.7. Thus, someone with a hobbyist's interest in aviation knows instantly that "Spitfire" is an E250 running Solaris 2.6, but "Sabre" is running 2.7.

The Oracle servers are twin-engine jets, to symboize their dual CPUs and that they are running 2.7 -- Banshee, Voodoo, Phantom, Tomcat. It helped that the author worked at McDonnell-Douglas for a while, granted; that made it easier to come up with enough names to make this work.

For the Sparc 10 desktops, we switch to another scheme: characters from the cult movie Buckaroo Banzai, Lord John Whorfin and Penny Priddy. I've found that giving a machine a common personal name -- like "John" -- leads to confusion, so a source of memorable but oddball names comes in handy.

[This entire document was originally installed as an intranet page hosted on 'whorfin'.]

Production Architecture

Not really relevant here; when I discuss the migration of content to production, it's enough to know that production-1, 2, and 3 are web servers running Apache and Netscape; production-4 is a staging machine with the same disk layout (but on which I have root!), and production-10 is a Netra T1 utility box. Except for -4, I do not have root on any of the production boxes.


Back: Index   -   Next: Software

Please Note

If a search engine dropped you directly into this document, you should go to the index page to find out what you're reading. This document is a record of a project from 1999-2000 -- it is not a current guide to installing any software product.