Established 1986
On servers only, update /etc/system to handle Oracle's memory requirements -- shmmax should be equal to total system RAM (1GB in the example below); this is a Solaris thing:
* Oracle changes 7/5/00:
set shmsys:shminfo_shmmax=1073741824
set shmsys:shminfo_shmmin=1
set shmsys:shminfo_shmmni=100
set shmsys:shminfo_shmseg=10
set semsys:seminfo_semmni=2000
set semsys:seminfo_semmsl=300
set semsys:seminfo_semmns=1024
set semsys:seminfo_semopm=100
set semsys:seminfo_semvmx=32767
* End Oracle changes
You'll need corrected "dbstart" and "dbshut" scripts from
the dba for Oracle 8.1.6 -- they are broken as delivered.
Once you have installed those, use this pair of
/etc/init.d scripts and link them
appropriately. That means creating symlinks into rc2.d and rc3.d as
S97oracle and S98tns for startup, and into rc0.d, rc1.d, and rcS.d as
K20tns and K21oracle for shutdown. The exact numbers aren't really
important, only their relative ordering matters -- they will be run
in numeric order, and you want Oracle to come up before TNS, and TNS
to shut down before Oracle. The rc.x scripts are only run when the
machine enters a new run level, so putting a K script into rc2.d
will NOT perform the desired clean shutdown!
To connect to the database (e.g. with sqlplus) the dba must set up
Update /etc/system:
* Increase max file descriptors in /etc/system according to
* http://www.weblogic.com/doc51/platforms/sun/ - KM 7/17/00
set rlim_fd_max = 4096
set rlim_fd_cur = 1024
To connect to Oracle, the db.properties file must be updated with the correct username and password. You may need to use 'find' because it's buried very deeply beneath the /weblogic directory:
# find /weblogic -name db.properties
For a clustered install, place the weblogic content under "/production/ftcluster".
Because we're doing this to fix the "chunk" exploit, we only want to rebuild as little as we have to. These instructions cover the new Apache, mod_ssl, and PHP. That's assuming you need SSL support.
Other Assumptions:
Okay, let's dive in. The order of the steps below is critical.
./configure --prefix=/usr/local/mm; make; make install
8/30/02 -- If you're trying to compile specifically for Solaris, and getting all sorts of errors that seem to be centered around the mm package, get yourself a 3.x version of gcc and start over! The new gcc package at sunfreeware.com is a godsend.
ln -s /usr/local/src/apache_1.3.26 /usr/local/src/apache ln -s /usr/local/src/mod_ssl-2.8.9-1.3.26 /usr/local/src/mod_ssl ln -s /usr/local/src/php-4.1.2 /usr/local/src/phpThis will save a little typing and a lot of hassle, since we no longer have to type all the hyphens and underscores and dots perfectly.
Find them (the path to them is in the existing Apache's httpd.conf) and copy them to the new Apache SOURCE tree. See next step.
mod_ssl is a glue module that connects Apache to openssl. I'm assuming you've already built openssl (e.g. openssl-0.9.6g) -- if not, and if the client is using SSL, DO THAT FIRST.
The order is important because this configure step puts source into the Apache tree (so it will be built when we build Apache), and it also "patches" the existing Apache source.
cd /usr/local/src/mod_ssl # Taking advantage of our symlinks!
./configure \
--with-apache=../apache \
--with-crt=/usr/local/apache/conf/ssl.crt/www.mydomain.com.crt \
--with-key=/usr/local/apache/conf/ssl.key/www.mydomain.com.key
PHP demands that you use apxs, a Perl-based Apache utility, in order to configure it as a DSO (dynamic module). But apxs doesn't exist until you build Apache... so you have to build Apache and tell it to take it on faith that libphp4.so *will* exist when it is needed.
Notice that we're telling Apache where to find openssl, NOT mod_ssl. Also notice that we are using the "Apache" layout. This should put the compiled httpd into /usr/local/apache/bin -- make sure this is not where your production httpd is currently running! If it is, you'll need to change the "layout" just enough to put your new httpd into a version-number directory. (Again, I'd do this the same way Linux handles its kernel: build into /usr/local/apache_1.3.26, then make /usr/local/apache a symlink to that directory. It's sweet for the next time you need to upgrade!)
Note that this should create the apxs binary which we will need in the next step.
cd /usr/local/src/apache
DO NOT 'enable-modules' that you do not want compiled in! If you're going to load them as DSO's, they don't belong in the configure command.
EAPI_MM="/usr/local/mm" \ SSL_BASE="../openssl" \ ./configure \ --with-layout=Apache \ --prefix=/usr/local/apache \ --enable-module=so \ --enable-rule=EAPI \ --enable-module=rewrite \ --enable-module=ssl \ --enable-shared=max \ --enable-shared=ssl \ --enable-shared=rewrite \ --disable-rule=SSL_COMPAT \ --enable-rule=SSL_SDBM \ --enable-rule=SSL_EXPERIMENTAL \ --enable-rule=SSL_VENDOR
Note that we're defining a couple of variables (EAPI_MM and SSL_BASE) as part of the same process as our .configure command. That is, we are entering all of that stuff as one long logical line. The continuation character, "\", is important. Don't leave it out.
If all goes well, then:
make
In the 1.3.26 version of Apache I ran into some old library-related problems that came back even though they had been fixed in the versions immediately prior. There is clearly a -L pointing to /usr/local/mm/lib, yet the compile and link of "gen_test_char" doesn't see it. Why? I don't know. Being a pragmatist, I fixed it by creating a symlink:
ln -s /usr/local/mm/lib/libmm.so.12 /usr/lib/libmm.so.12
If you try to build the mod_auth_mysql module you'll hit another couple of snags -- the first is a complaint about "No rule to make target `../../include/alloc.h'". It's already there, but as "ap_alloc.h". You can solve it by copying it:
cp ap_alloc.h alloc.h
(I tried just making a symlink, but Apache's 'make install' copies the include library (.h) files for future use -- and that copy step doesn't like symlinks.)
The second problem is more serious; including mysqlclient.a will blow up your build because someone forgot to add two required libraries to the Makefile. See markround.com for one solution. (I'm grateful for Mark's help in finally beating this problem.) The symptom, for those of you using Google, is unresolved references to "floor, compress, and uncompress." The first comes from the math library, the latter two from zlib. The fix is to edit Apache's "src/Makefile" to insert these two link libraries, -lm and -lz, like so (-l is a hyphen and a lowercase letter 'L'):
LIBS1= -L/usr/local/mysql/lib -lmysqlclient -lm -lz -lcrypt [and so forth]
At least, that's supposed to do it. I found that just adding them isn't enough -- it also matters that you put them after -lmysqlclient rather than in front of it.
This may not be related, but I recently encountered an issue trying to add Perl's DBD::mysql package to my CPAN toolkit. Even though it did include -lz it still wouldn't build until I symlinked libz.so.1 to create a "/usr/lib/libz.so" with no number. Perhaps doing that helped the problem above as well...? Figured I'd better include it as a tip, just in case.
Whew! If the sucker finally builds, you can test a few things at this point by running "src/httpd -l" (shows the compiled-in modules; there should be very few) and "src/httpd -t" (see that it likes your httpd.conf file). If it's OK, the next step is:
make install
cd /usr/local/src/php
./configure --prefix=/usr/local/php \
--with-mysql=/usr/local/mysql \
--with-EAPI \
--with-config-file-path=/usr/local/src/php \
--with-apxs=/usr/local/apache/bin/apxs \
--enable-track-vars
make
make install
This "make install" step puts libphp4.so into the Apache libexec subdirectory - check that it's there.
You do NOT want the static libphp4.a module, so don't use the "activate-module" version of the configure command.
/usr/local/apache/bin/httpd -t # assumes /usr/local/apache/conf
# as the location of httpd.conf
Did you get a "Syntax OK"? Give it a try. If not, figure out why. It's pretty good about telling you where it hurts.
cd /usr/local/apache-1.3.24/bin # the old or production Apache # Stop the old Apache, wait a few seconds for things to wind down... ./apachectl stop cd /usr/local/apache/bin ./apachectl startssl # or just "start" if they don't use SSL
Tip: Cut and paste the SSL passphrase somewhere on screen so you can enter it correctly and quickly!
CHECK, CHECK, CHECK! Does it answer on :80? :443? Be prepared to bring the old one back up if the new one doesn't start!!!
Okay, at this point we go back to the old notes from Fall 2000...
As with much software in the LAMP (Linux Apache MySQL PHP) family, you start an Apache "build" by running a script called "configure". It figures out your operating system and which C compiler you have, among other things. It's quite clever, actually. Each time you run the Apache configure command, it saves the parameter values you gave it as a file called "config.status"; running that as a shell script will recreate the last run of the "./configure" command. Of course if you are running it for the first time, you have to provide those parameters yourself. Here's my own (heavily customized) layout:
# ./configure --prefix=/opt/apache \
--sysconfdir=/httpd/conf \
--enable-rule=SHARED_CORE \
--enable-module=most \
--enable-shared=max \
--htdocsdir=/httpd/htdocs \
--localstatedir=/var/lock/apache \
--logfiledir=/var/log/apache \
--cgidir=/httpd/cgi-bin \
--runtimedir=/var/lock/apache \
--proxycachedir=/var/lock/apache
To upgrade to a new version of Apache, I should be able to take
"config.status", copy it to the new source directory, and run it there
as a shell script to recreate my prior configuration. Very handy!
After running ./configure, you'll need to do a 'make' and 'make install':
# make
# make install
We updated to 4.0.3 on 10/12/2000 due to an announcement in Bugtraq of a "format string" compromise (this is a generic C-library issue which has affected many applications in Q3 2000, not just PHP). To unpack it:
# cd /opt/src
# cp php-4.0.3.tar.gz .
# gunzip php-4.0.3.tar.gz
# cd ..
# tar xf src/php-4.0.3.tar
# cd /opt/php-4.0.3
Then configure it -- you must define ORACLE_HOME for the OCI_8
database interface to work. These configuration instructions are saved
in the source directory as an executable file named "DO_CONFIG":
ORACLE_HOME=/oracle/app/oracle/product/8.1.6 ; export ORACLE_HOME
./configure --with-config-file-path=/opt/php \
--with-oci8 --with-apxs=/opt/apache/bin/apxs \
--with-config-file=/opt/apache/conf \
--enable-track-vars \
--enable-sigchild \
--without-mysql
Then do a 'make' and 'make install' as usual.
You'll need to copy /opt/php-4.0.3/.libs/libphp4.so to /opt/apache/libexec and enable it by making the recommended changes to /httpd/conf/httpd.conf (see the php INSTALL document for details - it's in the php-4.0.3 directory).
chown -R nobody:web /httpd
find /httpd -type f | xargs -i chmod 664 {}
find /httpd -type d | xargs -i chmod 775 {}
If you want to be fancy about it, here's a script that provides a
minimal sanity check:
#!/bin/sh
if [ -d /httpd/htdocs ] ; then
cd /httpd
else
echo Cannot find docroot
exit 2
fi
chown -R nobody:web /httpd
find /httpd -type f | xargs -i chmod 664 {}
find /httpd -type d | xargs -i chmod 775 {}
The source for the Apache proxy is on whorfin in "/opt/src/avsproxy". The compiled avsproxy.so module is copied to /opt/apache/libexec as usual. We encountered one problem which turned out to be due to a bad switch in the AltaVista Makefile; the symptom is a conflict with PHP that gives a "relocation error" message at runtime. To fix it, just remove the "-lc" switch from the avsproxy Makefile and rebuild avsproxy.so (this keeps it from trying to install the libc library twice).
AltaVista's own instructions for incorporating the module into Apache are here.
Once installed and started, you manage the product by accessing the "Management User Interface" or MUI. It's a java application running on a high port (default is 9000, but we're using 8999 in production).
Be warned that the default install is configured to write logs into the install directory, and it leaves far too many files as WORLD-WRITEABLE. I tightened up the file permissions and moved the logs to /log/av by editing the file /app/AVSearchEngine30/httpd/config. When I finished, running "./avsetup -s" would no longer start the engine, but "./bin/startController" would, and the 'restart' button on the MUI still works. You must restart the engine after any changes, including changes to the static html templates; it caches all of the templates.
A very odd thing: the way AltaVista has set up their scripts, you must be logged into the "AVSearchEngine30" directory, not in the bin directory below it. "cd bin ; ./startController" will not work.
We're running the engine on production-4, owned by root but running as daemon; an attempt to install it on production-16 just plain didn't work, even when the colo admin tried to start it as root (it wants to run as daemon, regardless of who owns the binary).
Styling of the results page to match the rest of the site was done "by hand" in the subdirectories beneath /app/AVSearchEngine30/httpd/avshe_languages/search, using the html guide provided by AltaVista. To make a long story short, [a designer] provided code for a complete sample "results" page, and I hacked it apart into the sections used by AltaVista.
Longer version: The output page is made up of multiple parts: init, preamble, form, results, postamble. Each of those parts resides in its own subdirectory. I found it helpful to insert comments into each section to help figure out which of the parts needed to be the target of a specific change. Use your browser's "View Source" feature! And don't forget that you have to do a restart on the engine after each change, or else you won't see any effect.
These files have to be edited in order to move the product to a different machine (assuming /app/AVSearchEngine30 is the working directory):
Warning: Running the "./avsetup -i" script will undo all the changes you made to the files above, unless you first update its template file. It will copy the contents of that template over your production files! Better yet, just don't run avsetup. Ever.
We have to use the MUI to configure indexes and update the database used by the search engine. I use port-tunneling provided by ttssh to forward my local ports to production-4. This means we did not open any ports in the firewall.
On main (sabre), Apache allows proxy access to the MUI, but it uses an .htaccess file. If you go to sabre's port 9000 directly, the .htaccess restriction does not apply; it only affects you when you use the proxy provided by Apache. (http://main.oursite.com:1080/mui/mui.html) When you use port 9000, you are talking directly to AltaVista's mhttpd engine, not to Apache.
In order to have separate indexes (e.g. JK Lasser, Breaking News, TaxPlanet Guide), we have to figure out the exact case-sensitive name we want to use, and when creating the index we have to do an "path_reject" for every link on the page that we do NOT want in that index. AltaVista is very crafty about finding links that you have not specifically excluded! The link that you DO want to index goes at the bottom of the list as a "path_include". (Follow the link to see a screen shot).
The standard I'm following for indexes and their associated collectors looks like this:
| <index name> | <collector name> |
| BreakingNews | news |
| Lasser | Lasser |
| TaxPlanet | taxplanet |
| EntireSite | allFTP |
admin:{SHA}C3LGQO8rAU6F5t7JwkJ6YILkMLM=
This contains a SHA-encrypted password for user "admin". Replace it
with this line to leave the same user with a null password:
admin:
The Weblogic hooks go into obj.conf, and you need to make a 'text/jsp' MIME type for the .jsp file extension.
Interesting feature: It uses the Unix password file to authenticate users. At install time you specify an administrative user who has a shell account, and then that user can add other accounts (who also have valid usernames) through the Web-based GUI.
Note: It should be able to analyze access logs from all our products including Apache, Netscape, Weblogic, and even AltaVista. It does NOT analyze error logs, however. Why not? Because error logs are allowed to have arbitrary text, whereas access logs have a common layout.
If a search engine dropped you directly into this document, you should go to the index page to find out what you're reading. This document is a record of a project from 1999-2000 -- it is not a current guide to installing any software product.