| Subcribe via RSS

When perl and rpm don’t get along

April 30th, 2008 | No Comments | Posted in Uncategorized

Sometimes when building rpm packages you will get an rpm that requires a file that it already contains. This seems pretty lame (which it is) but here is an example and a workaround.

Building a package for freepbx we see this output:

rpmbuild -ba freepbx.spec
--------snip-----------------
Provides: config(freepbx) = 2.4.0-0
Requires(interp): /bin/sh /bin/sh
Requires(rpmlib): rpmlib(CompressedFileNames) <= 3.0.4-1 rpmlib(PayloadFilesHavePrefix) <= 4.0-1
Requires(post): /bin/sh
Requires(postun): /bin/sh
Requires: /bin/bash /usr/bin/env /usr/bin/perl /usr/bin/php config(freepbx) = 2.4.0-0 perl(DBI) perl(FindBin) perl(retrieve_parse_amportal_conf.pl)

The file we don’t want is: “perl(retrieve_parse_amportal_conf.pl)”

What rpmbuild does is go through the list of files and run “ldd” against all executables to find required libraries.
It also goes through each perl file and looks for “use / require” flags to pull out required perl modules.
When a developer does a legitimate thing like ‘require “retrieve_parse_amportal_conf.pl“‘ to include functions and such into their program, rpmbuild sees that this file is needed adds it too it’s list of required files.

Now rpmbuild also goes through and looks for what packages/files perl programs provide. It does this by scanning through the files and looking for “package“. If you just have an include file with functions, you don’t have a complete module and won’t have the package statement either. Thus rpmbuild will never see your file provides itself! Fortunatly there are a couple work arounds.

In your perl file that rpmbuild is requiring you can define ‘ our $RPM_Provides = “yourfilename.pl” ‘. rpmbuild will pick this up and happily add it to the provided file list. The other method is slightly more complicated but works well if you don’t want to patch the source code.

In your rpm spec file under the %prep section after the %setup add the following code:

cat << \EOF > %{name}-req
#!/bin/sh
%{__perl_requires} $* |\
sed -e '/perl(yourperlfile.pl)/d'
EOF
%define __perl_requires %{_builddir}/%{name}-%{version}/%{name}-req
chmod 755 %{__perl_requires}

Where yourperlfile.pl is the file you want to exclude from the rpm requires check.
This should make your rpm build hapily and exlude that file from the requires check.

If you want to see the actuall files rpmbuild runs take a look at:
/usr/lib/rpm/perl.prov
and
/usr/lib/rpm/perl.req

Tags: , , , ,

3Ware Sucks!

April 18th, 2008 | No Comments | Posted in Misc Rants

FYI Im not going to provide benchmarks or anything remotely useful here.

3Ware raid controllers suck when it comes to random IO r/w. I recently installed a new mail delivery server with 15 1TB Hitachi hard drives and a 3ware 9650SE-24M8. Raid 6 for the mail spool raid 10 for the mail indexes and raid 1 for / and everything else.

I always heard that Random IO sucked on these cards from other colleagues but brushed it off thinking I could live with less then optimal disk r/w throughput. Little did I know it was SO bad that the machine wasn’t even usable. I/O wait was through the roof! Like over 90%! Waiting to write mail to mail spools was timing out from locks that were still being written. It was REALLY BAD!

I quickly tore down the Hardware Raid and ran the raid 6 and raid 10 JBOD on the 3ware controller with software raid with the same I/O mail load. I/O wait is never over 10% now.

Note: 3Ware cards are pretty solid cards with nice management. They just have some random i/o issues to work out. Their support is not that helpful either on the issue. I turned all the nobs they suggested with little or no results. The cards seem to be tuned and geared towards high sequential I/O. And they do work great for that. 3ware will probably never fix the problem as long as people keep buying them though.

This was done on Centos 5 with dovecot and postfix. Was tested w/ same results on FreeBSD.

BTW: Munin is pretty nice! Thanks to dforbes for turning me on to it.

Filtering WebServer Spam

April 17th, 2008 | No Comments | Posted in Fighting Spam

Lately I have been tasked with filtering outgoing email coming from our webcluster. Why does it seem that nobody has figured out how to effectively filter outgoing email from webservers? Every solution seems to fall flat. The best I have seen so far came from a custom plugin I wrote for Spam Assassin that uses Akismet (Sorry if I was pounding on your servers guys) to do the filtering. The Akismet filter worked probably about 50% or so better then the commercial cloud mark plugin that I tried.  Not to mention it caught way fewer false positives then cloud mark. I wish I had my notes still cloud marks false positive rate was over 25% if I remember correctly. This was with all the Spam Assassin rules turned off by the way for testing on cloud mark and akismet.

My current setup is a sendmail wrapper that adds extra environment variables for outgoing email to use for filtering purposes and also does some script logging. This was pretty easy and I can post if anyone is interested. The rather difficult thing was making sendmail auto add the headers for me based on the environment variables since I didn’t want to manually alter the email. Ill post the sendmail magic here later on.

I have a VOIP project im working on right now that has pulled me away from this but when I get back to it I will post more info on what im doing now.