FreeBSD 8.0

I've been waiting with baited breath for this to come out of it's RC cycle, and now it's finally here.

I deployed a few RC1, 2 and 3 copies as test platforms doing minor tasks, and was impressed with the snappiness of the OS in general. I really feel there is a better networking stack underneath the system now.

With 8.0 out, it's just a simple matter to use

freebsd-update -r 8.0-RELEASE upgrade

follow the instructions, two reboots, and you're done.

I'm liking it more and more as the years go on. Glad I found BDSi years ago, which started me on the BSD route instead of being pulled into one of the many Linux camps.

Benchmarks? FreeBSD ZFS vs GEOM vs Hardware Speed

I'm currently running through some disk benchmarks to determine the speed of various ZFS arrays in FreeBSD 7.1 compared to GEOM or Hardware arrays.

I'm selecting bonnie and dbench for my testing at the moment - any other benchmarks come to mind that would be useful?

ZFS on FreeBSD 7.1

I've been flirting with ZFS on FreeBSD 7.1-RELEASE AMD64, and I must say I like it so far.

I'm hoping and dreaming of using it as my main filestore in a 'raidz' array of 1.5 TB drives to give me a massive temporary file store for my ever increasing Video and Music files.

Why ZFS?

I'm excited about ZFS mostly for being transactional, and for the raidz support - This is an enhanced version of RAID-5. I could have ran a GENOM RAID-5 array, but when you add in the other features of ZFS (and it has many - go check) it's very tempting to go with ZFS instead.

Unfortunately it's still experimental in FreeBSD, which may have some sticking with more tested technologies.

I should give a few tips if you're trying out ZFS

1) Don't bother with i386

You can run ZFS under i386 with some tweaks, but why? It's going to be limited, and I think you're asking for trouble. ZFS needs a lot of memory address space. Give it 64 Bits and you just have to put

zfs_enable="YES"

in /etc/rc.conf to make it start up.

2) Give it some power

I love using old hardware to run FreeBSD servers, but ZFS needs some oomph to run proprly. Sure people are running it in i386 with 384 Meg, but why? You can grab a decent Socket 775 board and 64 bit chip for $120, throw in 2 gigs of RAM for $60, and you're still under $200

3) Give it respect

This isn't UFS. It's still experimental, don't put your only copy of critical data on it. I'm going to put things that I really don't want to lose - MP3's and Videos, but not accounting data or similar. Will the world end if I lost it? No, but I'd be pretty pissed, so you know I'm doing a lot of testing on my hardware so I know how to recover from faults. Learn how to use it if you're going to want any dependability from it.

4) Don't try and boot from it

It won't. There are tricks to make the /usr and other partitions ZFS and the main boot on UFS, but why frig around and play with fire? We all have an old 80 or 40 gig HD around. Make that your UFS boot, and setup your other drives as dedicated ZFS.

5) Speed

In a very slap-dash test, my GEOM stripe was delivering data via samba at the same speed as my ZFS raidz pool. In some cases, even faster. Of course I'd have to do proper tests to make the playing field level, so this is just a quickie test to see if a ZFS raidz could hold up in speed to my GEOM stripe on my box for samba network use, and it does that just fine. It may suck for a MySQL db, but I haven't tested that yet.

6) Moving to different hardware

- If you build your pool on /dev/ad0, /dev/ad2, ad4 (etc) and then move to a new controller that is discovered as da0, da1, etc. You simply do this;

zpool tank export
zpool tank import

presto. My ad* based zpool is now showing up on my da* based controller.

7) Failure Redundancy

I setup my raidz, transferred a few gig of data to it, then pulled a drive while it was running. A few complaints on the console, but my data was still there. Speed seemed okay, and data was intact without a lot of rebuilding necessary. I later on rebooted and reconnect the drive, added it back into the pool, and life went on.

GEOM is annoying after a crash as it has to check the entire array before it lets you back in. On a 1.5 TB array, that takes some time. With ZFS, you can still fire up and access data.

You can also run a background check by typing

zfs scrub

No more off-line fscks! Schedule a low-priority scrub to run in the background, and you're ensuring your data is always clean and matching.

This simple crash test proved ZFS can do two main things for me ; 1) Notify me when something fails, and 2) accept the drive back after it's been fixed.

I now need to see if I pull a drive, format it, and put it back if it accepts it properly and rebuilds the array on it. One would hope, but I'd like to manually test it before putting all that media on it.


Conclusions:

Get FreeBSD 7.1-RELEASE AMD64 and grab 2 or more drives and play with it. Check the links below for setup info - it's really quite simple. I'm loving it, and I guess I will until it screws me over. :-)


Links for more info:

http://blog.thefrog.net/2008/04/zfs-on-freebsd.html

(Top link references the two bottoms ones which I used to setup my ZFS)

http://wiki.freebsd.org/ZFSQuickStartGuide
http://wiki.freebsd.org/ZFSTuningGuide

I also recommend

http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide
http://kerneltrap.org/FreeBSD/BSDCan_2008_ZFS_Internals
http://opensolaris.org/os/project/sdosug/past_meetings/ZFS-general.20070919s.pdf
http://www.tech-recipes.com/rx/1405/zfs-how-to-fsck-or-check-filesystem-integrity-with-scrub/

MSP Offerings: Zenith Infotech in more detail

I signed up for Zenith's trial over a month ago, and only had a few hours to play with it before running out of research time. My life is pretty tightly scheduled so I had to shelve my MSP demo for almost a month.

I'm back to it now, but my trial expired. No big deal. Zenith is dirt cheap to use. Check these prices:

http://www.saaztraining.com/zenith deliverables 2007.pdf

So I just signed up and am now paying a few bucks a month to continue to try out Zenith. I'm also now seeing Zenith from the side of a "real user", not a "prospective user" so I should be seeing the true system, not the salesman-enhanced side of things.

I like the fact that I'm not pressured to buy anything big upfront. It's all SAAS, so there is next to no requirement on my end, other than using IE (Firefox doesn't work) for their management console.

True, with SAAS you're always paying for it. But remember; software is always evolving. I have a few products I can keep running for a few years without upgrading, but not too many. Usually you're always in need of the new features or the bug fixes, or the new platform support that the new version brings.

I'm starting to become very aware of the internal costs we pay for maintaining our our software - The time we spend frigging with our servers, setting them up, maintaining them, etc. That time is far more productive being spent on customer's needs, which can be billed. Part of me is a bit apprehensive about the lack of control, but most of me is happy that I don't have to deal with putting it all back together if it crashes.

So what do I like so far about Zenith?

Well, I like the reports - Very similar to what I generate for my clients, except it's all there. I can output nearly the same thing with 5 clicks, and I'm done. This task is simple enough that it can now be given to the admin staff, not needing a technician to build the reports.

The dashboard/overview of what is going on with my servers is rather simple, but you know, it tells me the main points I need. I'm sure this is customizable, I'm still in the process of figuring that out.




I will keep learning this package and then write a better report on it soon. Once that's done, I'm off to check out Kaseya, Bigfix, and probably Level in the same detail. It's going to be a lot of time, but this is a big decision. These are complex packages, and they need proper attention to compare.

Switching Firewalls from ipf to pf on FreeBSD

It's time to deploy a new FreeBSD firewall, and I thought I'd check out pf instead of my standard ipf package.

pf is written by the OpenBSD team, and was designed to replace ipf because of the licensing issues with that code not being a proper BSD license. There's also some political issues, but I'll leave those along.

The Background:

Since the early days my office firewalls have been FreeBSD platforms. I've always loved the BSD platform, back from the days where I built an ISP around the BSDi package in the early 90's.

For as long as I can remember, I've been using ipf (ipfilter) by Darren Reed. It became so standard that FreeBSD started including it in their releases which makes life much easier.

Unix based firewalls always had my vote - They are infinitely configurable and extensible. They are not overly hard to setup, and they don't need a lot of hardware to run. In the day my options were a very expensive Cisco-class router, or Unix.

But as NAT routers kept dropping in price and improving in quality, my Unix based router/firewalls started slowly disappearing from sites. Most small offices don't need much - basic NAT with one port for remote access from us, and they are happy. A simple D-Link DI-604 goes for $50. That's a far cry cheaper than a decent computer and the charge for my time to setup a custom firewall.

So the Unix-Based firewalls only exist in some of my more complex, larger, or secure customer sites. The rest have Linksys or D-Link routers.

A few months ago my aged FreeBSD firewall finally died. Heck, it was a Pentium-233Mhz - I think I got my money's worth out of it. I knew it's days were numbered so I had a spare DI-604 ready to go, and popped it into place when needed.

I didn't really notice much of a difference in speed of the network (my main concern against the little routers) and it was handy to be able to use a web interface to configure it. I know you can do this with Unix, but it was one of the things I didn't bother updating as I upgraded the boxes over the years, relying on SSH instead. I don't pay a lot of attention to my firewall, and I didn't want to worry about security issues in the web interface screwing me up.

Life's been fine on the little D-Link until I started getting heavier into uTorrent. You put a few Windows 2003 machines on a network running uTorrent, and you can bring a little NAT router to it's knees. They just don't have enough memory and processor power.

The answer was to go back into some sort of Unix based firewall. Wanting something easier and FreeBSD based, I found pfSense. That didn't work out well for me, but it's another blog.

I made some compromises on how I use uTorrent, and my little DI-604 was still useful...

..until I had some heavy Spammer hacking going on. Talk about dragging your network into the ground. The DI-604 can't handle large amounts of traffic, even if it's rejecting it, and I was getting a lot of traffic. One old Windows 2000 IIS server (ticking time bomb really) was compromised, and the scum of the world was lining up at my router to use my bandwidth.

Back to Today - Trying pf over ipf:

So I have to build a new FreeBSD firewall. A quick download of 7.1-RELEASE i386 Disc1 (no need for 64 bit on a fw as far as I can see), and I'm on my way.

(btw msk support in 7.1 seems to be much better than 7.0 - no watchdog timeouts for me so far)

I decided to make this one a pf test platform as pf is also included by default with FreeBSD 7.1. I've heard mixed reviews of which is better, but the thing that intrigued me the most was that pf was a superset of ipf - my knowledge of rules for ipf wouldn't' be wasted, and I'd have more things to play with. ipf bugs me at times, as it doesn't seem to be evolving (like qmail). There is something to be said for a stable codebase, but I also don't think we've reached the pinacle of these packages.

Setup is simple;

in rc.conf;

pf_enable="YES"
pflog_enable="YES"

and your config file is /etc/pf.conf

pfctl is your control program for start, stop, and status.

If you want to be lazy, I'm quite sure a standard ipf.conf would work if you wanted to play with this.

So far I like pf. It's got a bit of a learning curve, but it has additional features that ipf lacks.

I like that the NAT portion is in the pf.conf - just looks cleaner to me to have one file for it. I like that I have more NAT rules. I have a "no nat" rule that blocks any outgoing SMTP traffic that doesn't go through one of our main mailservers. Just a different way of doing it, but nice.

pf also has the ability to do QoS routing via the ALTQ interface.

I'm currently checking out fwanalog so I can get a better look at pf's logs (also works for ipf) and I think I'm going to have to move into a Snort setup so I can detect these attacks and prevent them in the future.

This link has some simple info for monitoring pf firewalls;

http://prefetch.net/articles/monitoringpf.html

I'll post more as I find them.

ATI - Oh why, oh why?

It's like a beloved family pet has died.

I'm done with ATI. Never again.

My shop was always a firm supporter of ATI from the early days. By the time the first Rage chipsets came out, we were almost exclusively selling ATI products. They were not only a Canadian company, but they produced solid, reliable video cards. They were not always the fastest, but they were trouble free, and that was exactly what I wanted.

When a new Microsoft OS came out, you had support for your ATI products built in - It was just a given. Things worked. The world was a happy place with your ATI video card.

The shift in ATI happened gradually - they started going for more speed, wanting to be more cutting edge. More corners were cut, more fickle markets were being explored.

Being bought by AMD was both a good and a bad thing - but they were going south long before AMD stepped in.

What's with the Catalyst driver suite? Why do we need .NET to install graphics drivers? Yes, it's just for the control panel, but frig - big & bloated does NOT appeal to the gaming segment, nor to the business segment. Sure it's flash and fun to look at, but that gets boring after 5 min. You're not in the control panel all day - you use it for a while, then it's left alone for months or years.

If you want to have some fun, try and get ATI All-in-Wonder cards working with Microsoft Medic Center. Even funnier - try with Vista Media Center. There is simply no support. These high end cards are totally abandoned. All the money put into these cards (and they were never cheap) is wasted. You will have better luck with some no-name TV input cards than you will with the mighty ATI brand name.

It's not Microsoft's fault. They are not here to make drivers for the hardware companies. The number of posts with ATI users struggling to watch TV on ATI AIW cards is astounding.

I frigged with my Radeon 9000 AIW and MCE2005 for days. I even went as far as using my x800XT - I could never get playback and TV Input working. I shouldn't have to frig with this. I'm not asking for my Rage II video card to have full Vista support - we're taking cards that are not that old here.

BUT - My old Radeon 9000 AIW works just fine with MCE 2002, because Dell cut some specific drivers for it that work. Dell made it work for that particular package, and since they didn't sell that machine to run MCE 2005, they don't support it. I don't blame Dell for that. This is where the graphics chipset manuf. is supposed to come in.

Then there is the whole list of 3rd party Radeon chipset cards made for Dell or other peripheral companies that just don't work with the default ATI drivers. You have to track down old drivers from Sun-Moon-Star or similar defunct 3rd party companies. My built-for-Dell Radeon 9000 is one of them. ATI's standard driver install won't see it. It's the same frigging card. I have to use Omega or other hacked drivers to get them to work with my card.

This doesn't happen with Nvidia. It doesn't matter who makes that GeForce 7300LE chipset card - the basic nVidia drivers work with them. And they work well.

When my ATi Radeon 9000 AIW won't play back video in MCE, I just have to drop in an OLDER nVidia card, and I can play video without any problems. And you know what, it plays better as well.

I could also recant my recent RMA problems with a workstation-class ATI Fire GL card in an engineering workstation. ATI had to send THREE cards to me before they got it straight. They couldn't figure out their own cryptic part numbers, and kept sending the wrong replacement card. We told them exactly what it was every time - part number and the basic description - ATI Fire GL V3350 PCIe card with 256M. We were pissed, customer was pissed (as the whole process took over a month to resolve).

Learn from my pain. Learn from the pain of literally 10's of thousands of others. Stay away from ATI. Let them die now, so we can reflect on the good times we had with ATI before there is nothing but bad memories.

If you're going for TV - get yourself a Hauppage TV input card. Don't bother with anything else, or you'll be frigging with it.

I love nVidia cards at the moment - they are going into our business and gaming machines. Matrox - I haven't done much with them lately, but I have nothing bad to say about them.

ATI's Crossfire is very cool. It's better than nVidia's SLI, but I won't support their crappy company anymore. I'll give the cash to nVidia, and hope that their SLI2 (or whatever replaces SLI) is a proper step up.

My stock of any ATI product is going on ebay/Kijij - I don't even want to see them in my shop anymore. Just a huge waste of time. Nvidia, Matrox, frig - even a 3dfx Voodoo card please!

AMD - You own this crap-pile now. Smarten up - Making Crossfire an open conept is excellent, but won't save your video arm if you keep going down this route. Look carefully at nVidia's drivers - they install. No frigging. No worries about who made the card. I say I have a 7300LE, and I download the drivers, and install. No mess! It's how it's susposed to work.

Gah. I feel empty inside.

MSP Offerings: Zenith Infotech vs Kaseya va Bigfix Enterprise

It's time.

The collection of little tools we use to manage our customer's networks is growing unwieldy.

Yes, there are free things out there like Spiceworks and WSUS. More tools that don't work together, requiring technician time to patch together reports and run different tools to get a idea of our networks. WSUS drives me up the wall - it keeps promising things that ultimately don't work or need a lot of maintenance to keep running. If it was just on company with a number of desktops and servers under one roof, we'd be fine - but I have multiple locations to work with.

We need something that is one interface. Ultimately like to have 1 program to run my entire business (CRM, Billing, MSP, etc). We started working towards this in 1990 when I and a partner custom coded a webserver based app to run all our time, billing, and even to poll machines for hardware configs via DMI/WMI.

Of course, the time/expense of maintaining your own custom software eventually killed that noble project, but when it was doing 80% of our work, it was very, very cool.

For now, I'd be happy with three applications: My MSP, my PSA, and my accounting software.

I have Quickbooks, and my PSA is CommitCRM, so I'm okay here so far.

The requirements for a MSP offering are:

- Centralized patch management and deployment
- Inventory of servers and desktops we maintain, including installed software
- Ability to check Anti-Virus, disk space, backup status, etc. Common flags

All from one package. It would be nice to have a script ability, and I don't care too much about remote access, as I have that taken care of already .


I've got demos for Zenith, Kaseya, and BigFix installed on a temp MSP server right now, and I'm working through their training and applications to see who does what.

I'll post updates as I progress with these three applications, and eventally choose one.

ASUS P5AB-Deluxe with DUAL nVidia 7300 Graphics Cards (Quad Monitor)

Okay, that was hell.

I've been running dual monitors for some time now, both on my work PC and my home PC. I am leaning towards nVidia for multi-displays these days, as I'm finding the ATI drivers to be really cumbersome, and their multi-monitor stuff isn't as simple and efficient as the nVidia stuff. Maybe if you want the options for virtual desktops, ATI is your thing, but I don't - it's why I got multiple monitors in the first place.

Anyway - I have a few extra LCD's kicking around, and some spare PCIe cards, so I thought I'd go for 4 monitors. I could really use the extra space when I'm monitoring and working on multiple servers at the same time. My current 7300 LE has 2 displays on it, and that works quite well.

I thought I'd just plug in the second card and away I'd go. Not so. Once you jump to more than 2 monitors, it gets tricky.

I won't go through all the permutations and combinations I had to run. I'll just jump to the end;

- Doesn't matter if you use ATI or nVidia on a crossfire, SLI, or regular board, if you're just looking for multiple displays without speed enhancement. My P5B is a Crossfire board, and I'm running two nVidia 7300's (without SLI of course).

- I didn't have to change any of the defaults on my ASUS P5B board - they are all good.

- I don't have a perfect match with my cards. One ASUS EN7300LE 512M, one knockoff unknown 7300 GS with 256M.

- You WILL have to uninstall ALL of your graphic drivers.

- You WILL have to hook up all the monitors and make sure they are powered up before you start the machine.

- Once you've dropped all your video drivers, Windows should detect both cards. Make sure to point it to your latest drivers (which you have handy someplace). Don't worry about errors and such, just make sure you install both before you reboot.

-On reboot, you may have to activate the extra monitors under Display Settings. My two other monitors were turned off by Windows by default.

- Give it a few reboots before you scream. Some reboots may hang. Mine did. I reset, and it kept trucking the next time.

- Have some older versions of drivers handy for install - maybe the latest won't do it for you.

Any mixup of cards should work, but you're going to have the best results with the same bus (all PCIe, all PCI, etc), and the same card chipsets. I've heard it's bad news to mix cards that use different Direct X versions, so if you have a DX10 and a DX9 card, it may be rough going.

I also heard that Vista doesn't like different video cards - this is something that only works under XP and maybe 2003.

This is waaaayyy cheaper than a $450 nVidia NVS 400 Quad-Monitor card. Old spare 7300's and 6600's can be picked up for $20 used.