I think by the true sense of the name, "Cattleman" and the style of hat, you may be a little pressed to make it look less cowboyish.
If you're looking for a more casual outdoors-man look, My suggestion would be to add a Campdraft to the collection if you haven't already. Just my $0.02.
Charlie, you can make it look more "outback" by changing the band to a satin pug. Easy to do with hooks on both ends. If the brim looks a little too wide you might have to add a little bit of packer draw to the front & back of the brim.
John Lofgren Monkey Boots Shinki Horsebuttt - $1,136 The classic monkey boot silhouette in an incredibly rich Shinki russet horse leather. 
Grant Stone Diesel Boot Dark Olive Chromexcel - $395 Goodyear welted, Horween Chromexcel, classic good looks. 
Schott 568 Vandals Jacket - $1,250 The classic Perfecto motorcycle jacket, in a very special limited-edition Schott double rider style. I have a 20 year old Cattleman that came with a combination braided leather hatband/chin strap - what I did was wind the chinstrap around the crown and tie a bow in front. Not sure if this achieves the look you are going for but it does get the chin strap out of the way:
