ie8 fix

"Geeks rule!"? Yes - but what that means depends.

By | January 16, 2010, 12:15am PST

Summary: A small homily about pain - and cards - and the spread of Linux

An odd experience this week: I wanted to get a bunch of people collecting roughly similar data (on donors) to produce samples for me that were both randomly selected and geospatially representative. Since that’s oxymoronic, what I actually asked them to do was to stratify by district and then select randomly within each stratum.

The striking thing about the exercise was that an easy majority have moved the database for this to MySQL on Linux with holdouts on Solaris (mostly also MySQL), HP-UX, and various Microsoft configurations - but the unexpected thing was that none of the Unix people had any difficulty either understanding it or doing it; while the Wintel people equally unanimously wanted meetings, paperwork, “a better understanding of the requirements”, and in something like three out of four cases additional monies from their bosses before they could see about getting it done.

I was contemplating the difference between the Wintel marketing image as the solution for do it yourselfers who want to avoid having to deal with systems managers and the reality of the inflexibility and burdens its protagonists impose on users when, just as I was explaining the joys of the formulation “I’m from Microsoft and I’m here to help”, some black ice shifted my focus to practicing breakfalls on the sidewalk.

So as I’m laying there wondering where the phone went, one of the PC people who’d come out behind me looked down all concerned like to ask “did you fall?” Well, I’m a Jeff Foxworthy fan, so I dug out somebody’s business card from a previous meeting and handed it to him.

I’ve no idea whose card it was or what he made of it - but the analogy between what really happens when businesses replace their Unix infrastructures with Microsoft people and that slickly invisible ice on the sidewalk? Yep: that I’ll buy into.

Kick off your day with ZDNet's daily e-mail newsletter. It's the freshest tech news and opinion, served hot. Get it.

Topics

Paul Murphy (a pseudonym) is an IT consultant specializing in Unix and related technologies.

Disclosure

Paul Murphy

I do not work for, or otherwise receive anything from, any of the companies I write about. I have some money in a number of funds that bet on the markets, including the technology market, but have no direct control over how these funds are administered or what investments are made. I use Sun and Apple technology both at home and at work.

Biography

Paul Murphy

Originally a Math/Physics graduate who couldn't cut it in his own field, Paul Murphy (a pseudonym) became an IT consultant specializing in Unix and related technologies after a stint working for a DARPA contractor programming in Fortran and APL. Since then he's worked in both systems management and consulting for a range of employers including KPMG, the government of Alberta, and his own firm. In those roles he's "been there and done that" for just about every aspect of systems management and operation.

30
Comments

Join the conversation!

Just In

Yes -
murph_z 20th Jan 2010
I think you hit several nails right on the head here -

As usual in evolved situations it's almost impossible to tell cause from result (because yesterday's result is today's cause). In general, however, I take it that most of the perceived needs for control and safety reflect a history of failure and are causative with respect to the on going transformation of wintel into a data processing discipline.

My general solution is to put the Unix guys right in with the users - make them emotionally users rather than distant techno priests - and the ability their tools give them to get the job done should then push IT evolution in those organizations toward more functional (or, at least, less disfunctional) relationships.
0 Votes
+ -
That black ice can be a b1tch
Roger Ramjet 16th Jan 2010
But I'm sure you landed safely on your (very hard) head! If you had written something positive about M$, I would have been concerned . . .
0 Votes
+ -
And white ice is not?
Agnostic_OS 16th Jan 2010
nt
0 Votes
+ -
Black ice is harder to see...
Erik Engbrecht 16th Jan 2010
...and therefore a bigger b1tch when driving or walking. I believe it's
actually thin, clear ice, usually in small patches, which appears black on
blacktop roads and hence is very hard to see. White ice is easy to see,
and tends to be in larger patches, and therefore no where near as much
of a b1tch.

So, for all the PC police from warm climates out there, this is *not* a
racially motivated term. Color is a physical attribute. Get over it.

I obviously should not stop at the computer in the morning before
having coffee.
0 Votes
+ -
Sounds icist to me!
Agnostic_OS 16th Jan 2010
nt
0 Votes
+ -
How do you randomly select data in SQL?
Erik Engbrecht 16th Jan 2010
A *nix admin who doesn't know can just dump to text and
use any number of little scripting languages, but what's a
Windows admin to do? He need a DBA who knows archaic
SQL or a software engineer who can right an application to
do it.
0 Votes
+ -
A *nix admin who doesn't know can just dump
to text and use any number of little scripting
languages

Well, so could a Windows admin. Ever heard of
PowerShell?

but what's a Windows admin to do? He need a
DBA who knows archaic SQL or a software
engineer who can right an application to
do it

Or he could just do

Invoke-Sqlcmd -ser dbserver -dat dbname
"select * from sometable" | random -count 100 |
ConvertTo-Csv

Yup. No need to dump to a file, just pipe it
directly to the random cmdlet which will
choose a number of random objects from the
stream. Finally the records are exported in csv
format. No archaic parsing.
0 Votes
+ -
What???
The Mentalist 16th Jan 2010
The command line in windows?

It's amazing to how you say that after telling us for so long that *nix doesn't stand a chance against windoze because it forces you to use the command line.
0 Votes
+ -
Did I say that?
honeymonster 16th Jan 2010
or are you setting up a strawman?

If you heard this argument I believe it must
have been referring to the instances where you
*can not* accomplish a task in Linux/Unix
without resorting to the CLI and editing text
configuration files (in all sorts of
strange formats - from INI over the
Apache pseudo-xml to peculiar colon-delimited
passwords files).

In Windows you can perform all
administrative tasks through the GUI.
Automating those tasks may require
scripting. This is where PowerShell comes in.
0 Votes
+ -
How's that different?
Erik Engbrecht 16th Jan 2010
Other than that the commands are a little different, that's exactly the
type thing I would expect a *nix admin to do.

It's good to see that Windows 2008 Server is finally catching up with
Unix from the seventies, and that MS is even being gracious enough to
backport such innovations to earlier versions of Windows.

What's a fancy new tool and associated techniques in Windows land is
a decades old culture in Unix land. The task may be equally easy in
Windows, but it will be years before average Joe Windows admin gets
out of his GUI and uses them - if he ever does given the Windows
culture.
0 Votes
+ -
Not much
honeymonster 16th Jan 2010
Windows lacked a good admin-oriented scripting
for many years. There as wsh with JScript and
VB scripting - but they lacked the useful
piping feature of *sh shells.

But PowerShell is more than just catching up.
It is waaaay more consistent and leverages the
object-oriented nature of Windows. Thanks to
that PowerShell accomplishes the same as bash
or zsh with far fewer and simpler commands.

That's because PS has 1-upped the *nix shells:
Object piping means that commands become
simpler (to implement as well as to learn) and
PS scripts become easier to write and read
because there is no need for strange parsing
and formatting to make ends meet (no need for
awk, sed, cut etc).
0 Votes
+ -
Sounds complicated
Erik Engbrecht 16th Jan 2010
Object orientation is often more of a problem than a solution.

I take my statement back. MS hasn't managed to catch up to *nix from
the seventies. Thanks for saving me the time of looking a PowerShell.
After your first post of was seriously considering it.
0 Votes
+ -
You just demonstrated why *nix will forever be
stuck in a 1970 mindset where me-us-everyone
security is sufficient, where the only
securable objects are the ones in the file
system, where every command overloads what -f
means and where every application *must* define
its own configuration file format.

Dismissing a tool by homing in on a single
phrase is sooooo mature. Way to go.
0 Votes
+ -
If you haven't noticed...
Erik Engbrecht 16th Jan 2010
...pretty much every "object" in Unix is represented as "a file." Which
would make pretty much ever object securable.

My objection to OO doesn't have anything to do with Unix versus
Windows. Plenty of Unix bigot programmers use languages like C++,
Java, Python, Ruby, and even C# - all of which are object oriented to
varying degrees and according to varying definitions.

My objection is that languages that are OO has significantly worse
compositional properties and weaker type systems than functional
languages like the ML family and Haskell while lacking the flexibility
and simplicity of text streams across a pipe. That's not to say I think
OO is useless, especially in hybrid OO languages (Python, Ruby, Scala,
etc) where you aren't completely forced into the object paradigm.

For administrative scripting tasks a type system just gets in the way,
and most of the time so do things like classes, even if they are
dynamic. A prototype-based OO system like JavaScript's might
actually work well, but I've never tried it for such tasks.
0 Votes
+ -
I object! Take a look
honeymonster Updated - 16th Jan 2010
Take a look at the man pages of some of the
most often used commands in *nix: ls and ps.

Now look closely at how many of those options
1) filter on some property not usually reported
in the listing
2) control output format so that both humans
and next-in-pipeline commands may understand
the output.
3) select related information, e.g. threads.

ps alone has some 60+ options!

Now consider the ps command in PowerShell:


Get-Process
-Name
-ComputerName
-FileVersionInfo [ ]
-Id
-InputObject
-Module [ ]


That's it.

Now, is the *nix command more powerful? Nah -
most of what it does is about output formatting
because some other command should be able to
parse it. Some options control whether a
header should be printed. Other options control
what information goes into the output. Many
options control how that information is
displayed, e.g. dates, sizes, string
(delimited?) etc.

In PowerShell, the ps command output
objects . Another command can readily
consume them without needing to sed, cut or
awk anything. See, the object-orientation
lifts the formatting and output selection out
of the commands. They become simpler to
implement and simpler to understand. They do
"one thing only" - unlike the *nix beasts which
must find the information and then
format in text.
0 Votes
+ -
So answer a question for me...
Erik Engbrecht 17th Jan 2010
I'm writing a script and I want do what in an OO language would be
effectively introducing a new class. For example, in the text processing
way, I might have a script that takes tabular data, removes some
columns, and adds some new ones in. Doing this is trivial using Unix
style scripts because it's just text. How do I do it in PowerShell?
0 Votes
+ -
Fair question.
honeymonster 17th Jan 2010
I'm writing a script and I want do what in
an OO language would be effectively introducing
a new class. For example, in the text
processing way, I might have a script that
takes tabular data, removes some columns, and
adds some new ones in. Doing this is trivial
using Unix style scripts because it's just
text. How do I do it in PowerShell?

First off you have to realize that you seldom
deal with raw text in PowerShell. Why? Because
the built-in cmdlets all return stream og
objects - not text. Hence I don't need
to (re)parse the output from ps or ls - I can
just work with the properties, methods and even
events directly from PS:

ps|where{$_.WorkingSet -gt 100MB}

ps returns a stream of Process objects.
Process happens to have a property called
WorkingSet which indicates how much memory it
uses. The where cmdlet filters using a
script block - $_ represents the
iterator.

So the above command tells me which processes
uses more than 100MB. Note that the properties
are typed. 100MB is actually a number.

The above command still returns a stream of
Process objects. When they "fall off" the end
of the pipeline they are being rendered
according to a default format, unless I decide
otherwise. Key point is that the objects still
retain all their properties and methods. Unlike
the *nix ps command where the only information
forwarded is the text that you see.

I could do

ps|where{$_.WorkingSet -gt
100MB}|fl

which would list all processes in list
format. This will list each property on a new
line and an extra blank line between each
process.

ps|where{$_.WorkingSet -gt 100MB}|ft
Name,WorkingSet,Threads

which would list 3 columns, the "threads"
column with multiple values (thread IDs).

I could do

ps|where{$_.WorkingSet -gt
100MB}|foreach{$_.WaitForExit()}

Which would wait for all of the "big" process
to terminate before the script continued.

Keep in mind that "text" is just special
objects. In PS, text is streams of strings. If
I really wanted to process text PS has
ample tools for that: I can import/export
delimited files with and without headers, I can
split strings on delimiters into arrays, join
them again or drop down to regular expressions.

I'm not trying to dodge the question, but you
have to be more specific if I am to demonstrate
the PowerShell equivalences to awk, sed, cut
etc.
0 Votes
+ -
awwwww..
civikminded 16th Jan 2010
Powershell?! That's so cute. I just want to grab your cheeks and squeeze em.

A for effort!
0 Votes
+ -
Agreed
murph_z 17th Jan 2010
Wintel people singing the praises of powershell while dissing Unix for the CLI are both tragic and funny in a sad kind of way.

Maybe MS should try a new slogan: "Making the past licensible again" or "Hide the origin" or "Facing Forward, Marching Backwards" ("Hypocrites is us"? )

And, of course, in reality, the line quoted far above is a CLI demo, not something that would work. One of the fun consequences of OO in this case is that the job explodes in complexity once you start scripting the sql command (multiple tables) and structuring the output line. - and note that you can't adopt either of the obvious solutions without complicating the command beyond redemption because you have to stratify and chose randomly within each stratum, not choose randomly from the output after stratify and select.) i.e. you'll be doing a foreach one way or the other.
0 Votes
+ -
Murph, please show me
honeymonster 17th Jan 2010
How to do this with your favorite Unix CLI? If you read the post I was
answering.

Erik was claiming that this about a Windows admin: "He need a DBA who knows
archaic SQL or a software engineer who can right an application to
do it."

Since Erik obviously is challenged on SQL I decided to keep the SQL really
simple .

You are of course correct that if the

If the admin is allowed to know a little more SQL (or allowed to look up the
doc) he could do the following instead (separating out the sql for clarity):


$sql = "select * from table t1 tablesample(100 rows) inner join table2 t2 on
t1.key=t2.key"

Invoke-Sqlcmd -ser sbserver -dat dbname $sql | Export-Csv "sample.csv"

Send-MailMessage -t "murph@unixzealots.org" -sub "Sample data" `
-f "admin@windowszealots.com" `
-body "hi murph, this was too easy." `
-smtp smtp.windowszealots.com`
-att sample.csv


Three lines and you have your sample data. It should be obvious that the sql
scales to joins over many tables.

So, murph, how would you expect your Unix admins to go about this with your
favorite stack?
0 Votes
+ -
mail honeym <assoc_array
murph_z Updated - 17th Jan 2010
The least error prone, and least complex, way I know of is to make an sql script file which dumps combined records into one file for each district and a perl script which selects a few for output to a common file.

The point of this approach is neither elegance nor efficiency, but functional separation, testing, and replication. In fact this is an example of something I do all the time: using the dumbest approach simply because it makes it easy to guarantee that the result is right.

You can do exactly the same (in fact the perl would be portable) with Wintel, the point of the blog wasn't that this couldn't be done or even is hard to do, the point was the wintel people didn't want to go an extra step without lots of formality, hand wringing, and their assertion of control over user decision making in the systems context - where the Unix people responded to the ph calls from their bosses by just getting the job done.

(In thinking about this, the most elegant and efficient way, assuming one had tens of thousands of records per district and hundreds of districts, would be to pipe all tables to a perl associative array and output the final file from that - not that I would actually do this: with today's gear even horrendous SQL queries often take only a few seconds - much less than checking that the array includes its the end points happy )
0 Votes
+ -
Translation??
Ross44 16th Jan 2010
Quote
none of the Unix people had any difficulty either understanding it or doing it; while the Wintel people equally unanimously wanted meetings, paperwork, ?a better understanding of the requirements?, and in something like three out of four cases additional monies from their bosses before they could see about getting it done
End Quote

Tongue in cheek I think what you self-censored into this was something like:

"Since only people with exceptional intelligence, memory capacity and tenacity (and touches of both masochism and over-confidence) can survive in the Unix world, the Unix people found this problem trivial. And since these nerds love problem-solving for its own sake (indeed many of them much prefer solving arcane problems to creating simple, dull systems that work reliably) they wished to take ownership of the definition of the problem ASAP, and were suckers for doing the extra work without extra pay.

Windows people are less intelligent but more realistic and practical. They have lives and families. They realised that it is important to ensure that one fully understands the client?s problem because that is what the client is paying for, and that is what the systems are for. Unlike the Unix people they have accepted that all (large) software is a lash-up and inevitably has failings. So they tend to trudge on through their jobs with an element of resignation. They also appreciate the need to fully pin down the client to an agreed definition of the task, so that there can be no complaints later about what work was done and what result achieved. They also wished to be fairly paid for what they did because they don't look for all their fulfilment in their work."
0 Votes
+ -
And they're all engineers, and the Windows team is the one that doesn't use the 'standard practices' while the engineers in the Linux side are the ones who do everything by the book and have endless meetings etc.

I think you may be confusing corporate culture with platform culture.

But that aside, it's also a little bit of comparing apples to oranges. Linux, being open source, naturally is going to attract people who aren't money oriented. Windows, being corporate, is going to attract the opposite.

Your story is a little like complaining that Buddhists aren't the same as Christians because Christians believe in some kind of deity which as a Buddhist, you find silly.
0 Votes
+ -
Linux is a top DP platform.
peter_erskine@... 16th Jan 2010
And Linux with mysql is a very powerful pairing. mysql was designed to be friendly and easy for command-line tools to work with. Thrown in perl or python and Bob's your uncle - you can knock up some significant data processing in a minutes or hours.
I agree - the Windows folks just don't think in DP terms at all. They are helpless and hopeless by comparison.
0 Votes
+ -
So true
Richard Flude 17th Jan 2010
"while the Wintel people equally unanimously wanted
meetings, paperwork, ?a better understanding of the
requirements?, and in something like three out of four cases
additional monies from their bosses before they could see
about getting it done."

Spot on again Murph! So true.
0 Votes
+ -
Amateurs and professionals
tonymcs@... 17th Jan 2010
Yep the days of the cowboy are gone Rudy and the number of suckers who believe your infallibility is falling towards absolute zero.

Aaaah we can all remember the days when we were the kings of the range and people bowed down as we walked by. Sure we didn't know anything about business or design or users, but we controlled the machines so we had the power.

Let it go Rudy and take some courses in OO and MS Windows, you don;t have to live in the 20C and I've always said your museum could do with more Windows.
After you swore multiple times that you wouldn't waste anymore time reading Murph's blogs???
0 Votes
+ -
Ok he's often obnoxious, juvenile, hypocritical, and dishonest - so what? zdnet has bloggers like that happy

You need to think of him as diversity relief - and, besides, he's useful: every time he writes something he proves my belief that BASIC is bad for your mind because the important thing about his behavior isn't that it reflects forty years of working on a multi-million line BASIC application, but that the application is educational in nature and after 40 years
still coded for Beginners.
0 Votes
+ -
OK. Your blog, your rules... [nt]
Coogol 18th Jan 2010
nt
0 Votes
+ -
Structure
jshaw4343 19th Jan 2010
I will agree with your assessment of the *nix versus Wintel crowd. However, I was struck by the parallel to your last post where the business leaders are obsessed with controls and security over their systems and data. In my experience, the *nix crowd tends to be more creative, dynamic and responsive to needs of the business/users/customers. But I've also found that they tend to be lacking at structure, processes, and controls (don?t have or need them) that make management feel safe at night. That's where the Wintel crowd earns their keep. I'm not saying the *nix infrastructure is weak or unsecure. But the Wintel folks have succeeded at building an entire bureaucracy around system administration where they can provide management (and the auditors) tons of documentation about how their systems are running. That bureaucracy then breeds more bureaucracy generating the meetings, documentation, and additional paperwork. The *nix folks can get things done. But the Wintel crowd makes management feel safe.
0 Votes
+ -
Yes -
murph_z 20th Jan 2010
I think you hit several nails right on the head here -

As usual in evolved situations it's almost impossible to tell cause from result (because yesterday's result is today's cause). In general, however, I take it that most of the perceived needs for control and safety reflect a history of failure and are causative with respect to the on going transformation of wintel into a data processing discipline.

My general solution is to put the Unix guys right in with the users - make them emotionally users rather than distant techno priests - and the ability their tools give them to get the job done should then push IT evolution in those organizations toward more functional (or, at least, less disfunctional) relationships.

Join the conversation!

Formatting +
BB Codes - Note: HTML is not supported in forums
  • [b] Bold [/b]
  • [i] Italic [/i]
  • [u] Underline [/u]
  • [s] Strikethrough [/s]
  • [q] "Quote" [/q]
  • [ol][*] 1. Ordered List [/ol]
  • [ul][*] · Unordered List [/ul]
  • [pre] Preformat [/pre]
  • [quote] "Blockquote" [/quote]
ie8 fix

The best of ZDNet, delivered

ZDNet Newsletters

Get the best of ZDNet delivered straight to your inbox

Facebook Activity

White Papers, Webcasts, & Resources
ie8 fix