A speculation about parallelism

What I think we need is a way to develop code and hardware that fully represents the solutions in all of its parts.
Written by Paul Murphy, Contributor on

If I tell you that:

a1x1+ a2x2 + a3x3 = 0
b1x1+ b2x2 + b3x3 = 0
c1x1+ c2x2 + c3x3 = 0

What's the order of execution you apply to understand the information?

Right, there isn't one - the information is spread equally across all the parts of the thing and is read serially but comprehended as one thing.

Now if we write a computer program to solve this thing for some (a,b,c) that program will be inherently serial - with parallel execution for the elements of the vectors aij if the co-efficients aren't scalar - and serial result assembly afterwards.

With present methods a solver for this kind of thing is naturally at its fastest when run on a CPU with an integrated array processor because there really isn't any parallel computing going on - array operations are processed concurrently, but each such process is both independent of the others and serial in itself.

Similarly people argue that having a machine like a T1 UltraSPARC handle 21 concurrent Sybase Engines represents parallel computing, but I think it's fairer to describe that kind of thing as concurrent processing because the threads involved are pretty much independent and completion time for a particular query depends only indirectly on the completion of other threads.

To get past this what I think we need is a way to develop code and hardware that fully represents the solutions in all of its parts.

This is a very difficult idea to expresss - let me try an analogy. Suppose you agreed that a serial computing process is like a ball rolling down a pipe - you put parameter values in at the top and bit later your answers roll out the bottom. Notice that if the pipe were made of transparent stuff you could follow the data as it rolled down the pipe and see, even if the ball breaks up into bits for part of the trip, exactly where each step happens as it morphs into your answer. In terms of that analogy what I'm suggesting is that we need a pipe from which we can read the answer anywhere we look at it, and in which multiple concurrent observers looking at different locations in the pipe would always see the same ball.

Three languages, or computing environments, that I know of do a bit of this. Thus both Maple and Mathematica support direct symbolic manipulation - you could enter the matrix above, for example, and use its determinant as an intermediate result in the development of a formula representing some relationship without giving the thing explicit value.

APL does that too - but goes simualtaneously one step further and one step less because the program itself does indeed represent the general solution, but it acquires reality only when processed all the way through with some set of inputs.

What's needed, however, is something that I have no idea how to do - and don't think any computing environments today even attempt - which is the simualtaneous availability of all information across the entire system.


Editorial standards