Computers are classified by how many instruction streams and data streams they handle at once. This lesson covers the four basic architectures, SISD, SIMD, MISD and MIMD, and the characteristics of the massively parallel computers used for weather forecasting and scientific simulation.
Flynn's classification describes a computer by its instruction streams and data streams. SISD (single instruction, single data): one processor executes one instruction on one data item at a time, as in the Von Neumann model. SIMD (single instruction, multiple data): one instruction is applied to many data items at the same time, as in array processors and graphics processors. MISD (multiple instruction, single data): several processors apply different instructions to the same data; it is rare. MIMD (multiple instruction, multiple data): several processors each execute their own instructions on their own data, as in multicore processors. A massively parallel computer links a very large number of processors, each usually with its own memory, by a high-speed network so that they work simultaneously on different parts of the same problem.
Fig. 1: Flynn's classification, by number of instruction streams (rows) and data streams (columns)
Section 01
The four basic architectures.
An instruction stream is a sequence of instructions being executed; a data stream is a sequence of data items being processed. Count each as single or multiple and you get four combinations.
Architecture
How it works
Example of use
SISD single instruction, single data
One processor fetches and executes one instruction at a time, operating on one data item. No parallelism (apart from pipelining).
An early personal computer with a single-core processor.
SIMD single instruction, multiple data
Many processing units execute the same instruction at the same time, each on a different data item (data-level parallelism).
Array or vector processors; a GPU changing the brightness of every pixel in an image; processing sound samples.
MISD multiple instruction, single data
Several processors execute different instructions on the same data. Very few real computers use it.
Fault-tolerant systems, such as several flight-control computers processing the same sensor data and comparing results.
MIMD multiple instruction, multiple data
Several processors each fetch and execute their own instructions on their own data, independently and at the same time.
Multicore processors in modern PCs and phones; clusters of computers; supercomputers.
Worked example Matching the architecture to the task
A program must add 10 to each of 1000 values in an array, all at the same time. Which architecture suits this best, and why?
The same instruction (add 10) is needed for every item.
There are many data items, so there are multiple data streams.
One instruction stream, many data streams: SIMD.
Each processing unit applies the add to its own element in the same clock cycle.
Answer: SIMD: one instruction applied to many data items at once
Your turn
A laptop has a quad-core processor. One core plays music while another runs a spreadsheet recalculation. Which architecture is this?
MIMD: each core fetches and executes its own instructions on its own data, at the same time.
Examiner trap
Do not confuse SIMD with MIMD. In SIMD every unit executes the same instruction in lock-step; in MIMD each processor can be running a completely different program.
Section 02
Massively parallel computers.
The fastest computers in the world are not built from one enormous processor. They are built from a huge number of ordinary processors working together.
A very large number of processors: thousands or more, often whole computers (nodes) linked together.
Each processor usually has its own memory, rather than all of them sharing one memory.
A high-speed interconnection network links the processors so that they can pass messages and results to each other.
They work simultaneously on different parts of the same problem, so the problem must be one that can be split into many independent parts.
Special software is needed: programs must be written (or compiled) so that the work is divided between processors and the results are combined.
Uses: weather forecasting and climate modelling, simulating molecules for new medicines, modelling the universe, analysing very large data sets.
Your turn
Give two reasons why a weather forecast is well suited to a massively parallel computer.
The atmosphere can be divided into many small regions (a grid), and each processor can calculate the conditions in its own region at the same time. There is a huge amount of calculation that must finish quickly, before the forecast time has passed, and the processors only need to exchange values at the edges of their regions.
Examiner trap
A massively parallel computer is not the same as a multicore PC. The mark points are a very large number of processors, working at the same time on different parts of the same program, communicating through a network.
Stage 1 · Learn
Check what the sim just showed you
Four quick checks. Each correct answer earns XP and records this skill on your profile.
Quick check+10 XP
Which architecture applies the same instruction to many data items at the same time?
Quick check+10 XP
A quad-core processor runs four different programs at the same time. Which architecture is this?
Quick check+10 XP
Which is a characteristic of a massively parallel computer?
Quick check+10 XP
In the lab, why does adding more cores give a smaller and smaller improvement?
Section 03
Why more processors is not always faster.
Splitting a task between processors only helps for the part of the task that can be split. Try it in the lab: change the number of cores and the percentage of the task that can run in parallel.
Sequential parts: some steps depend on the result of the step before, so they must run one after another on one processor.
Communication overhead: processors must send each other data and wait for results, which takes time.
Harder programming: the program must be designed to split the work, and errors such as two processors updating the same data are harder to find.
Worked example How much faster?
A task takes 100 s on one processor. 80% of the work can be split equally between processors; the rest cannot. How long does it take on 4 processors, and on 1000?
The part that cannot be split always takes 20% of 100 s = 20 s.
On 4 processors the other 80 s becomes 80 ÷ 4 = 20 s, so the total is 20 + 20 = 40 s (2.5 times faster).
On 1000 processors it becomes 80 ÷ 1000 = 0.08 s, so the total is 20.08 s.
However many processors are added, the task can never take less than 20 s: at most 5 times faster.
Answer: 40 s on 4 processors; 20.08 s on 1000 processors
Examiner trap
Doubling the number of processors does not halve the time unless the whole task can be split and the processors never need to communicate.
Stage 2 · Exam
Exam-style questions
Unlocks once the four checks above are done. Worth more XP, written to A Level Paper 3 and Paper 4 standard.
Finish the four checks above to unlock the exam questions
Exam style+20 XP
Which architecture is rarely used in practice, but can be found in some fault-tolerant systems?
Exam style+20 XP
Which description matches SISD?
Exam style+20 XP
A task takes 200 s on one processor and half of it can be split between processors. How long does it take on two processors?
Exam style+20 XP
Why must each processor in a massively parallel computer be linked by a fast network?
Skill unlocked
Parallel processing architectures, mastered.
This skill is now recorded on your profile. Keep the chain going.