Introduction to Superscalar Execution

Tom Kelliher, CS 240

Apr. 16, 2004

Administrivia

Announcements

Assignment

Read 7.1--2.

From Last Time

Data hazards in pipelines.

Outline

  1. Introduction to superscalar pipelining

  2. Data dependencies.

  3. Out of order execution.

  4. Multiple Instruction Issue

  5. In-order execute pipeline and instruction scheduling.

Coming Up

Caches.

Introduction to Superscalar Pipelining

  1. Historical Progression of IPC: < 1, = 1, > 1. The entire pipeline must be widened.

    Challenges: small register files, multiple-branch predictions, multiple line fetches from caches.

  2. Range of parallelism: coarse- to fine-grained.

  3. Superscalar techniques address ILP. Let's parallelize a sequential binary.

  4. What's the upper bound on IPC? It depends.

    Text processing: low, mostly.

    Image processing, multimedia: high.

    Median operation on an image example: 

    medianImage(image dest, image src)
{
   for each pixel, p, in src
      p in dest = medianPixel(p in src);
}

medianPixel(pixel p)
{
   find the <= 8 neighboring pixels of p;
   compute and return the median value;
}
    Challenges: exposing potential ILP to the compiler.

    Example. Parallelize the following: 

    sum = 0;

for (i = 0; i < last; ++i)
   sum += array[i];

  5. Compiler techniques: loop unrolling, invariant code migration, strength reduction, etc.

Types of Data Dependencies

  1. RAR. Not a problem at all.

  2. RAW. A ``true'' dependency.

  3. WAR. A ``false'' dependency.

  4. WAW. Another ``false'' dependency.

Consider the code segment: 
      r1 = r2 + r3
      r4 = r1 + r5
      r1 = r6 + r7
      r8 = r1 + r4
ISA registers vs. physical registers. Register renaming?

Rename the previous example where the Register Alias Table (RAT) is initially: 

r1  -> p12     r2  ->  p6     r3  ->  p9     r4  -> p15
r5  -> p1      r6  -> p10     r7  ->  p8     r8  -> p14

Free List: p5, p11, p13, p4.
Which dependencies were removed? Which remain?

Out of Order execution

  1. What is it?

  2. In-order completion.

  3. How is it done?

Multiple Instruction Issue

  1. In-order execution case.

    Structural hazard stalls.

  2. Out of order execution case.

    Only stall if no free list entries.

In-Order Execute Pipeline

  1. This is ``simple'' case.

    Consider two-instruction issue.

    Must consider:

    1. Structural hazards.

    2. Data hazards.

    3. Control hazards.

    How to handle?

  2. Instruction pairs must be aligned.
    1. First instruction: R-format or branch.

    2. Second instruction: Memory access. (Add an address adder.)

  3. If first instruction stalls, both stall.

  4. Second instruction may stall due to data, control dependencies.

The pipeline:

What are the inter- and intra-instruction pair dependencies?

What are our options in increasing the functionality of that second ALU? (reg = reg op immed instrs, reg = reg op reg instrs) Additional dependencies?

Instruction Scheduling Example

Consider the code segment: 

sum = 0;

for (i = 0; i < last; ++i)
   sum += array[i];
Which might compile to: 
top:     lw $t0, 0($s1)
         addu $s2, $s2, $t0
         addi $s1, $s1, -4
         bne $s1, $0, top
How will the code be scheduled?

The addi could be raised, but what's it gain?

Suppose we unroll once: 

top:     lw $t0, 0($s1)
         addu $s2, $s2, $t0
         lw $t0, -4($s1)
         addu $s2, $s2, $t0
         addi $s1, $s1, -8
         bne $s1, $0, top
Where are the stalls? How can we introduce temp variables to eliminate some stalls?

How will the improved code schedule?

Is this an improvement?

By the way, what happens with the lw offsets?

Unroll twice more:

Is this an improvement?

When do you stop?


Thomas P. Kelliher
Thu Apr 15 15:29:44 EDT 2004
Tom Kelliher