Caches

Tom Kelliher, CS 240

Mar. 31, 2000

Administrivia

Announcements

Assignment

Read 7.3.

From Last Time

Memory hierarchy, terminology, cache basics.

Outline

1. Direct mapped caches, DECStation 3100. Exploiting spatial locality.

2. Handling cache misses.

3. Memory system design.

Coming Up

Caches: Handling misses. Write-through, write back caches. Memory design support for caches.

Direct Mapped Caches

Direct mapped cache:

A cache structured such that a memory block is associated with exactly one cache block, based upon the address of the memory block. The usual algorithm is:

First Cache

Let's design our first cache. Parameters:

1. Direct mapped --- a memory block maps to exactly one cache block.

2. Block size is one word.

3. Eight blocks.

4. Memory size is 64 words, byte addressable (always the case).

Questions:

1. How many bits for the cache?

2. Consider the memory trace: 22, 26, 22, 26, 16, 4, 16, 18, run on a cold cache. How many hits, misses? What's the hit rate? What are we missing from the cache design (tag bits).

3. Why don't we need to store the entire memory address?

4. What about a valid bit?

5. Now, how many bits for the cache?

A Real, Simple Cache

DECStation 3100:

1. MIPS R2K.

2. 64KB cache D- and I-caches. Block size is 4B.

3. 32-bit address space.

Questions:

1. How is the address partitioned?

2. How many bits per block?

3. How many bits for the cache?

Consider:

1. Design tradeoff of unified, split caches: higher hit rate (better management of cache lines), higher cache bandwidth.

2. Memory addresses come from PC or ALU.

3. Read hit sequencing.

4. Read miss sequencing.

   Write-through, write-back caches: memory consistency (multiple bus masters) and memory utilization reduction. Tradeoffs.

5. Write hit/miss sequencing for a write-through cache.

Handling Cache Misses

Three cases:

1. Instruction fetch miss: stall pipeline until we load the instruction from a lower position in the hierarchy.

   PC-4 business in the text.

2. Data fetch miss: stall pipeline as for a lw hazard. Or, keep going, until the data is actually needed.

3. Data store miss: stall pipeline. Or use a write buffer (L3 cache), to perform writes ``later.'' Must stall if write buffer is full.

   Loads must snoop the write buffer.

Second Cache

Let's design another cache. Parameters:

1. Direct mapped.

2. Block size is four words. Why would we want a larger block size? (Exploit spatial locality)

   Consider the hit rate when fetching eight words consecutively with block sizes of one and four words.

   To a point, increasing block size increases the hit rate. At what point does this break down?

3. 4K blocks.

4. 32 bit memory space.

Questions:

1. How many bits/block? Total size of the cache?

2. Size of the data portion of the cache, in bytes? (This is the number you see quoted.)

Organization of the cache:

Consider:

1. Why would we want a larger block size? (Exploit spatial locality)

   Consider the hit rate when fetching eight words consecutively with block sizes of one and four words.

2. To a point, increasing block size increases the hit rate. At what point does this break down? Why? (Fewer blocks.)

   Also, the miss penalty time will increase because of larger memory transfers.

3. How can a load instruction result in a memory write? How can a store instruction result in a memory read?

Memory System Design

Three memory organizations:

What is interleaving and its history?

Memory access delays:

1. One cycle to send an address. To access a block, we send just one address.

2. 15 cycles to allow DRAM operation to complete.

3. One cycle to transmit data.

Compute the bytes/cycle for each organization for four word blocks.

Points to consider:

1. The real point of interleaving is that we now have several independent memories.

   So we can perform several writes simultaneously, assuming we can write single words. This increases our write bandwidth, leading to fewer stalls.

2. But, DRAM sizes keep increasing, decreasing the number of banks for a given memory size.