Caches

Tom Kelliher, CS 240

Apr. 3, 2000

Administrivia

Announcements

Assignment

Read 7.3.

From Last Time

Simple cache, hit/misses.

Outline

1. Handling cache misses.

2. Multiple words/block cache.

3. Memory system design.

Coming Up

Caches: Virtual Memory.

Direct Mapped Caches

A Real, Simple Cache

Consider:

1. Design tradeoff of unified, split caches: higher hit rate (better management of cache lines), higher cache bandwidth.

2. Memory addresses come from PC or ALU.

3. Read hit sequencing.

4. Read miss sequencing.

   Write-through, write-back caches: memory consistency (multiple bus masters) and memory utilization reduction. Tradeoffs.

5. Write hit/miss sequencing for a write-through cache.

   Consider multiprocessor caches, DMA, bus mastering in write-back case. Snooping caches, MESI protocol (modified, exclusive, shared, invalid).

Handling Cache Misses

Three cases:

1. Instruction fetch miss: stall pipeline until we load the instruction from a lower position in the hierarchy.

   PC-4 business in the text.

2. Data fetch miss: stall pipeline as for a lw hazard. Or, keep going, until the data is actually needed.

   How do you associate the incoming word with a reg file position? (Load/Store units)

3. Data store miss: stall pipeline. Or use a write buffer (L3 cache), to perform writes ``later.'' Must stall if write buffer is full.

   Loads must snoop the write buffer.

Second Cache

Let's design another cache. Parameters:

1. Direct mapped.

2. Block size is four words. Why would we want a larger block size? (Exploit spatial locality)

   Consider the hit rate when fetching eight words consecutively with block sizes of one and four words.

   To a point, increasing block size increases the hit rate. At what point does this break down?

3. 4K blocks.

4. 32 bit memory space.

Questions:

1. How many bits/block? Total size of the cache?

2. Size of the data portion of the cache, in bytes? (This is the number you see quoted.)

Organization of the cache:

Consider:

1. Why would we want a larger block size? (Exploit spatial locality)

   Consider the hit rate when fetching eight words consecutively with block sizes of one and four words.

2. To a point, increasing block size increases the hit rate. At what point does this break down? Why? (Fewer blocks.)

   Also, the miss penalty time will increase because of larger memory transfers.

3. How can a load instruction result in a memory write? How can a store instruction result in a memory read?

Memory System Design

Three memory organizations:

What is interleaving and its history?

Memory access delays:

1. One cycle to send an address. To access a block, we send just one address.

2. 15 cycles to allow DRAM operation to complete.

3. One cycle to transmit data.

Compute the bytes/cycle for each organization for four word blocks.

Points to consider:

1. The real point of interleaving is that we now have several independent memories.

   So we can perform several writes simultaneously, assuming we can write single words. This increases our write bandwidth, leading to fewer stalls.

2. But, DRAM sizes keep increasing, decreasing the number of banks for a given memory size.