DEA-C01 Practice Question 46: Redshift Copy Parallel Files

Redshift parallelizes COPY across slices by assigning files to slices. A compressed file cannot be split, so one gzip file means one slice does all the work while the rest idle, precisely what the metrics show. The documented remedy is explicit: "When you load compressed data, we recommend that you split the data for each table into multiple files… Split your data into files so that the number of files is a multiple of the number of slices in your cluster. That way, Amazon Redshift can divide the data evenly among the slices." Size matters too: "Amazon Redshift doesn't take file size into account when dividing the workload. Thus, you need to make sure that the files are roughly the same size, from 1 MB to 1 GB after compression."

Option A makes the imbalance worse, not better. Doubling the nodes doubles the slices, but the single compressed file is still loaded by one of them; the cluster costs twice as much and the load takes just as long.

Option B is a legitimate tuning option: skipping automatic compression analysis and statistics updates shaves some time off a load, but it does nothing about the serialization. One slice would still read all 200 GB.

Option D swaps a purpose-built bulk loader for a much slower path and does not address the root cause either: Spectrum cannot split a single gzip object across readers, so the scan is just as serialized, and INSERT ... SELECT is far less efficient than COPY for bulk ingestion.

Reference: Loading data from compressed and uncompressed files