In task switching, extending the response–cue interval (RCI) reduces the switch cost—the detriment to performance when switching compared to repeating tasks. This reduction has been used as evidence for the existence of task-set decay processes. Recently, this has been challenged by the observation of sequential dependencies on the RCI effect: switch cost is only reduced at longer RCIs when the previous trial had a short RCI. This trial-wise variation of RCI is thought to affect the temporal distinctiveness (TD) of a previous task’s episodic trace, affecting the probability of its automatic retrieval on the current trial; importantly, TD is thought to be independent of the current trial’s RCI. The present study highlights a dependency between the current RCI and TD, and demonstrates that a decay model can reproduce some patterns of data attributed to TD. Further, the decay account makes a strong prediction when TD is held constant: repetition response times should slow as the RCI increases, and switch response times should be facilitated. This prediction was tested via re-analysis of extant data and three experiments. The re-analysis provided some evidence for the decay account, but Experiments 1 and 2 report slowing for task repetition and switch trials, which cannot be explained by a task-set decay process. Experiment 3, which utilized tasks requiring perceptual judgements, showed small evidence for decay. We conclude that the data are largely consistent with the TD account and that the evidence for decay of higher-level task-sets is not convincing.