We don't have to keep track of this much data between rects, so restructure things to make it more simple. This allows the JPEG parsing code to be a pure function which only depends on the input.