This timing concept is mentioned in PHY layer spec 36.213, but as far as i can assume the reason of 4 SF is something like this:
Let me start with this,In LTE, in FDD, RTT is 8 msec. The question arises why 8 HARQ processes? The answer is- consider the case of downlink, when eNodeB sends data, UE takes 3msec as processing time, therefore, it sends ACK/NACK 4 msec later, similarly after receiving ACK/NACK, eNodeB takes 3msec processing time and sends a new transmission/ re-transmission in a total of 8 msec
So suppose if eNB has send DCI at SF0,(1ms), UE will take 3ms processing time, at send UL after 4SF.
I don't know whether i am correct or not, but i can conclude some thing like that. Do post if you have correct answer.