Sri Aurobindo has clearly and succinctly stated the issues with respect to the options of rebirth: “If the physical universe were the sole manifested world, or if it were a quite separate world, rebirth as a part of the evolutionary process would be confined to a constant succession of direct transmigrations from one body to another; death would be immediately followed by a new birth without any possibility of an interval,–the passage of the soul wold be a spiritual circumstance in the uninterrupted series of a compulsory, mechanical, material procedure. The soul would have no freedom from Matter; it would be perpetually bound to its instrument, the body, and dependent on it for the continuity of its manifested existence. But we have found that there is a life on other planes after death and before the subsequent rebirth, a life consequent on the old and preparatory of the new stage of terrestrial existence. Other planes co-exist with ours, are part of one complex systems and act constantly upon the physical which is their own final and lowest term, receive its reactions, admit a secret communication and commerce.”
Even if the soul is not, from the beginning of its evolutionary journey, capable of transferring itself to these other planes or worlds, as it grows in its capacity and consciousness, it finds not only the capability but the necessity to sojourn in these other planes between births. Until that time we could envision a direct and immediate rebirth into a new form on the material plane, but once the evolutionary force has developed far enough, the process would clearly involve this interactive process.
We see in scriptures such as the Tibetan Book of the Dead a serious attempt to understand and interact with the soul in the worlds beyond the physical after death. The Tibetan Book of the Dead calls the inter-regnum the “bardo” and it has various stages which correspond with the systematic dissolution of the hold of the material frame, the vital being, the emotional ties, and the mental organisation that corresponded to the prior birth, thereby freeing the soul to carry the subtle impressions of the past lifetime forward in its choosing of a new birth.
Sri Aurobindo, The Life Divine, Book 2, Part 2, Chapter 22, Rebirth and Other Worlds; Karma, the Soul and Immortality