Look, this is really complicated, but the over-simplified version is that, on a cellular level the tissue for one set of crotch bits is roughly homologous in both sexes. They just develop different shapes based on developmental signals. A full surgical transition usually involves taking bits from the original equipment and grafting it in such a way that it forms the opposite set of equipment. in M-F they usually go as far as reconstructing a vaginal canal and no further, because a uterus and the rest are complicated structures to create. Technically vestigial tissue for both types of gonads generally remain in both sexes, so it theoretically could be possible to synthetically create a full uterus, ovaries and Fallopian tubes and all, but the ability to synthesize organs of that complexity is currently beyond our technological capability. Functionally you’re left with the equivalent of what a woman would have after a hysterectomy (eg. for something like ovarian or cervical cancer).

And yes, they still have a perineum.