Note: This is no longer an issue in vSphere 5.1.
Today while doing a manual DR failback test at one of the customers sites where they use Virtual Distributed Switches, I have noticed when removing/adding a Virtual Machine to inventory it lose the configuration for its network interface of which port group its connected to and the like. As the customer was running vSphere 4.1u1, I went ahead and tested the same in my lab running vSphere 5.0 which end up having the exact same behavior. Its important to note that this behavior only affect Virtual Machines that is connected to a Distributed Switch port group, & it does not affect port group configuration for Standard Switches. This behavior although look odd seems to be nothing but as intended by engineering as it makes the assumption that you are restoring the VM to a different Infrastructure where these Distributed Switches might not exist.
Agree or not agree is not the purpose of this post, but to confirm this behavior as well to be in front warning that you should keep that configuration handy before hand in case you need for any reason to remove your VMs from inventory. In my case was carrying out a manual disaster recovery fallback without involving vCenter Site Recovery Manager. The good part in my scenario these configuration was well documented and we were able to recover it quickly, though if we did not have that it could take us days to recover. I hope you get to read this before doing such a thing to save you the time from failing to recover.
At the end, I hope this behavior change in future releases as it will make our life easier in the field. Other than that, SRM 5 make up for this gap if you failover & failback using SRM 5 then this is nothing you have to worry about.
If you have a different thought, reasoning, or a better way of doing this then please leave it in the comment area below.