I heard that smoking stems in marijuana plants leads to penis shrinkage.  Is this true?

The most common cause of acute penis shrinkage is ejaculation. But I guess that is not what you are concerned about.

In an adult, there is nothing that will make a penis "smaller" but there are plenty of medications, drugs, behaviors and lifestyle choices that can interfere with libido or sexual drive, the ability to get and maintain erections and successfully ejaculate. Marijuana is one of them. But I guess that is also not what you are concerned about.

There are reports of testicle size shrinkage in adult and adolescent chronic marijuana users due to the estrogenic effects of the drug--whether stem or leaf has not been formally studied to my knowledge. There is no evidence this actually affects penis size, though one could potentially theorize that using marijuana regularly as a developing adolescent could interfere with normal hormonal and genital development.