As an attempt to settle the debate of whether or not silicone will damage your driver if you don't let it sit out enough I bring to you a couple of facts.#1
A plastic silicone sealent has a concentration of 77.9 mg per cubic meter of VOC's (Voletile Organic Compounds, meaning they are corrosive to organic compounds) and an emission rate of 26.0 mg per square meter per hour. This is a roughly low emmission rate, but this is after it cures.#2
I will quote this straight from the site,
Silicone sealant (acid curing)
By monitoring the emission over time for a material, it is possible to estimate the time needed for the material to offgass until the emission rate of pollutants reaches a low enough level that the material can be considered safe.
As an example of this the emission profile for an acid curing silicone sealant (Bostik Silicone 2680) is shown below (fig. 5). The sealant was exposed in the chamber as a 0.5 m joint cast into an aluminium U-profile.
Fig. 5: Emission profile of Bostik 2680 silicone sealant over 29 days.
As can be seen acid curing silicone sealants (which I beleive is what most are) releases Acetic acid. Acitic acid is defined as follows from Wikipedia,#3
Acetic acid, also known as ethanoic acid, is an organic chemical compound best recognized for giving vinegar its sour taste and pungent smell. Pure water-free acetic acid (glacial acetic acid) is a colorless hygroscopic liquid and freezes below 16.7°C (62°F) to a colourless crystalline solid. Acetic acid is corrosive, and its vapour causes an irritation to the eyes, dry and burning nose, sore throat and congestion to the lungs, although it is a weak acid based on its ability to dissociate in aqueous solutions.
Acetic acid is one of the simplest carboxylic acids (the second-simplest, next to formic acid). It is an important chemical reagent and industrial chemical that is used in the production of polyethylene terephthalate mainly used in soft drink bottles; cellulose acetate, mainly for photographic film; and polyvinyl acetate for wood glue, as well as many synthetic fibres and fabrics. In households diluted acetic acid is often used in descaling agents. In the food industry acetic acid is used under the food additive code E260 as an acidity regulator.
As they said and as I have said many times before, the gas emitted by the silicone sealant (acetic acid) is corrosive. Wikipedia goes on to explain,
Concentrated acetic acid is corrosive and must therefore be handled with appropriate care, since it can cause skin burns, permanent eye damage, and irritation to the mucous membranes. These burns or blisters may not appear until several hours after exposure. Latex gloves offer no protection, so specially resistant gloves, such as those made of nitrile rubber, should be worn when handling the compound.
I beleive that acetic acid will easily deteriorate foam surrounds, which are still common (Dayton DVC series for instance).If your drivers use foam surrounds let the silicone cure for 4 - 6 days
As for rubber surrounds it is pointed out that you need special rubber gloves to handle acetic acid. This would lead me to beleive that it can still deteriorate most rubbers, so One could assume that even rubber surrounds could be damaged, but most likely only weakened.
I hope this clears some things up.