Crossovers are tuned to a specific frequency, but in the calculation for that frequency, the speaker load comes into play.
For example, if I take a crossover tuned to 800hz between 8 ohm woofers and 8 ohm tweeters, and replace the woofer with a 4 ohm driver, that effects the crossover.
In the above crossover the Tweeter will stay at 800hz, but the woofer crossover will drop down to 400hz, leaving a very large gap in the response between 400hz and 800hz.
However, if you replace the woofer with a 16 ohms driver, then the crossover will jump to 1600hz leaving a large overlap between 800hz and 1600hz.
So, while it will probably work, it won't sound right.
You could correct the load by substituting a 8 ohm power resistor in place of the driver. Assuming that the drivers are 8 ohms. I'm guessing for casual use you would need minimum a 20w resistor, though more likely a 50w to 100w resistor would be better.
Then we come to the issue of Cabinet Tuning. The port in a cabinet is very tightly tied to the internal volume of the cabinet. Like Pipe Organ pipe, the length and diameter of the Port are very specific. If yo leave a large opening where the missing speaker once was, the cabinet becomes tuned to a whole new frequency.
Say as an example, the cabinet tuning would rise from 30hz up to about 80hz.